Cloud-edge collaborative embodiment navigation dynamic replanning control method and system

Abstract

This invention belongs to the field of embodied robot control technology, and relates to an embodied navigation dynamic replanning control method and system based on cloud-edge collaboration. The method includes: decoupling memory state output control commands for navigation control and pre-simulating the expected latent features of the environment at the next moment; acquiring the actual latent features at the next moment, filtering environmental noise based on the actual and expected latent features, comparing the geometric differences between the purified actual and expected geometric features, and determining whether to trigger a cloud-based replanning request based on a dynamic threshold; upon triggering, the cloud-based large model generates multiple obstacle avoidance action sequences based on abnormal data packets uploaded from the edge, parallelly extrapolating trajectory evolution and selecting the optimal path to generate incremental memory patches; receiving the incremental memory patches and writing them into the synchronized cloud cognitive state through non-parametric methods, executing the optimal obstacle avoidance action, and then resuming autonomous navigation; thereby achieving zero-sample adaptation to unfamiliar dynamic environments.

Description

Technical Field

[0001] This invention relates to the field of embodied robot control technology, and in particular to an embodied navigation dynamic replanning control method and an embodied navigation dynamic replanning control system based on cloud-edge collaboration. Background Technology

[0002] In related technologies, when solving the problem of embodied navigation in complex and unknown environments, multimodal sensor fusion (for static 3D modeling), end-to-end networks, or endpoint alignment fine-tuning are mainly used to optimize path planning; however, these technologies have the following core shortcomings in real-time dynamic replanning at the edge under extremely low computing power: It relies heavily on redundant hardware and external systems and lacks an endogenous mechanism for sensing environmental topology changes. Existing solutions mainly rely on stacking physical hardware (such as binoculars, LiDAR, etc.) or coupling with external IoT for "passive physical collision avoidance," and the underlying logic still assumes that the baseline path is safe. In unstructured environments, if a topological change occurs that exceeds the sensor's detection range (such as the passage ahead being suddenly blocked by a moving obstacle), the system cannot proactively trigger an alarm based on the inherent logical differences in the model, making its environmental generalization and perception capabilities extremely fragile.

[0003] The model weights are statically fixed, and the edge device lacks the ability to adapt to the environment in real time without fine-tuning. The weight parameters of the existing pre-trained model are fixed after deployment to the edge device. When the robot faces an unfamiliar dynamic scene outside the training set data distribution (OOD), it is prone to collisions due to the lack of a dynamic memory update mechanism. If gradient backpropagation is performed on the edge device to adapt to the new environment, its computational power consumption is completely insufficient to meet the engineering requirements of millisecond-level dynamic replanning for mobile robots.

[0004] The lack of a counterfactual multi-hypothesis inference mechanism leads to a computational paradox between "high-dimensional inference" and "low-latency control." Most existing navigation strategies can only output a single deterministic action, lacking the "counterfactual foresight" capability to simulate the future consequences of multiple paths in advance, resulting in extremely low decision-making tolerance. If a generative world model is forcibly introduced into the existing system, the massive decoding computational power will cause a surge in edge-side inference latency, severely compromising the high-frequency real-time performance (>20Hz) of chassis control. Summary of the Invention

[0005] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies. To this end, one objective of this invention is to propose a cloud-edge collaborative embodied navigation dynamic replanning control method. This method enables high-frequency closed-loop navigation by a lightweight model on the edge during normal environments. When encountering sudden environmental changes, it calculates the difference between expected and actual characteristics, triggers a high-computing-power generative world model in the cloud for multi-hypothesis deduction as needed, and distributes the local memory of the safe path to the edge, achieving zero-sample adaptation to unfamiliar dynamic environments.

[0006] The second objective of this invention is to propose an embodied navigation dynamic replanning control system based on cloud-edge collaboration.

[0007] To achieve the above objectives, the first embodiment of this invention proposes a cloud-edge collaborative embodied navigation dynamic replanning control method, comprising the following steps: The edge device integrates multimodal observations and natural language navigation commands, decouples memory state output control commands for navigation control, and pre-simulates the expected latent features of the environment at the next moment; It acquires the actual latent features at the next moment, filters environmental noise based on the actual latent features and the expected latent features of the environment at the next moment, compares the geometric differences between the purified actual geometric features and the expected geometric features, and determines whether to trigger a cloud replanning request based on a dynamic threshold; When a cloud replanning request is triggered, the cloud-based large model generates multiple obstacle avoidance action sequences based on abnormal data packets uploaded from the edge device, performs parallel trajectory evolution simulations, selects the optimal path, and generates incremental memory patches; The edge device receives the incremental memory patches, writes them into the synchronized cloud cognitive state through non-parametric methods, executes the optimal obstacle avoidance action, and then resumes autonomous navigation.

[0008] The embodied navigation dynamic replanning control method based on cloud-edge collaboration proposed in this application has the following advantages: it can perform high-frequency closed-loop navigation by a lightweight model on the edge in normal environments; when encountering sudden environmental changes, it can trigger a high-computing-power generative world model in the cloud to perform multiple hypothesis deductions as needed by calculating the difference between expected and actual features, and send the local memory of the safe path to the edge to achieve zero-sample adaptation to unfamiliar dynamic environments.

[0009] In addition, the embodied navigation dynamic replanning control method based on cloud-edge collaboration proposed in the above embodiments of the present invention may also have the following additional technical features: Optionally, the edge-side integrates multimodal observations and natural language navigation commands, decoupling memory state output control commands for navigation control. This includes: acquiring the current real visual observations and natural language navigation commands, wherein the real visual observations include RGB images and depth images; extracting global semantic features from the RGB images using a visual Transformer, and extracting spatial geometric features from the depth images using a deep residual network; concatenating the global semantic features and the spatial geometric features, then projecting them using an MLP and aligning them with the command latent features to generate the current real latent features, wherein the command latent features are pre-trained CLIP text. The encoder encodes the input natural language navigation commands and maps them to a unified semantic space through a projection layer to obtain command latent features. A gated recurrent unit is used to recursively calculate the current deterministic memory state in conjunction with the state of the previous time step. The posterior distribution network combines the current deterministic memory state and the current true latent features to sample the current random state. The current deterministic memory state and the current random state together constitute a working memory pool slice. A multimodal cross-attention network is constructed, and using the command latent features as the query, the current true latent features and the working memory pool slice are fused to generate navigation features. The navigation features are then mapped by an MLP to output control commands for navigation control.

[0010] Optionally, the expected latent features of the environment at the next moment are pre-simulated, including: inputting the working memory pool slice and the control command into the transfer network on the end side, forward extrapolating the expected deterministic state at the next moment, and outputting and caching the prior distribution of the expected latent features from the prior distribution network.

[0011] Optionally, filtering environmental noise based on the next-moment real latent features and the next-moment expected latent features of the environment, and comparing the geometric differences between the purified real geometric features and the expected geometric features, and determining whether to trigger a cloud replanning request based on a dynamic threshold, includes: constructing a semantic attention mask generation network, inputting the next-moment real latent features into the semantic attention mask generation network to generate a semantic attention mask matrix; multiplying the semantic attention mask matrix element-wise with the next-moment real latent features and the next-moment expected latent features of the environment to suppress unstructured noise, and then projecting it to the spatial occupancy subspace through a geometric dimension filter to obtain the purified real geometric features and expected geometric features; calculating a dynamic trigger threshold based on the current chassis linear velocity and angular velocity, and simultaneously calculating the difference between the purified real geometric features and expected geometric features through cosine distance; comparing the difference with the dynamic trigger threshold, if the difference is greater than the dynamic trigger threshold, triggering a cloud replanning request, immediately issuing an emergency braking command, suspending the end-side policy network, and packaging abnormal data packets for uploading to the cloud; if the difference is less than or equal to the dynamic trigger threshold, determining that there is no sudden change in the environment, and continuing the end-side navigation loop.

[0012] Optionally, when a cloud-based replanning request is triggered, the cloud-based large model generates multiple obstacle avoidance action sequences based on the abnormal data packets uploaded from the edge, and performs parallel trajectory evolution deduction and selects the optimal path to generate incremental memory patches. This includes: using a conditional diffusion backbone network with abnormal data packets as context, generating N sets of counterfactual action candidate sets that satisfy chassis physical constraints through K-step back diffusion; performing rolling temporal prediction on the N sets of counterfactual action candidate sets that satisfy chassis physical constraints simultaneously using a parallelized temporal Transformer network, and deducing N state topology evolution trajectory clusters in the latent space; and scoring the N state topology evolution trajectory clusters based on a comprehensive evaluation function from three dimensions: instruction alignment, collision penalty, and action smoothness, selecting the optimal trajectory with the highest score, and encapsulating its corresponding latent state sequence as an incremental memory patch.

[0013] Optionally, the edge receives the incremental memory patch and synchronizes the cognitive state in the cloud through non-parametric writing. After executing the optimal obstacle avoidance action, it resumes autonomous navigation, including: skipping the gradient backpropagation process and directly splicing the hidden state sequence in the incremental memory patch to the edge's local working memory pool to complete the real-time synchronization of the cognitive state; temporarily blocking the output of the edge policy network and forcibly extracting the first optimal action instruction from the incremental memory patch and issuing it for execution to drive the embodied agent to leave the danger zone; after the obstacle avoidance sequence is executed, verifying the consistency between the current observation and the injected memory, clearing the abnormal state cache and old expected features, and switching back to the edge navigation mode.

[0014] To achieve the above objectives, a second aspect of the present invention proposes an embodied navigation dynamic replanning control system based on cloud-edge collaboration, comprising: an edge-side navigation and expected representation generation module, used to fuse multimodal observations and natural language navigation commands, decouple memory state output control commands for navigation control, and pre-simulate the expected latent features of the environment at the next moment; a virtual-real difference monitoring and triggering module, used to acquire the real latent features at the next moment, filter environmental noise based on the real latent features and the expected latent features of the environment at the next moment, compare the geometric differences between the purified real geometric features and the expected geometric features, and determine whether to trigger a cloud replanning request based on a dynamic threshold; a counterfactual detection and multi-hypothesis evolution module, used to generate multiple sets of obstacle avoidance action sequences based on abnormal data packets uploaded from the edge side when it is determined that a cloud replanning request has been triggered, and to perform parallel trajectory evolution and select the optimal path to generate incremental memory patches; and a gradient-free memory overwriting and control takeover module, used to receive incremental memory patches, write them into the synchronous cloud cognitive state in a non-parametric manner, and resume autonomous navigation after executing the optimal obstacle avoidance action. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the embodied navigation dynamic replanning control method based on cloud-edge collaboration according to an embodiment of the present invention. Figure 2 This is a block diagram of an embodied navigation dynamic replanning control system based on cloud-edge collaboration according to an embodiment of the present invention. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0017] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0018] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0019] Figure 1 This is a flowchart illustrating the embodied navigation dynamic replanning control method based on cloud-edge collaboration according to an embodiment of the present invention, as shown below. Figure 1 As shown, the embodied navigation dynamic replanning control method based on cloud-edge collaboration includes the following steps: S101 integrates multimodal observations and natural language navigation commands at the edge, decouples memory state output control commands for navigation control, and pre-simulates the expected latent features of the environment at the next moment.

[0020] As one embodiment, the edge-side integrates multimodal observations and natural language navigation commands, decoupling memory state output control commands for navigation control. This includes: acquiring the current real visual observations and natural language navigation commands, where the real visual observations include RGB images and depth images; extracting global semantic features from the RGB images using a visual Transformer, and extracting spatial geometric features from the depth images using a deep residual network; concatenating the global semantic features and spatial geometric features, then projecting them using an MLP and aligning them with the command latent features to generate the current real latent features, where the command latent features are pre-trained CLIP text. The encoder encodes the input natural language navigation commands and maps them to a unified semantic space through a projection layer to obtain command latent features. A gated recurrent unit is used to recursively calculate the current deterministic memory state in combination with the state of the previous time step. The posterior distribution network combines the current deterministic memory state and the current true latent features to sample the current random state. The current deterministic memory state and the current random state together constitute a working memory pool slice. A multimodal cross-attention network is constructed, and the command latent features are used as queries to fuse the current true latent features and the working memory pool slice to generate navigation features. The navigation features are then mapped by an MLP to output control commands for navigation control.

[0021] As an example, the expected latent features of the environment at the next moment are predicted by: slicing the working memory pool and the transfer network at the control command input end, forward deducing the expected deterministic state at the next moment, and outputting the prior distribution of the expected latent features by the prior distribution network and caching it.

[0022] As a specific embodiment, the following detailed steps are included: S1011 proposes a state decoupling method based on joint encoding of ViT and ResNet and cyclic state space. First, the end system receives the current time. Real visual observations, including RGB images With depth images The pre-trained Visual Transformer (ViT) is used to extract global semantic features from the RGB image, and a deep residual network (ResNet) is used to extract spatial geometric features from the depth image. The two features are concatenated and passed through a projection mapping layer, and aligned with the instruction features in the same high-dimensional semantic space to obtain the current true latent features. :

[0023] Secondly, a gated recurrent unit is used to extract deterministic memory states containing historical topological context. The random state of the previous moment Actions performed at the previous moment The pieces are then stitched together and combined with the definitive memory from the previous moment. Perform recursive calculations:

[0024] Next, a posterior distribution network is constructed to extract and capture the random states that capture the uncertainty of the current environment's blind spots. The deterministic memory obtained from the current calculation With the latest real hidden features The input is a multilayer perceptron network, and the output is the mean of a multidimensional Gaussian distribution. Log-square of the standard deviation :

[0025] To support end-to-end gradient backpropagation in the network, a reparameterization technique is used to sample the current stochastic state from the Gaussian distribution:

[0026] in, For random noise that follows a standard normal distribution, Finally, the extracted deterministic memory states are combined with the random states to determine the current moment. End-side working memory pool slice .

[0027] S1012, First, the edge system uses a pre-trained CLIP text encoder to encode the input natural language instruction sequence, and then maps it to a unified semantic space through a projection layer to obtain the instruction latent features. Secondly, a multimodal cross-attention network is constructed. This network incorporates the current real latent features. Slice of working memory pool on the end side (Including deterministic states) With randomness The features are concatenated along the feature dimension to form a cascaded state tensor. : Using instruction hidden features As a query vector, in the form of a cascaded state tensor As key and value vectors, they are projected using a linear transformation matrix:

[0028]

[0029]

[0030] in, This represents the learnable weight matrix in the policy network. The dot product of the query vector and key vector is calculated, scaled, and normalized using Softmax to obtain the attention weights. These weights are then weighted and summed with the value vectors to extract key navigation features containing strong textual guidance intent.

[0031] in, The feature dimension of the key vector. For normalized exponential functions, For matrix The transpose of the key navigation features. Finally, the fused key navigation features are... Input a multi-layer residual fully connected network (MLP), map it to a continuous motion space, and output the chassis control command at the current moment:

[0032] in, Indicates the chassis linear velocity. This indicates the chassis angular velocity. The end-side control unit will transmit this command. The data is sent to the underlying actuators in real time to drive the robot chassis to perform physical displacement and complete the high-frequency closed-loop navigation at the current time step.

[0033] S1013, firstly, in Time control command Simultaneously with the distribution, the edge-side transition network immediately initiates a pure forward inference of the environmental dynamics. It slices the working memory pool at the current moment. (Including deterministic memory states) With randomness ) and control instructions to be executed Feature splicing is performed, and the expected deterministic state at the next time step is deduced using gated recurrent units: :

[0034] Secondly, construct a priori distributed network. Predict the expected latent feature distribution at the next time step without involving pixel-level rendering. This involves predicting the deterministic state. Input a multilayer perceptron network and output the mean of the expected multidimensional Gaussian distribution (prior distribution). logarithm of standard deviation :

[0035] Then, the next moment is determined. The expected prior distribution of latent features This represents a purely forward-looking projection of future environmental structural changes and uncertainties in blind spots:

[0036] Finally, the expected feature distribution parameters are written to the edge-side feature buffer. When proceeding to the next time step t+1 and obtaining the true posterior distribution... Then, calculate the KL divergence between the two:

[0037] This divergence serves as a core constraint for state verification feedback and dynamic self-evolution, forcing the model's forward inference to align with real physical laws.

[0038] S102, obtain the true latent features of the next time step, filter environmental noise based on the true latent features and the expected latent features of the environment at the next time step, compare the geometric difference between the purified true geometric features and the expected geometric features, and determine whether to trigger a cloud replanning request based on the dynamic threshold.

[0039] As an example, environmental noise is filtered based on the real latent features and expected latent features of the environment at the next moment. The geometric difference between the purified real geometric features and the expected geometric features is compared. A dynamic threshold is used to determine whether to trigger a cloud replanning request. This includes: constructing a semantic attention mask generation network; inputting the real latent features at the next moment into the semantic attention mask generation network to generate a semantic attention mask matrix; multiplying the semantic attention mask matrix element-wise with the real latent features and expected latent features of the environment at the next moment to suppress unstructured noise, and then projecting it to the spatial occupancy subspace through a geometric dimension filter to obtain the purified real geometric features and expected geometric features; calculating a dynamic trigger threshold based on the current chassis linear velocity and angular velocity, and simultaneously calculating the difference between the purified real geometric features and expected geometric features through cosine distance; comparing the difference with the dynamic trigger threshold. If the difference is greater than the dynamic trigger threshold, a cloud replanning request is triggered, an emergency braking command is immediately issued, the end-side policy network is suspended, and an abnormal data packet is packaged and uploaded to the cloud. If the difference is less than or equal to the dynamic trigger threshold, it is determined that there is no sudden change in the environment, and the end-side navigation loop continues.

[0040] As a specific embodiment, the following detailed steps are included: S1021, firstly, at time The edge system obtains the latest real hidden features. To eliminate interference from unstructured noise such as abrupt changes in ambient lighting, dynamic shadow depth, or material texture on the monitoring of virtual-real differences, a semantic attention mask generation network is constructed. Inputting this network, through global average pooling and multilayer perceptron mapping, calculates the spatial saliency weights of each dimension of the features, generating a semantic attention mask matrix:

[0041] Next, perform the feature masking operation. With current real hidden features And the expected hidden features cached in the previous time step. Element-wise product is performed separately to suppress visually redundant dimensions that do not contribute to navigation in the feature flow, retaining only salient features representing physical occupancy and topology:

[0042] Next, a pre-defined geometric filter is used to further reduce the dimensionality of the masked features. Using a projection operator with fixed weights, the high-dimensional semantic features are projected onto a geometric subspace representing occupancy and obstacle boundaries, resulting in a geometrically filtered subset of features.

[0043] in, Operators for geometric feature extraction are then used. Finally, the geometric feature subsets are compared. With expected geometric feature subset This establishes a physical-to-virtual consistency metric based on the current environment. The process employs a dual filtering mechanism to eliminate visual interference, ensuring that the system only triggers subsequent computational rerouting logic when it detects unexpected changes in the physical topology (such as newly appearing obstacles).

[0044] S1022, First, establish a dynamic trigger threshold adjustment mechanism. The system's fault tolerance should be positively correlated with the intensity of the robot's movement to compensate for sensor motion ambiguity and positioning drift caused by high-speed movement or sharp turns. This is based on the current chassis linear velocity. With angular velocity Set dynamic trigger threshold:

[0045] in, As a static base threshold, These are the compensation coefficients for linear velocity and angular velocity, respectively. This formula ensures that the robot has more stringent monitoring sensitivity when stationary or cruising at low speeds. Secondly, the cosine difference between actual observations and expected projections within the calculated geometric subspace is used. By measuring a subset of true geometric features With expected geometric feature subset The angle within a high-dimensional manifold is used to assess the degree of deviation between the evolution of the physical environment and the model's dynamic predictions.

[0046] The range of this difference is: The larger the value, the more serious the deviation between the current observation and the expected physical laws.

[0047] S1023, trigger and suspend logic. If The system determines that the current physical topology change is within the expected prediction range of the world model (such as known corners or static corridors), and no structural abrupt change has occurred. At this time, the edge system maintains autonomous operation and continues to execute the hidden space closed-loop control logic in S101 in a loop without requesting support from the cloud.

[0048] like The system determines that an unpredictable dynamic change has occurred in the environment (such as a suddenly appearing pedestrian, an unmodeled falling obstacle, or complete obstruction of the sensor's field of view). To ensure the robot's safety, the edge system immediately triggers an emergency protection mechanism: Emergency Braking: Directly send an emergency stop command to the underlying motor controller and clear the current action buffer.

[0049] State Suspension: Immediately suspend the decision-making process of the end-side policy network to prevent erroneous operations caused by incorrect features.

[0050] Cloud Routing: Redirecting the current environment's hidden features Historical memory status The navigation command data to be completed is packaged and uploaded to a high-computing cloud engine via a high-speed wireless link to request deeper large-scale model perception and global path replanning.

[0051] S103, when a cloud-based replanning request is triggered, the cloud-based large model generates multiple sets of obstacle avoidance action sequences based on the abnormal data packets uploaded from the device side, and performs parallel trajectory evolution and selects the optimal path to generate incremental memory patches.

[0052] As an example, when a cloud-based replanning request is triggered, the cloud-based large model generates multiple obstacle avoidance action sequences based on the abnormal data packets uploaded from the edge, and performs parallel trajectory evolution deduction and selects the optimal path to generate incremental memory patches. This includes: using a conditional diffusion backbone network with abnormal data packets as context, generating N sets of counterfactual action candidate sets that satisfy chassis physical constraints through K-step back diffusion; performing rolling temporal prediction on the N sets of counterfactual action candidate sets that satisfy chassis physical constraints simultaneously using a parallelized temporal Transformer network, and deducing N state topology evolution trajectory clusters in the latent space; and scoring the N state topology evolution trajectory clusters based on a comprehensive evaluation function from three dimensions: instruction alignment, collision penalty, and action smoothness, selecting the optimal trajectory with the highest score, and encapsulating its corresponding latent state sequence as an incremental memory patch.

[0053] As a specific embodiment, the following detailed steps are included: S1031 proposes a conditional diffusion backbone network. A high-dimensional action manifold generation method is proposed. This network aims to evolve standard normally distributed isotropic noise into counterfactual action sequences that conform to robot kinematic constraints through constrained physical semantic guidance.

[0054] Network architecture constitutes conditions for the diffusion of backbone networks A hybrid symmetric architecture based on Temporal Residual MLP and Cross-Attention is adopted: Action Encoding Operator: for the action trajectory of the current denoising step. Sine and cosine positional encoding is performed to capture temporal correlations. The diffusion step embedding operator utilizes multiple linear projection layers to count the diffusion iteration steps. Mapping to high-dimensional embedding vectors Condition-aware module: Utilizes a cross-attention mechanism to detect and process hidden anomalies uploaded from the endpoint. With instruction characteristics Injecting feature streams as context guides the semantic direction of action generation.

[0055] Mathematical Expression and Deductive Logic Diffusion Backbone Network The core task is to fit the noise prediction function, whose input is the first... Noisy motion trajectory of the step Diffusion step embedding and conditional feature mapping Predicted noise term Expressed as:

[0056] Within this backbone network, the progressive states of neurons in each layer are calculated as follows:

[0057] pass Step-by-step iterative reverse diffusion utilizes a reparameterized mean estimator to obtain pure noise. The clean counterfactual action sequence is restored from the original text.

[0058] in, The preset variance scheduling parameters, This is standard normal noise.

[0059] Physical constraint mapping to ensure the generation The candidate action set satisfies the chassis physical limits, and the output of the backbone network is connected to a nonlinear mapping layer (Tanh Squashing Layer) to force the control quantity to be constrained to speed. With angular velocity Within this range, through this generative architecture, the model can "explode" in the latent space, producing phenomena such as sudden, sharp turns. Rapid reversing Emergency obstacle avoidance schemes that are completely mutually exclusive on the topology.

[0060] S1032, Rolling Temporal Prediction (MPC Derivation): Proposes a parallel temporal Transformer Transition Network. The latent space evolution extrapolation method abandons the single-path recursive structure and adopts a multi-layer multi-head self-attention (MSA) architecture to achieve massive concurrent simulation. It utilizes a causal masking module to ensure that state extrapolation strictly follows physical causality. In each prediction step... The cloud engine will Each candidate action is packaged into a tensor batch using the formula:

[0061] The dynamic coupling feature flow is obtained, and then the expected hidden state at the next time step is deduced using a nonlinear mapping layer:

[0062] This deduction process performs vector drift entirely within the manifold space without involving pixel-level decoding, ultimately generating N parallel clusters of latent space state topological evolution trajectories. Furthermore, a geometric consistency loss is introduced during training to constrain the physical coherence of the trajectory.

[0063] S1033 proposes a policy evaluation and memory archiving method based on a multi-criteria cost function, with the cloud system using a lightweight occupancy head. Perform physical safety checks on each projected trajectory and output the spatial occupancy probability at each time step:

[0064] Construct a comprehensive evaluation function:

[0065] The trajectory clusters are scored, and while considering command alignment and collision penalties, motion oscillations are suppressed using the second derivative of the action sequence. The system then selects the optimal trajectory index with the highest score. The corresponding expected latent feature evolution sequence is then encapsulated as an incremental memory patch. This patch, as a strong verification logic, is sent to the edge device. It directly refreshes the hidden state register of the edge device's recurrent network through non-parametric overwriting, thereby enabling cognitive reshaping and guiding the robot out of dangerous areas caused by sudden environmental changes.

[0066] S104 receives incremental memory patches on the device side and writes them into the synchronous cloud cognitive state in a non-parametric manner. After performing the optimal obstacle avoidance action, it resumes autonomous navigation.

[0067] As one example, the device receives incremental memory patches and writes them to the synchronized cloud cognitive state in a non-parametric manner. After performing optimal obstacle avoidance actions, it resumes autonomous navigation, including: Skip the gradient backpropagation process and directly concatenate the hidden state sequence in the incremental memory patch to the local working memory pool on the edge to complete the real-time synchronization of the cognitive state; temporarily block the output of the edge policy network, forcibly extract the first optimal action instruction from the incremental memory patch and issue it for execution, driving the embodied agent to leave the danger zone; after the obstacle avoidance sequence is executed, verify the consistency between the current observation and the injected memory, clear the abnormal state cache and old expected features, and switch back to the edge navigation mode.

[0068] As a specific embodiment, the following detailed steps are included: S1041, the cloud engine will select the optimal inference sequence. The data is distributed to the edge system, which performs cognitive refresh through a non-parametric register rewriting mechanism. The system skips the gradient backpropagation process for all model weights and directly writes the hidden feature vector sequence within the patch into the memory unit of the Recurrent State Space Model (RSSM). The backbone network employs an attention architecture and uses KV-Cache injection technology to concatenate the derived feature key-value pairs into the current observation cache.

[0069] This process enables the edge model to obtain the latent space context containing the topological laws of the new environment in advance before physical displacement, realizing real-time synchronization between the cognitive state and the cloud-based inference results.

[0070] S1042, Forced Open-Loop Execution. After memory injection is complete, the end-side system immediately exits the suspended state and enters the strongly guided open-loop control mode. At this time, the system temporarily blocks the decision output of the end-side policy network and forcibly extracts and issues the corresponding first-segment optimal action instruction from the cloud patch. The underlying actuator responds to this command via a high-frequency servo interface, driving the robot to perform physical displacement according to a safe trajectory verified in the cloud.

[0071] This takeover mechanism ensures that, within the consistency window during which the perception system has not yet been fully rebuilt, the robot can a priori evade the current collision hazard zone based on high-confidence counterfactual inferences.

[0072] S1043, Restore High-Frequency Closed-Loop Operation on the End-Side. After the obstacle avoidance sequence is completed and the robot enters the safe zone, the end-side monitoring unit clears abnormal state alarms and old expected feature caches. This is due to the hidden state of the end-side recurrent neural network. The predictions have been fully updated in the patch, and the edge-side Visual Language Model (VLM) has completed its perceptual adaptation to the current environmental topology. The system validates the latest visual observation features. After confirming the restoration of physical consistency and maintaining continuity with the injected memory, the robot automatically switches back to the low-power, high-frequency navigation mode defined by S101. This logic achieves cognitive evolution through instantaneous intervention of cloud computing power, enabling the robot to continuously navigate in unfamiliar and rapidly changing scenarios without online training.

[0073] In summary, this application presents a cloud-edge collaborative embodied navigation dynamic replanning control method. The edge system is responsible for low-power closed-loop navigation and environment prediction generation. The edge system uses a visual encoder to extract global semantic and spatial geometric features of observations, and combines them with instructions to map them to a high-dimensional aligned space to generate real latent features. Subsequently, deterministic memory states containing historical topological context and random states capturing uncertainty in blind spots are separated to form a slice of the working memory pool at the current moment. On this basis, the policy network fuses instructions and memory features through a multimodal cross-attention mechanism to output high-frequency chassis control instructions. At the same time, the transfer network synchronously starts a decoder-less dynamic pure forward inference to generate the prior distribution of the expected latent features for the next moment, and calculates the KL divergence with the subsequent real posterior distribution to drive the model self-evolution. The virtual-real difference monitoring mechanism is responsible for triggering cloud-edge collaboration on demand. Upon entering the next time step, the system purifies the real latent features through semantic masks and geometric dimension filters, removing unstructured noise such as illumination, and extracting the geometric subspace features representing the space occupied. The system dynamically sets the trigger threshold based on the chassis's real-time linear and angular velocities and calculates the cosine difference between the actual observations and expected projections within the geometric subspace. When the difference is below the trigger threshold, it is determined that there are no topological changes in the environment, maintaining autonomous navigation on the edge. When the difference exceeds the trigger threshold, it is determined that a sudden change has occurred in the environment, and the edge immediately issues an emergency braking command, suspends the policy network, and uploads the context features to the cloud to trigger computational rerouting. The cloud engine is responsible for receiving edge requests and executing high-computing-power multi-hypothesis projections. The cloud conditional diffusion backbone network uses the edge's abnormal latent features and commands as context to generate N sets of counterfactual action sequences that satisfy the chassis's physical limits within the action latent space. Subsequently, the parallelized temporal Transformer transfer network performs rolling temporal predictions on these N candidate actions, concurrently simulating their future state topology evolution trajectory clusters within the manifold space. Ultimately, the system scores trajectory clusters based on comprehensive command alignment, space occupancy penalty, and motion smoothness, selects the globally optimal escape path, and encapsulates its corresponding expected latent feature evolution sequence as an incremental memory patch for deployment. Thus, the system achieves on-demand scheduling of cloud-edge computing power through an endogenous triggering mechanism based on virtual-real differences. It achieves high-frequency, low-power navigation by relying on the decoupled state architecture of the edge side, and achieves zero-sample adaptation in topological change scenarios by leveraging counterfactual multi-hypothesis deduction in the cloud. Without requiring online fine-tuning on the edge side, it simultaneously meets the triple requirements of real-time performance, safety, and environmental generalization for mobile robot navigation.

[0074] To implement the above embodiments, this invention also proposes a cloud-edge collaborative embodied navigation dynamic replanning control system, such as... Figure 2As shown, the embodied navigation dynamic replanning control system based on cloud-edge collaboration includes: an edge-side navigation and expected representation generation module 10, a virtual-real difference monitoring and triggering module 20, a counterfactual detection and multi-hypothesis evolution module 30, and a gradient-free memory overwrite and control takeover module 40.

[0075] The terminal-side navigation and expected representation generation module 10 is used to integrate multimodal observations and natural language navigation commands, decouple memory state output control commands for navigation control, and pre-simulate the expected latent features of the environment at the next moment; the virtual-real difference monitoring trigger module 20 is used to obtain the real latent features at the next moment, filter environmental noise based on the real latent features and the expected latent features of the environment at the next moment, and compare the geometric differences between the purified real geometric features and the expected geometric features, and determine whether to trigger a cloud replanning request based on a dynamic threshold; the counterfactual detection and multi-hypothesis evolution module 30 is used to generate multiple sets of obstacle avoidance action sequences based on the abnormal data packets uploaded from the terminal side when it is determined that a cloud replanning request has been triggered, and to deduce the trajectory evolution in parallel and select the optimal path to generate incremental memory patches; the gradient-free memory overwriting and control takeover module 40 is used to receive incremental memory patches, and to write them into the synchronous cloud cognitive state through non-parametric writing, and to resume autonomous navigation after executing the optimal obstacle avoidance action.

[0076] It should be noted that the above description and examples of the embodied navigation dynamic replanning control method based on cloud-edge collaboration are also applicable to the embodied navigation dynamic replanning control system based on cloud-edge collaboration in this embodiment, and will not be repeated here.

[0077] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0079] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0080] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0081] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0082] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0083] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0084] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0085] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0086] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0087] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0088] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A dynamic replanning control method for embodied navigation based on cloud-edge collaboration, characterized in that, Includes the following steps: The system integrates multimodal observations and natural language navigation commands at the edge, decouples memory state output control commands for navigation control, and pre-simulates the expected latent features of the environment at the next moment. Obtain the true latent features of the next moment, filter environmental noise based on the true latent features of the next moment and the expected latent features of the environment of the next moment, compare the geometric difference between the purified true geometric features and the expected geometric features, and determine whether to trigger a cloud replanning request based on a dynamic threshold. When a cloud-based replanning request is triggered, the cloud-based large model generates multiple obstacle avoidance action sequences based on the abnormal data packets uploaded from the device, and performs parallel trajectory evolution and selects the optimal path to generate incremental memory patches. The device receives the incremental memory patch and writes it into the synchronous cloud cognitive state in a non-parametric manner. After performing the optimal obstacle avoidance action, it resumes autonomous navigation.

2. The embodied navigation dynamic replanning control method based on cloud-edge collaboration as described in claim 1, characterized in that, The system integrates multimodal observations and natural language navigation commands at the edge, and decouples memory state output control commands for navigation control, including: Acquire real-time visual observations and natural language navigation instructions, wherein the real-time visual observations include RGB images and depth images; A visual Transformer is used to extract global semantic features from RGB images, and a deep residual network is used to extract spatial geometric features from depth images. The global semantic features and the spatial geometric features are concatenated and then projected by an MLP to align with the instruction latent features to generate the current true latent features. The instruction latent features are generated by encoding the input natural language navigation instructions using a pre-trained CLIP text encoder and mapping them to a unified semantic space through a projection layer. The current deterministic memory state is calculated recursively by using a gated recurrent unit combined with the state of the previous time step. The posterior distribution network combines the current deterministic memory state and the current true latent features to sample the current random state. The current deterministic memory state and the current random state together constitute a working memory pool slice. A multimodal cross-attention network is constructed, and the command latent features are used as queries. The current real latent features and working memory pool slices are fused to generate navigation features. The control commands are then output through MLP mapping for navigation control.

3. The embodied navigation dynamic replanning control method based on cloud-edge collaboration as described in claim 2, characterized in that, The implicit features of the expected environment in the next moment include: The working memory pool slice and the control command are input into the end-side transfer network to forward extrapolate the expected deterministic state at the next time step. The prior distribution network outputs the prior distribution of the expected latent features and caches it.

4. The embodied navigation dynamic replanning control method based on cloud-edge collaboration as described in claim 1, characterized in that, Based on the actual latent features of the next time step and the expected latent features of the environment of the next time step, environmental noise is filtered, and the geometric difference between the purified actual geometric features and the expected geometric features is compared. A dynamic threshold is used to determine whether to trigger a cloud-based replanning request, including: Construct a semantic attention mask generation network, and input the real latent features of the next time step into the semantic attention mask generation network to generate a semantic attention mask matrix; The semantic attention mask matrix is ​​multiplied element-wise with the real latent feature and the expected latent feature of the environment at the next time step to suppress unstructured noise, and then projected onto the spatial occupancy subspace through the geometric dimension filter to obtain the purified real geometric feature and expected geometric feature. The dynamic trigger threshold is calculated based on the current chassis linear velocity and angular velocity, and the difference between the purified real geometric features and the expected geometric features is calculated using the cosine distance. Compare the difference degree with the dynamic trigger threshold. If the difference degree is greater than the dynamic trigger threshold, a cloud replanning request is triggered, an emergency braking command is immediately issued, the end-side policy network is suspended, and an abnormal data packet is packaged and uploaded to the cloud. If the difference degree is less than or equal to the dynamic trigger threshold, it is determined that there is no sudden change in the environment, and the end-side navigation loop continues.

5. The embodied navigation dynamic replanning control method based on cloud-edge collaboration as described in claim 1, characterized in that, When a cloud-based replanning request is triggered, the cloud-based large model generates multiple obstacle avoidance action sequences based on abnormal data packets uploaded from the endpoint. It then performs parallel trajectory evolution analysis and selects the optimal path to generate incremental memory patches, including: Using a conditional diffusion backbone network with anomalous data packets as context, N sets of counterfactual action candidate sets that satisfy chassis physical constraints are generated through K-step back diffusion. Parallelized temporal Transformer network performs rolling temporal prediction on N sets of counterfactual action candidates that satisfy chassis physical constraints, and derives N clusters of state topology evolution trajectories in the latent space; Based on a comprehensive evaluation function, N state topology evolution trajectory clusters are scored from three dimensions: instruction alignment, collision penalty, and action smoothness. The optimal trajectory with the highest score is selected, and its corresponding hidden state sequence is encapsulated as an incremental memory patch.

6. The embodied navigation dynamic replanning control method based on cloud-edge collaboration as described in claim 1, characterized in that, The terminal receives the incremental memory patch and writes it to the synchronized cloud cognitive state in a non-parametric manner. After performing the optimal obstacle avoidance action, it resumes autonomous navigation, including: Skip the gradient backpropagation process and directly splice the hidden state sequence in the incremental memory patch into the local working memory pool on the edge to complete the real-time synchronization of cognitive state; Temporarily block the output of the end-side policy network, forcibly extract the first optimal action instruction from the incremental memory patch and send it out for execution, driving the embodied agent to escape the danger zone; After the obstacle avoidance sequence is completed, verify the consistency between the current observation and the injected memory, clear the abnormal state cache and old expected features, and switch back to the end-side navigation mode.

7. A cloud-edge collaborative embodied navigation dynamic replanning control system, characterized in that, include: The edge-side navigation and expected representation generation module is used to integrate multimodal observations and natural language navigation commands, decouple memory state output control commands for navigation control, and pre-simulate the expected latent features of the environment at the next moment; The virtual-real difference monitoring trigger module is used to obtain the real latent features of the next moment, filter environmental noise based on the real latent features and the expected latent features of the environment in the next moment, compare the geometric difference between the purified real geometric features and the expected geometric features, and determine whether to trigger a cloud replanning request based on the dynamic threshold. The counterfact detection and multi-hypothesis evolution module is used to generate multiple sets of obstacle avoidance action sequences based on the abnormal data packets uploaded from the edge when it is determined that a cloud replanning request is triggered. The cloud-based large model generates multiple sets of obstacle avoidance action sequences, infers trajectory evolution in parallel, and selects the optimal path to generate incremental memory patches. The gradient-free memory overwrite and control takeover module is used to receive incremental memory patches and write them into the synchronous cloud cognitive state in a non-parametric manner. After performing the optimal obstacle avoidance action, it resumes autonomous navigation.

Images