A visual navigation method and system based on a denoising diffusion bridge model

Abstract

The application discloses a visual navigation method and system based on a denoising diffusion bridge model. The method comprises the following steps: acquiring initial image data; performing feature coding processing on the initial image data to generate a context vector; performing linear modulation processing on the context vector to generate a conditional variable; generating a prior action according to the context vector and a preset motion rule; inputting the prior action and the conditional variable into the denoising diffusion bridge model for diffusion training to generate a target navigation action; and wherein the prior action is used as an initial state of the denoising diffusion bridge model when performing reverse denoising processing. The application can effectively reduce the redundant iteration in the diffusion process, reduce the cumulative error, greatly improve the efficiency and stability of action generation, and can be widely applied in the field of computer technology.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a visual navigation method and system based on a denoising diffusion bridge model. Background Technology

[0002] Visual navigation is a crucial technology for autonomous mobile robots to achieve target-oriented movement in complex environments. Current visual navigation technologies primarily employ methods based on mapping and path planning, or deep learning. While traditional path planning methods are stable, they heavily rely on accurate maps and environmental perception, exhibiting poor adaptability in unknown or dynamic environments and struggling to adjust navigation strategies in real time. Deep reinforcement learning-based methods, although adaptable to complex environments, suffer from high training costs, low sample efficiency, and limited generalization ability, requiring large amounts of training data to adapt to new environments. Imitation learning methods, while capable of learning navigation strategies from expert demonstration data, suffer from significant data quality impacting final navigation performance and are difficult to generalize to unknown environments. In recent years, diffusion-based visual navigation methods have become a research hotspot. However, these methods rely on Gaussian noise as initial input, leading to a significant deviation between the distribution of target actions and actual navigation requirements, thus increasing the computational complexity of the denoising step. Furthermore, the target actions generated by diffusion models are relatively sparse, making them difficult to directly apply to complex navigation tasks, especially in applications requiring precise control and immediate response.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The embodiments of this application aim to at least partially solve one of the technical problems in the related art. Therefore, the main objective of the embodiments of this application is to propose a visual navigation method and system based on a denoising diffusion bridge model, which can effectively reduce redundant iterations in the diffusion process, reduce accumulated errors, significantly improve the efficiency and stability of action generation, and adapt to complex dynamic environments.

[0005] To achieve the above objectives, one aspect of this application proposes a visual navigation method based on a denoising diffusion bridge model, the method comprising the following steps:

[0006] Obtain initial image data;

[0007] The initial image data is subjected to feature encoding processing to generate a context vector;

[0008] The context vector is linearly modulated to generate condition variables;

[0009] Based on the context vector and preset motion rules, generate prior actions;

[0010] The prior actions and the conditional variables are input into the denoising diffusion bridge model for diffusion training to generate target navigation actions; wherein, the prior actions are used as the initial state of the denoising diffusion bridge model when performing reverse denoising processing.

[0011] In some embodiments, the initial image data includes an initial observation image and an initial target image, and the feature encoding process of the initial image data to generate a context vector includes:

[0012] The initial observation image is subjected to feature extraction processing to obtain the observation image features;

[0013] The initial target image is subjected to feature extraction processing to obtain target image features;

[0014] The observed image features and the target image features are subjected to feature fusion processing to generate the context vector.

[0015] In some embodiments, the linear modulation processing of the context vector to generate condition variables includes:

[0016] The context vector is input into the feature linear modulation module;

[0017] The characteristic linear modulation module generates modulation parameters based on the context vector.

[0018] The characteristic linear modulation module determines the condition variable based on the modulation parameters.

[0019] In some embodiments, generating prior actions based on the context vector and preset motion rules includes:

[0020] The context vector is mapped to the action space through the fully connected layer of the neural network model, and low-dimensional action features corresponding to the context vector are generated.

[0021] Based on the low-dimensional motion features and the preset motion rules, an initial candidate path is generated;

[0022] The prior action is determined based on the initial candidate path.

[0023] In some embodiments, the method further includes:

[0024] Based on Gaussian prior rules, data is sampled from a Gaussian white noise distribution to obtain the prior action;

[0025] or,

[0026] The prior action is obtained by training the initial observation image input conditional variational autoencoder model from the initial image data.

[0027] In some embodiments, the step of inputting the prior action and the condition variable into a denoising diffusion bridge model for diffusion training to generate target navigation actions includes:

[0028] The prior actions and the condition variables are input into the denoising diffusion bridge model;

[0029] The denoising diffusion bridge model is trained by positive noise addition based on the real path point sequence to generate noisy actions;

[0030] The noise diffusion bridge model performs reverse noise reduction on the noise action based on the prior action and the condition variable to generate the target navigation action.

[0031] In some embodiments, after inputting the prior action and the condition variable into the denoising diffusion bridge model for diffusion training to generate the target navigation action, the method further includes:

[0032] Generate navigation action instructions based on the target navigation action;

[0033] The target navigation action and the navigation action command are sent to the robot's motion control module, which then controls the robot to execute the target navigation action according to the navigation action command.

[0034] To achieve the above objectives, another aspect of this application proposes a visual navigation system based on a denoising diffusion bridge model, the system comprising the following modules:

[0035] The initial image data acquisition module is used to acquire initial image data;

[0036] The feature encoding processing module is used to perform feature encoding processing on the initial image data to generate a context vector;

[0037] A linear modulation processing module is used to perform linear modulation processing on the context vector to generate condition variables;

[0038] The prior action generation module is used to generate prior actions based on the context vector and preset motion rules;

[0039] The target navigation action generation module is used to input the prior actions and the condition variables into the denoising diffusion bridge model for diffusion training to generate target navigation actions; wherein, the prior actions are used as the initial state of the denoising diffusion bridge model when performing reverse denoising processing.

[0040] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0041] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0042] The embodiments of this application include at least the following beneficial effects: This application provides a visual navigation method and system based on a denoising diffusion bridge model. This scheme acquires initial image data; performs feature encoding processing on the initial image data to generate a context vector; performs linear modulation processing on the context vector to generate condition variables; generates prior actions based on the context vector and preset motion rules; inputs the prior actions and condition variables into the denoising diffusion bridge model for diffusion training to generate target navigation actions; wherein, the prior actions are used as the initial state of the denoising diffusion bridge model during reverse denoising processing. The embodiments of this application utilize the denoising diffusion bridge model, using the prior action distribution as the initial state of the diffusion bridge model for the reverse diffusion process, enabling the initial state to be efficiently corrected to the target action distribution, thereby achieving a fast and stable visual navigation strategy. That is, the embodiments of this application transform the traditional visual navigation method's diffusion denoising process starting from Gaussian noise into a diffusion denoising process starting from an informative prior action distribution, effectively reducing redundant iterations in the diffusion process, reducing accumulated errors, significantly improving the efficiency and stability of action generation, and adapting to complex dynamic environments. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the steps of a visual navigation method based on a denoising diffusion bridge model provided in an embodiment of this application.

[0044] Figure 2 This is a schematic diagram of the overall process of a visual navigation method based on a denoising diffusion bridge model provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the structure of a visual navigation system based on a denoising diffusion bridge model provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0048] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0049] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0051] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0052] 1) Embodied Intelligence refers to the ability of an intelligent agent to make autonomous decisions and execute tasks through real-time interaction with the environment via a perception-action closed loop. In the embodiments of this application, it specifically refers to an intelligent system that acquires environmental information through visual sensors and combines it with physical motion to achieve navigation goals.

[0053] 2) Visual navigation is a technology that uses visual sensors (such as cameras or RGB-D (RGB-Depth, color-depth) sensors) to acquire environmental information and combines it with path planning and motion control algorithms to guide an intelligent agent from a starting point to a target location. In this application embodiment, it specifically refers to visual navigation based on pure RGB images.

[0054] 3) Diffusion Model: A deep learning framework based on a probabilistic generative model, which learns the data distribution by progressively adding noise (forward process) and denoising (backward process). In this embodiment, the Denoising Diffusion Bridge Model (DDBM) is used to generate the optimal action sequence from the current state to the target state.

[0055] 4) Imitation Learning: A method of training agent behavior by imitating expert demonstration data (such as human operations or predefined strategies). In the embodiments of this application, a diffusion model is used to model the noise distribution of expert trajectories as the basis for navigation strategy generation.

[0056] 5) Action Policy: The agent generates rules or probability distributions for its next action based on the current state. In this embodiment, the action policy is generated using a denoised diffusion bridge model, and the long-term cumulative reward is optimized using a Markov Decision Process (MDP) framework.

[0057] 6) Denoising Process: The reverse operation in the diffusion model that gradually restores noisy data to the original data. In this embodiment, this process is used to iteratively optimize from the initial trajectory with noise interference to generate a smooth navigation path that conforms to physical constraints.

[0058] Visual navigation is a crucial technology for autonomous mobile robots to achieve target-oriented movement in complex environments. Current visual navigation technologies primarily employ methods based on mapping and path planning, or deep learning. While traditional path planning methods are stable, they heavily rely on accurate maps and environmental perception, exhibiting poor adaptability in unknown or dynamic environments and struggling to adjust navigation strategies in real time. Deep reinforcement learning-based methods, although adaptable to complex environments, suffer from high training costs, low sample efficiency, and limited generalization ability, requiring large amounts of training data to adapt to new environments. Imitation learning methods, while capable of learning navigation strategies from expert demonstration data, suffer from significant data quality impacting final navigation performance and are difficult to generalize to unknown environments. In recent years, diffusion-based visual navigation methods have become a research hotspot. However, these methods rely on Gaussian noise as initial input, leading to a significant deviation between the distribution of target actions and actual navigation requirements, thus increasing the computational complexity of the denoising step. Furthermore, the target actions generated by diffusion models are relatively sparse, making them difficult to directly apply to complex navigation tasks, especially in applications requiring precise control and immediate response.

[0059] For example, traditional mapping and path planning techniques typically rely on high-precision sensors (such as LiDAR) for accurate ranging and positioning. However, in unknown or dynamic environments, maps are difficult to update in real time, computational complexity is high, and they are sensitive to noise and lighting conditions. Deep Reinforcement Learning (DRL) methods optimize navigation strategies through trial and error learning by the agent in a simulated environment, but training costs are high, sample efficiency is low, and it is difficult to generalize to different environments. Imitation Learning (IL) methods learn from expert demonstration data and can adapt to the target task relatively quickly, but they are highly dependent on the quality of expert data. Within the framework of imitation learning, visual navigation methods based on diffusion models (DM) have become a new research direction in recent years. These methods utilize the diffusion process to model data, employing a denoising diffusion probabilistic model (DDPM). During the training phase, Gaussian noise is added to the data to make its distribution tend towards a standard Gaussian distribution. During the inference phase, target navigation actions are generated through a denoising process. However, traditional diffusion models start with random Gaussian noise, requiring numerous steps to "correct" the noise during the generation process, resulting in inefficient generation. The generated actions often lack sufficient guidance, making it difficult to achieve stable navigation in complex scenes. Due to the significant difference between the initial noise and the target action, the model needs to make numerous corrections during the gradual denoising process, increasing inference time and computational overhead.

[0060] In view of this, this application provides a visual navigation method and system based on a denoising diffusion bridge model. This scheme acquires initial image data; performs feature encoding on the initial image data to generate a context vector; performs linear modulation on the context vector to generate condition variables; generates prior actions based on the context vector and preset motion rules; and inputs the prior actions and condition variables into the denoising diffusion bridge model for diffusion training to generate target navigation actions. The prior actions serve as the initial state of the denoising diffusion bridge model during reverse denoising. This application utilizes a denoising diffusion bridge model, using the prior action distribution as the initial state of the diffusion bridge model for reverse diffusion, enabling the initial state to be efficiently corrected to the target action distribution. This achieves a fast and stable visual navigation strategy. Specifically, this application transforms the traditional visual navigation method's diffusion denoising process starting from Gaussian noise into a diffusion denoising process starting from an informative prior action distribution. This effectively reduces redundant iterations in the diffusion process, lowers accumulated errors, significantly improves the efficiency and stability of action generation, and can adapt to complex dynamic environments.

[0061] This application provides a visual navigation method based on a denoising diffusion bridge model, relating to the field of computer technology. This visual navigation method based on a denoising diffusion bridge model can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster consisting of multiple physical servers, or a distributed system. It can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a visual navigation method based on a denoising diffusion bridge model, but is not limited to the above forms.

[0062] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframes, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0063] Please see Figure 1 , Figure 1 This is an optional flowchart of a visual navigation method based on a denoising diffusion bridge model provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S105.

[0064] Step S101: Obtain initial image data;

[0065] The initial image data refers to the original image without any processing, and includes the initial observation image and the initial target image.

[0066] The initial observation image is the raw image information about the observed scene or object directly acquired through visual means (such as cameras, sensors, etc.). It reflects the visual characteristics of the object's appearance, position, and posture in the actual scene and is an intuitive visual record of the real world. In the embodiments of this application, the initial observation image is a series of images observed by the robot at different times, i.e., an RGB (Red Green Blue Color Mode) observation sequence.

[0067] The initial target image refers to a pre-given single RGB image used to guide the output of the denoising diffusion bridge model to move towards the target image.

[0068] Step S102: Perform feature encoding processing on the initial image data to generate a context vector;

[0069] The initial image data includes the initial observation image and the initial target image.

[0070] In some embodiments, step S102 may include: performing feature extraction processing on the initial observation image to obtain observation image features; performing feature extraction processing on the initial target image to obtain target image features; and performing feature fusion processing on the observation image features and the target image features to generate a context vector.

[0071] In the specific implementation, the robot acquires continuous RGB observation sequences and target images. Visual features of the RGB observation sequences and target images are extracted via a convolutional network or a Transformer encoder, respectively. These visual features are then fused to generate a context vector C describing the current scene and target information. t .

[0072] Step S103: Perform linear modulation processing on the context vector to generate condition variables;

[0073] In some embodiments, step S103 may include: inputting a context vector into a feature linear modulation module; generating modulation parameters based on the context vector using the feature linear modulation module; and determining condition variables based on the modulation parameters using the feature linear modulation module.

[0074] In the specific implementation, the context vector C t The input is passed to the characteristic linear modulation module, which modulates the input context vector C. t Modulation parameters a and b are generated, and then condition variables are formed by characteristic linear modulation based on modulation parameters a and b. The condition variables are used to adjust the behavior of the generation strategy. The condition variables obtained after modulation are then input into the diffusion bridge module to guide the generation of the final navigation action.

[0075] Step S104: Generate prior actions based on the context vector and preset motion rules;

[0076] In some embodiments, step S104 may include: mapping the context vector to the action space through the fully connected layer of the neural network model and generating low-dimensional action features corresponding to the context vector; generating an initial candidate path based on the low-dimensional action features and preset motion rules; and determining the prior action based on the initial candidate path.

[0077] In some embodiments, the method may further include: sampling data from a Gaussian white noise distribution based on Gaussian prior rules to obtain prior actions; or training an input conditional variational autoencoder model of the initial observed image in the initial image data to obtain prior actions.

[0078] In its implementation, the prior action generation module provides three prior generation strategies: Gaussian prior generation strategy, rule-based prior generation strategy, and learning-based prior generation strategy.

[0079] Among them, the Gaussian prior generation strategy refers to directly sampling from the Gaussian white noise distribution as a benchmark for comparison with traditional diffusion models. The rule-based prior generation strategy refers to mapping the context vector C through a fully connected layer. t The algorithm moves into the action space and, combined with pre-defined rules (such as using the predicted path length ρ and the confidence θ of the motion direction classification, like going straight, turning left, turning right, U-turn, etc.), generates a parabolic path as the initial action. The learning-based prior generation strategy refers to using a lightweight Conditional Variational Autoencoder (CVAE) model to learn the relationship between observations and actions from expert data, thereby obtaining prior actions that better match the requirements of the actual task.

[0080] Among them, the rule-based prior generation strategy maps C through a fully connected layer. t The initial motion is generated by combining preset motion rules (such as the parabola generation formula). For example, a parabola can be generated according to the standard form y = ax². 2 +bx+c or vertex form y=a(xh) 2 +k is used to solve for parameters, ensuring that the path is smooth and meets kinematic requirements.

[0081] In practical implementation, the specific generation principle and process of the rule-based prior generation strategy are as follows:

[0082] 1) Design principle: A fully connected layer (FC) is used to map environmental perception information (such as observed image features) to the action space, extract low-dimensional features to generate preliminary prior actions; the prior actions follow preset trajectory constraints, which are usually set as parabolic trajectories to simulate natural motion trends and improve motion smoothness; the preliminary path is determined based on the predicted path length ρ and motion behavior classification (e.g., straight, left turn, right turn, right U-turn, left U-turn).

[0083] 2) Trajectory modeling: using the parabolic function y = a(xh) 2 +k defines a path that passes through the current robot position and the target point. The direction and shape of the parabola are affected by the path length and angle perturbations, thus generating different types of trajectories.

[0084] Where h represents the axis of symmetry of the set path, k represents the vertex height of the set path, and a < 0 is constrained to ensure that the trajectory bends downward, thereby simulating a reasonable path for the robot to move towards the target point.

[0085] 3) Noise modeling: To improve generalization ability, an adaptive noise mechanism based on prediction confidence was designed. Specifically, when the model has high prediction confidence for the current state, noise perturbation is reduced to maintain accuracy; when the prediction confidence is low, noise perturbation is increased to encourage the exploration of different paths.

[0086] The noise standard deviation σ is calculated using the following formula:

[0087] σ=min_std+(max_std-min_std)·(1-confidence)

[0088] Where max_std is the predefined maximum standard deviation, min_std is the predefined minimum standard deviation, and confidence is the behavior classification confidence score output by the fully connected layer.

[0089] Rule-based prior generation strategies are applicable to noise adjustment for path length and angle direction, ensuring that the path is both constrained and adaptable to different scenarios.

[0090] In the embodiments of this application, by providing a variety of prior generation strategies (including Gaussian, rule-based, learning-based, etc.), the system can achieve better performance in different scenarios and adapt to complex dynamic environments.

[0091] Step S105: Input the prior action and the condition variable into the denoising diffusion bridge model for diffusion training to generate target navigation actions; wherein, the prior action is used as the initial state of the denoising diffusion bridge model when performing reverse denoising processing.

[0092] In some embodiments, step S105 may include: inputting prior actions and conditional variables into a denoising diffusion bridge model; performing forward noise-adding training on the denoising diffusion bridge model based on the real path point sequence to generate noisy actions; and performing reverse denoising processing on the noisy actions based on the prior actions and conditional variables to generate target navigation actions.

[0093] In some embodiments, after step S105, the method may further include: generating navigation action instructions based on the target navigation action; sending the target navigation action and the navigation action instructions to the robot's motion control module, and controlling the robot to execute the target navigation action based on the navigation action instructions through the motion control module.

[0094] In this embodiment of the application, the prior action a is... TAs the initial state, backpropagation is performed using the DDBM framework. In each step of backpropagation, the neural network adjusts the current state 'a'. t Calculate the scoring function s(a) at time step t. t ,t,a T ,T), and simultaneously combined with h(a t ,t,a T The h-transform term of Doob is used to adjust the direction of action updates. After k iterations (usually fewer steps than the traditional diffusion model), the target action a0 is output. The generated target action sequence a0 will be directly used for robot control to achieve the navigation task from the current state to the target position.

[0095] It should be noted that the initial action is used as a constraint term for the network only in the first step of backpropagation, i.e., the current state is equal to the initial action (prior action). During training, the intermediate states between the target action and the initial action are first obtained based on Doob's h-transform term. Then, the neural network is input with the current state, the time step, and the changes of the two adjacent intermediate states corresponding to the learning time step.

[0096] It should be noted that the forward operation involves adding noise (diffusion) based on the true values ​​in the dataset during model training, resulting in an intermediate state between the true value and the prior, allowing the model to learn how to gradually transform from the prior action into the true value through a denoising process. Since this is the default operation of the diffusion model, it will not be described in detail in the embodiments of this application.

[0097] In its implementation, the diffusion training process of the denoising diffusion bridge model includes a forward diffusion process and a reverse denoising process. Specifically, the typical forward diffusion process and reverse denoising process are as follows:

[0098] (1) Forward diffusion process: The data starts from the original distribution p0, and noise is gradually added to eventually transform it into a simple distribution (usually a Gaussian distribution P). T =N(0,I), where P T Let N represent the initial data distribution in the diffusion model, where N represents the normal distribution and I represents the identity matrix. This forward diffusion process can be represented by a stochastic differential equation (SDE), as shown in the following expression:

[0099] dx t =f(x) t ,t)dt+g(t)dw t

[0100] Where, x t This represents the system state or sample variable at time t, which changes over time; dx t Represents the state x between time t and t+dt.t Tiny changes; f(x) t ,t) represents the drift term, a function of the current state and time, indicating the trend of deterministic change; g(t) represents the diffusion coefficient, controlling the intensity of random perturbations, and is a function of time; dw t Represented as a Wiener process, it represents the noise disturbance and satisfies... That is, a normal distribution with zero mean and variance dt, where d represents the differential.

[0101] (2) Reverse denoising process: In order to recover the original data from the noise, reverse SDE is used, and the specific expression is as follows:

[0102]

[0103] in, Indicates about x t The logarithmic probability density gradient, also known as the score function, is used to indicate the direction of the fastest ascent in the probability distribution for the current state point. This represents the Wiener process (Brownian motion) in reverse time, compared to the w in the forward process. t Different, used for inverse simulation.

[0104] In this embodiment, to make the initial action distribution (prior distribution) closer to the target action distribution, Doob's h-transform is introduced. The forward diffusion process and reverse denoising process of the diffusion bridge in this embodiment are as follows:

[0105] (1) The forward process of the diffusion bridge in the embodiments of this application can be written as the following expression:

[0106] da t =[f(a t ,t)+g(t) 2 h(a t ,t,a0,T)]dt+g(t)dw t

[0107] in, Used to force a process to a fixed endpoint y; a t It represents the action state at time t, and is an action variable rather than a state variable (as opposed to x mentioned earlier). t ); a0 represents the target endpoint action of the diffusion process, which is usually the actual action in the data or provided by experts; T represents the total time or endpoint time of the diffusion process.

[0108] (2) In the imitation learning scenario, the backdiffusion bridging process can be represented by the following expression:

[0109]

[0110] Among them, s(a t ,t,a T T) is a conditional scoring function (which can be approximated as...) In this embodiment, h(a) is a convolutional neural network; t ,t,a T ,T) is also used to guide the smooth transition of the distribution, and serves as the target in the reverse denoising process for the convolutional neural network to fit.

[0111] In this embodiment of the application, in order to quantify the prior distribution π s (a T The difference between the target distribution π(a0) and the target distribution π(a0) is used to introduce the KL (Kullback-Leibler Divergence), which is calculated using the following formula:

[0112]

[0113] Specifically, the closer the prior distribution is to the target distribution, the smaller the correction required at each step of the diffusion process, resulting in a lower lower bound on the error. That is:

[0114]

[0115] Where C represents a constant related to the noise scheduling g(t) and the time step; D represents the expected value of a sample taken from a certain distribution (usually the initial distribution π(a0)); KL (π s (a T )||π(a0)) represents the Kullback–Leibler divergence (KL divergence), also known as relative entropy, which is an indicator that measures the difference between two probability distributions.

[0116] This application embodiment generates an action sequence that better meets the task requirements by constructing an initial state that is closer to the target action distribution and reducing the error caused by KL divergence during the diffusion process.

[0117] Furthermore, this application embodiment also designs a training loss function for the entire visual navigation system network. The training loss of the entire network consists of the following three parts (diffusion bridge loss L). b Prior generation loss L p and time distance loss L d )constitute:

[0118] (1) Diffusion bridge loss L b ;

[0119] The weighted mean squared error (MSE) loss is used to measure the difference between the target action a0 and the network's denoised output D. θ (a t ,t,a T Differences between:

[0120]

[0121] Where w(t) is the time-weighted scheduling function, and a0 represents the true value in the dataset.

[0122] (2) Prior generation loss L p ;

[0123] For rule priors, when generating initial actions using fully connected layers, the MSE loss and cross-entropy classification loss are calculated, as shown in the following expressions:

[0124]

[0125] Where, p c One-hot encoding for the actual motion categories; To predict probabilities; This represents the difference between the predicted value and the true value of the i-th sample; a represents the predicted value of the i-th sample; i λ represents the true value of the i-th sample; c and λ a These are the weighting coefficients for classification loss and regression loss, respectively, used to balance the loss effects of these two different tasks.

[0126] When learning priors and generating priors using CVAE, the loss consists of reconstruction error and KL divergence, as shown in the following expression:

[0127]

[0128] Where N represents the number of samples; O represents the condition variable, which in this embodiment refers to the features of the image; a represents the known action information; z represents the latent variable; KL(·) is a divergence calculation used to calculate the difference between two distributions. KL divergence is used to force the posterior distribution to be close to the prior distribution, ensuring that the model does not depend on the known action information a when it is generated. π(z|O) is the posterior distribution estimated by the CVAE inference model; π(z|O) is the conditional prior distribution of CVAE, which depends only on the condition variable O.

[0129] (3) Temporal distance loss L d ;

[0130] To better capture the temporal relationship between the target image and the current observation, an MSE loss is introduced between the predicted temporal distance and the true distance, as shown in the following expression:

[0131]

[0132] Where, f(c t,i ) represents the temporal distance predicted using context vectors, and d i This corresponds to the actual distance.

[0133] The final overall loss of the entire network is:

[0134] L=λ b L b +λ p L p +λ d L d

[0135] Among them, the weights λ of each item b , λ p With λ d Used to control the balance between modules.

[0136] This application embodiment achieves coordinated optimization of each module by comprehensively using diffusion bridge loss, prior generation loss and temporal distance loss, thereby improving the stability and accuracy of the overall navigation.

[0137] This application provides a visual navigation method based on a denoising diffusion bridge model. This model transforms the action generation problem in visual navigation tasks into a continuous denoising process starting from arbitrary informative priors (rather than traditional Gaussian noise). By introducing multiple prior generation strategies, designing corresponding denoising diffusion processes, and jointly optimizing the loss function, it not only accelerates policy reasoning (reducing denoising steps) but also significantly improves the smoothness of the navigation path and the task success rate. Furthermore, it exhibits high robustness and efficiency in various environments (including indoor / outdoor, zero-shot, and adaptive tasks).

[0138] Steps S101 to S105 of this embodiment involve: acquiring initial image data; performing feature encoding on the initial image data to generate a context vector; performing linear modulation on the context vector to generate condition variables; generating prior actions based on the context vector and preset motion rules; and inputting the prior actions and condition variables into a denoising diffusion bridge model for diffusion training to generate target navigation actions. The prior actions serve as the initial state of the denoising diffusion bridge model during reverse denoising. This embodiment utilizes a denoising diffusion bridge model, using the prior action distribution as the initial state for the reverse diffusion process. This allows the initial state to be efficiently corrected to the target action distribution, thereby achieving a fast and stable visual navigation strategy. In other words, this embodiment transforms the traditional visual navigation method's diffusion denoising process, which starts with Gaussian noise, into a diffusion denoising process starting with an informative prior action distribution. This effectively reduces redundant iterations in the diffusion process, lowers accumulated errors, significantly improves the efficiency and stability of action generation, and adapts to complex dynamic environments.

[0139] To explain in detail the principle of the technical solution of the present invention, the overall process of the present invention will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and should not be regarded as a limitation of the present invention.

[0140] The core idea of ​​the visual navigation method based on the denoising diffusion bridge model (DDBM) is to transform the traditional denoising process, which starts with Gaussian noise, into a continuous denoising process starting with informative prior actions, thereby generating action sequences that meet the requirements of the target task more efficiently and accurately. Please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of the overall process of a visual navigation method based on a denoising diffusion bridge model provided in an embodiment of this application; as shown below. Figure 2 As shown, the entire visual navigation system mainly includes a feature extraction module, a feature linear modulation module, a priori generation module, and a diffusion bridge module. Figure 2 In the feature extraction module, the orange square columns represent the features of all images after passing through the feature encoder. The red vertical bars in the learned prior generation strategy module represent the feature encoder, and the blue-green vertical bars represent the feature decoders. z represents the latent space of the features. The feature extraction module uses RGB observations and the target image, extracting features from both through a transform encoder and generating a context vector. The feature linear modulation module applies learned conditions to improve path accuracy. The prior generation module ensures the strategy output better aligns with the target action, starting with alternative previous actions generated using optional prior strategies. Finally, in the strategy output module, the system takes the current action and the features extracted from the image as input, performs denoising processing by the network, and outputs the final action sequence.

[0141] In specific implementations, such as Figure 2 As shown, firstly, the feature extraction module receives the RGB observation image and the target image, and extracts key features from the RGB observation image and the target image through a visual encoder. These key features are then aggregated to form an embedding vector (context vector) that describes the current state and the target state. This context vector is then passed as input to the feature linear modulation module. Next, in the feature linear modulation module, the system generates condition variables based on the input state features (context vector). These condition variables adjust the path generation process in the subsequent diffusion bridge module through feature linear modulation. By adjusting the policy conditions, the feature linear modulation module makes the generated path more accurately point to the target, improving the accuracy and efficiency of the task. Subsequently, the prior generation module generates alternative action trajectories based on the current state. These trajectories are provided by the prior policy generator, providing a reference for policy optimization. Finally, in the policy output module, the system takes the current action and the features extracted from the image as input, processes them through network denoising, and outputs the final action sequence. This action sequence effectively guides the agent to perform the task and achieve the expected goal in the environment.

[0142] Specifically, the implementation processes of the feature extraction module, feature linear modulation module, prior generation module, and diffusion bridge module are as follows:

[0143] 1) Feature extraction module and feature linear modulation module;

[0144] In the specific implementation, the robot acquires continuous RGB observation sequences and target images. Visual features of the RGB observation sequences and target images are extracted via a convolutional network or a Transformer encoder, respectively. These visual features are then fused to generate a context vector C describing the current scene and target information. t Subsequently, the context vector C t The input is passed to the characteristic linear modulation module, which modulates the input context vector C. t Modulation parameters a and b are generated, and then condition variables are formed by characteristic linear modulation based on modulation parameters a and b. The condition variables are used to adjust the behavior of the generation strategy. The condition variables obtained after modulation are then input into the diffusion bridge module to guide the generation of the final navigation action.

[0145] 2) Prior action generation module;

[0146] Using context vector C t Generate preliminary motion estimate a T (Prior actions, i.e.) Figure 2(Source action in the process). The prior action generation module provides three prior generation strategies: Gaussian prior generation strategy, rule-based prior generation strategy, and learning-based prior generation strategy.

[0147] Among them, the Gaussian prior generation strategy refers to directly sampling from the Gaussian white noise distribution as a benchmark for comparison with traditional diffusion models. The rule-based prior generation strategy refers to mapping the context vector C through a fully connected layer. t The system moves into the action space and, combined with pre-defined rules (such as using the predicted path length ρ and the confidence θ of the motion direction classification, like going straight, turning left, turning right, U-turn, etc.), generates a parabolic path as the initial action. The learning-based prior generation strategy refers to using a lightweight conditional variational autoencoder model to learn the relationship between observations and actions from expert data to obtain prior actions that better match the requirements of the actual task. Figure 2 In the learning-based prior generation strategy, the input image is the initial RGB observation sequence.

[0148] 3) Noise reduction and diffusion bridge module;

[0149] With prior action a T As the starting point of the diffusion process, the target action a0 is eventually obtained through multi-step back diffusion (using back-diffusion (Stochastic Differential Equation) or probabilistic flow ODE (Ordinary Differential Equation)). During this process, the network iteratively updates the actions, gradually converging them to the target action distribution, thus generating high-quality action sequences for the robot to execute.

[0150] like Figure 2 As shown, the overall implementation process of a visual navigation method based on a denoising diffusion bridge model includes the following four steps:

[0151] (1) Input processing;

[0152] Specifically, first, collect the RGB image sequence within the current time period. and target image I g Then, RGB image sequences are extracted using either a convolutional network or a Transformer encoder. and target image I g The features of each frame of the image are then analyzed using aggregation techniques to extract the RGB image sequence. and target image I g The extracted features are aggregated to generate a context vector C. t .

[0153] (2) Prior generation (generating preliminary actions a through different strategies) T ):

[0154] The first is the Gaussian prior generation strategy: directly sampling a. T ~N(0,I).

[0155] Second, a rule-based prior generation strategy: fully connected layer mapping C t The initial motion is generated by combining preset motion rules (such as the parabola generation formula).

[0156] For example, a parabola can be defined in the standard form y = ax 2 +bx+c or vertex form y=a(xh) 2 +k is used to solve for parameters, ensuring that the path is smooth and meets kinematic requirements.

[0157] In practical implementation, the specific generation principle and process of the rule-based prior generation strategy are as follows:

[0158] 1) Design principle: A fully connected layer is used to map environmental perception information (such as observed image features) to the action space, and low-dimensional features are extracted to generate preliminary prior actions. The prior actions follow preset trajectory constraints, which are usually set as parabolic trajectories to simulate natural motion trends and improve motion smoothness. The preliminary path is determined based on the predicted path length d and the motion behavior classification (e.g., straight, left turn, right turn, right U-turn, left U-turn).

[0159] 2) Trajectory modeling: using the parabolic function y = a(xh) 2 +k defines a path that passes through the current robot position and the target point. The direction and shape of the parabola are affected by the path length and angle perturbations, thus generating different types of trajectories.

[0160] Where h represents the axis of symmetry of the set path, k represents the vertex height of the set path, and a < 0 is constrained to ensure that the trajectory bends downward, thereby simulating a reasonable path for the robot to move towards the target point.

[0161] 3) Noise modeling: To improve generalization ability, an adaptive noise mechanism based on prediction confidence was designed. Specifically, when the model has high prediction confidence for the current state, noise perturbation is reduced to maintain accuracy; when the prediction confidence is low, noise perturbation is increased to encourage the exploration of different paths.

[0162] The noise standard deviation σ is calculated using the following formula:

[0163] σ=min_std+(max_std-min_std)·(1-confidence)

[0164] Where max_std is the predefined maximum standard deviation, min_std is the predefined minimum standard deviation, and confidence is the behavior classification confidence score output by the fully connected layer.

[0165] Rule-based prior generation strategies are applicable to noise adjustment for path length and angle direction, ensuring that the path is both constrained and adaptable to different scenarios.

[0166] Third, a learning-based prior generation strategy: using a trained CVAE model to generate prior actions that better match expert demonstrations based on input observation data.

[0167] (3) Noise reduction and diffusion bridging process: the prior action a T As the initial state, backpropagation is performed using the DDBM framework. In each step of backpropagation, the neural network adjusts the current state 'a'. t Calculate the scoring function s(a) at time step t. t ,t,a T ,T), and simultaneously combined with h(a t ,t,a T ,T)(Doob's h-transform term), adjusts the action update direction, and after k iterations (usually fewer steps than the traditional diffusion model), outputs the target action a0.

[0168] (4) Output and decision: The generated target action sequence a0 will be directly used for robot control to realize the navigation task from the current state to the target position.

[0169] It should be noted that this embodiment is only a brief illustrative description of the general process of a visual navigation method based on a denoising diffusion bridge model. Detailed descriptions of each step can be found in the relevant content of the foregoing embodiments, and will not be repeated here. It is understood that the present invention does not limit this.

[0170] This application embodiment acquires initial image data; performs feature encoding on the initial image data to generate a context vector; performs linear modulation on the context vector to generate condition variables; generates prior actions based on the context vector and preset motion rules; and inputs the prior actions and condition variables into a denoising diffusion bridge model for diffusion training to generate target navigation actions. The prior actions serve as the initial state of the denoising diffusion bridge model during reverse denoising. This application embodiment utilizes a denoising diffusion bridge model, using the prior action distribution as the initial state for the diffusion bridge model's reverse diffusion process. This allows the initial state to be efficiently corrected to the target action distribution, thereby achieving a fast and stable visual navigation strategy. In other words, this application embodiment transforms the traditional visual navigation method's diffusion denoising process, which starts with Gaussian noise, into a diffusion denoising process starting with an informative prior action distribution. This effectively reduces redundant iterations in the diffusion process, lowers accumulated errors, significantly improves the efficiency and stability of action generation, and can adapt to complex dynamic environments.

[0171] This application provides a visual navigation method based on a denoising diffusion bridge model, which can be applied to mobile robots, autonomous driving systems, and other application scenarios. For example, the visual navigation method based on the denoising diffusion bridge model provided in this application is integrated into a mobile robot system. The mobile robot system includes a visual sensor, a navigation strategy module, and a motion control module. Specifically, the visual sensor (such as an RGB camera) is responsible for acquiring images of the current environment and the target; the navigation strategy module processes the image features using the denoising diffusion bridge model to generate navigation action commands; the motion control module receives the navigation action commands and drives the robot to execute corresponding actions, thereby achieving autonomous navigation and target localization of the robot in the environment. The entire system forms a closed-loop process of perception-decision-execution, improving the real-time performance and robustness of navigation.

[0172] In this embodiment, by introducing prior action information, the target action is generated in a more efficient manner, improving the computational efficiency and generalization ability of the model. Compared with traditional methods, this embodiment reduces denoising steps, optimizes the generation quality of the target action, and enhances adaptability to different environments.

[0173] Furthermore, the core of the visual navigation method based on the denoised diffusion bridge model (DDBM) proposed in this application lies in utilizing informative prior actions as the initial state of the diffusion process, and guiding these prior actions to the target action distribution through Doob's h-transform, thereby achieving more efficient and accurate action generation. While some alternative solutions exist, they all have significant shortcomings, as detailed below:

[0174] 1) For traditional diffusion model schemes, such as NoMaD (Goal-Masked Diffusion Policies for Navigation and Exploration), the NoMaD method starts the diffusion process directly from Gaussian noise as the initial state. Since the initial state and the target action distribution are very different, these methods require many denoising iterations, resulting in long inference time, poor real-time performance, and difficulty in guaranteeing generation accuracy.

[0175] 2) For Generative Adversarial Networks (GANs) or Flow-based Models, these methods can also be used to generate continuous actions. However, the training process of GANs is unstable, and there is often a pattern collapse problem in high-dimensional action spaces. Although Flow-based Models have reversibility, they have high computational complexity and weak real-time response capability, making it difficult to meet the requirements of visual navigation in complex dynamic environments.

[0176] In summary, the visual navigation method based on a denoising diffusion bridge model provided in this application simultaneously possesses the following technical advantages: it significantly reduces the distribution difference between the initial state and the target state by utilizing prior information (such as rule-based or learning-based prior generation); it effectively reduces the number of denoising iterations through diffusion bridge technology, improving generation efficiency and action accuracy; and it meets the high requirements of visual navigation in complex environments in terms of real-time performance and robustness. Therefore, the denoising diffusion bridge model guided by informative priors in this application can achieve faster, more stable, and higher-precision action generation in visual navigation, while also adapting to complex dynamic environments.

[0177] The key point of the visual navigation method based on the denoising diffusion bridge model (DDBM) proposed in this application is:

[0178] 1) By using the denoising diffusion bridge model, the traditional diffusion denoising process that starts from Gaussian noise is transformed into starting from the informational prior action distribution. That is, the prior action distribution is used as the initial state of the diffusion bridge model. The reverse diffusion process is guided by Doob's $h$-transform, so that the initial state is efficiently corrected to the target action distribution, realizing a fast and stable visual navigation strategy, thereby greatly improving the efficiency and stability of action generation.

[0179] 2) Multi-strategy prior generation methods mainly include Gaussian prior generation methods, rule-based prior generation methods, and learning-based prior generation methods. Among them, rule-based prior generation (using fully connected mapping combined with parabolic path generation, real-time analysis of scene information and adding appropriate noise to ensure action diversity) and learning-based prior generation (using a lightweight CVAE model to learn expert data) are the two main prior generation mechanisms.

[0180] 3) Joint optimization training scheme: By designing diffusion bridge loss, prior generation loss (including action prediction error and motion direction classification loss) and time distance loss, the visual feature extraction, prior generation and denoising diffusion bridge modules are optimized in a coordinated manner, thereby improving the consistency between the generated action and the target action distribution.

[0181] 4) The KL divergence is introduced as a measure of the proximity between the prior and target distributions, and the relationship between error definition and noise scheduling is demonstrated, providing theoretical support for the method design. The influence of the KL divergence between the prior distribution and the target action distribution on the diffusion process error is theoretically proven, ensuring that the closer the prior is to the target distribution, the fewer denoising correction steps are required, thereby improving the accuracy of the generated actions.

[0182] Based on these four key points, the visual navigation method based on the denoised diffusion bridge model (DDBM) proposed in this application can improve the adaptability and robustness of visual navigation in complex dynamic environments. Even in the presence of noise and environmental changes, it can effectively reduce redundant iterations in the diffusion process, reduce accumulated errors, and ensure the smoothness of the navigation path and the success rate of the task.

[0183] Therefore, the visual navigation method based on the denoised diffusion bridge model (DDBM) proposed in this application effectively avoids the redundant denoising steps caused by the traditional diffusion model starting with Gaussian noise by introducing informative prior action distributions (including rule-based and learning-based priors), thereby significantly improving the efficiency and accuracy of target action generation. Furthermore, by jointly optimizing the diffusion bridge loss, prior generation loss, and temporal distance loss, the model's ability to fit the target action distribution and its adaptability to different environments are further enhanced, achieving faster and more stable real-time visual navigation overall. Compared with traditional visual navigation methods, the embodiments of this application have significant advantages in real-time performance, accuracy, and adaptability to complex dynamic environments.

[0184] Please see Figure 3 This application also provides a visual navigation system 300 based on a denoising diffusion bridge model, which can implement the above-mentioned visual navigation method based on a denoising diffusion bridge model. The system includes the following modules:

[0185] The initial image data acquisition module 301 is used to acquire initial image data;

[0186] Feature encoding processing module 302 is used to perform feature encoding processing on the initial image data to generate a context vector;

[0187] The linear modulation processing module 303 is used to perform linear modulation processing on the context vector to generate condition variables;

[0188] The prior action generation module 304 is used to generate prior actions based on the context vector and preset motion rules;

[0189] The target navigation action generation module 305 is used to input the prior action and the condition variable into the denoising diffusion bridge model for diffusion training to generate target navigation actions; wherein, the prior action is used as the initial state of the denoising diffusion bridge model when performing reverse denoising processing.

[0190] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0191] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned visual navigation method based on a denoising diffusion bridge model. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0192] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] Please see Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0194] The processor 401 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0195] The memory 402 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 to implement a visual navigation method based on a denoising diffusion bridge model according to an embodiment of this application.

[0196] Input / output interface 403 is used to implement information input and output;

[0197] The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0198] Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404);

[0199] The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0200] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned visual navigation method based on a denoising diffusion bridge model.

[0201] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0202] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0203] This application provides a visual navigation method and system based on a denoising diffusion bridge model. The method involves acquiring initial image data; performing feature encoding on the initial image data to generate a context vector; performing linear modulation on the context vector to generate condition variables; generating prior actions based on the context vector and preset motion rules; and inputting the prior actions and condition variables into the denoising diffusion bridge model for diffusion training to generate target navigation actions. The prior actions serve as the initial state of the denoising diffusion bridge model during reverse denoising. This application utilizes the denoising diffusion bridge model, using the prior action distribution as the initial state for the reverse diffusion process. This allows the initial state to be efficiently corrected to the target action distribution, thereby achieving a fast and stable visual navigation strategy. Specifically, this application transforms the traditional visual navigation method's diffusion denoising process, which starts with Gaussian noise, into a diffusion denoising process starting with an informative prior action distribution. This effectively reduces redundant iterations in the diffusion process, lowers accumulated errors, significantly improves the efficiency and stability of action generation, and adapts to complex dynamic environments.

[0204] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0205] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0206] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0208] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0209] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0210] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0211] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0213] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0214] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A visual navigation method based on a denoising diffusion bridge model, characterized in that, The method includes the following steps: Obtain initial image data; The initial image data is subjected to feature encoding processing to generate a context vector; The context vector is linearly modulated to generate condition variables; Based on the context vector and preset motion rules, generate prior actions; The prior actions and the conditional variables are input into the denoising diffusion bridge model for diffusion training to generate target navigation actions; wherein, the prior actions are used as the initial state of the denoising diffusion bridge model when performing reverse denoising processing.

2. The method according to claim 1, characterized in that, The initial image data includes an initial observation image and an initial target image. The feature encoding process on the initial image data to generate a context vector includes: The initial observation image is subjected to feature extraction processing to obtain the observation image features; The initial target image is subjected to feature extraction processing to obtain target image features; The observed image features and the target image features are subjected to feature fusion processing to generate the context vector.

3. The method according to claim 1, characterized in that, The step of performing linear modulation processing on the context vector to generate condition variables includes: The context vector is input into the feature linear modulation module; The characteristic linear modulation module generates modulation parameters based on the context vector. The characteristic linear modulation module determines the condition variable based on the modulation parameters.

4. The method according to claim 1, characterized in that, The step of generating prior actions based on the context vector and preset motion rules includes: The context vector is mapped to the action space through the fully connected layer of the neural network model, and low-dimensional action features corresponding to the context vector are generated. Based on the low-dimensional motion features and the preset motion rules, an initial candidate path is generated; The prior action is determined based on the initial candidate path.

5. The method according to claim 1, characterized in that, The method further includes: Based on Gaussian prior rules, data is sampled from a Gaussian white noise distribution to obtain the prior action; or, The prior action is obtained by training the initial observation image input conditional variational autoencoder model from the initial image data.

6. The method according to claim 1, characterized in that, The step of inputting the prior actions and the conditional variables into the denoising diffusion bridge model for diffusion training to generate target navigation actions includes: The prior actions and the condition variables are input into the denoising diffusion bridge model; The denoising diffusion bridge model is trained by positive noise addition based on the real path point sequence to generate noisy actions; The noise diffusion bridge model performs reverse noise reduction on the noise action based on the prior action and the condition variable to generate the target navigation action.

7. The method according to claim 1, characterized in that, After inputting the prior actions and the conditional variables into the denoising diffusion bridge model for diffusion training to generate target navigation actions, the method further includes: Generate navigation action instructions based on the target navigation action; The target navigation action and the navigation action command are sent to the robot's motion control module, which then controls the robot to execute the target navigation action according to the navigation action command.

8. A visual navigation system based on a denoising diffusion bridge model, characterized in that, The system includes the following modules: The initial image data acquisition module is used to acquire initial image data; The feature encoding processing module is used to perform feature encoding processing on the initial image data to generate a context vector; A linear modulation processing module is used to perform linear modulation processing on the context vector to generate condition variables; The prior action generation module is used to generate prior actions based on the context vector and preset motion rules; The target navigation action generation module is used to input the prior actions and the condition variables into the denoising diffusion bridge model for diffusion training to generate target navigation actions; wherein, the prior actions are used as the initial state of the denoising diffusion bridge model when performing reverse denoising processing.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images