Training method and using method of end-to-end path planning model, device and medium

Abstract

The present application relates to the field of artificial intelligence and robot navigation technology, and more particularly to a kind of end-to-end path planning model training method, use method, device and medium, comprising: obtaining agent information in simulation environment and B spline path for subsequent path planner training;Real environment depth map and camera pose data are used to generate safety corridor block as the instructor signal of encoder training, and a deep vision encoder is trained;B spline path is used as the instructor signal, task critical features are extracted through the trained deep vision encoder, and the path planner is input into the path planner for end-to-end training, and a path planner for outputting obstacle avoidance path according to input depth map and dynamic information is obtained.The present application effectively solves the problem of end-to-end path planning prone to overfitting, and improves the generalization ability and perception accuracy of the model.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and robot navigation technology, and in particular to a training method, usage method, device and medium for an end-to-end path planning model. Background Technology

[0002] In recent years, embodied intelligence has emerged as a key pathway connecting digital intelligence and the physical world. Its core task is to enable intelligent agents to achieve high-quality closed-loop perception, decision-making, and interaction through physical entities and the environment. Traditional methods typically employ a modular, hierarchical optimization architecture, decoupling perception, planning, and control into independent modules and optimizing them stage by stage through manually designed interfaces and loss functions. However, as the complexity of application scenarios increases, this paradigm reveals significant limitations: in modular systems, small errors in the perception module can be amplified in the planning and control stages, leading to cascading failures; interfaces between modules rely on expert experience for design, resulting in high engineering costs for cost functions and state representations, and are heavily dependent on specific platforms and tasks. Cross-scenario migration requires the reconstruction of a large amount of logic, limiting generalization capabilities; information compression and abstraction in intermediate modules inevitably lead to the loss of details, resulting in high-level decision-making lacking a global understanding of the underlying dynamics, making it difficult to achieve truly optimal control.

[0003] To overcome these bottlenecks, end-to-end methods directly map raw sensor inputs to low-level control commands, achieving global joint optimization through differentiable computational graphs. While end-to-end approaches demonstrate potential in simulation, their physical implementation faces significant challenges: acquiring physical interaction data is extremely costly, while methods like reinforcement learning suffer from low sample efficiency; current end-to-end reinforcement learning often employs simulation for training, making end-to-end neural networks prone to overfitting to simulated environments, and their generalization ability remains unstable when faced with unfamiliar objects or terrains; the global model has a large number of parameters, and edge deployments face the dual pressures of latency and power consumption.

[0004] Existing technologies, such as EGO-planner, rely on traditionally optimized path planners that depend on Simultaneous Localization and Mapping (SLAM) algorithms, consuming significant computational resources and making deployment on lightweight platforms difficult. Another approach, end-to-end visual motion policy learning methods based on differentiable simulation, downsample depth maps to extremely low resolution to maintain a lightweight network architecture and prevent overfitting, resulting in the loss of excessive detail. This leads to a loss of sensitivity in the agent when facing small obstacles, and the end-to-end network is strongly coupled with the robot's dynamic parameters, making it difficult to transfer to robots of different configurations, sizes, and masses. Therefore, there is an urgent need for an end-to-end path planning solution that can solve the overfitting problem, reduce training difficulty, achieve platform decoupling, and possess high perceptual accuracy. Summary of the Invention

[0005] To address the technical problems of existing end-to-end path planning methods, such as overfitting to simulation environments, loss of perception details, and poor generalization ability due to strong coupling with dynamic parameters, this application proposes a training method, usage method, device, and medium for an end-to-end path planning model. It utilizes real-world data to automatically label and train a feature extraction network, and trains an end-to-end decision network using a pre-trained sequence decision strategy as a mentor signal. This achieves end-to-end path planning with high generalization, high perception accuracy, and platform decoupling.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] Firstly, the present invention provides a training method for an end-to-end path planning model, the technical solution of which is as follows:

[0008] S100: A simulation environment that is consistent with the real environment is used to generate a safe corridor information based on the depth map in the simulation environment, as well as the position and dynamic information of the agent, to obtain a B-spline path for subsequent path planner training.

[0009] S200: Based on the depth map collected from the real environment and the camera pose data of the physical agent, a navigation target is assigned to the depth map. A safe corridor block is generated based on the depth map, camera pose data and navigation target, which serves as the mentor signal for encoder training. The depth visual encoder is trained to obtain the trained depth visual encoder.

[0010] S300. Using the acquired B-spline path as the mentor signal, based on the depth map and the position and dynamics information of the agent in the simulation or real environment, the key features of the task are extracted by the trained depth vision encoder, and the key features of the task and the position and dynamics information of the agent are input into the path planner for end-to-end training to obtain a path planner for outputting obstacle avoidance path based on the input depth map and dynamics information.

[0011] The above scheme adopts a phased training strategy. First, it acquires high-quality B-spline paths as prior knowledge in a simulation environment. Then, it uses real-world data to train a deep visual encoder to extract key features of the task, thus solving the problem of overfitting to the simulation environment in end-to-end methods. At the same time, it uses B-spline paths as mentor signals to train the planner, thereby decoupling the perception features from the control objectives, reducing the dependence on specific platform dynamic parameters, and improving the model's generalization ability and deployment flexibility.

[0012] As an optional implementation, the depth vision encoder includes:

[0013] The block mapping module is configured to perform block processing and linear projection on the input depth map to generate a perceptual feature sequence.

[0014] The feature fusion module is configured to concatenate the guidance features, which include the navigation target and camera attitude, with the perception feature sequence, and to perform position encoding on the concatenated feature sequence.

[0015] The encoding extraction module is configured to perform attention feature extraction on the position-encoded feature sequence and obtain the key features of the task from the extraction results.

[0016] The above scheme effectively processes high-resolution depth maps by using a ViT-like encoder structure, preserving perceptual details, and enhances the specificity of feature representation by fusing navigation target and attitude information through guided features.

[0017] As an optional implementation, S200 includes:

[0018] Collect depth maps and camera pose data of physical agents in real-world environments;

[0019] Randomly assign navigation targets to the depth map;

[0020] Based on the depth map, camera pose data, and navigation target, a path search algorithm is invoked to generate a reference path and the corresponding safe corridor block as a mentor signal for encoder training.

[0021] Using the depth map, camera pose data, and navigation target as input, and the mentor signal as the target, the depth visual encoder is trained to obtain the trained depth visual encoder.

[0022] The above solution effectively solves the problem of the gap between simulation and reality by collecting data in a real environment and automatically labeling it. It can obtain a large amount of high-quality training data without manual labeling, and significantly improves the generalization ability of the encoder in a real environment.

[0023] As an optional implementation, training the depth visual encoder based on the depth map, camera pose data, and navigation target as input, and using the mentor signal as the target, includes:

[0024] The depth map, camera pose data, and navigation target are input into the depth visual encoder to obtain the task key features output by the depth visual encoder.

[0025] The network parameters of the deep visual encoder are updated with the goal of minimizing the mean square error between the key features of the task and the mentor signal trained by the encoder.

[0026] The above scheme adopts a supervised learning approach and uses mean squared error as the loss function to achieve alignment between encoder output features and safety corridor features, thus ensuring the accuracy of feature extraction.

[0027] As an optional implementation, the key features of the task include a first feature group, a second feature group, and a third feature group;

[0028] The first feature group includes the coordinate information of the reference path nodes;

[0029] The second feature group includes the coordinate information of the nearest obstacle on the left side of the safety corridor block;

[0030] The third feature group includes the coordinate information of the nearest obstacle to the right of the safety corridor block.

[0031] The above scheme divides the encoder output features into three sets of clear coordinate information, filters out the obstacle location features that are most critical to the path planning task, ignores invalid complex environmental information, and further improves the planner's decision-making efficiency and accuracy.

[0032] As an optional implementation, the path planner includes a decoder network;

[0033] The decoder network is configured to receive the feature sequence corresponding to the key features of the task and the position and dynamic feature sequence of the agent as input, perform position encoding and multi-layer attention decoding processing on the input, and output the B-spline path control point sequence.

[0034] The above scheme adopts a decoder network structure to directly output a complete sequence of B-spline path control points, realizing end-to-end path generation. At the same time, by fusing dynamic features, the executability of the path is guaranteed.

[0035] As an optional implementation, S300 includes:

[0036] The B-spline path corresponding to the depth map and the position and dynamics information of the agent in the simulation or real environment is used as the mentor signal for training the path planner.

[0037] The depth map is input into the trained depth visual encoder to extract key features for the task.

[0038] The key features of the task are fused with the position and dynamic information of the agent to form an input feature sequence, which is then input into the path planner to obtain the predicted path output by the path planner.

[0039] The network parameters of the path planner are updated with the goal of minimizing the mean square error between the predicted path and the mentor signal used to train the path planner, so as to obtain the path planner through end-to-end training.

[0040] The above scheme uses the B-spline path generated by the reinforcement learning strategy as the mentor signal to train the Transformer path planner, realizing a direct mapping from perceptual features to path output. This avoids the overfitting problem that may be faced when training the planner alone, while ensuring the smoothness and executability of the output path.

[0041] Secondly, the present invention also provides a method for using a physical intelligent device, the method comprising: deploying a path planner trained by a training method of an end-to-end path planning model in the physical intelligent device; acquiring depth map and position and attitude data of the physical intelligent device, and inputting the depth map and position and attitude data into the path planner to obtain path information output by the path planner.

[0042] The above solution deploys the trained planner on physical intelligent devices, realizing the application of end-to-end path planning in real-world scenarios, and can quickly generate traceable paths based on real-time sensing data.

[0043] Thirdly, the present invention also provides a training apparatus for an end-to-end path planning model, the apparatus comprising:

[0044] The first acquisition module simulates a simulation environment consistent with the real environment, and obtains B-spline paths for subsequent path planner training based on the safety corridor information generated from the depth map in the simulation environment, as well as the position and dynamic information of the agent.

[0045] The encoder training module assigns navigation targets to the depth map based on the depth map collected in the real environment and the camera pose data of the physical agent. Based on the depth map, camera pose data and navigation targets, it generates safe corridor blocks as mentor signals for encoder training, trains the depth visual encoder, and obtains the trained depth visual encoder.

[0046] The path planner training module is used to extract key task features based on the acquired B-spline path as the mentor signal, the depth map in the simulation environment or the position and dynamics information of the agent, and the trained depth vision encoder. The key task features and the position and dynamics information of the agent are then input into the path planner for end-to-end training to obtain a path planner that outputs obstacle avoidance paths based on the input depth map and dynamics information.

[0047] The above scheme realizes the deviceization of the training method by dividing it into functional modules, which facilitates its deployment and execution on computing devices.

[0048] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the training method for the end-to-end path planning model.

[0049] The above scheme uses storage media to carry computer programs, providing a carrier for the dissemination and application of the method.

[0050] Beneficial effects:

[0051] The technical solution provided by this invention effectively solves the problem of overfitting to simulation environments in end-to-end methods by automatically annotating and training a deep visual encoder based on real-world data, preserving high-resolution perceptual details and improving perceptual accuracy. By training an end-to-end planner using reinforcement learning strategies as mentor signals, it achieves decoupling between perceptual features and control objectives, reduces dependence on specific platform dynamic parameters, and improves the model's generalization and cross-platform deployment capabilities. At the same time, the entire training framework does not require a high-precision physical simulation environment, reducing training difficulty and cost, and providing an efficient and robust solution for embodied intelligence navigation in complex real-world environments.

[0052] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0053] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0054] Figure 1 This is a flowchart illustrating an embodiment of a training method for an end-to-end path planning model according to the present invention.

[0055] Figure 2 This is a flowchart illustrating the deep visual encoder training method according to an embodiment of the present invention.

[0056] Figure 3 This is a flowchart illustrating the path planner training method according to an embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram of an embodiment of a training device for an end-to-end path planning model according to the present invention. Detailed Implementation

[0058] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0059] Figure 1The diagram shows a flowchart of an embodiment of a training method for an end-to-end path planning model provided by the present invention, which is executed by a training device for the end-to-end path planning model.

[0060] Example 1:

[0061] like Figure 1 As shown, this embodiment provides a training method for an end-to-end path planning model. This method aims to address the technical problems of existing end-to-end path planning methods, such as overfitting to simulation environments, loss of perception details, and poor generalization ability due to strong coupling with dynamic parameters. The method specifically includes the following steps:

[0062] Step S100: Simulate a simulation environment consistent with the real environment, and obtain B-spline paths for subsequent path planner training based on the safety corridor information generated from the depth map in the simulation environment, as well as the position and dynamic information of the agent.

[0063] Specifically, this step is the foundational stage of the entire training process. Its core objective is to leverage the low-cost and high-efficiency advantages of simulation environments to acquire high-quality prior knowledge of paths. In the simulation environment, scenarios containing various complex obstacles can be constructed to simulate the physical motion characteristics of intelligent agents (such as drones and autonomous vehicles). A reference path is generated by calling a path search algorithm (such as the A* algorithm), and a safe corridor block is constructed based on this reference path as state input. Then, a smooth and executable B-spline path is generated using a reinforcement learning strategy. This B-spline path not only contains obstacle avoidance information but also implicitly contains a motion trajectory that conforms to dynamic constraints, providing high-quality "mentor signals" for subsequent path planner training.

[0064] It should be understood that although a simulation environment is mentioned in this embodiment, in other embodiments, if sufficient real environment data and computing power are available, prior knowledge acquisition at this stage can be performed partially or entirely based on high-fidelity real environment data, as long as a B-spline path that meets safety constraints can be generated.

[0065] Step S200: Based on the depth map collected from the real environment and the camera pose data of the physical agent, assign navigation targets to the depth map, generate safe corridor blocks based on the depth map, camera pose data and navigation targets, use them as mentor signals for encoder training, train the depth visual encoder, and obtain the trained depth visual encoder.

[0066] Specifically, this step is crucial for bridging the "simulation-to-real-world gap." Traditional end-to-end methods often train directly in a simulation environment, leading to model failure when faced with textures, lighting, or obstacles unseen in real-world environments. This step innovatively introduces depth map data collected from real-world environments, acquiring a large amount of depth map and camera pose data in real-world scenes using handheld devices or other mobile platforms. For each frame of data, a navigation target is randomly assigned, and a pathfinding algorithm is automatically invoked to generate corresponding safe corridor blocks as "mentor signals" for encoder training. The depth vision encoder uses this depth map, pose, and target as input to learn how to extract features (i.e., safe corridor information) that have a critical impact on path planning from complex real-world visual information. In this way, the encoder "sees" the complex details of the real world during the training phase, effectively avoiding overfitting and improving the model's generalization ability in real-world environments.

[0067] Step S300: Using the acquired B-spline path as the mentor signal, based on the depth map and the position and dynamic information of the agent in the simulation environment or real environment, the key features of the task are extracted by the trained depth vision encoder, and the key features of the task and the position and dynamic information of the agent are input into the path planner for end-to-end training to obtain a path planner for outputting obstacle avoidance path based on the input depth map and dynamic information.

[0068] Specifically, this step is the final end-to-end integration and training phase. In this phase, the parameters of the deep visual encoder are frozen, and its function is to compress high-dimensional visual information into low-dimensional task-critical features (such as obstacle coordinates, path nodes, etc.). The path planner receives these visual features along with the agent's own dynamic information (such as position, velocity, and acceleration), directly outputting a complete B-spline path. During training, the B-spline path generated by the reinforcement learning strategy in step S100 serves as the mentor signal for the path planner's training, guiding its learning.

[0069] This design cleverly decouples perception and control: the deep vision encoder focuses on visual perception tasks, while the path planner focuses on decision-making. Because the path planner's input includes explicit dynamic information, the model can adapt to physical platforms with different dynamic parameters (such as drones of varying masses or autonomous vehicles with different drive mechanisms), achieving platform decoupling. By minimizing the error between the predicted path output by the planner and the mentor signal used to train the path planner, the model ultimately learns a direct mapping from the raw sensor input to the executable path.

[0070] Understandably, through the above three steps, the differentiated design of this method in terms of both deployment environment and monitoring signals can be clearly seen:

[0071] In terms of deployment environment, the training of the reinforcement learning policy network in S100 is entirely based on the simulation environment. It uses the simulation depth map to generate safe corridor information as state input, thereby obtaining a large amount of trial and error experience at extremely low cost. The training of the depth vision encoder in S200 is entirely based on the depth map data collected in the real environment, enabling the encoder to learn to extract effective features from the original input containing real noise, texture and lighting changes, thus bridging the gap between simulation and reality. The training of the path planner in S300 integrates data from two types of environments. It can use simulation environment data or introduce real environment data collected in S200, enabling the planner to learn both the path optimization strategy of the simulation environment and the perceptual feature representation of the real environment.

[0072] Regarding the mentor signals, although there are two supervised training phases in this method, the sources, forms, and uses of the mentor signals differ. The mentor signal used in S200 is a safe corridor block and its corresponding reference path node coordinates generated using the A* algorithm based on the real environment depth map. Its function is to guide the depth visual encoder to extract low-dimensional geometric constraint features from the high-dimensional depth map. The mentor signal used in S300 is a B-spline path generated by the reinforcement learning policy network trained in S100. Its function is to guide the end-to-end path planner to learn a direct mapping from perceptual features to executable paths. These two phases belong to different training phases, serve different trained networks, and are not confused with each other.

[0073] It is this design, which separates "strategy training in the simulation environment" from "perception training in the real environment" and separates "geometric feature supervision of the encoder" from "path output supervision of the planner," that allows this method to benefit from the efficient trial-and-error advantages of the simulation environment while effectively avoiding overfitting of the end-to-end model to the simulation texture, ultimately obtaining a path planner that can be deployed across platforms.

[0074] It's important to note that both S100 and S200 utilize the A* path search algorithm to generate reference paths and safe corridor blocks, but their purposes are entirely different. In S100, the safe corridor blocks serve as the state input to the reinforcement learning policy network, enabling it to learn decisions within a structured, low-dimensional environmental representation and quickly converge to a high-quality path generation policy. In S200, the reference paths and safe corridor blocks generated from the real depth map are used as mentor signals for encoder training, guiding the encoder to extract the truly crucial geometric constraints for path planning from the high-dimensional, noisy real depth map. Simply put, A* is used in S100 to construct the state space for reinforcement learning, while in S200 it's used to generate supervisory labels for the encoder. Although the algorithms are the same, their functional roles within the overall training framework are completely different.

[0075] Furthermore, for clarity, a brief comparison of the "safe corridor blocks" involved in both S100 and S200 is provided here. The safe corridor blocks in S100 are generated based on depth maps from a simulated environment. Their function is to provide structured state input information to the reinforcement learning policy network, helping it efficiently learn path planning strategies in a low-dimensional, noise-free environment. The safe corridor blocks in S200, on the other hand, are generated based on depth maps collected in a real environment. Their function is to serve as mentor signals for training the depth visual encoder, i.e., supervising the "standard answer" of the encoder's output. Although their names are similar, their sources differ: the former comes from simulated depth maps, while the latter comes from real depth maps. Their roles in training also differ: the former is the input to the reinforcement learning network, while the latter is the supervisory label for the encoder network. They are generated independently at each stage and are not used interchangeably.

[0076] By organically combining the above three stages, the method provided in this embodiment not only solves the overfitting problem in end-to-end planning, but also realizes flexible deployment across platforms, providing reliable technical support for the autonomous navigation of physical intelligent devices in complex and unknown environments.

[0077] Example 2:

[0078] In one alternative embodiment, the depth visual encoder includes:

[0079] The block mapping module is configured to perform block processing and linear projection on the input depth map to generate a perceptual feature sequence.

[0080] The feature fusion module is configured to concatenate the guidance features, which include the navigation target and camera attitude, with the perception feature sequence, and to perform position encoding on the concatenated feature sequence.

[0081] The encoding extraction module is configured to perform attention feature extraction on the position-encoded feature sequence and obtain the key features of the task from the extraction results.

[0082] This embodiment, based on Embodiment 1, elaborates on the specific internal structure of the depth visual encoder and its feature extraction process in step S200. As described in step S200, the core design of this depth visual encoder lies in how to efficiently extract truly effective "task-critical features" for path planning tasks from high-resolution depth maps, while eliminating redundant background noise, thereby solving the problem of overfitting in end-to-end models.

[0083] Specifically, the depth visual encoder described in this embodiment includes a block mapping module, a feature fusion module, and an encoding extraction module.

[0084] First, the block mapping module is configured to perform block processing and linear projection on the input depth map to generate a perceptual feature sequence.

[0085] In practical implementation, considering that common depth cameras (such as the RealSense series) output high-resolution depth maps (e.g., 1280×720), directly processing the entire image would result in enormous computational overhead and difficulty in capturing local details. Therefore, this invention employs processing logic similar to Vision Transformer (ViT) to reduce the high-resolution depth i to a moderate resolution (e.g., 640×360) and divide it into several fixed-size image blocks. For example, the image can be divided into 32×18 pixel blocks, each containing 20×20 pixels. Subsequently, the pixel values ​​within each image patch are flattened into a one-dimensional vector (e.g., 400-dimensional), and then linearly projected onto a high-dimensional hidden feature space (e.g., 768-dimensional) through a linear projection layer (i.e., a fully connected layer, or a single-layer feedforward network (FFN)). In this way, the original two-dimensional image is transformed into a sequence of multiple feature vectors (e.g., 576), i.e., a perceptual feature sequence. The advantage of this block-based processing method is that it preserves the local spatial structure information of the image while allowing subsequent attention mechanisms to process each local region in parallel, significantly improving computational efficiency.

[0086] Secondly, the feature fusion module is configured to concatenate the guidance features containing the navigation target and camera attitude with the perception feature sequence, and to perform position encoding on the concatenated feature sequence.

[0087] This step is crucial for the "task-oriented" feature extraction in this invention. Traditional image encoders often focus only on the visual features of the image itself, neglecting the agent's current flight intention and state. This invention innovatively introduces guidance features, which specifically include the relative coordinates of the navigation target point (e.g., the x and y offsets relative to the agent's current position) and the camera's real-time attitude information (e.g., pitch and roll angles), totaling four dimensions.

[0088] During processing, these guiding features are mapped through a linear layer to the same dimensional space as the perceptual feature sequence, generating a special "guiding token" (CLS token). This token is then concatenated with the original image patch feature sequence (e.g., making the input sequence 577×768), typically placed at the beginning of the sequence. Subsequently, positional encoding is applied to the entire concatenated sequence to assign positional information to each feature vector within the sequence, enabling it to distinguish the spatial arrangement order of image patches. Through this fusion method, the encoder can "know" the agent's target direction and current viewpoint in subsequent processing, thus focusing on regions in the image relevant to the navigation target while ignoring irrelevant background.

[0089] Finally, the encoding extraction module is configured to perform attention feature extraction on the position-encoded feature sequence and extract the key features of the task from the extraction results.

[0090] This module typically employs a multi-layer Transformer encoder structure, capturing long-range dependencies within the feature sequence through a multi-head self-attention mechanism. For example, the network may contain several layers (e.g., 3 layers) of Transformer Blocks, each Block containing a multi-head attention layer and a feedforward neural network layer. After multi-layer attention computation, each feature vector in the sequence incorporates global contextual information. At this point, the encoding extraction module extracts specific feature vectors from the output sequence (e.g., extracting the output vector corresponding to the "leading term" at the beginning of the sequence, or extracting the average output of all terms), and maps them to the final output space through a multi-layer feedforward network to obtain the key features of the task.

[0091] Regarding the specific physical meaning of the key features of the task, this embodiment clearly divides them into a first feature group, a second feature group, and a third feature group.

[0092] Specifically, the first feature group includes the coordinate information of the reference path nodes.

[0093] These coordinates correspond to key nodes on a reference path (such as a polyline path generated by the A* algorithm) generated based on the current depth map and navigation target. For example, this feature set has 60 dimensions, consisting of the 3D coordinates of 20 nodes arranged in order from near to far (3D × 20 = 60 dimensions). These nodes are arranged according to their distance from the agent, outlining an ideal travel trajectory from the current position to the target.

[0094] Since the default coordinates of the first node are [0,0], this feature group actually includes nodes 2 through 21. When there are fewer than 20 reference path nodes, the last node will be automatically repeated to fill the remaining 20 nodes.

[0095] The encoder extracts these features, which means it has learned to "see the way," that is, to identify the skeleton of passable passageways in the environment.

[0096] The second feature group includes the coordinate information of the nearest obstacle to the left of the safety corridor block.

[0097] It should be noted that these coordinates correspond to the three-dimensional position of the nearest Euclidean distance obstacle grid on the left side in the camera coordinate system when calculating the distance to the nearest obstacle on the left side for the 1st to 20th safety corridor blocks, totaling 60 dimensions (3D × 20 = 60 dimensions).

[0098] The third feature group includes the coordinates of the nearest obstacle to the right of the safety corridor block.

[0099] It should be noted that these coordinates correspond to the three-dimensional position of the nearest Euclidean distance obstacle grid on the right side in the camera coordinate system when calculating the distance to the nearest obstacle on the right side for the 1st to 20th safety corridor blocks, totaling 60 dimensions (3D × 20 = 60 dimensions).

[0100] Understandably, the second and third sets of features correspond to the safety boundaries on either side of the reference path, respectively. When generating a safe corridor, the positions of the nearest obstacles on the left and right sides of each corridor block are calculated. The encoder extracts these coordinates, meaning it has learned to "see the walls," that is, to accurately locate the boundaries of obstacles that pose a threat to passage. It should be understood that "left" and "right" as used here are relative to the direction the agent travels along the reference path.

[0101] The three sets of features above have an index-aligned correspondence. For ease of understanding, let i be the corridor block number, i = 1, 2, ..., 20. Since the coordinates of each point are represented by three-dimensional numerical values, and the three sets of features are arranged in corridor block order, therefore:

[0102] The (3i-2)th to 3ith dimensions of the first feature group correspond to the reference path node u associated with the ith security corridor block. i+1 3D coordinates;

[0103] The (3i-2)th to 3ith dimensions of the second feature group correspond to the three-dimensional coordinates of the nearest obstacle to the left of the ith safety corridor block in the camera coordinate system.

[0104] The (3i-2)th to 3ith dimensions of the third feature group correspond to the three-dimensional coordinates of the nearest obstacle on the right side of the i-th safety corridor block in the camera coordinate system.

[0105] Among them, u i+1 This represents the (i+1)th node on the reference path, and the definition of the reference path node u is consistent with the notation used in the aforementioned safe corridor block generation process. Through this index alignment design, three sets of features together fully describe the geometric constraint information of each corridor block: the reference path point provides the direction of travel, and the coordinates of the obstacles on the left and right sides provide the obstacle avoidance boundaries.

[0106] By outputting the three sets of explicit coordinate features mentioned above, the deep visual encoder of this invention completes an efficient "information distillation." It no longer attempts to understand the texture details of every tree and every brick in the image (these details often lead to overfitting noise), but directly outputs the coordinates of the "road" and "wall" that have direct geometric constraints on path planning decisions. This design not only greatly reduces the input dimensionality of subsequent planners and alleviates the computational burden, but more importantly, it fundamentally avoids the risk of end-to-end models failing in real-world environments due to excessive focus on irrelevant visual details, significantly improving the model's generalization ability and robustness.

[0107] It should be noted that when there are no obstacles on one side or the obstacles are too far away, there is an upper limit to the distance between the left and right sides, which is generally set to 1.5 meters when the speed does not exceed 10 meters per second.

[0108] Example 3:

[0109] This embodiment, based on the above embodiment, elaborates on the specific training process of the depth visual encoder in step S200. The core of this step lies in how to utilize real-world data to efficiently train the encoder through automatic annotation, thereby solving the problems of overfitting during training in simulation environments and high annotation costs in real-world environments for end-to-end models.

[0110] Specifically, step S200 includes the following sub-steps:

[0111] Step S210: Collect depth maps and camera pose data of physical agents in a real environment.

[0112] At this stage, data acquisition does not rely on expensive positioning equipment or complex simulation environments. In practice, a handheld acquisition device can be used, integrating a depth camera (such as the RealSense series) and an inertial measurement unit (IMU). The operator moves this device through various real-world scenarios (such as indoor offices, outdoor forests, and construction sites) to simulate different viewpoints and postures the intelligent agent might encounter. The depth camera records the depth information of the environment in real time, while the IMU simultaneously records the camera's pitch, roll, and other attitude data.

[0113] It should be understood that, in addition to handheld methods, data acquisition devices can also be mounted on mobile robots or vehicles for automated data collection, as long as depth map data with pose information can be obtained. This data, collected directly in the real environment, includes textures, lighting variations, and sensor noise unique to the real world, which is a key foundation for improving the model's generalization ability.

[0114] Step S220: Randomly assign navigation targets to the depth map.

[0115] For each frame of depth map data acquired, a navigation target point is randomly assigned. This target point is typically represented by coordinate offsets relative to the current position and can be located outside the camera's field of view. The ingenuity of this design lies in the fact that by randomly assigning targets, the same depth map frame can correspond to multiple different navigation intentions, thereby greatly expanding the diversity of training samples. For example, for a narrow corridor, a "go straight" target, or a "turn left" or "turn right" target, can be assigned, forcing the encoder to learn how to extract different environmental features according to different task requirements, rather than simply memorizing the static structure of the environment.

[0116] Step S230: Based on the depth map, camera pose data and navigation target, call the path search algorithm to generate a reference path and the corresponding safe corridor block as the mentor signal for encoder training.

[0117] This is a crucial step in achieving "automatic annotation" in this embodiment. Since the data acquisition process does not record the actual driving trajectory, it cannot be directly used as a supervisory label. This invention utilizes traditional path search algorithms (such as the A* algorithm) combined with depth map information to automatically generate a reference path from the current location to the target. Based on this reference path, the system further calculates the corresponding safe corridor blocks. The safe corridor block defines the minimum collision-free spatial boundary required for the agent to traverse this area. This process is entirely automated by the algorithm, requiring no manual annotation, thus constructing a massive amount of "depth map-safe corridor" data pairs at extremely low cost. These safe corridor blocks serve as mentor signals for encoder training, containing the most critical geometric constraints for navigation in the environment.

[0118] Step S240: Based on the depth map, camera pose data and navigation target as input, and with the mentor signal as the target, train the depth visual encoder to obtain the trained depth visual encoder.

[0119] After constructing the training dataset, the specific network training phase begins. This training process employs a supervised learning paradigm, using the depth map, camera pose data, and navigation target as input, and the mentor signal as the target, to train the depth visual encoder. Specifically, it includes the following sub-steps:

[0120] Step S241: Input the depth map, camera pose data and navigation target into the depth vision encoder to obtain the task key features output by the depth vision encoder.

[0121] It should be noted that the key feature of this task is the encoder's compressed representation of environmental information.

[0122] Step S242: Update the network parameters of the deep visual encoder with the goal of minimizing the mean square error between the key features of the task and the mentor signal.

[0123] Specifically, the loss function calculates the sum of squared Euclidean distances between the feature vectors output by the encoder (such as predicted path node coordinates and obstacle coordinates) and the real safe corridor feature vectors generated in step S230. Through backpropagation, the encoder weights are continuously adjusted so that its output features increasingly accurately reflect traversable areas and obstacle boundaries in the environment. This mean squared error-based regression training method enables the encoder to accurately learn spatial geometric relationships, rather than fuzzy classification features. After training, the encoder possesses the ability to quickly extract key navigation features from complex real-world visual input, providing reliable environmental perception input for the subsequent path planner.

[0124] To more intuitively demonstrate the complete training process of a depth visual encoder, we will now combine... Figure 2 Please provide an explanation. Figure 2 A schematic flowchart of a depth vision encoder training method according to an embodiment of the present invention is shown.

[0125] like Figure 2 As shown, the training process of a depth visual encoder includes the following steps:

[0126] First, initialize the network parameters of the depth vision encoder; then, use a depth camera to acquire and store depth maps and camera pose data in the real world.

[0127] For each depth map data collected, a navigation target is randomly assigned, and a path search algorithm (such as the A* algorithm) is called to generate an obstacle avoidance reference trajectory and its corresponding safe corridor block. Based on the reference trajectory and the safe corridor block, 180-dimensional target features are extracted as the mentor signal for training the depth visual encoder (the mentor signal corresponds to the target of minimizing the mean square error in the aforementioned S242).

[0128] A training round begins by dividing all data into batches (each batch contains 128 depth data points). For each batch, the depth map, camera pose data, and navigation target are input into the depth vision encoder. The task key features output by the encoder are obtained through forward propagation of the network. The mean squared error loss function (corresponding to the aforementioned "mean squared error") between the task key features and the mentor signal is calculated, and the network parameters are updated through backpropagation.

[0129] After all batches of training are completed, determine whether the network has converged. If it has not converged, continue to the next round of training. If it has converged, end the training, obtain the trained depth visual encoder, and freeze its parameters for use in subsequent stages.

[0130] This training process enables automatic annotation and efficient training of massive amounts of real-world depth map data, eliminating the need for manual or semi-manual annotation.

[0131] Example 4:

[0132] In one alternative approach, the path planner includes a decoder network; the decoder network is configured to receive a feature sequence corresponding to the key features of the task and a position and dynamic feature sequence of the agent as input, perform position encoding and multi-layer attention decoding processing on the input, and output a B-spline path control point sequence.

[0133] This embodiment, based on the above embodiments, elaborates on the specific network structure and end-to-end training process of the path planner (Transformer path planner) in step S300. The core objective of this step is to construct an end-to-end planning network that can directly output executable paths from perceptual features, utilizing the deep visual encoder and reinforcement learning strategy trained in the preceding stages.

[0134] Specifically, the path planner described in this embodiment includes a decoder network.

[0135] The decoder network employs a Transformer architecture containing only decoder layers, i.e., a decoder-only structure. This choice is based on the sequence generation characteristics of path planning tasks: the planner needs to progressively generate a series of B-spline path control points based on the input context information, which has similar logic to text generation tasks in language models. The decoder network is configured to receive feature sequences corresponding to the key features of the task, as well as positional and dynamic feature sequences of the agent, as input.

[0136] In constructing the input data, this embodiment employs a feature fusion strategy. Specifically, the key task features output by the deep visual encoder (such as reference path node coordinates and obstacle coordinates) are converted into several visual feature tokens, while the agent's current position, velocity, acceleration, and other dynamic information are encoded into a dynamic feature token. These tokens together constitute the planner's input sequence.

[0137] It should be understood that this design, which introduces dynamic information as an independent token, is key to achieving "platform decoupling." Because the dynamic information is input into the network as an explicit feature, rather than implicitly coupled into the network weights, when the planner is deployed to physical agents of different qualities and driving capabilities, only the input dynamic token needs to be changed, without retraining the entire network, greatly improving the model's cross-platform transferability.

[0138] The decoder network performs positional encoding and multi-layer attention decoding on the input, outputting a sequence of B-spline path control points. Specifically, the network internally uses rotational positional encoding to mark the positions of the input sequence to capture the relative positional relationships between features. Subsequently, through multiple stacked decoder layers, a multi-head attention mechanism is used to perform deep feature extraction and fusion on the input sequence. Each decoder layer focuses on the correlation within the sequence through a self-attention mechanism and gradually generates predictions of future paths through cross-attention mechanisms (if encoder output exists) or direct self-attention computation. Finally, the network's output layer maps high-dimensional features into a three-dimensional coordinate sequence, i.e., the sequence of control points for the B-spline path, which defines a smooth, continuous spatiotemporal trajectory that satisfies dynamic constraints.

[0139] It should be noted that the design of using dynamic information as an independent token input also demonstrates significant advantages during training. Since the path planner's input explicitly includes the agent's position, velocity, and acceleration, the network learns to adjust the output path based on the input dynamic state, rather than internalizing specific dynamic parameters into the network weights. When this planner is subsequently deployed on physical agents of different configurations and masses, only the corresponding dynamic information needs to be provided to generate B-spline paths adapted to the platform's motion capabilities.

[0140] For the training process of the path planner, i.e., the specific execution of step S300, this embodiment adopts a supervised training paradigm "with reinforcement learning strategy as the mentor". Specifically, it includes the following sub-steps:

[0141] Step S310: Use the B-spline path corresponding to the depth map and the position and dynamics information of the agent in the simulation environment or real environment as the mentor signal for training the path planner.

[0142] This step is crucial for generating supervisory labels. This invention does not rely on manually labeled path data, but instead uses the B-spline path obtained in step S100 as the mentor signal. Specifically, depth maps, agent positions, and dynamic information collected from a simulated or real environment are input into a reinforcement learning policy network. This network, through its internal sequential decision logic, outputs a high-quality B-spline path. This path has been optimized by the reinforcement learning reward function, exhibiting good obstacle avoidance performance and smoothness. Because the reinforcement learning policy is obtained through extensive trial and error training in a simulated environment, the quality of its output path is far superior to randomly generated paths, and the generation cost is far lower than manual labeling. Therefore, using it as a mentor signal can provide high-quality supervisory guidance for the planner.

[0143] It should be noted that the training data comes from two sources: first, depth maps and corresponding dynamic information acquired directly in the simulation environment; and second, depth map data collected in the real environment during the second stage, for which a set of matching dynamic data is randomly generated. The dynamic data includes velocity, acceleration, and jerk, and its generation must satisfy the following constraints: acceleration must satisfy a kinematic coupling relationship with the agent's pitch and roll angles, and the angle between the velocity direction and the depth camera direction must not exceed 20 degrees. This approach effectively expands the diversity of training samples, further improving the path planner's generalization ability in real-world environments.

[0144] Step S320: Input the depth map into the trained depth visual encoder to extract key features for the task.

[0145] In this step, the parameters of the depth visual encoder are frozen and no longer participate in gradient updates. Its role is to compress the high-dimensional depth map data into low-dimensional task-critical features, such as the reference path node coordinates and obstacle coordinates mentioned earlier. This feature extraction process acts as an "information filter," removing texture noise from the image that is irrelevant to navigation and retaining only geometric constraint information valuable for planning decisions.

[0146] Step S330: The key features of the task are fused with the position and dynamic information of the agent to form an input feature sequence, which is then input into the path planner to obtain the predicted path output by the path planner.

[0147] The planner receives the fused feature sequence, passes it through the forward propagation of the decoder network, and directly outputs a predicted B-spline path control point sequence. This process achieves an end-to-end mapping from perceived features to path output.

[0148] Step S340: Update the network parameters of the path planner with the goal of minimizing the mean square error between the predicted path and the mentor signal trained by the path planner, so as to obtain the path planner through end-to-end training.

[0149] During training, the mean square error between the predicted path control point sequence output by the planner and the mentor signal control point sequence generated in step S310 is calculated. This error reflects the geometric deviation between the predicted path and the ideal path. This error is propagated back to the planner network via backpropagation to update its weight parameters. After multiple rounds of iterative training, the planner gradually learns to mimic the decision-making logic of reinforcement learning strategies, enabling it to directly output high-quality paths based on the input visual features and dynamic states.

[0150] It's worth noting that this training method has significant technical advantages. Firstly, it avoids the problems of low sampling efficiency and high overfitting risk encountered when training an end-to-end planner directly in a real-world environment. This is because the reinforcement learning strategy has already learned robust planning logic in the simulation environment, and the planner only needs to "imitate" this logic through supervised learning. Secondly, it achieves an effective connection between perception and decision-making: the deep visual encoder handles complex real-world visual data, while the planner generates paths. The two are connected through task-critical features, ensuring both the generalization of perception and the reliability of decision-making. The resulting path planner can be deployed in physically intelligent devices such as drones and autonomous vehicles, achieving efficient and safe autonomous navigation in real-world environments.

[0151] To more intuitively demonstrate the complete training process of the Transformer path planner, we will now combine... Figure 3 Please provide an explanation. Figure 3A flowchart illustrating the Transformer path planner training method according to an embodiment of the present invention is shown.

[0152] like Figure 3 As shown, after completing the first stage of reinforcement learning training and the second stage of deep visual encoder training, the parameters of the Transformer path planner network are initialized.

[0153] Take a batch of [depth map, agent dynamics] data from the experience pool; use the B-spline obstacle avoidance path generated by the first-stage reinforcement learning path planning method and corresponding to the [depth map, agent dynamics] data as the mentor signal for training the path planner. This path is represented in the form of 21×3-dimensional control point data.

[0154] A batch of depth map and corresponding target point information are fed into the trained deep visual encoder network to obtain 180-dimensional feature output, which is then reshaped into a matrix of 20 tokens × 9-dimensional features, i.e., the feature sequence corresponding to the key features of the task. An agent dynamics token with one token is added to the front of the matrix to form a 21 × 9-dimensional input matrix (corresponding to the aforementioned S330 "fusing the key features of the task with the position information and dynamics information of the agent into an input feature sequence").

[0155] The 21×9 dimensional input matrix is ​​fed into the Transformer path planner (specifically, the decoder network), and after forward propagation of the network, a 21×3 dimensional output matrix is ​​obtained, which is the predicted B-spline path control point sequence (corresponding to the aforementioned "predicted path").

[0156] Calculate the mean square error between the predicted path control point sequence and the mentor signal used to train the path planner (corresponding to S340 "aiming to minimize the mean square error between the predicted path and the mentor signal used to train the path planner"), backpropagate this error, update the network parameters of the Transformer path planner, and obtain the path planner through end-to-end training.

[0157] Determine if the path planner network has converged; if not, continue to the next round of training; if it has converged, the training is complete, and an end-to-end path planner that can be practically deployed on physical intelligent devices is obtained.

[0158] Example 5:

[0159] This embodiment provides a method for using a physical intelligent device, which demonstrates the deployment and application process of the path planner obtained by the above training method on a real physical platform.

[0160] Specifically, the method first deploys a path planner trained by the training method described above in a physical intelligent device.

[0161] It should be noted that the physical intelligent devices mentioned here can be aerial robots (such as drones) or ground mobile robots (such as unmanned vehicles and service robots). Because the training method of this invention decouples perception and dynamics, the deployment process is highly flexible. For drone platforms, their dynamic parameters (such as maximum acceleration and mass) may differ from the simulation parameters used during training. In this case, simply providing the planner with its real-time dynamic information as input features allows the planner to adaptively output a path that conforms to the platform's constraints, without needing to retrain the network weights. For ground robots, since they move in a two-dimensional plane, the planner is equally applicable; simply adapting the input depth map and attitude information to a ground-based perspective is sufficient.

[0162] After deployment, the system enters the real-time operation phase. It acquires the depth map, position, and attitude data of the physical intelligent device and inputs this data into the path planner to obtain the path information output by the planner.

[0163] This process is a high-frequency closed-loop control flow. The depth camera (such as the RealSense D435i) on the physical intelligence device acquires depth images of the environment at a high frequency (e.g., 30Hz), while the inertial measurement unit (IMU) simultaneously acquires the device's attitude data (such as pitch and roll angles) and position information (which can be obtained via GPS, SLAM, or odometry). This raw sensor data, after preprocessing, is directly input into the end-to-end path planner. The planner uses its internally frozen depth vision encoder to quickly extract key task features (i.e., traversable path nodes and obstacle boundaries in the environment) and, combined with the current dynamic state, outputs a complete, smooth sequence of B-spline path control points in one go through the decoder network.

[0164] To more clearly illustrate the practical application effects of this invention, the following detailed explanation is provided in conjunction with two typical deployment scenarios:

[0165] Scenario 1: Deployment with spatial positioning (applicable to ground service robots or large drones).

[0166] In this scenario, the physically intelligent device possesses precise spatial positioning capabilities, such as obtaining global coordinates through SLAM (Simultaneous Localization and Mapping) algorithms, UWB (Ultra-Wideband) positioning, or high-precision GPS. The planner performs path replanning at a relatively low frequency (e.g., 1 Hz). After each planning outputs a long-distance B-spline path, the device utilizes its precise positioning system to track the path in a closed-loop manner using a PID (Proportional-Integral-Derivative) controller.

[0167] Simultaneously, the safety of the current tracking path is verified in real time at a high frequency (e.g., 30Hz). Specifically, a local grid map is constructed in real time using newly acquired depth maps, and the collision between the currently executed B-spline path and obstacles in the grid map is detected. This verification process reuses the same safe corridor construction logic as the training phase. Once a collision risk is detected (e.g., a moving person suddenly enters the path), replanning is immediately triggered, generating a new obstacle avoidance path. This "low-frequency planning + high-frequency verification" strategy ensures both the stability of long-distance navigation and the real-time response capability to dynamic obstacles, greatly improving operational safety.

[0168] Scenario 2: Deployment without spatial positioning (applicable to small and light drones).

[0169] In this scenario, physically intelligent devices are limited by computing power or payload, making it impossible to run complex SLAM algorithms or achieve high-precision GPS positioning. The planner of this invention exhibits unique advantages. The planner performs path replanning at a high frequency (e.g., 10Hz). Each plan outputs a short-duration B-spline path (e.g., the trajectory within the next second). The device employs open-loop tracking, directly calculating the expected acceleration at each moment in the future based on the parametric equations of the B-spline path, and sending this acceleration sequence to the flight control system for execution. Due to the lack of position feedback correction, tracking errors accumulate over time. However, thanks to the high-frequency replanning mechanism of this invention, each path has a very short validity period (only a small segment is executed). Before the error accumulates to a dangerous level, a new planning cycle is initiated, generating a new path based on the latest sensor data, thus "refreshing" the tracking benchmark. This "high-frequency planning + open-loop tracking" mode allows lightweight UAVs to achieve robust autonomous obstacle avoidance flight in complex environments without expensive positioning sensors, relying solely on depth cameras and IMUs, significantly reducing hardware costs and computing power barriers.

[0170] As can be seen from the descriptions of the two scenarios above, the path planner provided by this invention has strong environmental adaptability and deployment flexibility. It can adaptively adjust the planning frequency and tracking strategy according to the hardware configuration and positioning conditions of the physical intelligent device. Under the premise of ensuring obstacle avoidance safety, it maximizes the use of the device's own perception and control capabilities, realizing a complete closed loop of end-to-end path planning technology from simulation training to physical deployment.

[0171] Example 6:

[0172] like Figure 4As shown, this embodiment provides a training device 500 for an end-to-end path planning model, which is used to execute the training method for the end-to-end path planning model described above. This device, through the division of functional modules, materializes the complex training process, facilitating deployment and execution in computer systems or server clusters.

[0173] Specifically, the device includes a first acquisition module, an encoder training module, and a planner training module.

[0174] The first acquisition module 510 simulates a simulation environment consistent with the real environment, and obtains a B-spline path for subsequent path planner training based on the safety corridor information generated from the depth map in the simulation environment, as well as the position and dynamic information of the agent.

[0175] Specifically, the first acquisition module serves as the data source and prior knowledge generator for the entire training system. At the hardware level, this module corresponds to the simulation engine interface in a high-performance computing server. Internally, it integrates a physics engine and sensor simulator, capable of constructing virtual scenes containing various obstacles. This module first generates the agent's initial state (dynamic information such as position, velocity, and acceleration), then calls a built-in path search algorithm (such as the A* algorithm) to generate a reference path, and constructs safe corridor blocks based on the reference path. Finally, this module outputs high-quality B-spline path data by running a reinforcement learning policy. This data is stored in a shared database or cache for subsequent modules to access.

[0176] It should be understood that the first acquisition module is not limited to generating data, but may also include a data preprocessing unit, which is used to format and clean the simulation output data to ensure that it meets the input standards of the subsequent neural network.

[0177] The encoder training module 520 assigns navigation targets to the depth map based on the depth map collected in the real environment and the camera pose data of the physical agent. Based on the depth map, camera pose data and navigation targets, it generates safe corridor blocks as mentor signals for encoder training, trains the depth visual encoder, and obtains the trained depth visual encoder.

[0178] Specifically, the encoder training module is the core processing unit that bridges the "simulation-reality" gap. Physically, this module connects to real-world data acquisition devices (such as handheld depth cameras or mobile robot platforms) via a data interface, or directly reads pre-stored real-world datasets. Internally, this module further includes an automatic annotation submodule and a network optimization submodule.

[0179] The automatic annotation submodule is responsible for randomly assigning navigation targets to the depth map input for each frame, and calling the same path search logic as the first acquisition module to generate safe corridor blocks, thereby achieving low-cost automatic annotation of massive data.

[0180] The network optimization submodule is responsible for building the deep visual encoder network, taking depth map, pose and target as input, automatically labeled safe corridor features as supervision, and updating network parameters through backpropagation algorithm.

[0181] The parameters of the trained depth vision encoder output by the encoder training module are frozen and saved as a model file for use by the planner training module.

[0182] The planner training module 530 is used to extract key task features by using the acquired B-spline path as the mentor signal, based on the depth map and the position and dynamic information of the agent in the simulation environment or real environment, through the trained depth vision encoder, and input the key task features and the position and dynamic information of the agent into the path planner for end-to-end training, so as to obtain a path planner for outputting obstacle avoidance path according to the input depth map and dynamic information.

[0183] Specifically, the planner training module is the final output unit of the entire device, responsible for integrating the results of the preceding modules and generating a deployable end-to-end model. This module shares storage space with the encoder training module to read the trained encoder parameters; it also shares a simulation environment interface or a real data interface with the first acquisition module to obtain training samples.

[0184] During training, the planner training module first freezes the encoder parameters, using only them to extract visual features. Then, it constructs a Transformer decoder network as the path planner, fusing visual features with dynamic information into an input sequence. This module uses the B-spline path generated by the first acquisition module as a mentor signal, optimizing the planner parameters by minimizing the error between the predicted path and the mentor signal. After training, the module outputs the final path planner model file, which can be directly ported to the embedded computing unit of a physical intelligent device for execution.

[0185] Through the collaborative work of the three modules described above, the training device provided in this embodiment achieves a complete closed loop from acquiring simulation prior knowledge and learning real-world environmental features to generating end-to-end decisions. The modules are tightly connected via data flow and control flow, ensuring both high training efficiency and the final model's generalization ability and robustness in real-world environments.

[0186] The parameters and steps for implementing the corresponding functions of each module in the training device 500 for the end-to-end path planning model in this embodiment can be referred to the parameters and steps in the embodiment of the training method for the end-to-end path planning model above, and will not be repeated here.

[0187] Example 7:

[0188] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the training method for the end-to-end path planning model described in any of the above embodiments.

[0189] Specifically, the computer-readable storage medium can be any tangible medium that can contain or store a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0190] In this embodiment, a computer-readable storage medium is used to store software code that implements the above training method. When the code is loaded and executed by the processor, it can drive the computing device to complete the entire process from acquiring prior knowledge of the simulation environment, learning features of the real environment, to generating end-to-end decisions.

[0191] It should be understood that the computer-readable storage medium may include, but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. For example, in a typical application scenario, the storage medium may be a high-performance storage array deployed on a cloud server to store massive amounts of real-world depth map data and trained depth vision encoder model parameters for download or access by multiple edge-end physical intelligent devices. In another scenario, the storage medium may also be a flash memory chip embedded in a physical intelligent device (such as a drone flight control system) to directly store a trained path planner model, enabling local real-time inference.

[0192] Through this storage medium, the technical solution proposed in this invention—which involves automatically labeling and training an encoder based on real-world data and training an end-to-end planner using reinforcement learning strategies as mentor signals—is solidified and disseminated. This allows any device with computing power to read and run the program on this medium, thereby replicating the technical effects of this invention, namely, achieving end-to-end path planning with high generalization, high perception accuracy, and platform decoupling.

[0193] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention, such as adjustments to network layers, equivalent replacements of loss functions, or adaptations to different types of physical intelligent devices, should all be covered within the scope of protection of the present invention.

Claims

1. A training method for an end-to-end path planning model, characterized in that, The method includes: S100: A simulation environment that is consistent with the real environment is used to generate a safe corridor information based on the depth map in the simulation environment, as well as the position and dynamic information of the agent, to obtain a B-spline path for subsequent path planner training. S200: Based on the depth map collected from the real environment and the camera pose data of the physical agent, a navigation target is assigned to the depth map. A safe corridor block is generated based on the depth map, camera pose data and navigation target, which serves as the mentor signal for encoder training. The depth visual encoder is trained to obtain the trained depth visual encoder. S300. Using the acquired B-spline path as the mentor signal, based on the depth map and the position and dynamics information of the agent in the simulation or real environment, the key features of the task are extracted by the trained depth vision encoder, and the key features of the task and the position and dynamics information of the agent are input into the path planner for end-to-end training to obtain a path planner for outputting obstacle avoidance path based on the input depth map and dynamics information.

2. The training method for an end-to-end path planning model according to claim 1, characterized in that, The depth vision encoder includes: The block mapping module is configured to perform block processing and linear projection on the input depth map to generate a perceptual feature sequence. The feature fusion module is configured to concatenate the guidance features, which include the navigation target and camera attitude, with the perception feature sequence, and to perform position encoding on the concatenated feature sequence. The encoding extraction module is configured to perform attention feature extraction on the position-encoded feature sequence and obtain the key features of the task from the extraction results.

3. The training method for an end-to-end path planning model according to claim 1, characterized in that, S200 includes: Collect depth maps and camera pose data of physical agents in real-world environments; Randomly assign navigation targets to the depth map; Based on the depth map, camera pose data, and navigation target, a path search algorithm is invoked to generate a reference path and the corresponding safe corridor block as a mentor signal for encoder training. Using the depth map, camera pose data, and navigation target as input, and the mentor signal as the target, the depth visual encoder is trained to obtain the trained depth visual encoder.

4. The training method for an end-to-end path planning model according to claim 3, characterized in that, The process of training the depth visual encoder based on the depth map, camera pose data, and navigation target as input, and using the mentor signal as the target, includes: The depth map, camera pose data, and navigation target are input into the depth visual encoder to obtain the task key features output by the depth visual encoder. The network parameters of the deep visual encoder are updated with the goal of minimizing the mean square error between the key features of the task and the mentor signal trained by the encoder.

5. The training method for an end-to-end path planning model according to claim 3, characterized in that, The key features of the task include a first feature group, a second feature group, and a third feature group; The first feature group includes the coordinate information of the reference path nodes; The second feature group includes the coordinate information of the nearest obstacle on the left side of the safety corridor block; The third feature group includes the coordinate information of the nearest obstacle to the right of the safety corridor block.

6. The training method for an end-to-end path planning model according to claim 1, characterized in that, The path planner includes a decoder network; The decoder network is configured to receive the feature sequence corresponding to the key features of the task and the position and dynamic feature sequence of the agent as input, perform position encoding and multi-layer attention decoding processing on the input, and output the B-spline path control point sequence.

7. The training method for an end-to-end path planning model according to claim 6, characterized in that, The S300 includes: The B-spline path corresponding to the depth map and the position and dynamics information of the agent in the simulation or real environment is used as the mentor signal for training the path planner. The depth map is input into the trained depth visual encoder to extract key features for the task. The key features of the task are fused with the position and dynamic information of the agent to form an input feature sequence, which is then input into the path planner to obtain the predicted path output by the path planner. The network parameters of the path planner are updated with the goal of minimizing the mean square error between the predicted path and the mentor signal used to train the path planner, so as to obtain the path planner through end-to-end training.

8. A method of using a physical intelligent device, characterized in that, The method includes: Deploy a path planner trained using the training method of an end-to-end path planning model according to any one of claims 1 to 7 in a physical intelligent device; Acquire the depth map and camera pose data of the physical intelligent device, and extract key features of the task through a trained depth vision encoder; The position and dynamics information of the physical agent are obtained, and the key features of the task and the position and dynamics information are input into the path planner to obtain the path information output by the path planner.

9. A training device for an end-to-end path planning model, characterized in that, The device includes: The first acquisition module simulates a simulation environment consistent with the real environment, and obtains B-spline paths for subsequent path planner training based on the safety corridor information generated from the depth map in the simulation environment, as well as the position and dynamic information of the agent. The encoder training module assigns navigation targets to the depth map based on the depth map collected in the real environment and the camera pose data of the physical agent. Based on the depth map, camera pose data and navigation targets, it generates safe corridor blocks as mentor signals for encoder training, trains the depth visual encoder, and obtains the trained depth visual encoder. The path planner training module is used to extract key task features based on the acquired B-spline path as the mentor signal, the depth map in the simulation environment or the position and dynamics information of the agent, and the trained depth vision encoder. The key task features and the position and dynamics information of the agent are then input into the path planner for end-to-end training to obtain a path planner that outputs obstacle avoidance paths based on the input depth map and dynamics information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a training method for an end-to-end path planning model according to any one of claims 1 to 7.

Images