An autonomous path planning method and system for an unmanned vehicle based on a large model guide

Abstract

The application provides a large model guidance-based unmanned vehicle autonomous path planning method and system, and belongs to the technical field of unmanned vehicle autonomous navigation and path planning. The method constructs a hierarchical autonomous navigation framework, including a high-level semantic waypoint generation module and a low-level reinforcement learning control module; the high-level module performs semantic understanding on the environment through a VLA model and generates a local trajectory point sequence, and selects trajectory points far away as local navigation targets; the low-level module constructs multi-modal features from visual images, laser radar distances and robot state information, jointly models using a Transformer multi-modal fusion encoding network, and outputs continuous control actions through an SAC algorithm to realize local trajectory tracking and real-time obstacle avoidance. The application effectively overcomes the dependence of traditional methods on environment maps and the problem of pure reinforcement learning being prone to local optimization, and improves the autonomous navigation ability and robustness of unmanned vehicles in complex map-free environments.

Description

Technical Field

[0001] This invention belongs to the field of autonomous navigation and path planning technology for unmanned vehicles, and relates to an autonomous path planning method and system for unmanned vehicles based on large model guidance. In particular, it is an autonomous path planning method for unmanned vehicles based on the fusion of vision-language-action model (VLA) and reinforcement learning, which is used for mapless autonomous navigation and real-time obstacle avoidance control of unmanned vehicles in complex environments. Background Technology

[0002] The field of autonomous navigation and path planning for unmanned vehicles mainly includes methods based on traditional optimization, methods based on reinforcement learning, and hierarchical navigation methods based on large models. Traditional path planning methods, such as Dijkstra's algorithm, A*, Artificial Potential Field (APF), and Restricted Path Planning (RRT), are widely used in unmanned vehicle navigation tasks in structured environments due to their mature principles and relatively simple implementation. However, these methods typically rely on pre-built environmental maps or accurate environmental models, resulting in poor adaptability to unknown scenarios and complex obstacle environments. They are also prone to problems such as high computational cost, local optima, and insufficient real-time performance.

[0003] Reinforcement learning-based methods aim to overcome the limitations of traditional planning methods, which heavily rely on environmental models. Typical methods include PPO, DDPG, TD3, and SAC. These methods learn navigation strategies through interaction between the agent and the environment, and can directly utilize sensor information to output control commands, exhibiting good autonomous decision-making and environmental adaptability in complex environments. However, these methods typically require large amounts of training data and computational resources, and lack global semantic understanding capabilities. They are prone to getting trapped in local optima during long-distance navigation or in complex maze environments, affecting navigation stability and path planning efficiency.

[0004] In recent years, hierarchical navigation methods based on visual language navigation and VLA have gained increasing attention. These methods utilize large models to semantically understand the environment and generate local waypoints or sub-objects to enhance the global planning capabilities of autonomous vehicles. Some studies further combine traditional controllers or reinforcement learning controllers to complete local motion execution, thereby improving navigation performance in complex scenarios. However, existing methods still suffer from insufficient coordination between high-level semantic planning and low-level motion control, easily leading to conflicts between waypoint generation and local control, trajectory oscillations, and insufficient obstacle avoidance capabilities. Simultaneously, the fusion and utilization of multimodal information such as vision, LiDAR, and robot state is still insufficient, limiting the system's robustness and generalization ability in complex environments.

[0005] To address the aforementioned issues and balance global semantic guidance capabilities with local real-time control performance, existing research has begun to explore layered fusion of large models and reinforcement learning. This involves high-level modules generating semantic waypoints and low-level modules executing motion control to improve the autonomous navigation capabilities of unmanned vehicles in map-free environments. However, existing methods still suffer from two key shortcomings: first, the lack of effective coordination between high-level waypoint generation and low-level control strategies can easily lead to local control instability; second, existing control methods do not adequately utilize multimodal perception information, and their obstacle avoidance capabilities and navigation robustness in complex environments still need improvement.

[0006] Therefore, it is necessary to address how to effectively integrate the semantic understanding capabilities of large models with the continuous control capabilities of reinforcement learning, and to achieve joint modeling of multimodal perception information, thereby improving the autonomous path planning capabilities and real-time obstacle avoidance performance of autonomous vehicles in complex environments. Summary of the Invention

[0007] The technical problem solved by this invention is as follows: In existing autonomous navigation and path planning technologies for unmanned vehicles, traditional optimization-based path planning methods rely on pre-built environmental maps, which have poor adaptability in environments and complex mapless scenarios, and are prone to local optima and insufficient real-time performance. Although reinforcement learning-based methods can learn navigation strategies through environmental interaction, they suffer from slow training convergence speed, low sample efficiency, and lack of global semantic understanding capabilities, making them prone to getting stuck in local optima in complex scenarios. Existing hierarchical navigation methods based on visual language navigation or VLA can provide certain global semantic guidance capabilities, but still suffer from insufficient coordination between high-level semantic planning and low-level motion control, insufficient utilization of multimodal perception information, and poor local obstacle avoidance stability.

[0008] In view of this, the purpose of this invention is to provide an autonomous path planning method and system for unmanned vehicles based on large model guidance. This method and system achieve efficient autonomous navigation and real-time obstacle avoidance control of unmanned vehicles in complex environments by constructing a hierarchical navigation framework with high and low layers decoupled, and combining multimodal information such as vision, lidar and robot state.

[0009] To achieve the above objectives, the present invention provides the following technical solution: An autonomous path planning method for unmanned vehicles based on a large model, the method specifically includes the following steps: S1. Construct a hierarchical autonomous navigation framework for unmanned vehicles, dividing the path planning process into a high-level semantic waypoint generation module and a low-level reinforcement learning control module. S2. In the high-level semantic waypoint generation module, the current environment image, historical observation information and target condition information of the unmanned vehicle are obtained and input into the VLA model for environmental semantic understanding and local trajectory prediction. S3. Utilize the VLA model to output a sequence of local trajectory points for the next few steps, and select the local trajectory points that are farther away as the tracking targets of the low-level control module to achieve global direction guidance. S4. In the low-level reinforcement learning control module, acquire the visual image information, lidar distance information and robot state information of the unmanned vehicle, and construct multimodal state input; S5. The visual image information, lidar distance information and robot state information obtained by the low-level reinforcement learning control module are preprocessed, normalized and feature embedded, and different modal information is uniformly mapped to the same feature dimension to construct a multimodal feature sequence for reinforcement learning control policy network. S6. Utilize a Transformer-based multimodal fusion coding network to jointly model visual features, laser features, and robot state features, and extract environmental spatial structure information and target guidance features; S7. Input the fused state features output by the Transformer multimodal fusion coding network into a low-level reinforcement learning control policy network constructed based on the SAC algorithm. Generate continuous control actions of the unmanned vehicle through the Actor policy network, and evaluate the value of the state-action pairs through the Critic value network. At the same time, combine the experience replay mechanism, the double Q network structure, the maximum entropy policy optimization and the target network soft update mechanism to train and update the low-level control policy, so that the unmanned vehicle can achieve local trajectory tracking, real-time obstacle avoidance and stable motion control when tracking high-level semantic waypoints. S8. Data interaction between the high-level semantic waypoint generation module and the low-level reinforcement learning control module is realized through the ROS communication mechanism, and autonomous navigation control of the unmanned vehicle is completed.

[0010] Furthermore, in step S2, the high-level semantic waypoint generation module performs the following steps: Step S21: Obtain the current environment image and historical image sequence of the unmanned vehicle. The current environment image is cropped and scaled to a size of 224×224 for extracting semantic features; the historical image sequence is cropped and scaled to a size of 96×96 for extracting motion features; and VLA input is constructed by combining natural language target description and target pose information. Step S22: Use the VLA model to perform semantic understanding of the environment and predict future local trajectory point sequences; Step S23: Select a preset trajectory point from the predicted local trajectory point sequence as the local navigation target, and periodically update the local waypoint information.

[0011] Furthermore, in step S5, the multi-frame visual images are divided into several image blocks and visual tokens are generated; the LiDAR scanning data is divided into multiple sectors according to a preset angle range, and a LiDAR token is generated based on the obstacle distance in each sector; the relative distance and relative orientation angle between the unmanned vehicle and the local navigation target, as well as the motion state information such as the current linear velocity and angular velocity of the unmanned vehicle, are mapped into state tokens; finally, the visual tokens, LiDAR tokens, and state tokens are concatenated in a preset order to form a unified multimodal input sequence for subsequent Transformer multimodal fusion coding.

[0012] Furthermore, in step S5, constructing the multimodal feature sequence for the reinforcement learning control policy network includes the following steps: Step S51: Preprocess the visual image information, acquire multiple consecutive frames of environmental images captured by the front camera of the unmanned vehicle, convert them into grayscale images and perform size scaling, cropping and pixel normalization processing. Step S52: Divide the preprocessed multi-frame visual images into image blocks. Divide the input image into multiple non-overlapping image blocks according to a preset size. Each image block is a local visual unit and is converted into a visual feature vector through a linear mapping method to form a visual token sequence. Step S53: Perform angle sectorization processing on the lidar distance information. Select a preset angle range in front as the effective sensing range with the direction of the unmanned vehicle's movement as the center, and divide the effective sensing range evenly into multiple angle sectors. For each sector, count the distance values ​​of all lidar ranging points in the sector and select the minimum distance as the representative distance of the sector to obtain the lidar distance vector. Step S54: Normalize and embed the distance vector of the LiDAR. Normalize the distance value of each sector according to the maximum effective ranging range of the LiDAR to reduce the impact of different scale data on the stability of network training. Then, map the normalized distance value of each sector to a unified feature dimension to generate a LiDAR token, so that each LiDAR token corresponds to local spatial distance information within a fixed angle range. Step S55: Normalize and embed the robot state information. The robot state information includes the relative distance and relative orientation angle between the unmanned vehicle and the local navigation target, the current linear velocity of the unmanned vehicle, and the current angular velocity of the unmanned vehicle. Among them, the relative distance is used to represent the distance of the unmanned vehicle from the high-level semantic waypoint, the relative orientation angle is used to represent the deflection direction of the local navigation target relative to the vehicle's coordinate system, and the linear velocity and angular velocity are used to characterize the current motion state of the unmanned vehicle. After normalizing the above state quantities, they are converted into state tokens through a fully connected mapping layer. Step S56: Concatenate the state token, visual token, and laser token to form a unified multimodal feature sequence; Step S57: Add positional encoding to the multimodal feature sequence, wherein the positional encoding is used to preserve the spatial order relationship between image blocks and laser sectors; Step S58: Input the spliced ​​and encoded multimodal feature sequence into the Transformer-based multimodal fusion coding network. This network jointly models the scene structure information in the visual image, the obstacle distance information in the LiDAR, and the target guidance information in the robot state, and outputs the fused state feature representation for the low-level reinforcement learning control policy network.

[0013] Furthermore, in step S51, the visual input adopts the form of stacking four consecutive grayscale images with an image size of 256×144, which is used to retain environmental change information in a short period of time; in step S52, the 256×144 image is divided into 16×16 sub-image blocks with a size of 16×9.

[0014] Furthermore, in step S56, the state token is set at the beginning of the sequence so that it plays a target guidance role in the subsequent Transformer encoding process; then the visual token and the laser token are concatenated in sequence so that the network can simultaneously focus on the visual scene structure and the distance distribution of local obstacles under the target constraint.

[0015] Furthermore, in step S7, the low-level reinforcement learning control policy network constructed based on the SAC algorithm includes the following steps: Step S71: Construct the state space of the low-level reinforcement learning control module. The state space consists of current multimodal observation information, including continuous multi-frame visual image information, lidar distance information, relative distance and relative orientation angle between the unmanned vehicle and the local navigation target, and the current linear velocity and angular velocity of the unmanned vehicle. The above information is used to extract features and jointly model through a multimodal fusion coding network to obtain a fusion state feature representation for input to the SAC control policy network. Step S72: Construct the continuous motion space of the unmanned vehicle, which includes linear velocity control quantity and angular velocity control quantity, wherein the linear velocity control quantity is used to control the forward speed of the unmanned vehicle, and the angular velocity control quantity is used to control the turning motion of the unmanned vehicle; Step S73: Generate a continuous action distribution through the Actor policy network, input the fused state features into the Actor policy network, and output the mean and standard deviation parameters of the action distribution from the Actor network. Then, use the reparameterized sampling method to sample the control action from the action distribution. Subsequently, use the hyperbolic tangent function or the action scaling function to constrain the range of the sampled action to obtain the linear velocity command and angular velocity command of the unmanned vehicle. Step S74: Send the linear velocity command and angular velocity command to the motion control interface of the unmanned vehicle, so that the unmanned vehicle can perform local motion according to the continuous action; during the execution, the unmanned vehicle moves according to the local navigation target provided by the high-level semantic waypoint, and at the same time combines the information of nearby obstacles sensed by the lidar to achieve real-time obstacle avoidance, so as to gradually approach the local navigation target while ensuring safety. Step S75: Evaluate the value of the current state-action pair through the Critic value network. The Critic value network includes two independent Q networks, which estimate the value of the current fused state features and control actions respectively, and take the smaller of the two Q values ​​as the basis for the target value estimation, so as to reduce the overestimation problem that may be generated by a single value network and improve the stability of low-level control policy training. Step S76: Construct a reward function for training the low-level control strategy. The reward function is determined based on factors such as the distance change between the autonomous vehicle and the local navigation target, whether the vehicle reaches the local navigation target, whether a collision occurs, the obstacle safety distance, the magnitude of linear velocity, the magnitude of angular velocity change, and the smoothness of motion. Specifically, a positive reward is given when the autonomous vehicle approaches the local navigation target; a larger target reward is given when the autonomous vehicle reaches the local navigation target; a penalty is given when the autonomous vehicle collides with an obstacle or gets too close to an obstacle; and a corresponding penalty is given when the motion changes too much or there is a long period of stagnation. This ensures that the autonomous vehicle simultaneously considers target tracking efficiency, obstacle avoidance safety, and motion smoothness during the learning process. Step S77: Store the state transition samples generated during the interaction between the autonomous vehicle and the environment into the experience replay pool. The state transition samples include the current state, current action, immediate reward, next state, and termination flag. During training, randomly sample a small batch of samples from the experience replay pool to update the Actor policy network and the Critic value network. Through the experience replay mechanism, the correlation between continuous interaction samples can be broken, improving the efficiency of training data utilization and the stability of policy updates. Step S78: The maximum entropy strategy is used to optimize the target update Actor policy network. In the policy optimization process, not only is the cumulative reward maximized, but also the policy entropy term is introduced to keep the policy appropriately random during the training phase, thereby enhancing the autonomous vehicle's ability to explore in complex obstacle environments and reducing the probability of getting stuck in local optimal paths or local oscillations. Step S79: The target network parameters of the Critic are updated using the target value network soft update mechanism. By smoothly updating the target network parameters according to the preset soft update coefficient, the calculation of the target Q value is more stable, the numerical oscillation during the training process is reduced, and the convergence stability of the SAC algorithm in the continuous action control task is improved. Step S710: After training is completed, the Actor policy network is deployed as a low-level control policy. During the actual navigation or testing phase, the Actor network outputs deterministic or near-deterministic linear velocity and angular velocity control quantities based on the current fusion state characteristics, enabling the unmanned vehicle to complete local waypoint tracking, real-time obstacle avoidance, and continuous stable motion control under the guidance of high-level semantic waypoints.

[0016] Furthermore, in step S72, the linear velocity control amount is constrained to... Within the range, the angular velocity control quantity is constrained to Within the specified range, to ensure that the output actions comply with the kinematic constraints of the unmanned vehicle chassis and the requirements for safe driving.

[0017] This invention also provides an autonomous path planning system for unmanned vehicles based on a large model. This system employs the method described above and includes: The high-level semantic waypoint generation module is used to acquire the current environmental image, historical observation information and target condition information of the unmanned vehicle, and input them into the vision-language-action model for environmental semantic understanding and local trajectory prediction. It outputs a sequence of local trajectory points for the next few steps, and selects the local trajectory points that are farther away as the tracking targets of the low-level control module. The low-level reinforcement learning control module is used to acquire visual image information, lidar distance information and robot state information of the unmanned vehicle and construct multimodal state input. It performs joint modeling of visual features, lidar features and robot state features through a multimodal fusion coding network, and generates continuous control actions of the unmanned vehicle through a reinforcement learning control policy network based on the SAC algorithm, so as to realize local trajectory tracking, real-time obstacle avoidance and stable motion control. The data interaction module is used to realize data interaction between the high-level semantic waypoint generation module and the low-level reinforcement learning control module through the ROS communication mechanism.

[0018] Furthermore, the low-level reinforcement learning control module includes: The multimodal feature construction unit is used to preprocess, normalize, and embed features of visual image information, LiDAR distance information, and robot state information, and to uniformly map different modal information to the same feature dimension to construct a multimodal feature sequence. The Transformer multimodal fusion coding unit is used to jointly model visual features, laser features, and robot state features to extract environmental spatial structure information and target guidance features. SAC reinforcement learning control unit is used to input fused state features into the Actor policy network to generate continuous control actions for the unmanned vehicle, and to evaluate the value of state-action pairs through the Critic value network. It combines experience playback mechanism, double Q network structure, maximum entropy policy optimization and target network soft update mechanism to train and update control policy. The multimodal feature construction unit includes: The visual feature extraction subunit is used to perform grayscale conversion, size scaling, cropping and pixel normalization on continuous multi-frame environmental images captured by the front camera of the unmanned vehicle, and divide the input image into multiple non-overlapping image blocks according to a preset size, and convert them into a visual token sequence through linear mapping. The laser feature extraction subunit is used to select a preset angle range in front of the unmanned vehicle as the effective perception range and divide it evenly into multiple angle sectors. The minimum distance value of the laser radar ranging point in each sector is counted as the representative distance. After normalizing the distance values ​​of each sector, they are mapped to a unified feature dimension to generate a laser token. The state feature extraction subunit is used to normalize the relative distance, relative orientation angle, current linear velocity, and current angular velocity between the unmanned vehicle and the local navigation target, and then convert them into state tokens through a fully connected mapping layer. The feature splicing subunit is used to splice the state token, visual token, and laser token into a unified multimodal feature sequence, and to add position encoding to the multimodal feature sequence.

[0019] The beneficial effects of this invention are as follows: Compared with existing autonomous vehicle path planning methods, the autonomous path planning method for autonomous vehicles proposed in this invention, based on the fusion of VLA and reinforcement learning, possesses both global semantic guidance capabilities and local real-time control capabilities. Specifically, this is reflected in the following aspects: 1) By introducing a VLA model to generate semantic waypoints, it provides global directional guidance for the autonomous vehicle, effectively alleviating the problem of pure reinforcement learning methods easily getting trapped in local optima; 2) It adopts a hierarchical decoupled navigation structure, enabling high-level semantic planning and low-level motion control to work collaboratively, improving navigation stability and path planning efficiency; 3) It combines multimodal information such as vision, LiDAR, and robot state for joint modeling, improving environmental perception, obstacle avoidance, and navigation robustness in complex environments; 4) It can achieve efficient autonomous navigation in mapless complex environments, possessing good generalization ability and practical application value.

[0020] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is an overall framework diagram of the method of the present invention; Figure 2 This is a pseudocode diagram of the method of the present invention; Figure 3 Reward curves for different algorithms during the training process; Figure 4 The diagram shows the planning results of different algorithms in three scenarios during the simulation experiment. Figure 5 The results show the performance comparison of different algorithms in three scenarios. Detailed Implementation

[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0023] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0024] This invention proposes an autonomous path planning method for unmanned vehicles based on the fusion of VLA and reinforcement learning. Compared with existing technologies, this invention exhibits better navigation performance and environmental adaptability. To verify the effectiveness of this invention, in this embodiment, multiple complex map-less navigation scenarios were constructed on the ROS-Gazebo simulation platform, and comparative experiments were conducted with VisionFormer, LaserFormer, and existing reinforcement learning navigation methods. Experimental results show that this invention outperforms existing methods in terms of navigation success rate, path planning efficiency, and training convergence speed.

[0025] Specifically, this invention introduces a VLA model to generate semantic waypoints, providing global directional guidance for autonomous vehicles and effectively alleviating the problem of traditional reinforcement learning methods easily getting trapped in local optima in complex maze environments. In complex scenarios such as U-shaped mazes and bow-shaped mazes, this invention can guide autonomous vehicles to complete obstacle avoidance navigation, while traditional reinforcement learning methods are prone to prolonged wandering, collisions, or navigation failures. Simultaneously, this invention employs a decoupled hierarchical navigation structure, enabling high-level semantic planning and low-level motion control to work collaboratively, improving path planning efficiency and motion continuity while ensuring navigation stability.

[0026] This invention also employs a multimodal fusion approach to jointly model visual images, LiDAR data, and robot state information, resulting in stronger environmental perception and robustness compared to single-modal methods. The VisionLaserFormer multimodal fusion coding network effectively integrates complementary information from different sensors, thereby improving real-time obstacle avoidance and navigation stability in complex environments. Experimental results demonstrate that this invention outperforms single-modal reinforcement learning methods in terms of navigation success rate and path smoothness in complex obstacle environments.

[0027] Furthermore, real-world scenario verification experiments were conducted. The experimental results show that the method proposed in this invention is not only applicable to simulation environments but also enables stable autonomous navigation on real unmanned vehicle platforms. Even in complex environments, it maintains good path planning and real-time obstacle avoidance capabilities, verifying the invention's significant engineering application value and practical deployment capability.

[0028] Although this invention is mainly designed and verified for unmanned vehicles in mapless autonomous navigation scenarios, the proposed hierarchical semantic navigation and multimodal reinforcement learning fusion method is also applicable to mobile robots, autonomous vehicles and intelligent delivery robots, and has broad application prospects.

[0029] The overall framework of the method of this invention is as follows: Figure 1As shown, it mainly includes a high-level semantic waypoint generation module and a low-level reinforcement learning control module. The high-level module generates local semantic waypoints based on a VLA model, while the low-level module uses multimodal reinforcement learning to achieve trajectory tracking and real-time obstacle avoidance control.

[0030] In this embodiment, the autonomous navigation task of the unmanned vehicle is first constructed, and the navigation problem of the unmanned vehicle in a complex map-free environment is modeled as a partially observable Markov decision process (POMDP). Let the robot at time t... The state is Execute actions Then obtain the next state and rewards The objective is to learn the optimal strategy. This maximizes the cumulative reward. The control variables for the autonomous vehicle consist of linear velocity and angular velocity.

[0031] In the formula, Indicates linear velocity. It represents angular velocity.

[0032] This embodiment uses the VLA model as the high-level semantic planning module, and the specific process is as follows: Figure 1 As shown. First, the current environment image, historical image sequence, and target pose information of the autonomous vehicle are acquired, and then input into the VLA model for environmental semantic understanding. The VLA model outputs a sequence of local trajectory points for the next few steps:

[0033] In this embodiment, the 8th trajectory point, which is relatively far away, is selected as the local tracking target to enhance the navigation efficiency and obstacle avoidance capability of the local controller.

[0034] Furthermore, this embodiment employs the VisionLaserFormer multimodal fusion coding network based on Transformer as the low-level control module, and its structure is as follows: Figure 1 As shown. The input information includes visual images, LiDAR distance information, and robot state information. The visual input consists of four consecutive stacked grayscale images, each with a size of 256×144. The LiDAR input is a distance vector of length 40. The robot state information includes target distance, target angle, linear velocity, and angular velocity.

[0035] In the visual feature extraction process, the input image is first divided into multiple image patches and mapped to a sequence of visual tokens. Let the input image be:

[0036] Where C represents the number of image channels, and H and W represent the image height and width, respectively. The image is then divided into several patches and mapped to a unified feature space to form visual tokens. Simultaneously, the LiDAR distance information and robot state information are mapped to LiDAR tokens and state tokens, respectively, and concatenated to form a unified multimodal feature sequence.

[0037] in, For state token, For visual tokens, This is a laser token. Subsequently, a Transformer coding network is used to jointly model multimodal information, enabling information interaction and feature fusion between different modalities.

[0038] In the reinforcement learning training process, this embodiment uses the SAC algorithm for policy optimization, and its training process is as follows: Figure 2 As shown. First, the simulation environment (Env), the SAC agent (Agent), and the experience replay buffer are initialized. Then, state transition samples are obtained through continuous interaction between the agent and the environment:

[0039] The data is then stored in the experience replay buffer. Once the experience replay buffer meets the sampling conditions, a small batch of experience data is randomly sampled to update the Critic network, Actor network, and target network. The SAC algorithm employs a maximum entropy strategy for optimization, with the following optimization objective:

[0040] In the formula, The entropy temperature coefficient Represents policy entropy.

[0041] This embodiment uses Ubuntu 20.04, ROS Noetic, and Gazebo simulation platforms for experimental verification. The unmanned vehicle platform adopts a differential drive model, with the control frequency set at 10Hz, the LiDAR field of view angle at 240°, and divided into 40 distance zones; the simulation scenarios include various mapless navigation environments such as U-shaped mazes, bow-shaped mazes, and complex dynamic obstacle scenarios. Figure 3 The reward curves for different algorithms during the training process are shown.

[0042] To verify the effectiveness of the method of this invention, this embodiment compares the proposed method with existing reinforcement learning navigation methods. The experimental results are as follows: Figure 4 and Figure 5As shown, this invention outperforms existing methods in terms of navigation success rate, path planning efficiency, and training convergence speed. In U-shaped maze scenarios, traditional reinforcement learning methods are prone to getting trapped in local optima due to local reward-driven approaches, while this invention generates semantic waypoints through VLA to guide the autonomous vehicle around obstacles to reach the target area, significantly improving navigation capabilities in complex environments.

[0043] Furthermore, this embodiment also included real-world scenario verification experiments. A real-world unmanned vehicle platform was equipped with a front-facing camera and a LiDAR sensor, and deployed with Ubuntu and ROS systems. Experimental results show that the method of this invention can stably complete autonomous navigation tasks in real-world complex environments, verifying that the method has good engineering application capabilities and practical deployment value.

[0044] The above embodiments clearly illustrate the significant advantages of the present invention in improving the autonomous path planning capability, real-time obstacle avoidance capability, and navigation robustness of unmanned vehicles in complex environments. Through the description of the above embodiments, the present invention achieves a hierarchical fusion of VLA semantic guidance and reinforcement learning motion control, effectively overcoming the problems of traditional path planning methods being highly dependent on environmental maps, pure reinforcement learning methods being prone to getting trapped in local optima, and existing hierarchical navigation methods failing to utilize multimodal information. This improves the autonomous path planning capability, real-time obstacle avoidance capability, and navigation robustness of unmanned vehicles in complex environments.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for autonomous path planning of unmanned vehicles based on large model guidance, characterized in that, The method specifically includes the following steps: S1. Construct a hierarchical autonomous navigation framework for unmanned vehicles, dividing the path planning process into a high-level semantic waypoint generation module and a low-level reinforcement learning control module. S2. In the high-level semantic waypoint generation module, the current environment image, historical observation information and target condition information of the unmanned vehicle are obtained and input into the VLA model for environmental semantic understanding and local trajectory prediction. S3. Utilize the VLA model to output a sequence of local trajectory points for the next few steps, and select the local trajectory points that are farther away as the tracking targets of the low-level control module to achieve global direction guidance. S4. In the low-level reinforcement learning control module, acquire the visual image information, lidar distance information and robot state information of the unmanned vehicle, and construct multimodal state input; S5. The visual image information, lidar distance information and robot state information obtained by the low-level reinforcement learning control module are preprocessed, normalized and feature embedded, and different modal information is uniformly mapped to the same feature dimension to construct a multimodal feature sequence for reinforcement learning control policy network. S6. Utilize a Transformer-based multimodal fusion coding network to jointly model visual features, laser features, and robot state features, and extract environmental spatial structure information and target guidance features; S7. Input the fused state features output by the Transformer multimodal fusion coding network into a low-level reinforcement learning control policy network constructed based on the SAC algorithm. Generate continuous control actions of the unmanned vehicle through the Actor policy network, and evaluate the value of the state-action pairs through the Critic value network. At the same time, combine the experience replay mechanism, the double Q network structure, the maximum entropy policy optimization and the target network soft update mechanism to train and update the low-level control policy, so that the unmanned vehicle can achieve local trajectory tracking, real-time obstacle avoidance and stable motion control when tracking high-level semantic waypoints. S8. Data interaction between the high-level semantic waypoint generation module and the low-level reinforcement learning control module is realized through the ROS communication mechanism, and autonomous navigation control of the unmanned vehicle is completed.

2. The autonomous path planning method for unmanned vehicles based on large model guidance according to claim 1, characterized in that, In step S2, the high-level semantic waypoint generation module performs the following steps: Step S21: Obtain the current environment image and historical image sequence of the unmanned vehicle. The current environment image is cropped and scaled to a size of 224×224 for extracting semantic features; the historical image sequence is cropped and scaled to a size of 96×96 for extracting motion features; and VLA input is constructed by combining natural language target description and target pose information. Step S22: Use the VLA model to perform semantic understanding of the environment and predict future local trajectory point sequences; Step S23: Select a preset trajectory point from the predicted local trajectory point sequence as the local navigation target, and periodically update the local waypoint information.

3. The autonomous path planning method for unmanned vehicles based on large model guidance according to claim 2, characterized in that, In step S5, multiple frames of visual images are divided into several image blocks and visual tokens are generated; the LiDAR scanning data is divided into multiple sectors according to a preset angle range, and LiDAR tokens are generated based on the obstacle distances in each sector; the relative distance and relative orientation angle between the unmanned vehicle and the local navigation target, as well as the motion state information such as the current linear velocity and angular velocity of the unmanned vehicle, are mapped into state tokens; finally, the visual tokens, LiDAR tokens, and state tokens are concatenated in a preset order to form a unified multimodal input sequence for subsequent Transformer multimodal fusion coding.

4. The autonomous path planning method for unmanned vehicles based on large model guidance according to claim 3, characterized in that, In step S5, constructing the multimodal feature sequence for the reinforcement learning control policy network includes the following steps: Step S51: Preprocess the visual image information, acquire multiple consecutive frames of environmental images captured by the front camera of the unmanned vehicle, convert them into grayscale images and perform size scaling, cropping and pixel normalization processing. Step S52: Divide the preprocessed multi-frame visual images into image blocks. Divide the input image into multiple non-overlapping image blocks according to a preset size. Each image block is a local visual unit and is converted into a visual feature vector through a linear mapping method to form a visual token sequence. Step S53: Perform angle sectorization processing on the lidar distance information. Select a preset angle range in front as the effective sensing range with the direction of the unmanned vehicle's movement as the center, and divide the effective sensing range evenly into multiple angle sectors. For each sector, count the distance values ​​of all lidar ranging points in the sector and select the minimum distance as the representative distance of the sector to obtain the lidar distance vector. Step S54: Normalize and embed the distance vector of the LiDAR. Normalize the distance value of each sector according to the maximum effective ranging range of the LiDAR to reduce the impact of different scale data on the stability of network training. Then, map the normalized distance value of each sector to a unified feature dimension to generate a LiDAR token, so that each LiDAR token corresponds to local spatial distance information within a fixed angle range. Step S55: Normalize and embed the robot state information. The robot state information includes the relative distance and relative orientation angle between the unmanned vehicle and the local navigation target, the current linear velocity of the unmanned vehicle, and the current angular velocity of the unmanned vehicle. Among them, the relative distance is used to represent the distance of the unmanned vehicle from the high-level semantic waypoint, the relative orientation angle is used to represent the deflection direction of the local navigation target relative to the vehicle's coordinate system, and the linear velocity and angular velocity are used to characterize the current motion state of the unmanned vehicle. After normalizing the above state quantities, they are converted into state tokens through a fully connected mapping layer. Step S56: Concatenate the state token, visual token, and laser token to form a unified multimodal feature sequence; Step S57: Add positional encoding to the multimodal feature sequence, wherein the positional encoding is used to preserve the spatial order relationship between image blocks and laser sectors; Step S58: Input the spliced ​​and encoded multimodal feature sequence into the Transformer-based multimodal fusion coding network. This network jointly models the scene structure information in the visual image, the obstacle distance information in the LiDAR, and the target guidance information in the robot state, and outputs the fused state feature representation for the low-level reinforcement learning control policy network.

5. The autonomous path planning method for unmanned vehicles based on large model guidance according to claim 4, characterized in that, In step S51, the visual input is in the form of stacked 4 consecutive grayscale images with an image size of 256×144, which is used to retain environmental change information in a short period of time; in step S52, the 256×144 image is divided into 16×16 sub-image blocks with a size of 16×9.

6. The autonomous path planning method for unmanned vehicles based on large model guidance according to claim 5, characterized in that, In step S56, the state token is set at the beginning of the sequence so that it plays a target-guiding role in the subsequent Transformer encoding process; then the visual token and the laser token are concatenated in sequence so that the network can simultaneously focus on the visual scene structure and the local obstacle distance distribution under the target constraint.

7. The autonomous path planning method for unmanned vehicles based on large model guidance according to claim 6, characterized in that, In step S7, the low-level reinforcement learning control policy network constructed based on the SAC algorithm includes the following steps: Step S71: Construct the state space of the low-level reinforcement learning control module. The state space consists of current multimodal observation information, including continuous multi-frame visual image information, lidar distance information, relative distance and relative orientation angle between the unmanned vehicle and the local navigation target, and the current linear velocity and angular velocity of the unmanned vehicle. The above information is used to extract features and jointly model through a multimodal fusion coding network to obtain a fusion state feature representation for input to the SAC control policy network. Step S72: Construct the continuous motion space of the unmanned vehicle, which includes linear velocity control quantity and angular velocity control quantity, wherein the linear velocity control quantity is used to control the forward speed of the unmanned vehicle, and the angular velocity control quantity is used to control the turning motion of the unmanned vehicle; Step S73: Generate a continuous action distribution through the Actor policy network, input the fused state features into the Actor policy network, and output the mean and standard deviation parameters of the action distribution from the Actor network. Then, use the reparameterized sampling method to sample the control action from the action distribution. Subsequently, use the hyperbolic tangent function or the action scaling function to constrain the range of the sampled action to obtain the linear velocity command and angular velocity command of the unmanned vehicle. Step S74: Send the linear velocity command and angular velocity command to the unmanned vehicle motion control interface, so that the unmanned vehicle can perform local motion according to the continuous action; during the execution, the unmanned vehicle moves according to the local navigation target provided by the high-level semantic waypoint, and at the same time combines the information of nearby obstacles sensed by the lidar to achieve real-time obstacle avoidance; Step S75: Evaluate the value of the current state-action pair through the Critic value network. The Critic value network includes two independent Q networks, which respectively estimate the value of the current fused state features and control actions, and take the smaller of the two Q values ​​as the basis for the target value estimation. Step S76: Construct a reward function for training the low-level control strategy. The reward function is determined based on factors such as the distance change between the autonomous vehicle and the local navigation target, whether the vehicle reaches the local navigation target, whether a collision occurs, the obstacle safety distance, the magnitude of linear velocity, the magnitude of angular velocity change, and the smoothness of motion. Specifically, a positive reward is given when the autonomous vehicle approaches the local navigation target; a larger target reward is given when the autonomous vehicle reaches the local navigation target; a penalty is given when the autonomous vehicle collides with an obstacle or gets too close to an obstacle; and a corresponding penalty is given when the motion changes too much or there is a prolonged pause. This ensures that the autonomous vehicle simultaneously considers target tracking efficiency, obstacle avoidance safety, and motion smoothness during the learning process. Step S77: Store the state transition samples generated during the interaction between the autonomous vehicle and the environment into the experience replay pool. The state transition samples include the current state, current action, immediate reward, next state, and termination flag. During training, randomly sample a small batch of samples from the experience replay pool to update the Actor policy network and the Critic value network. Step S78: The maximum entropy strategy is used to optimize the target update Actor policy network. In the policy optimization process, not only is the cumulative reward maximized, but also the policy entropy term is introduced to keep the policy appropriately random during the training phase, thereby enhancing the autonomous vehicle's ability to explore in complex obstacle environments and reducing the probability of getting stuck in local optimal paths or local oscillations. Step S79: The target network parameters of the Critic are updated using the target value network soft update mechanism. By smoothly updating the target network parameters according to the preset soft update coefficient, the calculation of the target Q value is more stable, the numerical oscillation during the training process is reduced, and the convergence stability of the SAC algorithm in the continuous action control task is improved. Step S710: After training is completed, the Actor policy network is deployed as a low-level control policy. During the actual navigation or testing phase, the Actor network outputs deterministic or near-deterministic linear velocity and angular velocity control quantities based on the current fusion state characteristics, enabling the unmanned vehicle to complete local waypoint tracking, real-time obstacle avoidance, and continuous stable motion control under the guidance of high-level semantic waypoints.

8. The autonomous path planning method for unmanned vehicles based on large model guidance according to claim 7, characterized in that, In step S72, the linear velocity control amount is constrained to Within the range, the angular velocity control quantity is constrained to Within the specified range, to ensure that the output actions comply with the kinematic constraints of the unmanned vehicle chassis and the requirements for safe driving.

9. An autonomous path planning system for unmanned vehicles based on a large model, characterized in that, The system employs the method as described in any one of claims 1 to 8, and the system comprises: The high-level semantic waypoint generation module is used to acquire the current environmental image, historical observation information and target condition information of the unmanned vehicle, and input them into the vision-language-action model for environmental semantic understanding and local trajectory prediction. It outputs a sequence of local trajectory points for the next few steps, and selects the local trajectory points that are farther away as the tracking targets of the low-level control module. The low-level reinforcement learning control module is used to acquire visual image information, lidar distance information and robot state information of the unmanned vehicle and construct multimodal state input. It performs joint modeling of visual features, lidar features and robot state features through a multimodal fusion coding network, and generates continuous control actions of the unmanned vehicle through a reinforcement learning control policy network based on the SAC algorithm, so as to realize local trajectory tracking, real-time obstacle avoidance and stable motion control. The data interaction module is used to realize data interaction between the high-level semantic waypoint generation module and the low-level reinforcement learning control module through the ROS communication mechanism.

10. The autonomous path planning system for unmanned vehicles based on large model guidance according to claim 9, characterized in that, The low-level reinforcement learning control module includes: The multimodal feature construction unit is used to preprocess, normalize, and embed features of visual image information, LiDAR distance information, and robot state information, and to uniformly map different modal information to the same feature dimension to construct a multimodal feature sequence. The Transformer multimodal fusion coding unit is used to jointly model visual features, laser features, and robot state features to extract environmental spatial structure information and target guidance features. SAC reinforcement learning control unit is used to input fused state features into the Actor policy network to generate continuous control actions for the unmanned vehicle, and to evaluate the value of state-action pairs through the Critic value network. It combines experience playback mechanism, double Q network structure, maximum entropy policy optimization and target network soft update mechanism to train and update control policy. The multimodal feature construction unit includes: The visual feature extraction subunit is used to perform grayscale conversion, size scaling, cropping and pixel normalization on continuous multi-frame environmental images captured by the front camera of the unmanned vehicle, and divide the input image into multiple non-overlapping image blocks according to a preset size, and convert them into a visual token sequence through linear mapping. The laser feature extraction subunit is used to select a preset angle range in front of the unmanned vehicle as the effective perception range and divide it evenly into multiple angle sectors. The minimum distance value of the laser radar ranging point in each sector is counted as the representative distance. After normalizing the distance values ​​of each sector, they are mapped to a unified feature dimension to generate a laser token. The state feature extraction subunit is used to normalize the relative distance, relative orientation angle, current linear velocity, and current angular velocity between the unmanned vehicle and the local navigation target, and then convert them into state tokens through a fully connected mapping layer. The feature splicing subunit is used to splice the state token, visual token, and laser token into a unified multimodal feature sequence, and to add position encoding to the multimodal feature sequence.

Images