Vehicle navigation method and electronic device

Abstract

The application discloses a vehicle navigation method and electronic equipment. The method is applied to the fields of vehicle navigation and automatic driving, and comprises the following steps: in response to a navigation request for a target operation area, acquiring map information of the target operation area, and an initial pose and a target pose corresponding to the navigation request; based on the map information, a passable area currently perceived by a vehicle, the initial pose and the target pose, generating a first navigation track by using a reinforcement learning module; based on the map information, an end point of the first navigation track and the target pose, generating a second navigation track by using a geometric curve planning module; and controlling the vehicle to travel based on the first navigation track and the second navigation track. The application solves the technical problem of poor availability of vehicle navigation in the related art.

Description

Technical Field

[0001] This invention relates to the fields of vehicle navigation and autonomous driving, and more specifically, to a vehicle navigation method and electronic device. Background Technology

[0002] In large-scale, long-distance navigation planning tasks for unmanned mining trucks, the mining environment is characterized by large open areas and complex path constraints.

[0003] Currently, rule-based navigation planning is commonly used. However, when the map area expands, obstacles become denser, or there is a need for long-distance, multi-directional navigation, rule-based navigation planning is prone to problems such as unreasonable navigation, such as sticking too close to obstacles or engaging in invalid exploration, resulting in poor navigation usability.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a vehicle navigation method and electronic device to at least solve the technical problem of poor usability of vehicle navigation in related technologies.

[0006] According to one aspect of the present invention, a vehicle navigation method is provided, comprising: responding to a navigation request for a target work area; acquiring map information of the target work area, and an initial pose and a target pose corresponding to the navigation request, wherein the target work area is an unstructured open area, and the map information is global map information of the unstructured open area; generating a first navigation trajectory using a reinforcement learning module based on the map information, a currently perceived traversable area of ​​the vehicle, the initial pose, and the target pose, wherein the starting position of the first navigation trajectory is the initial pose, and the distance between the ending position of the first navigation trajectory and the target pose is less than a preset distance; generating a second navigation trajectory using a geometric curve planning module based on the map information, the ending position of the first navigation trajectory, and the target pose; and controlling the vehicle to drive based on the first navigation trajectory and the second navigation trajectory.

[0007] According to another aspect of the present invention, an electronic device is also provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention during runtime.

[0008] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0009] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0010] According to another aspect of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0011] According to another aspect of the present invention, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of the present invention.

[0012] In this embodiment of the invention, in response to a navigation request for a target work area, map information of the target work area, as well as the initial pose and target pose corresponding to the navigation request, are obtained. Based on the map information, the currently perceived traversable area of ​​the vehicle, the initial pose, and the target pose, a first navigation trajectory is generated using a reinforcement learning module. Based on the map information, the endpoint of the first navigation trajectory, and the target pose, a second navigation trajectory is generated using a geometric curve planning module. Based on the first and second navigation trajectories, the vehicle's movement is controlled. This application employs a hierarchical navigation mechanism of coarse planning using a reinforcement learning module and fine planning using a geometric curve planning module. The reinforcement learning module rapidly generates a first navigation trajectory near the initial pose and target pose in an unstructured open area, and then the geometric curve planning module generates a precise second navigation trajectory near the target pose, achieving the purpose of reasonable navigation and thus improving the technical effect of navigation usability. This solves the technical problem of poor usability of vehicle navigation in related technologies. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0014] Figure 1 This is a flowchart of a vehicle navigation method according to an embodiment of the present invention;

[0015] Figure 2 This is a schematic diagram of an optional reinforcement learning training and interaction process according to an embodiment of the present invention;

[0016] Figure 3 This is a schematic diagram of an optional single-step iterative reinforcement learning process according to an embodiment of the present invention;

[0017] Figure 4This is a simplified diagram of an optional global grid map according to an embodiment of the present invention;

[0018] Figure 5 This is a schematic diagram of an optional map processing procedure according to an embodiment of the present invention;

[0019] Figure 6 This is a schematic diagram of an optional passage area according to an embodiment of the present invention;

[0020] Figure 7 This is a schematic diagram of an optional feature fusion processing procedure according to an embodiment of the present invention;

[0021] Figure 8 This is a schematic diagram of an optional network processing procedure according to an embodiment of the present invention;

[0022] Figure 9 This is a schematic diagram of an optional model loss processing procedure according to an embodiment of the present invention;

[0023] Figure 10 This is a schematic diagram of an optional navigation trajectory according to an embodiment of the present invention;

[0024] Figure 11 This is a schematic diagram of an optional navigation trajectory in a narrow scene according to an embodiment of the present invention;

[0025] Figure 12 This is a schematic diagram of an optional irregular scene trajectory according to an embodiment of the present invention;

[0026] Figure 13 This is a schematic diagram of the trajectory of another optional irregular scene according to an embodiment of the present invention;

[0027] Figure 14 This is a schematic diagram of a vehicle navigation device according to an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] According to an embodiment of the present invention, a method embodiment of a vehicle navigation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 1 This is a flowchart of a vehicle navigation method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0032] Step S102: Respond to the navigation request for the target work area, obtain the map information of the target work area, as well as the initial pose and target pose corresponding to the navigation request.

[0033] The target operation area is an unstructured open area, and the map information is the global map information of the unstructured open area.

[0034] The aforementioned navigation requests are navigation task trigger signals initiated by humans, vehicle intelligent driving systems, or fleet dispatching systems. Navigation requests can be issued by upper-level dispatching algorithms or drivers through human-machine interfaces or voice commands. Navigation requests reflect the vehicle's driving intentions, including but not limited to navigation vehicles, work tasks, starting points, and target points. Through navigation requests, the initial pose (such as the coordinates and angle of the navigation start point) and target pose (such as the coordinates and angle of the destination point) of the vehicle navigation can be determined. Navigation requests serve as the trigger source for subsequent perception, planning, and control processes, such as activating trajectory planning.

[0035] The aforementioned target work area refers to the range within which vehicles can conduct operational activities. In a mining scenario, the target work area is an unstructured open area, such as a spoil heap loading area, parking lot, maintenance area, main road, and various work points. Unstructured means that the ground texture and terrain undulations, as well as the distribution of obstacles, in this target work area exhibit high uncertainty and dynamic change characteristics, posing a high driving risk. Within the target work area, there may be different terrain features, as well as static infrastructure such as walls and fences, and dynamic objects such as other vehicles and personnel.

[0036] The map information mentioned above refers to a description of the target work area and its surrounding environment. Map information includes, but is not limited to: map boundary information, static obstacle information, and map topology information. This map information provides the reinforcement learning module with global map information, enabling proactive path planning in conjunction with vehicle perception data.

[0037] In one optional embodiment, upon receiving a navigation request, map loading is triggered, and map information associated with the target work area is retrieved from local storage. The map information includes not only a two-dimensional planar topology but also integrated three-dimensional elevation information and semantic tags, such as static obstacle information like the boundaries of the spoil heap and the geometric contours of the loading area. Simultaneously, the vehicle utilizes a Global Navigation Satellite System combined with an inertial navigation system and wheel speedometer data for tightly coupled fusion positioning, calculating the vehicle's initial pose in real time. Furthermore, by combining the map information with the navigation request, the target pose of the vehicle is calculated.

[0038] In another optional embodiment, upon receiving a navigation request for a target work area, pre-stored map information of the target work area is obtained based on the navigation request identifier. Furthermore, the vehicle's onboard camera begins high-frequency acquisition of image streams of the surrounding environment, and a semantic segmentation network identifies road boundaries, obstacles, and passable areas in real time. Through feature point matching, the vehicle's initial pose is estimated in real time. Additionally, by identifying visual markers of the target work area, such as QR codes, landmarks, or specific color-coded areas, and combining this with visual odometry, the target pose corresponding to the target point is calculated and then converted to a global coordinate system.

[0039] In another alternative embodiment, distributed positioning data is acquired based on vehicle-to-infrastructure (V2I) communication and cloud-based map services. Navigation requests are sent to a cloud edge server via a communication network. The cloud server, using a digital twin model of the mining area, generates a map in real time containing static obstacles, road users, and traffic signs, and distributes it to the vehicle. Then, the cloud server uses stored historical planning data and the navigation request to calculate the initial and target poses. This V2I-cloud collaboration reduces the computational burden on the vehicle and enables real-time map updates to adapt to changes in the mining environment, ensuring the timeliness of navigation request responses and the accuracy of map information.

[0040] Step S104: Based on map information, the passable area currently perceived by the vehicle, the initial pose and the target pose, the first navigation trajectory is generated using a reinforcement learning module.

[0041] The starting position of the first navigation trajectory is the initial pose, and the distance between the ending position of the first navigation trajectory and the target pose is less than a preset distance.

[0042] The currently perceived passable area refers to the space that the vehicle can safely traverse in its surrounding environment, as perceived by onboard sensors such as LiDAR and cameras. The passable area provides obstacle avoidance information and motion constraints. It can be determined based on the distribution of obstacles within a certain radius around the vehicle's center. The passable area compensates for the limitations of global map information in dynamic scenarios, enabling the reinforcement learning module to handle frequently moving vehicles, pedestrians, or temporarily stored materials in the mining area during trajectory generation, thereby improving vehicle safety.

[0043] The aforementioned first navigation trajectory refers to the path generated by the reinforcement learning module, which guides the vehicle from its initial pose to a point approaching the target pose. This first navigation trajectory can quickly determine the approximate direction of travel and key waypoints, avoiding getting stuck in local obstacles. It can be represented as a series of discrete pose points or a continuous sequence of steering angle and arc length commands, thus decomposing the complex long-distance planning problem into multiple short-cycle decision steps, reducing the computational burden of each decision.

[0044] The reinforcement learning module described above is a component for coarse-grained waypoint exploration in unstructured open areas. It autonomously learns navigation strategies through interaction with the environment. The reinforcement learning module can predict the vehicle's trajectory from its initial pose to its target pose using map information and traversable areas.

[0045] In one optional embodiment, the reinforcement learning module generates a first navigation trajectory based on a long-tail scenario reinforcement learning scheme using a curriculum learning mechanism. A dynamically adaptive training and inference strategy is designed for long-tail difficult examples (such as narrow passages and dense obstacle areas) present in the mining environment.

[0046] First, the difficulty level of the navigation is assessed. The safety cost distribution is estimated using historical success and failure rates, and the scene sampling weights are dynamically adjusted. For scenarios that are currently difficult to solve, exploration noise is increased or a specific set of difficult examples is introduced for local policy adjustments. Specifically, the reinforcement learning module maintains an experience replay buffer, which stores interaction data for scenarios of different difficulty levels. During inference, if the current state is detected to be in a high-cost region, the algorithm temporarily adjusts the action distribution, thus favoring conservative exploration. By introducing a resampling mechanism based on failure history, experience samples that lead to high costs or failures are replayed preferentially during the training phase, thereby accelerating the convergence of the reinforcement learning module's learning of difficult examples. When generating the first navigation trajectory, the reinforcement learning module can dynamically adjust the learning rate or exploration strategy based on the real-time perceived risk level, ensuring that a feasible and smooth path can still be generated under complex constraints.

[0047] In another alternative embodiment, the reinforcement learning module generates a first navigation trajectory based on a coarse-grained waypoint exploration scheme using hierarchical reinforcement learning. Employing a hierarchical architecture, long-distance path planning is decomposed into two stages: high-level waypoint selection and low-level action execution. The first navigation trajectory generation stage primarily focuses on the rapid convergence of the high-level policy.

[0048] The high-level policy of the reinforcement learning module makes decisions on key waypoints in an abstract state space, discretizing the continuous control space into several directions (such as left turn, straight ahead, right turn, acceleration, and deceleration). At each step, the expected reward of each discrete action in the current state is first evaluated, and the action with the highest reward is selected and executed for a certain number of steps, forming a coarse-grained trajectory. Under the instructions of the high-level policy, the low-level policy combines the vehicle's currently perceived traversable area and uses fast lookup tables or simple geometric rules to generate specific steering angle and speed commands, ensuring the vehicle avoids collisions and moves smoothly. This hierarchical design can focus on long-term temporal dependencies and avoids getting lost in local details.

[0049] When generating the initial navigation trajectory, the high-level policy quickly determines the approximate path, while the low-level policy ensures execution feasibility. By decoupling long-range planning and short-range control, the difficulty of exploration in high-dimensional continuous space by the reinforcement learning module is reduced, and the planning efficiency and success rate in unstructured open regions are improved.

[0050] Step S106: Based on map information, the endpoint of the first navigation trajectory, and the target pose, the second navigation trajectory is generated using the geometric curve planning module.

[0051] The aforementioned geometric curve planning module is a component used to plan the vehicle's journey from the end of the first navigation trajectory to the target pose and generate a precise end-point trajectory. The geometric curve planning module performs high-precision positioning and attitude adjustment based on the coarse path generated by the reinforcement learning module. This module ensures that the vehicle can smoothly and accurately reach the target pose, addressing the issue of insufficient end-point control precision in the reinforcement learning module.

[0052] The aforementioned second navigation trajectory refers to the precise path generated by the geometry curve planning module from the endpoint of the first navigation trajectory to the target pose, thereby helping the vehicle to accurately reach the target pose. The second navigation trajectory may be represented as a continuous and smooth geometric curve to complete the final fine-tuning action, ensuring that the vehicle's attitude conforms and guaranteeing the final completion of the navigation request, enabling the vehicle to achieve high-precision automated operations in complex mining environments.

[0053] In one alternative embodiment, the geometric curve planning module generates a second navigation trajectory based on a mixed-integer linear programming end-path search scheme. The geometric curve planning module transforms the generation problem of the second navigation trajectory into a discretized optimization problem, and finds the global optimum through a solver.

[0054] The geometry curve planning module first discretizes the map information into a grid or graph structure. Then, it defines the state variables as the vehicle's position, heading, and curvature, and the action variables as the discretized steering angle and velocity. The optimization problem aims to minimize the total cost from the end of the first navigation trajectory to the target pose, including distance cost, steering cost, and collision cost. Constraints include the vehicle kinematics model, map boundary constraints, and obstacle avoidance constraints.

[0055] A heuristic pruning technique is employed to search the subspace that may contain the optimal solution. When generating the second navigation trajectory, the solver returns a set of discrete action sequences, which are interpolated to generate a continuous trajectory as the second navigation trajectory, thus enabling the search of a theoretically globally optimal path under complex constraints.

[0056] In another optional embodiment, a second navigation trajectory is generated based on polynomial interpolation and kinematic constraints. The geometric curve planning module uses the endpoint, target pose, and intermediate transition points of the first navigation trajectory as boundary conditions. To ensure trajectory smoothness, fifth- or seventh-order polynomial interpolation is used to ensure continuity of position, velocity, and acceleration at the trajectory's start and end points. A solver is introduced, using vehicle kinematic model constraints (such as maximum steering angle and minimum turning radius) as inequality constraints for the optimization problem. The objective function of the optimization problem is to minimize the integral curvature or rate of change of acceleration of the trajectory to ensure path smoothness. During the generation of the second navigation trajectory, the solution is iteratively solved until polynomial coefficients satisfying the constraints are found. Furthermore, the generated trajectory is checked for conflicts with local obstacles in the map information. If conflicts occur, the boundary conditions are adjusted or obstacle avoidance costs are increased, and the trajectory is readjusted to reduce vehicle jitter at the end of the process, thereby improving operational efficiency and safety.

[0057] In another alternative embodiment, the geometry curve planning module generates a second navigation trajectory based on spline curve fitting and continuous curvature adjustment. B-splines are used to fit the path from the endpoint of the first navigation trajectory to the target pose, aiming to generate a trajectory with high-order continuity and ease of vehicle tracking.

[0058] The geometric curve planning module inserts several control points between the endpoint of the first navigation trajectory and the target pose. The positions of these control points are determined through an optimization algorithm to minimize trajectory length and curvature variations. Subsequently, a smooth geometric path is generated using a spline curve algorithm. To ensure stable vehicle tracking, the control point weights are adjusted to achieve a uniform curvature distribution on the trajectory and avoid sharp turns. During the generation of the second navigation trajectory, the vehicle's kinematic constraints are transformed into constraints on the spline coefficients, ensuring that the generated second navigation trajectory is physically executable. Therefore, the second navigation trajectory possesses smoothness and trackability, reducing the vehicle's lateral errors and improving the overall stability and accuracy of navigation.

[0059] Step S108: Control the vehicle's movement based on the first navigation trajectory and the second navigation trajectory.

[0060] In one optional embodiment, the macroscopically coarse-grained first navigation trajectory generated by the reinforcement learning module and the microscopically precise positioning second navigation trajectory generated by the geometry curve planning module are used as reference benchmarks for vehicle driving control. This leverages the reinforcement learning module's ability to handle long-distance exploration in complex open environments, combined with the deterministic advantages of the geometry planning module in high-precision docking and obstacle avoidance at the end point of travel, to control vehicle driving.

[0061] The first navigation trajectory provides the vehicle with a path from its initial pose to its target pose, preventing it from blindly driving in unstructured areas, reducing the probability of planning failure, and improving stability over long distances. Then, the second navigation trajectory takes over control as the vehicle approaches the target pose. Through precise geometric curve fitting, it corrects potential jitter, deviations, or unevenness issues in the reinforcement learning module's end-point control, ensuring the vehicle reaches the target pose smoothly and accurately. This improves the overall navigation success rate of the unmanned driving system in the mining area and guarantees the safety, comfort, and efficiency of the vehicle's operation.

[0062] In this embodiment of the invention, in response to a navigation request for a target work area, map information of the target work area, as well as the initial pose and target pose corresponding to the navigation request, are obtained. Based on the map information, the currently perceived traversable area of ​​the vehicle, the initial pose, and the target pose, a first navigation trajectory is generated using a reinforcement learning module. Based on the map information, the endpoint of the first navigation trajectory, and the target pose, a second navigation trajectory is generated using a geometric curve planning module. Based on the first and second navigation trajectories, the vehicle's movement is controlled. This application employs a hierarchical navigation mechanism of coarse planning using a reinforcement learning module and fine planning using a geometric curve planning module. The reinforcement learning module rapidly generates a first navigation trajectory near the initial pose and target pose in an unstructured open area, and then the geometric curve planning module generates a precise second navigation trajectory near the target pose, achieving the purpose of reasonable navigation and thus improving the technical effect of navigation usability. This solves the technical problem of poor usability of vehicle navigation in related technologies.

[0063] Optionally, based on map information, the currently perceived traversable area, the initial pose, and the target pose, a first navigation trajectory is generated using a reinforcement learning module. This includes: using the initial pose as the current pose, and using the multimodal feature extractor in the reinforcement learning module to extract multimodal features from the map information, traversable area, current pose, and target pose to obtain the vehicle's current state information; inputting the current state information into the execution network of the reinforcement learning module, and using the execution network to generate the vehicle's current execution action, wherein the current execution action includes: front wheel steering angle and single-step arc length; and substituting the current execution action, the vehicle's wheelbase, and the preset steering angle into... The process involves solving the vehicle's kinematic model to obtain a new vehicle pose; using this new pose as the current pose, and repeating the process of extracting multimodal features from the map information, passable area, current pose, and target pose using a multimodal feature extractor to obtain the vehicle's current state information; inputting this current state information into the execution network to generate the vehicle's current action; and then substituting the current action, the vehicle's wheelbase, and a preset steering angle into the vehicle's kinematic model to solve for a new vehicle pose, continuing until the new pose meets the iteration stopping condition; and finally, determining the first navigation trajectory based on the updated poses from multiple iterations.

[0064] The aforementioned multimodal feature extractor can fuse map information, traversable areas, current pose, and target pose into a unified feature representation, obtaining current state information that reflects the current state, thereby reducing the gap between different modal data and providing a rich and comprehensive state-aware foundation for the execution network, thus improving the accuracy and robustness of decision-making.

[0065] The aforementioned current action is the predicted action of the vehicle by the reinforcement learning module's execution network, based on the current state information. This current action drives the vehicle to move in order to approach the target pose.

[0066] The aforementioned iteration stopping condition refers to the logical judgment condition used to determine whether the current iteration process should terminate during the generation of the first navigation trajectory by the reinforcement learning module. The iteration stopping condition provides a clear criterion for convergence or failure, ensuring that the reinforcement learning module can output valid results within a finite number of steps. The iteration stopping condition involves multiple dimensions, including whether the vehicle exceeds the preset map boundary, whether a geometric collision occurs, whether it enters a specific judgment area near the target endpoint, and whether the maximum allowed number of iterations has been reached. For example, Euclidean distance or Manhattan distance can be used to determine whether the vehicle has entered a specific judgment area near the target endpoint.

[0067] Reasonable iterative stopping conditions can effectively prevent vehicles from getting stuck in local deadlocks or engaging in meaningless stationary exploration in complex mining environments, thus meeting the real-time requirements of autonomous vehicles. Furthermore, iterative stopping conditions can reduce the uncertainty risk of reinforcement learning models in end-point control, ensuring that the generated initial navigation trajectory is executable. Iterative stopping conditions can be determined through simulation training and field testing, thereby balancing planning efficiency and path quality, ultimately achieving the goal of efficient, safe, and stable autonomous driving operations in mining areas.

[0068] In an optional embodiment, in order to reduce the differences in format, scale and semantics of data from different sources, and to provide the subsequent execution network with an information-rich and semantically consistent state input, ensuring that the execution network can simultaneously understand the macroscopic global topology and the microscopic local obstacle details, a multimodal feature extractor can be used to transform heterogeneous map information, passable areas, current pose and target pose into a unified high-dimensional feature vector that the execution network can process.

[0069] Then, the current state information is input into the execution network of the reinforcement learning module to achieve end-to-end mapping from perception to decision-making, thereby generating the current execution action containing the front wheel steering angle and single-step arc length, which improves the vehicle's decision-making efficiency and intelligence level in complex scenarios.

[0070] Therefore, based on the kinematic model, the vehicle's new pose is predicted using the generated current action for the current pose. By simulating the vehicle's actual motion in the physical world, updated state estimates are provided for the next step of perception and decision-making, forming a closed-loop feedback between perception, decision-making, and execution. The kinematic model accurately calculates the vehicle's displacement and rotation under the current action, ensuring that the generated trajectory conforms to the vehicle's mechanical constraints, thus guaranteeing that the planned path is physically executable.

[0071] Furthermore, the new pose is used as the current pose, and the aforementioned steps of feature extraction, action generation, and kinematic model solution are repeated until the iteration stopping condition is met. Thus, through iterative rolling temporal control or online planning mechanisms, the vehicle is progressively moved from the initial pose to the target pose, and the environmental state is reassessed at each step to adjust subsequent strategies. By gradually approximating the target pose, the path is continuously adjusted in a dynamically changing environment, ensuring that the vehicle can overcome local obstacles and cope with environmental uncertainties.

[0072] Furthermore, by continuously accumulating pose points during the iteration process, a coherent path sequence from the initial pose to the near-target pose is formed. This allows for robust navigation in dynamic environments using the first navigation trajectory, improving the success rate and adaptability of path planning.

[0073] Optionally, multimodal feature extraction is performed on map information, passable areas, current pose, and target pose to obtain the vehicle's current state information, including: constructing a global grid map based on map information, current pose, and target pose, and extracting features from the global grid map to obtain global features; extracting features from the passable area to obtain local features; constructing target geometric relationships based on the current pose and target pose, and extracting features from the target geometric relationships to obtain relational features, wherein the target geometric relationships include the relative distance, relative angle, and heading angle deviation between the current pose and the target pose; determining the first priority corresponding to the global features, the second priority corresponding to the local features, and the third priority corresponding to the relational features based on the navigation scenario corresponding to the navigation request; and using a self-attention mechanism, combining the first priority, the second priority, and the third priority to perform feature fusion on the global features, local features, and relational features to obtain the current state information.

[0074] The aforementioned global raster map is a form of environmental representation, which can be a digital map of discrete grid cells.

[0075] The aforementioned target geometric relationships refer to the distance, relative angle, and heading difference between the vehicle's current pose and the target pose. Target geometric relationships can transform abstract navigation targets into concrete geometric relationships, thereby providing clear directional guidance and distance feedback for subsequent trajectory planning. Constructing target geometric relationships can reduce scale differences and directional ambiguity caused by global map coordinates, enabling reinforcement learning modules to generalize learning across a unified relative geometric feature space under different poses, thus reducing the reinforcement learning module's dependence on coordinates.

[0076] The aforementioned navigation scenarios refer to the environments in which vehicles operate during specific transportation or operational tasks, characterized by specific environmental features, terrain constraints, obstacle distribution, and dynamic interaction requirements. Navigation scenarios can be categorized into open, narrow, and near-target pose scenarios. For example, in a narrow loading area scenario, obstacles may be dense, requiring high obstacle avoidance accuracy. In an open spoil heap scenario, long-distance exploration efficiency is emphasized. Determining the navigation scenario allows for targeted adjustments to perception, planning, and control strategies.

[0077] In one alternative embodiment, the complex unstructured environment of the mining area can be discretized into a global raster map to extract high-level semantic and topological information, thereby reducing data noise and irregularities. Global features retain the core geometric and semantic features of the environment, reflecting the distribution of distant obstacles and road boundaries. Furthermore, traversable areas can reflect environmental details within the short-range surroundings of the vehicle, particularly dynamic obstacles, temporary barriers, and road surface textures. Therefore, feature extraction from traversable areas to obtain local features can compensate for the limitations of the global raster map in terms of real-time performance and detail resolution, ensuring that subsequent trajectory planning can respond to changes in the vehicle's surrounding environment, such as moving vehicles, pedestrians, or newly appearing obstacles.

[0078] Then, in order to quantify the spatial set relationships such as the relative distance, relative angle, and heading angle deviation between the current vehicle pose and the target pose, the target geometric relationship can be constructed based on the current pose and the target pose and the relationship features can be extracted, thereby providing clear target guidance and guiding the strategy to converge toward the target pose.

[0079] Furthermore, since the contribution of each modal feature to decision-making varies across different scenarios, the priorities of global, local, and relational features are determined based on the specific navigation scenario corresponding to the navigation request. This allows for dynamic adjustment of the weights or importance of different modal features during the fusion process. For example, global features are more important in open areas, while local and relational features are more crucial in narrow passages. Through scene recognition or attention mechanisms, feature weights are adaptively allocated, ensuring that the fused current state information better matches the current needs and preventing irrelevant features from interfering with decision-making.

[0080] Finally, a self-attention mechanism is employed to deeply fuse priority-weighted global, local, and relational features, effectively capturing the complex interactions between features. By explicitly modeling the dependencies between different features through self-attention, more discriminative joint features are extracted, noise is suppressed, and the expressive power of key features is enhanced, thereby outputting a high-quality and robust current state information.

[0081] Optionally, feature extraction is performed on the global raster map to obtain global features, including: multi-scale feature extraction of the global raster map to obtain feature maps of multiple scales, wherein the feature maps of different scales correspond to different resolutions; feature fusion of feature maps of multiple scales to obtain fused features; two-dimensional unfolding of the fused features to obtain two-dimensional features; feature processing of multiple preset features using a self-attention mechanism to obtain self-attention features; and feature fusion of the fused features and self-attention features using a cross-attention mechanism to obtain global features.

[0082] In one optional embodiment, multi-scale feature extraction of the global grid map can be performed using deep learning architectures such as convolutional neural networks or visual transducers. Local details and macroscopic topological structures within the global grid map are captured at different network levels, generating feature maps with varying resolutions and receptive fields to balance long-distance environmental context awareness with precise identification of nearby obstacles. Multi-scale feature extraction effectively reflects road boundaries and large obstacles while also including nearby small obstacles and texture details. By extracting features at different scales, low-resolution feature maps contain rich global semantic information and long-range dependencies, while high-resolution feature maps retain precise spatial locations and geometric details. This hierarchical extraction mechanism provides a rich and complementary information source for subsequent feature fusion, improving the completeness and robustness of global feature representation and avoiding navigation errors caused by missing or ambiguous information.

[0083] Then, feature pyramid networks or skip connections can be used to align, stitch together or weight sum feature maps of different resolutions and semantic levels in the spatial dimension to generate fused features. This can reduce the semantic gap and spatial misalignment between features of different scales, and organically combine macro-level global context information with micro-level local detail information to form a fused feature that includes both overall layout and local precision.

[0084] Furthermore, the fused features can be expanded in two dimensions to obtain two-dimensional features. This means that the fused multidimensional feature tensor (which usually includes height, width and channel dimensions) can be flattened or reshaped into a two-dimensional matrix form, which can then be adapted to subsequent self-attention mechanisms.

[0085] The system utilizes a self-attention mechanism to process multiple preset features to obtain self-attention features. By introducing a set of learnable query vectors, it can dynamically extract a subset of features related to preset semantics or task objectives, thereby generating semantically filtered and weighted self-attention features. This allows the self-attention features to more accurately reflect key elements in the environment, such as road centerlines, obstacle boundaries, or potential travel paths, thus improving the semantic purity of feature representation.

[0086] Therefore, by using a cross-attention mechanism to fuse fused features and self-attention features, the semantic information represented by the self-attention features can be mapped to the spatial location of the fused features, thereby achieving deep interaction and complementary fusion of the two types of features and ultimately generating a unified global feature. Combining the semantic focusing ability of self-attention features and the spatial richness of fused features, the bidirectional interaction of cross-attention ensures that the final global feature contains both precise spatial geometric information and semantic guidance information, thus achieving deep integration at the feature level. Through the cross-attention mechanism, the global feature can more accurately reflect the key semantic structure of the environment, such as the connectivity of passable areas or the semantic category of obstacles, thereby improving the semantic consistency of feature expression.

[0087] Optionally, the above method further includes: constructing a target detection range centered on the vehicle and based on a preset distance; performing obstacle detection on the target detection range according to multiple preset directions to obtain first boundary points corresponding to multiple preset directions; interpolating the area between two adjacent preset directions to obtain multiple second boundary points; and constructing a passable area based on the first boundary points and multiple second boundary points corresponding to multiple preset directions.

[0088] In one alternative embodiment, to construct a passable area, firstly, a target detection range is constructed centered on the vehicle and based on a preset distance. This preset distance can be determined by the sensing capabilities of the onboard sensors. A circular or polygonal area with a certain radius is delineated around the vehicle's current position as the local field of view boundary for subsequent obstacle detection and environment modeling. This avoids processing irrelevant information from the entire map or at infinity, thereby reducing computational complexity and improving real-time performance, while ensuring coverage of the safe braking distance and reaction time required by the vehicle at its current speed.

[0089] Then, obstacle detection is performed on the target detection range according to multiple preset directions to obtain the first boundary points corresponding to the preset directions. For example, the target detection range can be divided into several discrete sectors or ray directions, and a search is performed along the ray in each direction until the first obstacle is detected or the detection boundary is reached. The intersection point is recorded as the first boundary point in that direction. The first boundary points reflect the safe passage distance of the vehicle in each direction, providing the original set of data points for the subsequent construction of a smooth passable area.

[0090] Furthermore, linear or spline interpolation of the region between two adjacent preset directions can generate additional second boundary points between two adjacent first boundary points to fill the gaps between discrete sampling points, thereby refining the description of obstacle contours or traversable boundaries. By increasing the density of boundary points, the constructed traversable region boundaries become smoother and more accurate. Through interpolation operations, a denser set of second boundary points than the first boundary points can be generated, thus more accurately describing the edge shape of obstacles and avoiding misjudging smooth obstacle edges as sharp angles, which would affect the smoothness of path planning. This improves the geometric accuracy and continuity of the local environment model, making the constructed traversable region more closely resemble the actual physical environment, reducing planning errors caused by boundary discretization, and contributing to the generation of smoother, more natural trajectories.

[0091] Finally, the refined boundary points are aggregated using interpolation. Then, methods such as convex hull algorithms, polygon fitting, or mesh mapping are employed to determine a connected, safe region—the passable area—starting from the vehicle's current pose and bypassing obstacles. This transforms discrete boundary points into a continuous and well-defined geometric region, clearly defining the spatial range within which the vehicle can move freely. This provides a safe boundary for subsequent trajectory planning, ensuring that the generated path will not collide with obstacles.

[0092] Optionally, the above method further includes: inputting the current state information and the current execution action into a reward function to determine a predicted reward value, and constructing a first loss function value based on the predicted reward value; inputting the current state information and the current execution action into a risk cost function to determine a predicted cost value, and constructing a second loss function value based on the predicted cost value; adjusting the preset weights corresponding to the second loss function value based on the deviation between the predicted cost value and the preset predicted value to obtain a target weight; weighting and summing the negative of the first loss function value and the second loss function value based on the target weight to obtain a total loss function; and updating the parameters of the execution network based on the total loss function.

[0093] The reward function described above can be used to evaluate the current state information and the current action to be performed during reinforcement learning.

[0094] The risk cost function described above is an evaluation function used to quantify collision risk.

[0095] In one optional embodiment, the current state information and the current action can be input into the reward function to determine the predicted reward value. This guides the reinforcement learning module to implement a behavioral strategy that accumulates rewards over a long period, ensuring that the vehicle effectively approaches the target pose, maintains a smooth path, and avoids violations when performing the current action. Feedback from the reward function ensures the effectiveness of the planning results in terms of task completion.

[0096] Furthermore, by inputting the current state information and the current action into the risk cost function, the safety risks associated with the vehicle performing the current action under the current state are assessed. Based on the predicted cost value, a second loss function value can then be constructed. This allows safety to be constructed as an independent objective in the total loss function, decoupling task rewards from safety constraints. This facilitates risk level control and improves vehicle safety in complex mining environments.

[0097] Then, based on the deviation between the predicted cost and the preset predicted value, the preset weights corresponding to the second loss function value are adjusted to obtain the target weights. That is, the weight coefficients of the safety constraints in the total loss function are dynamically adjusted according to the deviation between the actual cumulative safety cost during the current action and the preset safety threshold, to avoid fixed weights being difficult to adapt to different training stages or different scenario complexities. Through the deviation feedback mechanism, adaptive adjustment of the tightness of the safety constraints is achieved, providing a robust safety constraint handling method and avoiding training instability caused by improper weight settings.

[0098] Finally, based on the target weights, the negatives of the first loss function value and the second loss function value are weighted and summed to obtain the total loss function. That is, maximizing the reward (i.e., the negative of the first loss function value) and minimizing the safety cost (i.e., the second loss function value) are combined through dynamic weights to form a unified loss function that can guide global parameter updates. This allows subsequent execution networks to simultaneously consider task performance and safety when updating parameters. Through weighted summation, the total loss function reflects overall performance, guiding the execution network to find a balance between reward and cost, ensuring that the final generated navigation is both efficient and safe. Furthermore, updating the execution network parameters based on the total loss function can gradually improve the network's performance, enabling it to output action sequences that achieve both high rewards and meet safety constraints.

[0099] Optionally, based on the deviation between the cost prediction value and the preset prediction value, the preset weight corresponding to the second loss function value is adjusted to obtain the target weight, including: when the cost prediction value is greater than the preset prediction value, a first gradient is determined based on the deviation, and the preset weight is increased based on the first gradient to obtain the target weight; when the cost prediction value is less than or equal to the preset prediction value, a second gradient is determined based on the deviation, and the preset weight is decreased based on the second gradient to obtain the target weight.

[0100] In one optional embodiment, if the predicted cost is greater than a preset predicted value, it can be considered that the accumulated safety risk cost of the current action exceeds a preset safety threshold. The weight coefficient of the safety constraint in the total loss function is dynamically increased based on the deviation between the predicted cost and the preset predicted value. For example, a first gradient is determined based on the deviation, and the preset weight is increased based on the first gradient to obtain the target weight. That is, when a potential safety risk is detected, the penalty of the safety constraint needs to be strengthened, forcing the execution network to be more inclined to generate low-risk, high-safety actions in subsequent updates, thereby quickly correcting behaviors that deviate from the safety boundary. Through the gradient ascent mechanism, the priority of safety costs in dangerous scenarios is ensured, thereby preventing safety accidents caused by accidental collisions or obstacle-hitting behaviors during the exploration process through an active safety defense mechanism.

[0101] If the predicted cost is less than or equal to the preset predicted value, it can be assumed that the accumulated safety risk cost of the current action is below or just reaches the preset safety threshold. Then, by calculating the negative or zero deviation between the actual cost and the threshold, this deviation is used as a gradient signal to dynamically reduce or maintain the weight coefficient of the safety constraint in the total loss function. This can be achieved by mapping the direction and magnitude of the gradient using a sign function or linear function, thus realizing closed-loop adaptive adjustment of the safety constraint weight. In other words, under the premise that the current strategy meets the safety constraints, the intensity of the safety penalty is appropriately reduced, allowing the execution network to focus more on improving navigation efficiency and avoiding performance degradation caused by excessive conservatism. Through the gradient descent mechanism, the preset weight automatically decreases as the safety margin increases, balancing the relationship between safety and efficiency. This achieves a dynamic balance between safety and efficiency, avoiding unnecessary deceleration, detours, or stalls caused by excessively high safety weights, ensuring that the vehicle maintains high operating efficiency in the mining environment.

[0102] Optionally, based on the target weights, a weighted sum is performed on the negative of the first loss function value and the second loss function value to obtain the total loss function, including: performing logarithmic processing on the target weights to obtain the target logarithmic value; obtaining the product of the second loss function value and the target logarithmic value to obtain the target product; and obtaining the sum of the target product and the negative of the first loss function value to obtain the total loss function.

[0103] In one optional embodiment, the target weights are logarithmically processed to obtain the target logarithmic value, thereby stabilizing the training process. This numerical stability mechanism ensures the convergence of the reinforcement learning module in long-term, high-complexity tasks, avoiding training failures caused by parameter explosion or vanishing parameters.

[0104] Then, the logarithmically transformed target logarithmic value is multiplied by the second loss function value representing security risk to construct a weighted security constraint loss term. This ensures that the security cost has an influence in the total loss function commensurate with its risk level. Through multiplication, the security cost is dynamically scaled. When the security risk is high, the target product is larger, thus amplifying the proportion of the security cost in the total loss and forcing the execution network to focus more on security. When the security risk is low, the target product is smaller, thus reducing the impact of the security cost and allowing the execution network to focus more on task rewards.

[0105] The sum of the product of the objectives and the negative of the first loss function value is then obtained to construct the final total loss function used for backpropagation. This unifies the two originally competing objectives into a single objective. Through summation, the total loss function becomes the guiding signal for updating the network parameters.

[0106] Optionally, the current state information and the current action are input into the reward function to determine the reward prediction value, including: inputting the current state information and the current action into the two reward evaluation models corresponding to the reward function respectively, and using the two reward evaluation models to generate two first prediction sets; determining the quantile mean of the two first prediction sets respectively, and obtaining two prediction means; obtaining the minimum value of the two prediction means to obtain the reward prediction value.

[0107] In one optional embodiment, the current state information and the current action are respectively input into two reward evaluation models corresponding to the reward function to generate two first prediction sets. Each reward evaluation model receives the current state information and the current action, and outputs a set of first predictions containing multiple quantiles (e.g., 16 quantiles). These two first prediction sets together constitute a distributional representation of the value estimate of the same state-action pair. By computing two independent distribution estimates in parallel, the differences between the models are utilized to capture the uncertainty of value estimation, thereby providing richer and more robust value information and avoiding misguidance caused by noise or bias in a single network.

[0108] Then, the quantile means of the two first prediction sets are determined to obtain two prediction means. This is achieved by compressing the original high-dimensional distribution vector into a single scalar value. The prediction means reflect the network's estimate of the expected reward for the current state-action pair. By calculating the means, the influence of extreme values ​​in the distribution is reduced, resulting in a more robust estimate of the expected reward. The prediction means reflect the network's average optimism or conservatism in the current state.

[0109] Furthermore, the minimum of the two predicted means is obtained to arrive at the predicted reward value. By comparing the expected reward scalar values ​​calculated by the two reward evaluation models and selecting the smaller predicted mean as the final predicted reward value, it is ensured that the execution network is based on a relatively safe and conservative value benchmark. Through this conservative selection mechanism, the variance in the policy update process is effectively reduced, making the learning process of the execution network more stable and reliable, thereby improving the stability and safety of the reinforcement learning module in long sequences and highly constrained tasks.

[0110] Optionally, the current state information and the current action are input into the risk cost function to determine the cost prediction value, including: inputting the current state information and the current action into two cost evaluation models corresponding to the risk cost function respectively, generating two second prediction values ​​using the two cost evaluation models; obtaining the maximum value among the two second prediction values ​​to obtain the cost prediction value.

[0111] In one optional embodiment, the current state information and the current action are respectively input into two cost evaluation models corresponding to the risk cost function to generate two second predicted values. The second predicted value represents a scalar value of the cumulative safety cost. The two second predicted values ​​constitute a dual independent estimate of the degree of safety risk associated with the current state-action relationship. By performing dual verification on the current state, uncertainties and potential biases in risk assessment are captured, ensuring that the estimation of safety costs in complex and variable mining environments is not misled by defects or noise in a single network. By computing the two independent cost estimates in parallel, risk quantification results from two perspectives can be obtained simultaneously, and these two results may exhibit different sensitivities under different network parameter initializations or training dynamics.

[0112] Then, the maximum of the two second predicted values ​​is obtained to get the cost prediction value. This ensures that during the safety constraint optimization process, the vehicle is prevented from entering a dangerous area or colliding due to underestimating safety risks. Through the conservative selection mechanism, the penalty for high-risk actions is effectively amplified, forcing the execution network to more strictly avoid high-cost behaviors during updates, thereby establishing a safety boundary awareness during training and improving the vehicle's safety in long-distance, complex path planning.

[0113] Optionally, the above method further includes: determining an immediate reward value based on the currently executed action and the reward function; constructing a reward target value based on the immediate reward value and historical reward predictions; constructing a reward loss function value based on the deviation between the reward target value and the reward predictions; and updating the parameters of the two reward evaluation models based on the reward loss function value.

[0114] In one optional embodiment, an immediate reward value is determined based on the currently executed action and the reward function. The immediate reward value reflects the degree to which the currently executed action contributes to the task objective at the immediate level, including positive rewards for reaching the destination, negative penalties for collisions, regularization terms for path smoothness, and rewards for progress improvement, thereby providing feedback signals to the reinforcement learning module and transforming the navigation task into a quantifiable numerical indicator.

[0115] Then, a reward target value is constructed based on the immediate reward value and historical reward predictions. This allows us to capture the temporal dependence of rewards, considering not only immediate benefits but also long-term development. This results in a self-consistent reward target value that reflects long-term cumulative returns, which guides the learning of the execution network and avoids short-sighted behavior caused by current interests. This enables a leap from single-step decision-making to multi-step planning.

[0116] Furthermore, a reward loss function can be constructed based on the deviation between the target reward value and the predicted reward value. This quantifies the accuracy of the reward evaluation model's estimation; a smaller deviation indicates that the model's prediction of long-term returns is closer to reality. This reward loss function value serves as a signal for backpropagation to adjust model parameters and reduce error. Thus, by updating the parameters of both reward evaluation models based on the reward loss function value, the reward evaluation models can reduce prediction errors and provide more accurate return predictions when facing new environmental conditions. Transforming the error signal calculated based on the target reward value and the predicted reward value into actual changes in the reward evaluation model parameters achieves the accumulation of knowledge and the improvement of evaluation capabilities.

[0117] Optionally, the above method further includes: determining the current era value based on the current execution action and risk cost function; constructing a cost target value based on the current era value and historical cost predictions; constructing a cost loss function value based on the cost target value and cost predictions; and updating the parameters of the two cost evaluation models based on the cost loss function value.

[0118] In one alternative embodiment, the epochal value is determined based on the current action and the risk cost function. The epochal value reflects the immediate negative impact of the current action on security. By transforming security risk into a quantifiable numerical metric, negative feedback signals are provided to the reinforcement learning module.

[0119] Then, a target cost value can be constructed based on the current era's value and historical cost predictions. For example, using the Bellman equation or the principle of temporal difference learning, the current era's value can be combined with historical cost predictions to calculate the target cost value. This captures the temporal cumulative effect of safety risks. By introducing historical cost predictions, a leap from single-step obstacle avoidance to multi-step safety planning is achieved.

[0120] Furthermore, based on the target cost value and the predicted cost value, a cost loss function is constructed to quantify the accuracy of the cost assessment model in estimating security risks. A smaller deviation indicates that the cost assessment model's prediction of long-term security risks is closer to reality. The cost loss function value can be used as a signal for backpropagation to adjust the parameters of the cost assessment model to reduce errors.

[0121] Therefore, the parameters of the two cost assessment models can be updated based on the cost loss function value. By using gradient descent, the performance of the cost assessment models is gradually adjusted, enabling them to provide more accurate safety cost predictions when facing new environmental conditions. This allows for the accumulation of knowledge and improvement of safety assessment capabilities within the cost assessment models, ensuring that they can continuously adapt to the complexity and dynamic changes of the mining environment as training progresses, maintaining the timeliness and accuracy of their assessment capabilities.

[0122] Optionally, the new pose satisfies the iteration stopping condition, including: detecting whether the new pose is outside the map boundary; if the new pose is outside the map boundary, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation fails; if the new pose is not outside the map boundary, detecting whether the new pose collides with an obstacle; if the new pose collides with an obstacle, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation fails; if the new pose does not collide with an obstacle, detecting whether the distance between the new pose and the target pose is less than a preset distance; if the distance between the new pose and the target pose is less than the preset distance, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation succeeds; if the distance between the new pose and the target pose is greater than or equal to the preset distance, detecting whether the reinforcement learning iteration duration has reached a preset duration; if the reinforcement learning iteration duration has reached the preset duration, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation succeeds.

[0123] The aforementioned new pose satisfying the iteration stopping condition may include, but is not limited to: detecting whether the new pose is outside the map boundary, that is, whether the new pose exceeds the physical or logical boundary of the global map. The map boundary serves as the first line of defense in the navigation process, used to quickly eliminate invalid exploration paths that cause the vehicle to drive out of the work area or enter unstructured unknown areas, preventing the execution network from performing meaningless state space searches.

[0124] If the new pose is outside the map boundary, satisfying the iteration stopping condition means that the new pose meets the iteration stopping condition and the first navigation trajectory generation fails. In other words, when it is confirmed that the new pose has exceeded the boundary, the current iteration process is marked as ended, and the result of this navigation attempt is judged as a failure. This clearly defines navigation failure cases caused by violating boundary constraints, and separates the failed first navigation trajectory from the valid paths for targeted handling in subsequent training or execution (such as adding boundary violation penalties or triggering replanning).

[0125] When the new pose is not outside the map boundary, collision detection is performed to identify dangerous paths that, while not crossing the boundary, could lead to the vehicle hitting or getting too close to obstacles. Collision detection filters out navigation schemes with physical conflicts, ensuring that the generated path is geometrically feasible.

[0126] In the event of a collision between the new pose and an obstacle, it can be determined that the new pose meets the iteration stopping condition and the generation of the first navigation trajectory has failed. This clearly defines the trajectory generation failure caused by physical collision.

[0127] If no collision occurs between the new pose and the obstacle, the distance between the new pose and the target pose is checked to determine if the vehicle has entered the target's successful assessment zone. This condition serves as the primary criterion for the success of the first navigation trajectory task, confirming whether the vehicle is sufficiently close to the target pose. This allows for the termination of long-distance reinforcement learning exploration, switching to more precise end-point planning, or directly determining task completion. By using distance threshold determination, the successful navigation process can be efficiently terminated, releasing computational resources and triggering subsequent end-point control logic. This achieves a smooth transition from global coarse planning to local fine planning, ensuring the timely termination of the navigation task when the predetermined accuracy is reached.

[0128] If the distance between the new pose and the target pose is less than a preset distance, the new pose is determined to meet the iteration stopping condition and the first navigation trajectory is successfully generated. Alternatively, the current iteration can be marked as ended, and the first navigation trajectory is considered successful and recorded as a high-quality sample. The success of planning due to successfully reaching the vicinity of the target pose is clearly defined, thus completing positive feedback for successful paths.

[0129] If the distance between the new pose and the target pose is greater than or equal to a preset distance, the iteration time of the reinforcement learning is checked to see if the preset time has been reached. A timeout protection mechanism in the navigation process prevents the reinforcement learning module from getting stuck in an infinite loop, engaging in ineffective exploration, or taking too long to plan in complex environments, thus ensuring the real-time requirements of vehicle navigation. The time limit terminates navigation attempts that fail to converge for extended periods, avoiding resource waste and ensuring the stability of the navigation algorithm's computational efficiency. It also prevents scheduling delays caused by planning timeouts, especially in long-distance, multi-directional mining scenarios, where this mechanism guarantees the real-time requirements of vehicle scheduling.

[0130] When the iteration time of reinforcement learning reaches the preset time, it can be determined that the new pose meets the iteration stopping condition and the first navigation trajectory is successfully generated. This means that a feasible path has been found within a limited time limit, enabling the reinforcement learning module to output a feasible navigation scheme within the specified time. Even if the scheme may not be globally optimal, it ensures the feasibility of the navigation task.

[0131] Optionally, the training samples of the reinforcement learning module include multiple sub-samples, with different sub-samples corresponding to different difficulty levels. The sampling rate of the multiple sub-samples is determined based on the historical success rate and historical cost prediction value of the reinforcement learning module.

[0132] In one optional embodiment, the training samples of the reinforcement learning module include multiple sub-samples, with different sub-samples corresponding to different difficulty levels. By dividing the training samples of the mining area navigation task into multiple levels according to the planning difficulty, each level contains several training scenarios (sub-samples) with specific challenging characteristics. For example, a simple difficulty level might be an open straight road, while a complex difficulty level might be a narrow bend or a dense obstacle area. Through structured, hierarchical training samples, the reinforcement learning module can learn in a targeted manner for scenarios of different complexities, avoiding overfitting on simple scenarios and underperforming on complex scenarios. By grading the difficulty, the diversity and representativeness of the training samples can be balanced, ensuring that the reinforcement learning module can establish basic navigation capabilities in the early stages of training and face high-difficulty challenges in the later stages, thereby improving the overall generalization performance of the reinforcement learning module.

[0133] The sampling rate for multiple subsamples can be determined based on the historical success rate and historical cost predictions of the reinforcement learning module. By combining the difficulty of task completion (historical success rate) and the level of security risk (historical cost prediction), a multi-dimensional sampling weight model is constructed to dynamically allocate training opportunities for subsamples of different difficulty levels. This balances the emphasis between task efficiency and security constraints, preventing the reinforcement learning module from becoming overly conservative due to excessive focus on security, and also preventing it from neglecting security due to excessive pursuit of efficiency. By comprehensively considering historical success rate and historical cost predictions, the sampling strategy can automatically adjust its focus according to changes in the training stage. Initially, it may emphasize success rate to establish basic capabilities, while later it may emphasize low cost to improve security.

[0134] The technical solution proposed in this application will be described below with reference to an optional embodiment. This application proposes a method for open area navigation in mining areas based on reinforcement learning.

[0135] Currently, most large-scale, long-distance path planning tasks still rely on rule-based or traditional graph search methods. These methods are highly reliable in scenarios with simple structures and regular environments, but as the map area expands and the environmental complexity increases, problems such as search failures and a surge in planning time can easily occur.

[0136] In large, open areas of mining areas, where the map often has a wide traversable area and complex path constraints, the performance of traditional planning algorithms degrades, easily leading to planning failures, unreasonable paths, or unstable planning time. Furthermore, mining areas often feature narrow, obstacle-rich, and long-distance, multi-directional scenarios, making traditional algorithms prone to obstacle avoidance, collisions, or ineffective exploration.

[0137] This embodiment designs a reinforcement learning-based navigation algorithm for open areas in mining areas. Modeling and strategy learning are performed for typical mining scenarios such as spoil heaps and loading areas. Reinforcement learning autonomously learns decision-making strategies through trial and error with the environment, enabling it to more fully explore environmental structure and traffic patterns. This allows for the planning of more reasonable and concise paths in complex scenarios, while improving planning efficiency and stability.

[0138] This algorithm improves the robustness and success rate of global path planning in large-scale, unstructured open areas of mining (such as spoil heaps and narrow loading areas), reducing the probability of planning failure. It also stabilizes planning time, maintaining relatively stable computational efficiency across scenarios of varying complexity, avoiding drastic fluctuations in time consumption, and meeting the real-time requirements of vehicle scheduling. Furthermore, the algorithm can autonomously learn traffic characteristics from historical environmental data and interactions, thereby generating more reasonable and smoother paths under complex constraints, reducing unnecessary detours and turns.

[0139] Reinforcement learning can effectively adapt to and solve long-distance global path planning problems in complex open areas of mining regions. For example, kinematic modeling of the route topology in reinforcement learning is combined with path planning algorithms near the destination. Reinforcement learning performs efficient and intelligent coarse-grained waypoint exploration in complex open environments, switching to precise and reliable geometric curve planning algorithms when approaching the target. This leverages the advantages of reinforcement learning in complex spatial exploration while avoiding its shortcomings in end-point control, thus reducing time consumption and meaningless searches. Furthermore, a multimodal feature extractor for open areas, by fusing a joint state input of a global semantic grid map, a local risk grid, and a target geometric relationship vector, can not only perceive the global topology but also focus on local details and understand its relative position to the target, thereby achieving cross-scale environmental understanding and providing an information foundation for making coherent and stable long-distance decisions in open areas. This allows the loss function to not only receive local distance and target vectors but also utilize globally traversable areas, obstacle boundaries, and narrow passage connectivity for long-term path decisions.

[0140] This embodiment proposes a deep learning architecture that combines rewards and penalties. Planning benefits and safety risks are modeled as two interrelated but independently learned objectives: a reward function estimates the task reward, and a risk cost function estimates the safety cost. This architecture simultaneously improves reachability and safety under Lagrange multiplier constraints. The use of a distributed reward function can characterize the reward distribution of the same state-action pair under different future trajectories, providing a richer value representation for long-term, multi-solution path planning tasks. Furthermore, a safety-constrained path planner based on a dual-depth deterministic policy gradient algorithm is proposed to address the problem of ordinary reinforcement learning easily encountering obstacle avoidance, collisions, or ineffective exploration in narrow, obstacle-dense, and long-distance multi-directional scenarios. A curriculum-based training mechanism for long-tail problems is implemented. Through a difficulty-adaptive training strategy for long-tail difficult maps, such as the high diversity of mine maps and the wide distribution of path planning difficulty, a failure history-driven resampling curriculum mechanism is used to strengthen the model's learning intensity in high-risk scenarios such as those prone to collisions and timeouts. This improves the generalization ability of the overall solution and the success rate in complex scenarios without changing the definition of the planning objective.

[0141] Specifically, in this embodiment, the first step is to construct the environment, model motion, and construct reward rules.

[0142] The relationships between the elements in this step are as follows: the environment provides observation → perception and processing transform into a usable state → decision (Actor) generates action → action drives environment (kinematic model) update → environment outputs new state / reward → evaluation generates feedback → gradient update optimizes decision → this process is repeated to achieve autonomous optimization of robot motion. These elements cover the logic of reinforcement learning's "environment-agent (perception-decision-evaluation)-closed loop" and also align with the "perception-planning-execution-feedback" process of robot motion control.

[0143] The reinforcement learning training and interaction process for motion control of mining trucks is as follows: Figure 2 As shown, the process includes: determining the scene environment using the initial position state; performing initial observations using the scene environment; obtaining the state space through an observation feature extractor; and inputting the state space into the execution network to obtain the execution action. Combining the state space and the execution action, reward and cost evaluations are performed to obtain predicted values. The predicted values ​​are used to update the policy gradient of the execution network. Furthermore, reward and cost predictions can be performed using the scene environment to determine the target value, and then the reward and cost evaluations are updated using historical values. The execution action is also used as input to the kinematic model to determine the position state and determine termination and termination conditions. If termination occurs, the environment is updated; otherwise, the environment is not updated. Finally, the reward and penalty rules are implemented and fed back to the scene environment.

[0144] Environment construction involves the initial starting and ending coordinates and angles, map boundary information, static obstacle information, and map topology information.

[0145] The action space and kinematic model are as follows.

[0146] action The normalized two-dimensional continuous control quantities are the normalized front wheel steering angle α. t steer and normalized arc length a t arc .

[0147] Position state estimation uses a kinematic model to estimate the front wheel steering angle. Single-step arc length Substituting the motion mapping, the recursive formulas for the final position and heading angle are as follows:

[0148] ;

[0149] in, This refers to the front wheel steering angle. This is the maximum steering angle; For single-step arc length, and The boundary is the arc length. L is the vehicle wheelbase. a t steer and at arc This is the predicted motion value. Heading angle. x and y are the coordinates, and t is the time.

[0150] The termination and success definitions adopt the following recommended checking order (which can be consistent with the implementation): if it goes out of bounds, it is considered a failure; if it collides, it is considered a failure; if the distance threshold to the nearest target is not reached, it continues to the next step; if it enters the decision area (e.g., the Euclidean distance is less than the preset value), for multiple candidate decision areas, the number of reversals is limited, and then collision detection is performed. If successful, it terminates and the path is recorded.

[0151] like Figure 3 The diagram illustrates an optional single-step iterative reinforcement learning process, as follows: A single-step iterative function triggers a parsing action. After calculating the new pose, a state check is performed. The process checks for boundary violations. If the pose is within the bounds, it checks for collisions. If no collision occurs, it checks for arrival. If arrival fails, it checks for timeouts. If a timeout occurs, the reward is calculated. If no timeout occurs, the distance is checked, such as whether it is less than 40m. If so, a path is searched, leading to a successful arrival. Rewards are calculated regardless of boundary violations, collisions, successful arrivals, or distances. After reward calculation, observations are generated, and the observations, rewards, and status are returned.

[0152] Furthermore, the reward function of this architecture is explained.

[0153] Task Rewards It consists of arrival rewards, terminal penalties, progress rewards, turn smoothing regularization, step penalties, and grid revisit penalties.

[0154] The reward functions include: (1) reward a for reaching the finish line; (2) collision penalty -b; and (3) distance progress reward, such as... If the current vehicle is less than the shortest distance to the destination in the search history, a single-step reward will be given. Euclidean algorithm can be used. (4) Smoothing penalty, such as a turning smoothing regularization term. It consists of the penalty for the difference in steering angle between adjacent steps: , k is a coefficient. (5) Step penalty, such as Current step count / maximum step count, e is the coefficient (6) grid re-exploration penalty -f For revisiting the explored grid indicator function, f is the coefficient. (7) Deviation penalty includes multiple routes; if one is reasonable, no penalty is imposed. This prevents multiple routes from being very close when exiting an intersection, or two reverse routes being very close together. The route penalty can be expressed by the following formula:

[0155] ;

[0156] in, The penalty / reward value for multiple routes. It is a set of nearby routes. It is a small threshold, such as 0.1, to avoid misjudgment due to noise. This can be used for weighted penalty strength. If all nearby routes... If so, then all routes will be reversed, and penalties will be applied. If any nearest route... Greater than If the direction is forward or perpendicular, no penalty will be imposed.

[0157] Security Cost Decoupled modeling from task reward, constructed from the nearest obstacle distance. Let The nearest obstacle distance, the risk boundary. .

[0158] The cost of single-step dense security is defined as:

[0159] ;

[0160] Among them, single-step dense security cost m is the risk boundary. Let be the straight-line distance from the vehicle to the nearest obstacle, and 'clip' be the cutoff function. The idle interval is [0,1]. When the vehicle approaches the obstacle... The risk gradient signal is provided before the obstacle is applied, as the value increases continuously from 0 to 1.

[0161] The formula for the safety cost is as follows, and this cost is also written into the experience replay buffer for learning:

[0162] ;

[0163] in, Total security cost, single-step dense security cost , This is a penalty for failure.

[0164] The second step in this embodiment is observation acquisition. Observations of the environment are acquired from the current moment.

[0165] First, obtain the global raster map. For example, based on map boundaries and obstacle (safety island) information, construct a raster map of size M x N, where each raster is c x c in size. Figure 4 As shown, an optional global raster map is presented. Figure 4 The solid lines in the diagram represent boundary lines, the dashed lines represent translated soil removal lines (considered as boundaries), and the triangles represent obstacles.

[0166] The global grid map has 5 channels. The first 3 channels are RGB, storing information about the start point, end point, obstacles, and boundary lines. The 4th and 5th channels are the cosine and sinine angles of the start and end points, respectively. After each action is completed, the start point position (green grid) is updated in real time on the map and input into the network.

[0167] Further extract key global features.

[0168] First, for the global raster map, cross-attention calculation is performed using location to obtain the corresponding features. This prevents blind spots and provides a global view.

[0169] For example, features are extracted using a visual transformer network to obtain multi-scale features. Low-resolution feature maps reflect the global context, while high-resolution feature maps reflect nearby details. Then, a feature pyramid network is used to fuse the multi-scale features, outputting a fused feature (H, W, c), which is then flattened and transformed into: (H W,c) features K,V. Then, a full cross-attention mechanism is applied, which, due to its small size, allows for full computation.

[0170] For base point generation, n u and v values ​​are randomly initialized. Position embedding is performed into (n, c), followed by self-attention and cross-attention to obtain fused features. The output is an (n, c) dimensional feature, using the ground truth trajectory as the supervision head. 100 trajectory points are sampled as ground truth. The attention formula is as follows, and the output shape of this attention is (n, H). W). The final output Attention @ V has a shape of (n, c):

[0171] Attention = Softmax( );

[0172] Where Attention represents attention, and Q is the query vector. This is a matrix multiplication operation. To perform matrix transpose on vector K, is the feature dimension, and Softmax is the normalization function. This is the scaling factor.

[0173] like Figure 5The diagram illustrates the map processing procedure. The process is as follows: Inputting the query sequence and location time-series information yields an input vector. This vector undergoes multi-head self-attention processing and residual normalization to obtain the query vector. The query vector is then combined with key-value pairs obtained by processing two-dimensional image features using global cross-attention and a mask matrix. After multi-head cross-attention, it passes through a single-layer feedforward neural network to obtain a fused query vector. This fused query vector is then passed through two layers of feedforward neural networks to obtain the ground truth trajectory. The fused query vector can be used as the input vector for further processing, repeated N times, to enhance the network's expressive power. The self-attention mechanism aims to prevent Q from becoming overly concentrated in two-dimensional space.

[0174] Then, for the distances to obstacles in m directions, the samples are uniformly divided into m directions according to 360 degrees in the polar coordinate system, with a maximum detection distance of M. Feature extraction is considered using boundaries, such as using a 50m range for the intrusion boundary, and uniform sampling is performed first. Then, the point closest to the vehicle is found in the two rays and interpolated, resulting in a total of 2m points. This ensures both sampling uniformity and that no convex points are missed, preventing the omission of convex obstacle points. Multilayer perceptron feature dimensionality increase is performed, and the output feature is (n, c).

[0175] For the distance / relative angle / heading difference between the current position and the destination, the dimensionality of the features is increased by multilayer perceptron through a 5-dimensional vector, which includes the straight-line distance between the current position and the destination, the cosine and sinine of the relative angle between the current position and the destination, and the cosine and sinine of the heading difference between the current position and the target position. The output feature is (n,c).

[0176] like Figure 6 As shown, an optional passage range is illustrated. Figure 6 The first intersection of the solid line and the boundary can be the difference point, which is the point closest to the vehicle. The second intersection of the dashed line and the boundary is the angle of a fixed-angle ray. The irregular shape in the figure is a closed polygon, and the optional passage area is formed by the first and second intersection points in sequence.

[0177] Then, self-attention is used to extract the features of each component, followed by feature fusion. The input is (n, 3c), and the output is (n, 3c).

[0178] like Figure 7 The diagram illustrates an optional feature fusion process. This process is as follows: the fused query vector is used to obtain the input vector (query vector, key vector, value vector), which is then processed through multi-head self-attention and residual normalization to obtain a new query vector. This new query vector is then passed through a single-layer feedforward neural network to obtain a new fused query vector. This new fused query vector is then used as input again to perform the process N times.

[0179] The third step in this embodiment implements the execution network. The network input is the output (n, d) of the neural network's self-attention mechanism, which is st, implicitly containing the position state at time t, the target point, and map information. The action output is input to a single-layer converter (one multi-head attention), which outputs 2D (front wheel angle, length) through multiple perceptrons. A deterministic double-delay deep deterministic policy gradient algorithm or a deep deterministic policy gradient algorithm is used.

[0180] like Figure 8 The diagram illustrates the processing procedure of an optional execution network. The process is as follows: the fused query vector is used to obtain the input vector (query vector, key vector, value vector), which is then processed through multi-head self-attention and residual normalization to obtain a new query vector. This new fused query vector is then obtained through a single-layer feedforward neural network. Finally, a multilayer perceptron is used to determine the execution action based on the new fused query vector.

[0181] The third step in this embodiment implements model loss.

[0182] A dual Q-reward network with a self-attention mechanism is employed, with a state space of obs(n,d). The network input consists of two-dimensional features, action(n,2), which are then merged into (n,dm). The network output is a single-layer transformer and a multiple perceptron (outputting a 1-dimensional Q-value). To reduce Q-value overestimation, two independent reward evaluation models are included, with inputs of Obs + Action and outputs of q.

[0183] like Figure 9 The diagram illustrates an optional model loss processing procedure. The procedure is as follows: The input vector (query vector, key vector, value vector) is obtained by fusing the query vector and the executed action. This is then processed through multi-head self-attention and residual normalization to obtain a new query vector. This new fused query vector is then passed through a single-layer feedforward neural network to obtain a new fused query vector. Finally, the new fused query vector is passed through a multilayer perceptron to update the query vector again.

[0184] To enhance the value representation capability in long-term, strongly constrained path planning tasks, the reward function (RewardCritic) employs a distributed reward representation. Two RewardCritics output N quantile reward estimates, instead of scalar Q-values.

[0185] ;

[0186] in, This represents the long-term cumulative return. Let be the weight parameter of the i-th reward function, s be the current environment state, and a be the action. This is the first quantile return estimate of the output of the i-th reward function.

[0187] The Critic outputs a single number representing the future reward obtained after performing a certain action. This is prone to errors in long-term tasks because a single local error can drastically change the final result, and "averaging" masks this risk. Therefore, N estimates are output to prevent large estimation errors; the Actor uses the quantile mean as the policy optimization objective during updates to maintain training stability.

[0188] ;

[0189] in, 'a' represents the average action value. 's' represents the current environment state, and 'a' represents the action. This is the estimate of the k-th quantile return output of the i-th reward function. These are the weight parameters for the first reward function.

[0190] The dual-Q safety constraint network also predicts the loss value. This is achieved by introducing two scalars: Cost Critic and Critic. and The expected cumulative safety cost of state-action pairs is estimated. The Cost Critic maintains a scalar output, and distributed modeling is mainly used on the task reward side, while the safety constraint side maintains a more stable scalar cost estimate. The Actor is updated jointly with the Lagrange multiplier λ.

[0191] The dual-path Critic network and the Actor employ the same multimodal embedding and attention architecture, but do not share parameters; the Critic network additionally takes the action as a modal input.

[0192] The fifth step in this embodiment involves feedback loop learning. This includes training optimization mechanisms and curriculum mechanisms. The focus is on optimizing training to address collision risk.

[0193] The overall training process involves constructing a scenario library using real-world map boundaries, including entry and exit scenarios for dumping grounds, loading areas, and parking lots. Initially, scenarios are randomly selected. After exploring more than K episodes, a learning mode is entered, taking the success rate of each map's past k searches; scenarios with higher failure rates are more likely to be selected. Iterative search planning is then performed within each individual scenario.

[0194] The target Q is obtained through reward and punishment rules. The Reward Critic target value follows the target policy smoothing mechanism of the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm.

[0195] Let the target Actor be Predict actions, target Critic is In the next step, truncated noise is added (to improve the model's training robustness) as input to the critic:

[0196] ;

[0197] in, For the target action in the next moment, For the state at the next moment, For the original action, It is Gaussian random noise. To limit noise, This indicates a normal distribution.

[0198] For distributed reward criters, compare the quantile means of the two target criters and select the one with the smaller mean as the TD3 conservative target distribution.

[0199] The formula for evaluating good actions, based on the selection of q, is as follows:

[0200] .

[0201] Furthermore, the quantile target is We used mean square error to regress the quantile outputs.

[0202] in, Instant rewards (obtained from the environment), Use discount factors. Whether it is successful or not, All future rewards after t+1 are the predicted values ​​of the target network (obtained from the target network, which acts as a stabilizer; the current network's value will keep changing, leading to non-convergence - the TD model aims to make the model self-consistent).

[0203] Furthermore, let's explain the loss function. Predict Q at time t. tar Is it for the Q in the next moment? cur The goal. Q tar The network uses soft updates, from Q cur The parameters were copied over from the network.

[0204] Specifically, the goal of the loss function is to make the predicted value... infinitely close to the target value It can be expressed by the following formula:

[0205] ;

[0206] in, For mean square error loss, To sample state s, action a, immediate reward r, and next state s from experience pool B’ . As expected.

[0207] Furthermore, the temporal difference objective is expressed by the following formula:

[0208] ;

[0209] in, For training purposes. For single-step instant rewards. It is a discount factor, if If the value is 0, then the formula fits the immediate reward, indicating direct feedback from the environment. If... If the value is 0.99, then the formula fits the prediction of immediate rewards and long-term value.

[0210] The quantile loss is as follows:

[0211]

[0212] Furthermore, evaluate the bad behavior. Using a similar approach to Cost Critic, select the distribution with the larger mean as the conservative target distribution for TD3:

[0213] .

[0214] Cost Critic employs the same TD3 objective mechanism, regressing cumulative security costs in scalar form. The target network is maintained through soft updates. Refer to the Reward Critic network.

[0215] Cost Critic Loss (MSE loss) is expressed by the following formula:

[0216] ;

[0217] in, This is the loss value. Output values ​​at the cost of safety. The target value is the cost.

[0218] In reinforcement learning, the loss function of policy gradient methods typically involves calculating the expectation of each state. To adjust the average performance of long-term rewards across possible states, empirical replay sampling of states can be used, followed by approximating the expectation using the sample mean. .

[0219] Furthermore, the paper introduces actor network updates and learning. The Lagrange multiplier method is used to transform the CMDP constrained optimization problem into an unconstrained problem. The original problem description is: to obtain the largest possible action score without collisions.

[0220] The formula for the Actor loss function is as follows:

[0221] ;

[0222] in, For the total loss, To obtain the expected state, For quantile mean return, The action corresponding to state s, For Lagrange multipliers, For the sake of safety, For the cost of network parameters, This is a safety threshold. The loss must be minimized while simultaneously satisfying: Reword q: increases; Cost q: decreases, but cannot increase to d. If it exceeds d, the loss penalty is increased. For adaptive Lagrange multipliers.

[0223] The Actor loss function, substituted with the Q-network formula, expands to:

[0224]

[0225] in, This is the loss value. For the predicted return value of the kth quantile, For the first security cost network, To obtain the expected state, For Lagrange multipliers, The action corresponding to state s.

[0226] This objective allows the strategy to simultaneously pursue high expected returns and low security costs.

[0227] When the cost of the strategy estimated by Cost Critic exceeds the threshold At that time, riding the son By increasing the gradient ascent, the policy update more strongly penalizes high-risk actions; conversely... Reduce and optimize the reward of the key regression task. Logarithmic parameterization is used in the implementation. To ensure :

[0228] ;

[0229] in, This indicates a parameter update. This represents the deviation of the mean cost from the threshold d. For example, 1 indicates a collision; a value greater than d=0.8 indicates danger, and the coefficient is increased to 100, which means the penalty term in the loss function is increased.

[0230] Furthermore, courses are designed to address long-tail challenges. Data is sampled according to difficulty, with training samples drawn from three difficulty levels assisted by VLM (Visual Learning Model). During training, sampling is performed using the following coefficients. The learning rate for difficult examples can be gradually increased (manually defined difficult examples, repeated learning). For examples with high failure rates, the sampling rate can be increased, such as further resampling based on recent success rates, and scenarios with higher failure rates receiving higher sampling probabilities (failed cases, repeated learning). Collision risk training focuses on: selecting difficult scenarios for learning through cost critics, i.e., training with more experience samples that lead to higher costs (repeated learning of scenarios with high cost critics).

[0231] Therefore, this embodiment possesses data-driven capabilities, enhancing its intelligence based on complex data. Action space and kinematic modeling generate coarse routes within the target trajectory range, achieving rapid computation. The feature extraction module completes multimodal fusion, providing both global information and local features. The refined state space guides the actor network in predicting good actions. Planning rewards and safety risks are modeled as two interrelated but independently learned objectives: the Reward Critic estimates the task reward, and the Cost Critic estimates the safety cost. The Actor simultaneously optimizes reachability and safety under Lagrange multiplier constraints.

[0232] Furthermore, taking a specific scenario navigation planning example, the vehicle navigation method of this embodiment will be explained by way of example.

[0233] like Figure 10 The image shows a trajectory output using this vehicle navigation method. Figure 10 The solid line in the center represents the navigation trajectory. For example... Figure 11 As shown, this demonstrates a trajectory output by the vehicle navigation method in a confined space. Figure 11 The medium circle represents the starting point and the ending point, and the dashed line represents the navigation trajectory. Figure 12 This shows a schematic diagram of a trajectory in an irregular scene. Figure 12 Trajectory 1 represents the trajectory obtained using the navigation method of this embodiment, while trajectory 2 represents the trajectory obtained using the conventional method. Figure 13 It also shows the trajectory of another irregular scene. Figure 13 Trajectory 3 represents the trajectory obtained using the navigation method of this embodiment, while trajectory 4 represents the trajectory obtained using the conventional method.

[0234] According to an embodiment of the present invention, a device embodiment for a vehicle navigation device is provided. It should be noted that this device can be used to execute the vehicle navigation method described above. For example... Figure 14 The diagram shown illustrates a vehicle navigation device, which includes:

[0235] The acquisition module 50 is used to respond to navigation requests for the target work area, acquire map information of the target work area, and the initial pose and target pose corresponding to the navigation request. The target work area is an unstructured open area, and the map information is the global map information of the unstructured open area.

[0236] The generation module 52 is used to generate a first navigation trajectory based on map information, the currently perceived passable area of ​​the vehicle, the initial pose and the target pose, using a reinforcement learning module. The starting position of the first navigation trajectory is the initial pose, and the distance between the ending position of the first navigation trajectory and the target pose is less than a preset distance.

[0237] Planning module 54 is used to generate a second navigation trajectory based on map information, the endpoint and target pose of the first navigation trajectory, and using the geometric curve planning module.

[0238] The control module 56 is used to control the vehicle's movement based on the first navigation trajectory and the second navigation trajectory.

[0239] Optionally, the generation module is further configured to: take the initial pose as the current pose, and use the multimodal feature extractor in the reinforcement learning module to extract multimodal features from the map information, passable area, current pose, and target pose to obtain the vehicle's current state information; input the current state information into the execution network of the reinforcement learning module, and use the execution network to generate the vehicle's current execution action, wherein the current execution action includes: front wheel steering angle and single-step arc length; substitute the current execution action, the vehicle's wheelbase, and the preset steering angle into the vehicle's kinematic model for solution to obtain the vehicle's new position. The process involves: taking the new pose as the current pose and repeatedly performing multimodal feature extraction on the map information, passable area, current pose, and target pose using a multimodal feature extractor to obtain the vehicle's current state information; inputting the current state information into the execution network to generate the vehicle's current action; substituting the current action, the vehicle's wheelbase, and the preset steering angle into the vehicle's kinematic model for solution to obtain the vehicle's new pose; and continuing until the new pose meets the iteration stopping condition. Based on the updated poses corresponding to multiple iterations, the first navigation trajectory is determined.

[0240] Optionally, the generation module is also used to: construct a global grid map based on map information, the current pose, and the target pose, and extract features from the global grid map to obtain global features; extract features from traversable areas to obtain local features; construct target geometric relationships based on the current pose and the target pose, and extract features from the target geometric relationships to obtain relational features, wherein the target geometric relationships include the relative distance, relative angle, and heading angle deviation between the current pose and the target pose; determine the first priority corresponding to the global features, the second priority corresponding to the local features, and the third priority corresponding to the relational features based on the navigation scenario corresponding to the navigation request; and use a self-attention mechanism to combine the first priority, the second priority, and the third priority to perform feature fusion on the global features, local features, and relational features to obtain the current state information.

[0241] Optionally, the generation module is also used for: performing multi-scale feature extraction on the global raster map to obtain feature maps of multiple scales, wherein the feature maps of different scales correspond to different resolutions; performing feature fusion on the feature maps of multiple scales to obtain fused features; performing two-dimensional unfolding on the fused features to obtain two-dimensional features; performing feature processing on multiple preset features using a self-attention mechanism to obtain self-attention features; and performing feature fusion on the fused features and self-attention features using a cross-attention mechanism to obtain global features.

[0242] Optionally, the above device further includes an interpolation module, used for: constructing a target detection range centered on the vehicle and based on a preset distance; performing obstacle detection on the target detection range according to multiple preset directions to obtain first boundary points corresponding to multiple preset directions; interpolating the area between two adjacent preset directions to obtain multiple second boundary points; and constructing a passable area based on the first boundary points and multiple second boundary points corresponding to multiple preset directions.

[0243] Optionally, the above device further includes a loss module, configured to: input the current state information and the current execution action into a reward function to determine a predicted reward value, and construct a first loss function value based on the predicted reward value; input the current state information and the current execution action into a risk cost function to determine a predicted cost value, and construct a second loss function value based on the predicted cost value; adjust the preset weight corresponding to the second loss function value based on the deviation between the predicted cost value and the preset predicted value to obtain a target weight; based on the target weight, perform a weighted summation of the negative of the first loss function value and the second loss function value to obtain a total loss function; and apply the total loss function to the execution network... The parameters are updated; preferably, the loss module is further configured to: determine a first gradient based on the deviation when the predicted cost is greater than a preset predicted value, and increase the preset weight based on the first gradient to obtain a target weight; determine a second gradient based on the deviation when the predicted cost is less than or equal to the preset predicted value, and decrease the preset weight based on the second gradient to obtain a target weight; preferably, the loss module is further configured to: perform logarithmic processing on the target weight to obtain a target logarithmic value; obtain the product of the second loss function value and the target logarithmic value to obtain a target product; obtain the sum of the target product and the negative of the first loss function value to obtain the total loss function.

[0244] Optionally, the loss module is further configured to: input the current state information and the current action to two reward evaluation models corresponding to the reward function, and generate two first prediction sets using the two reward evaluation models; determine the quantile mean of the two first prediction sets respectively, and obtain two prediction means; obtain the minimum value among the two prediction means to obtain the reward prediction value; preferably, the loss module is further configured to: input the current state information and the current action to two cost evaluation models corresponding to the risk cost function, and generate two second prediction values ​​using the two cost evaluation models; obtain the maximum value among the two second prediction values ​​to obtain the cost prediction value.

[0245] Optionally, the above-mentioned device further includes an update module, used for: determining an immediate reward value based on the currently executed action and the reward function; constructing a reward target value based on the immediate reward value and historical reward prediction values; constructing a reward loss function value based on the deviation between the reward target value and the reward prediction value; and updating the parameters of the two reward evaluation models based on the reward loss function value;

[0246] Preferably, the update module is further configured to: determine the current era value based on the current action and risk cost function; construct a cost target value based on the current era value and historical cost predictions; construct a cost loss function value based on the cost target value and cost predictions; and update the parameters of the two cost evaluation models based on the cost loss function value.

[0247] Optionally, the new pose satisfies the iteration stopping condition, including: detecting whether the new pose is outside the map boundary; if the new pose is outside the map boundary, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation fails; if the new pose is not outside the map boundary, detecting whether the new pose collides with an obstacle; if the new pose collides with an obstacle, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation fails; if the new pose does not collide with an obstacle, detecting whether the distance between the new pose and the target pose is less than a preset distance; if the distance between the new pose and the target pose is less than the preset distance, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation succeeds; if the distance between the new pose and the target pose is greater than or equal to the preset distance, detecting whether the reinforcement learning iteration duration has reached a preset duration; if the reinforcement learning iteration duration has reached the preset duration, determining that the new pose satisfies the iteration stopping condition and the first navigation trajectory generation succeeds.

[0248] Optionally, the training samples of the reinforcement learning module include multiple sub-samples, with different sub-samples corresponding to different difficulty levels. The sampling rate of the multiple sub-samples is determined based on the historical success rate and historical cost prediction value of the reinforcement learning module.

[0249] Embodiments of this application also provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention during runtime.

[0250] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0251] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0252] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of the present invention.

[0253] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of the present invention.

[0254] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0255] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0256] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0257] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0258] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0259] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A vehicle navigation method, characterized in that, include: In response to a navigation request for a target work area, the system obtains map information of the target work area, as well as the initial pose and target pose corresponding to the navigation request, wherein the target work area is an unstructured open area, and the map information is global map information of the unstructured open area; Based on the map information, the passable area currently perceived by the vehicle, the initial pose, and the target pose, a first navigation trajectory is generated using a reinforcement learning module. The starting position of the first navigation trajectory is the initial pose, and the distance between the ending position of the first navigation trajectory and the target pose is less than a preset distance. Based on the map information, the endpoint of the first navigation trajectory, and the target pose, a second navigation trajectory is generated using the geometric curve planning module; The vehicle is controlled to move based on the first navigation trajectory and the second navigation trajectory.

2. The method according to claim 1, characterized in that, Based on the map information, the currently perceived passable area of ​​the vehicle, the initial pose, and the target pose, a first navigation trajectory is generated using a reinforcement learning module, including: The initial pose is used as the current pose, and the multimodal feature extractor in the reinforcement learning module is used to extract multimodal features from the map information, the passable area, the current pose, and the target pose to obtain the current state information of the vehicle. The current state information is input into the execution network of the reinforcement learning module, and the execution network is used to generate the current execution action of the vehicle, wherein the current execution action includes: front wheel steering angle and single-step arc length; The current action being executed, the wheelbase of the vehicle, and the preset steering angle are substituted into the kinematic model of the vehicle for solution to obtain the new pose of the vehicle; The new pose is used as the current pose, and the process of extracting multimodal features from the map information, the passable area, the current pose, and the target pose using the multimodal feature extractor is repeated to obtain the current state information of the vehicle. The current state information is then input into the execution network, which generates the current action of the vehicle. The current action, the wheelbase of the vehicle, and the preset steering angle are then substituted into the kinematic model of the vehicle for solution to obtain the new pose of the vehicle. This process continues until the new pose meets the iteration stopping condition. The first navigation trajectory is determined based on the updated poses corresponding to multiple iterations.

3. The method according to claim 2, characterized in that, Multimodal feature extraction is performed on the map information, the passable area, the current pose, and the target pose to obtain the vehicle's current state information, including: Based on the map information, the current pose, and the target pose, a global grid map is constructed, and features are extracted from the global grid map to obtain global features; Feature extraction is performed on the passable area to obtain local features; Based on the current pose and the target pose, a target geometric relationship is constructed, and feature extraction is performed on the target geometric relationship to obtain relationship features. The target geometric relationship includes the relative distance, relative angle, and heading angle deviation between the current pose and the target pose. Based on the navigation scenario corresponding to the navigation request, a first priority corresponding to the global feature, a second priority corresponding to the local feature, and a third priority corresponding to the relation feature are determined. By utilizing a self-attention mechanism and combining the first priority, the second priority, and the third priority, feature fusion is performed on the global features, the local features, and the relational features to obtain the current state information.

4. The method according to claim 3, characterized in that, Feature extraction is performed on the global raster map to obtain global features, including: Multi-scale feature extraction is performed on the global raster map to obtain feature maps of multiple scales, wherein the feature maps of different scales correspond to different resolutions; Feature fusion is performed on the feature maps at the multiple scales to obtain fused features; The fused features are unfolded in two dimensions to obtain two-dimensional features; The self-attention mechanism is used to process multiple preset features to obtain self-attention features. The global features are obtained by fusing the fused features and the self-attention features using a cross-attention mechanism.

5. The method according to claim 2, characterized in that, The method further includes: A target detection range is constructed centered on the vehicle and based on a preset distance; Obstacle detection is performed on the target detection range according to multiple preset directions to obtain the first boundary points corresponding to the multiple preset directions; Interpolate the region between two adjacent preset directions to obtain multiple second boundary points; The passable area is constructed based on the first boundary points corresponding to the multiple preset directions and the multiple second boundary points.

6. The method according to claim 2, characterized in that, The method further includes: The current state information and the current action are input into the reward function to determine the predicted reward value, and a first loss function value is constructed based on the predicted reward value; The current state information and the current action are input into the risk cost function to determine the cost prediction value, and a second loss function value is constructed based on the cost prediction value; Based on the deviation between the predicted cost value and the preset predicted value, the preset weight corresponding to the second loss function value is adjusted to obtain the target weight; Based on the target weights, the negatives of the first loss function value and the second loss function value are weighted and summed to obtain the total loss function. The parameters of the execution network are updated based on the total loss function; Preferably, based on the deviation between the predicted cost value and the preset predicted value, the preset weight corresponding to the second loss function value is adjusted to obtain the target weight, including: If the predicted cost is greater than the preset predicted value, a first gradient is determined based on the deviation, and the preset weight is increased based on the first gradient to obtain the target weight. If the predicted cost is less than or equal to the preset predicted value, a second gradient is determined based on the deviation, and the preset weight is reduced based on the second gradient to obtain the target weight. Preferably, based on the target weights, a weighted sum is performed on the negatives of the first loss function value and the second loss function value to obtain the total loss function, including: The target weights are logarithmically processed to obtain the target logarithmic value; Obtain the product of the second loss function value and the target logarithm value to get the target product; The total loss function is obtained by summing the target product with the negative of the first loss function value.

7. The method according to claim 6, characterized in that, Input the current state information and the currently executed action into the reward function to determine the predicted reward value, including: The current state information and the current action are respectively input into the two reward evaluation models corresponding to the reward function, and the two reward evaluation models are used to generate two first prediction sets. The quantile mean of the two first prediction sets is determined respectively to obtain two prediction means; The minimum of the two predicted means is obtained to obtain the predicted reward value; Preferably, the current state information and the currently executed action are input into a risk cost function to determine the cost prediction value, including: The current state information and the current action are respectively input into the two cost evaluation models corresponding to the risk cost function, and two second prediction values ​​are generated using the two cost evaluation models. The maximum value of the two second predicted values ​​is obtained to obtain the cost predicted value.

8. The method according to claim 7, characterized in that, The method further includes: Based on the currently executed action and the reward function, determine the immediate reward value; Based on the instant reward value and the historical reward prediction value, a reward target value is constructed; Based on the deviation between the target reward value and the predicted reward value, a reward loss function value is constructed; The parameters of the two reward evaluation models are updated based on the reward loss function value; Preferably, the method further includes: Based on the current action being performed and the risk cost function, the value of that era is determined; Based on the aforementioned contemporary value and historical cost predictions, a cost target value is constructed. Based on the target cost value and the predicted cost value, a cost loss function value is constructed; The parameters of the two cost evaluation models are updated based on the cost loss function value.

9. The method according to claim 2, characterized in that, The new pose satisfies the iteration stopping conditions, including: Detect whether the new pose is outside the map boundary; If the new pose is outside the map boundary, it is determined that the new pose satisfies the iteration stopping condition, and the first navigation trajectory generation fails. If the new pose is not outside the map boundary, detect whether the new pose collides with an obstacle; If the new pose collides with an obstacle, it is determined that the new pose satisfies the iteration stopping condition, and the first navigation trajectory generation fails. If the new pose does not collide with the obstacle, detect whether the distance between the new pose and the target pose is less than a preset distance; If the distance between the new pose and the target pose is less than the preset distance, it is determined that the new pose satisfies the iteration stop condition, and the first navigation trajectory is successfully generated; If the distance between the new pose and the target pose is greater than or equal to the preset distance, it is detected whether the iteration time of the reinforcement learning has reached the preset time. If the iteration duration of the reinforcement learning reaches the preset duration, it is determined that the new pose satisfies the iteration stopping condition, and the first navigation trajectory is successfully generated.

10. The method according to claim 1, characterized in that, The training samples of the reinforcement learning module include multiple sub-samples, with different sub-samples corresponding to different difficulty levels. The sampling rate of the multiple sub-samples is determined based on the historical success rate and historical cost prediction value of the reinforcement learning module.

11. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 10.

Images