Adaptive parking method and system based on hierarchical reinforcement learning and kinematic constraints

Abstract

The application provides an adaptive parking method and system based on hierarchical reinforcement learning and kinematic constraints, relates to the technical field of automatic driving, and aims to solve the problems of poor adaptability, weak safety and low efficiency in the prior art.The method obtains a bird's eye view environment representation of a parking scene and vehicle state information, extracts and fuses features through a perception module to obtain a joint state representation, inputs the joint state representation into a trained high-level strategy network to generate a high-level sub-target instruction, performs kinematic feasibility verification on the high-level sub-target instruction through an adaptive steering boundary module to obtain a verified sub-target, and inputs the verified sub-target and the joint state representation into a trained low-level strategy network to generate continuous vehicle control actions.The application solves the problems of poor adaptability, weak safety and low efficiency in the prior art, and enables the vehicle to complete an efficient, safe and reliable parking task in a dynamic dense scene.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and particularly relates to an adaptive parking method and system based on hierarchical reinforcement learning and kinematic constraints. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the autonomous driving technology system, the Automated Parking Assist System (APS) is a key component for achieving autonomous vehicle movement. APS enables vehicles to perceive, understand, and proactively adapt to complex and changing parking environments in real time, autonomously planning and executing a safe, efficient, and kinematically sound parking path. Traditional solutions often employ rule-based path planning methods, integrating perception, planning, and control modules to generate a feasible path from the initial position to the target parking space. However, such methods often struggle to ensure real-time path feasibility in narrow and dynamic urban parking environments.

[0004] Current mainstream automated parking systems primarily employ a geometric model-based trajectory planning framework, relying on structured decision pipelines to achieve environmental perception, path search, and trajectory tracking. Optimization methods have been systematically applied to the path planning field, with the core idea being the establishment of a mathematical optimization model for the path planning problem. While these methods perform well in conventional scenarios, they significantly increase the probability of parking failure in dynamic and variable environments.

[0005] While reinforcement learning methods can improve generalization ability through data interaction, their end-to-end paradigm suffers from inherent problems such as training instability and poor interpretability. As an improvement, hierarchical reinforcement learning enhances the hierarchy and clarity of decision-making through task decomposition. However, the lack of explicit safety constraints in its high-level policies may lead to unsafe sub-objective selections, and it suffers from fundamental limitations such as low search efficiency and poor control precision in high-dimensional discrete action spaces, thus limiting its practical applicability in real-world scenarios.

[0006] To overcome the above shortcomings, a hybrid paradigm combining rule-based verification with learning-based strategies is proposed. Although these methods improve security and exploration capabilities, their rigid constraints and conflict-based optimization objectives limit their adaptability to dynamic scenarios and often lead to local optima.

[0007] End-to-end imitation learning methods use large-scale expert demonstration data to build behavioral clone models for parking tasks. Accurate driving models are developed using imitation learning methods by collecting and processing large end-to-end driving datasets. However, the policy generation process heavily relies on strong prior knowledge and lacks the ability to reason about new scenarios, resulting in poor model performance when handling unexpected situations in dynamic and complex driving environments.

[0008] It is evident that existing technologies still have some shortcomings:

[0009] (1) Pure rule-based methods have weak adaptability and are computationally intensive, making it difficult to handle dynamic and complex scenarios.

[0010] (2) Learning-based reinforcement methods suffer from sparse rewards, unstable training, poor interpretability, and questionable security.

[0011] (3) Although the hierarchical reinforcement learning (HRL) method is structured, the lack of kinematic safety constraints in high-level decision-making makes sub-goals infeasible.

[0012] (4) The rule-learning hybrid learning paradigm suffers from poor generalization ability and low decision-making efficiency due to conflicting optimization goals. Summary of the Invention

[0013] To overcome the shortcomings of the prior art, this invention provides an adaptive parking method and system based on hierarchical reinforcement learning and kinematic constraints. It combines high-level policies and low-level control policies, and integrates hierarchical gating and hierarchical discriminant networks. The high-level policy selects sub-objectives to guide the behavior of the low-level control policy, and the low-level control policy executes specific control actions according to the selected sub-objectives. This solves the adaptability and accuracy problems of traditional parking systems in dynamic environments, enabling vehicles to complete efficient, safe, and reliable parking tasks in dynamic and dense scenarios.

[0014] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0015] The first aspect of the present invention provides an adaptive parking method based on hierarchical reinforcement learning and kinematic constraints, comprising:

[0016] Obtain a bird's-eye view of the parking scene and vehicle status information;

[0017] The perception module extracts and fuses features from the bird's-eye view environment representation and vehicle status information to obtain a joint state representation;

[0018] The joint state representation is input into the trained high-level policy network to generate high-level sub-objective instructions;

[0019] The kinematic feasibility of the high-level sub-target command is verified by the adaptive steering boundary module, and the verified sub-target is obtained.

[0020] The verified sub-objectives and joint state representations are input into the trained low-level policy network to generate continuous vehicle control actions.

[0021] During the training process, based on the reward mechanism, the high-level policy network and the low-level policy network are jointly trained using a hierarchical reinforcement learning framework.

[0022] As one implementation method, a bird's-eye view representation of the parking scene is obtained, and the specific process is as follows:

[0023] The sensor acquires 3D point cloud data containing information on obstacle location, vehicle orientation, lane boundaries, and road markings.

[0024] Based on the vehicle's position and orientation, the 3D point cloud data is processed and mapped onto a 2D plane centered on the vehicle.

[0025] The mapped data is integrated and encoded to generate a two-dimensional bird's-eye view representation of the parking scene.

[0026] As one implementation method, the vehicle status information includes at least the real-time coordinates of the rear axle center, the real-time heading angle, the vehicle wheelbase, the maximum front wheel steering angle, and the minimum turning radius.

[0027] As one implementation method, a perception module is used to extract and fuse features from the bird's-eye view environment representation and vehicle status information. This perception module includes a first convolutional layer, four residual blocks, a global average pooling layer, a first fully connected layer, and a second fully connected layer. The specific process is as follows:

[0028] The first feature map is obtained by extracting features from the bird's-eye view environment representation through the first convolutional layer;

[0029] The first feature map is semantically enhanced by four residual blocks to obtain the enhanced feature map.

[0030] The enhanced feature map is compressed and transformed by a global average pooling layer and a first fully connected layer to obtain the bird's-eye view feature vector.

[0031] The vehicle state information is converted into an ontology measurement vector, which is then input into the second fully connected layer for feature extraction to obtain the ontology embedding vector.

[0032] The feature vector of the bird's-eye view and the ontology embedding vector are concatenated to obtain the joint state representation.

[0033] As one implementation method, the joint state representation is input into a trained high-level policy network to generate a high-level sub-objective instruction. The specific process is as follows:

[0034] By using the Actor network in the high-level policy network, the joint state representation is generated into a two-dimensional logits vector, which is then normalized to obtain the probability distribution of the high-level policy sub-objectives.

[0035] Based on joint state representation, a state value function is constructed through the Critic network in the high-level policy network, and sub-objectives are selected according to the state value function.

[0036] Based on the reward mechanism, a high-level policy optimization loss function is constructed using a near-end policy optimization algorithm to optimize the high-level policy.

[0037] As one implementation method, the kinematic feasibility of high-level sub-target commands is verified through an adaptive steering boundary module. The specific process is as follows:

[0038] In the adaptive steering boundary module, the vehicle's center position is determined based on vehicle state information;

[0039] Determine the coordinates of the parking target based on the vehicle's center position;

[0040] Based on the coordinates of the parking target, kinematic constraints are applied to the high-level sub-target commands through kinematic modeling to generate kinematically constrained sub-targets.

[0041] The optimal shift boundary is calculated based on the vehicle's minimum turning radius and wheelbase to ensure that the sub-objectives satisfy the kinematically feasible set.

[0042] As one implementation method, the verified sub-objectives and joint state representations are input into the trained low-level vehicle network to generate continuous vehicle control actions. The specific process is as follows:

[0043] In the low-level policy network, a parking feasibility module based on kinematic constraints is used to delineate the safe zone during the vehicle parking process;

[0044] Based on the safe area during vehicle parking, the verified sub-objectives and joint state representations are input into the Critic network of the low-level policy network to construct the joint state-sub-objective value function.

[0045] Based on the joint state-sub-objective value function, the verified sub-objective and joint state representations are input into the Actor network of the low-level policy network to generate sub-policy control vectors, which control continuous longitudinal and steering actions.

[0046] Based on the low-level reward mechanism, a low-level policy optimization loss function is constructed using a proximal policy optimization algorithm to train and optimize the low-level policy.

[0047] As one implementation method, the low-level reward mechanism has the following formula:

[0048] ;

[0049] ;

[0050] ;

[0051] in, Indicates the reward for the forward sub-strategy; This indicates a reward for the reversing strategy; This represents the first reward function, used to quantify the deviation between the forward search and the target parking space; This represents the collision reward function, used to penalize collisions. This represents a reward function that encourages autonomous vehicles to maintain vertical alignment with the target parking space. This represents the reward function based on smoothing operations; This represents the second reward function, used to quantify the deviation between the forward search and the target parking space; This represents the second collision reward function, used to penalize collisions. express; This represents the reward function based on the parking feasibility module.

[0052] As one implementation method, a low-level policy optimization loss function is constructed using a proximal policy optimization algorithm to optimize the low-level policy. The specific process is as follows:

[0053] Based on the low-level sub-policy control vector, calculate the low-level policy importance sampling ratio;

[0054] Calculate the low-level policy advantage function based on the joint state-sub-objective value function;

[0055] Based on the sampling ratio of low-level policy importance and the low-level policy advantage function, an optimization objective for low-level policies is constructed, namely the low-level policy optimization loss function.

[0056] A second aspect of the present invention provides an adaptive parking system based on hierarchical reinforcement learning and kinematic constraints, comprising:

[0057] The data acquisition module is used to acquire a bird's-eye view of the parking scene and vehicle status information;

[0058] The joint state representation acquisition module is used to extract and fuse features from the bird's-eye view environment representation and vehicle state information through the perception module to obtain a joint state representation.

[0059] The high-level policy module is used to input the joint state representation into the trained high-level policy network and generate a high-level sub-objective instruction.

[0060] The sub-target verification constraint module is used to verify the kinematic feasibility of high-level sub-target commands through the adaptive steering boundary module, and obtain the verified sub-target.

[0061] The low-level policy module is used to input the verified sub-objectives and joint state representations into the trained low-level policy network to generate continuous vehicle control actions.

[0062] During the training process, based on the reward mechanism, the high-level policy network and the low-level policy network are jointly trained using a hierarchical reinforcement learning framework.

[0063] The above one or more technical solutions have the following beneficial effects:

[0064] In this embodiment, a hierarchical reinforcement learning decision architecture is adopted to decompose the complex parking task into two relatively simple sub-problems: high-level sub-objective selection and low-level continuous control. This greatly reduces the policy search space of each layer, enabling the agent to learn efficiently. At the same time, the key geometric parameters of the vehicle are used as configurable inputs to the ATB and PF modules, decoupling the core decision algorithm from specific vehicle models. Only one training is required, and the model can be seamlessly adapted to various vehicle models with different wheelbases and turning radii by adjusting the parameters. The success rate in cross-vehicle tests exceeds 96%, improving the model training stability and generalization ability.

[0065] In this embodiment, an adaptive steering boundary module is used to verify the kinematic feasibility of the high-level strategy decision loop. This module performs online calculations based on the vehicle's precise geometric parameters (wheelbase, minimum turning radius) and real-time pose, actively filtering out all kinematically infeasible sub-objectives. This eliminates dangerous commands that could lead to collisions or planning failures from the decision-making source, improving the safety of parking decisions and paths compared to traditional learning base methods.

[0066] In this embodiment, a parking feasibility module is introduced to provide refined spatial guidance for low-level strategy control, elevating passive obstacle avoidance (avoiding collisions) to active guidance (driving within safe areas). This module dynamically divides parameterized multi-stage feasible regions in a bird's-eye view and uses the matching degree between the vehicle trajectory and these regions as the core reward signal to drive the low-level strategy to generate smooth, accurate parking paths that conform to human driving habits. Experimental verification shows that this scheme achieves a parking success rate consistently exceeding 97.8% in narrow real-world scenarios containing dynamic obstacles, with path smoothness significantly superior to traditional optimization methods. It effectively solves the problems of high failure rates and trajectory oscillations in dynamic environments, significantly improving parking success rate and path quality in complex dynamic scenarios.

[0067] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0068] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0069] Figure 1This is a schematic diagram of the overall framework of an adaptive parking method based on hierarchical reinforcement learning and kinematic constraints in this embodiment.

[0070] Figure 2 This is a schematic diagram of the parking process in Embodiment 1.

[0071] Figure 3 This is a schematic diagram of the sensing module structure in Embodiment 1;

[0072] Figure 4 This is a schematic diagram of the high-level strategy network structure in this embodiment.

[0073] Figure 5 This is a schematic diagram of the low-level policy network structure in this embodiment.

[0074] Figure 6 This is a demonstration diagram illustrating the calculation of the minimum turning radius for the adaptive steering boundary module in Embodiment 1.

[0075] Figure 7 This is a diagram demonstrating the trajectory of the vehicle controlled by the kinematic constraint-based parking feasibility module in this embodiment 1 during the parking process.

[0076] Figure 8 This is a schematic diagram of parking spaces in multiple scenarios according to Embodiment 1;

[0077] in, Figure 8 (a) is a narrow indoor scene; Figure 8 (b) is an outdoor, vacant scene. Figure 8 (c) is a narrow outdoor scene. Detailed Implementation

[0078] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0079] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0080] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0081] Example 1

[0082] This embodiment discloses an adaptive parking method based on hierarchical reinforcement learning and kinematic constraints.

[0083] To more clearly illustrate this embodiment, the adaptive parking implementation process based on hierarchical reinforcement learning and kinematic constraints can be specifically described as follows:

[0084] Adaptive parking methods based on hierarchical reinforcement learning and kinematic constraints include:

[0085] S1. Obtain a bird's-eye view of the parking scene and vehicle status information;

[0086] S2. The perception module extracts and fuses features from the bird's-eye view environment representation and vehicle status information to obtain a joint state representation.

[0087] S3. Input the joint state representation into the trained high-level policy network to generate a high-level sub-objective instruction;

[0088] S4. The kinematic feasibility of the high-level sub-target command is verified by the adaptive steering boundary module to obtain the verified sub-target.

[0089] S5. Input the verified sub-objectives and joint state representations into the trained low-level policy network to generate continuous vehicle control actions.

[0090] like Figure 1 As shown, in step S1, a bird's-eye view of the parking scene and vehicle status information are obtained.

[0091] S1-1. Obtain a bird's-eye view of the parking scene. The specific process is as follows:

[0092] (1) Obtain three-dimensional point cloud data containing information on obstacle location, vehicle orientation, lane boundary and road markings through sensors.

[0093] In this embodiment, environmental information surrounding the vehicle is obtained using a Velodyne VLP-16 LiDAR sensor. This information is provided in the form of point cloud data, which includes the location of obstacles, vehicle orientation, vehicle lane boundaries, and road markings.

[0094] (2) Based on the vehicle position and vehicle orientation, the three-dimensional point cloud data is processed and mapped onto a two-dimensional plane centered on the vehicle.

[0095] In this embodiment, by processing the point cloud data and combining it with the vehicle's position and orientation, coordinate transformation is performed to project the point cloud data in three-dimensional space onto a two-dimensional plane parallel to the ground, thereby forming a preliminary top-down view space table.

[0096] (3) Integrate and encode the mapped data to generate a two-dimensional bird's-eye view representation of the parking scene.

[0097] The mapped data is integrated and encoded into a unified two-dimensional bird's-eye view, generating a bird's-eye view environment representation.

[0098] After the above steps, by converting real-time data collected from sensors such as LiDAR into a two-dimensional bird's-eye view, we can gain a more intuitive understanding of the vehicle's driving space and environment, providing structured environmental input for subsequent path planning and decision-making modules.

[0099] S1-2. Based on the bird's-eye view environment representation, delineate the drivable area around the vehicle.

[0100] In a bird's-eye view (BEV), the drivable area around the vehicle is defined using the following formula:

[0101]

[0102] In the BEV diagram, each point This corresponds to the accessible area around the vehicle at a certain angle. For example, Indicates the reachable area ahead. This indicates the accessible area to the right front. The system uses information from these areas to define a safe passage space for the vehicle and dynamically adjusts its path planning. The BEV map environment is constructed dynamically; as sensor data is continuously updated, the vehicle can adjust its surrounding drivable area in real time to cope with constantly changing environments, such as the appearance of new obstacles or changes in road conditions.

[0103] After the above steps, the bird's-eye view environmental representation data is converted into... The form describes a set of discrete points in the reachable area in each direction around the vehicle. In this way, the vehicle can assess the drivable area in real time in a dynamic environment, providing important support for the decision-making process, thereby avoiding collisions and optimizing the driving route.

[0104] S1-3. Obtain vehicle status information.

[0105] In this embodiment, vehicle status information is obtained from the accelerator, brake, and steering wheel via onboard sensors and inertial navigation. The vehicle status information includes at least the real-time coordinates of the rear axle center, real-time heading angle, vehicle wheelbase, maximum front wheel steering angle, and minimum turning radius. The specific acquisition process is as follows:

[0106] (1) Obtain the real-time coordinates and real-time heading angle of the rear axle center of the vehicle through the vehicle positioning system and sensors.

[0107] (2) Fixed parameters such as vehicle wheelbase and maximum front wheel steering angle are obtained from factory information or vehicle network.

[0108] (3) Determine the minimum turning radius based on the vehicle's geometric parameters. The calculation formula is:

[0109] ;

[0110] in, Indicates the maximum front wheel steering angle of the vehicle. This indicates the vehicle's wheelbase.

[0111] (4) Determine the maximum feasible radius.

[0112] 1) Determine the safe turning distance of the vehicle based on the drivable area around the vehicle.

[0113] Vehicle safe turning distance The calculation formula is:

[0114] ;

[0115] in, This represents the circumferential depth that can be reached in all directions around the vehicle.

[0116] 2) The maximum feasible radius is determined by the sum of the vehicle's safe turning distance and the distance between the parking space and the vehicle.

[0117] Maximum feasible radius The calculation formula is:

[0118] ;

[0119] in, For safe turning distance of the vehicle, This indicates the distance between the parking space and the vehicle.

[0120] like Figure 1 As shown, in step S2, the perception module extracts and fuses features from the bird's-eye view environment representation and vehicle state information to obtain a joint state representation.

[0121] like Figure 3 , Figure 4 and Figure 5 As shown, this embodiment constructs an adaptive parking model based on hierarchical reinforcement learning and kinematic constraints, which mainly includes a data acquisition module, a joint state representation acquisition module, a perception module, a high-level policy network, and a low-level policy network.

[0122] S2-1. Construct a perception module based on an improved neural network.

[0123] like Figure 3 As shown, in this embodiment, the perception module uses an improved ResNet-50 as the backbone network for feature extraction. The perception module includes two branches, the first branch including... The system consists of a convolutional layer, four residual blocks, a global average pooling layer, and a first fully connected layer. Each of the four residual blocks comprises sequentially connected... Convolutional layers Spatial convolutional layer The convolutional layers each employ Batch Normalization and ReLU activation functions; the second branch includes a second fully connected layer.

[0124] S2-2. The perception module extracts and fuses features from the bird's-eye view environment representation and vehicle status information.

[0125] like Figure 3 As shown, the specific process of feature extraction and fusion is as follows:

[0126] (1) Input the bird's-eye view environment representation into the first branch for feature extraction to obtain the bird's-eye view feature vector.

[0127] 1) The first feature map is obtained by extracting features from the bird's-eye view environment representation through the first convolutional layer.

[0128] Input the bird's-eye view environment representation into a A convolutional layer (96 channels, stride = 2) generates a The feature map, i.e., the first feature map.

[0129] 2) The first feature map is semantically enhanced by four residual blocks to obtain the enhanced feature map.

[0130] Semantic abstraction is progressively enhanced through four residual blocks. Each residual block employs a bottleneck structure, first passing through one... The convolutional layer performs dimensionality reduction, and then passes through a... Spatial convolutional layer and one The convolutional layers are dimensionally increased with a channel expansion ratio of 4:1 and employ skip connections, passing through four residual blocks for enhancement to mitigate the vanishing gradient problem. The output channels of the residual blocks are 64, 128, 256, and 512, respectively. Furthermore, each residual block is processed using Batch Normalization and ReLU activation functions.

[0131] 3) The enhanced feature map is compressed and transformed by a global average pooling layer and a first fully connected layer to obtain the bird's-eye view feature vector.

[0132] The enhanced feature map is first compressed into a 512-dimensional vector by a global average pooling layer (GAP), and then converted into a 256-dimensional intermediate representation, i.e., the bird's-eye view feature vector, through a fully connected layer.

[0133] (2) Convert the vehicle state information into an ontology measurement vector and input it into the second branch for feature extraction to obtain the ontology embedding vector.

[0134] Based on vehicle state information, throttle, steering, linear / angular velocity, and position are converted into six-dimensional body measurement vectors. Based on vehicle state information, environmental perception is achieved using the Velodyne VLP-16 LiDAR sensor, while motion data is obtained through an onboard odometer and inertial measurement unit (IMU). The collected throttle, steering, linear / angular velocity, and position information are normalized to ensure that all input information is within the same scale range. Subsequently, this processed state information is combined into a six-dimensional body measurement vector. It is then projected through another fully connected layer into a 256-dimensional embedding vector, namely the ontology embedding vector.

[0135] (3) The bird's-eye view feature vector and the ontology embedding vector are concatenated to obtain the joint state representation.

[0136] The feature vectors of the bird's-eye view and the ontology embedding vectors are concatenated to form a 512-dimensional joint state representation. This is fed as input into the Actor and Critic networks.

[0137] like Figure 1 , Figure 4 As shown, in step S3, the joint state representation is input into the trained high-level policy network to generate a high-level sub-target instruction.

[0138] like Figure 4 , Figure 5 As shown, both the high-level and low-level policy networks include Actor networks and Critic networks, which jointly optimize the decision-making process. A unified Actor-Critic architecture is used to achieve collaborative optimization of high-level and low-level policies, optimizing sub-objective planning and continuous control. Specifically, the high-level policy Actor network outputs the probability distribution of sub-objective selection based on the current environmental characteristics, while the Critic network outputs the long-term value estimate of the corresponding sub-objective, used to evaluate the contribution of the sub-objective to the overall parking task. Subsequently, the low-level policy Actor network outputs continuous action vectors, and the Critic network outputs the short-term value estimate under the current sub-objective, used to guide the optimization of low-level fine-grained control.

[0139] like Figure 4 As shown, the joint state representation is input into the trained high-level policy network to generate a high-level sub-objective instruction. The specific process is as follows:

[0140] S3-1. Using the Actor network in the high-level policy network, the joint state representation is used to generate a two-dimensional logits vector, which is then normalized to obtain the probability distribution of the high-level policy sub-objectives.

[0141] The probability distribution of sub-targets in the high-level strategy is given by the following formula:

[0142] ;

[0143] in, It is a high-level strategy to select sub-objectives The probability of; It is a sub-target preference score calculated through a convolutional neural network (CNN); Represented as a joint state; It is a kinematically feasible set; It is a binary mask function used to filter out infeasible sub-targets; Represents the sub-objectives computed through a neural network. Preference assessment; Indicates the restriction and filtering of sub-target candidates A set of.

[0144] S3-2. Based on joint state representation, a state value function is constructed through the Critic network in the high-level policy network, and sub-objectives are selected according to the state value function.

[0145] Constructing state value functions The formula is:

[0146] ;

[0147] in, Indicates the state The state value below Represents the expected value, calculated in state Cumulative rewards earned after the start; It is a discount factor that controls the influence of future rewards; In time step The immediate reward at that time; T is the task's end time.

[0148] According to the state value function To select parking sub-target The state value function Used to calculate the advantage function of high-level strategies and guide strategy optimization.

[0149] S3-3. Based on the reward mechanism, a high-level policy optimization loss function is constructed using the near-end policy optimization algorithm to optimize the high-level policy network.

[0150] A high-level policy optimization loss function is constructed using a near-end policy optimization algorithm, wherein the high-level policy optimization loss function includes the high-level policy loss. High-level strategy value loss and high-level policy entropy regularization The formula is:

[0151] 。

[0152] (1) Constructing high-level strategy loss .

[0153] Calculate the high-level strategy advantage function based on the state-value function. The formula is:

[0154] ;

[0155] in, Indicates time step The higher-level reward function below, Indicates time step The state-value function under the following conditions Indicates time step The state-value function under +1, This represents the discount factor.

[0156] Calculate the sampling ratio of the importance of the high-level strategy based on the probability distribution of the sub-objectives of the high-level strategy. The formula is:

[0157] ;

[0158] in, For the joint state representation, Indicates the current strategy. Indicates the old strategy, These represent the current state. Next, select sub-targets using the new and old strategies. The probability of.

[0159] High-level strategy losses The calculation formula is:

[0160] ;

[0161] in, The loss represents the high-level policy loss, i.e., the optimization objective of the high-level policy, expressed as the loss at time step [missing information]. The objective function is as follows; It is the expected value calculation, which represents the cumulative loss over multiple time steps; It is the high-level strategy importance sampling ratio, which measures the ratio between new and old strategies and is used to regulate the stability of the learning process; It is the high-level strategy advantage function, reflecting the advantage of the current strategy's chosen sub-objectives relative to the benchmark strategy; It is the pruning threshold of the near-end policy optimization algorithm to prevent excessive changes during updates and maintain training stability.

[0162] (2) Constructing high-level strategic value loss .

[0163] The high-level strategy value loss is represented as:

[0164] ;

[0165] in, This indicates minimizing the current state. Indicates a future state. It is an expected value calculation. Indicates at time step At that time, state The state-value function under the following conditions Indicates at time step At that time, state The state-value function under the following conditions This indicates a time step. Instant rewards on the platform.

[0166] The goal of high-level strategy value loss is to minimize the current state. and future state The mean squared error between the value functions is used to improve the accuracy of predicting future rewards.

[0167] (3) Provide reward guidance to high-level strategy networks.

[0168] High-level reward function Aimed at encouraging the feasibility and smoothness of sub-objective selection, it is defined as:

[0169] ;

[0170] in, For indicator functions; This represents the reward for generating sub-goals at a higher level. , Current state To sub-target Euclidean distance, To maximize the possible distance, encourage approaching sub-targets; When the state satisfies the kinematic feasible set Timely rewards promote safe decision-making; This indicates a sudden change in course as a form of punishment. To ensure a smooth path.

[0171] The design of this high-level reward function balances exploration efficiency with constraint compliance, avoiding unstable convergence.

[0172] like Figure 1 , Figure 6As shown, in step S4, the kinematic feasibility of the high-level sub-target command is verified by the adaptive steering boundary module, and the verified sub-target is obtained.

[0173] To mitigate the risk of generating suboptimal sub-objectives in high-level strategies, an adaptive steering boundary module is introduced and integrated into the high-level strategy. Based on kinematic modeling, the timing of reversing switching is dynamically optimized, and redundant path exploration is minimized by evaluating the real-time steering geometry constraints of the autonomous vehicle.

[0174] like Figure 6 As shown, the specific process is as follows:

[0175] (1) Determine the center position of the vehicle body based on the vehicle status information.

[0176] Specifically, firstly, the vehicle's steering boundary coordinates are calculated using the vehicle's geometric center and wheelbase length.

[0177] The center position of the vehicle body The calculation formula is:

[0178] ;

[0179] in, This indicates the current coordinates of the vehicle's rear axle center. Indicates the vehicle's heading angle. This indicates the vehicle's wheelbase. It determines the position of the vehicle's center of gravity.

[0180] (2) Determine the coordinates of the parking target based on the center position of the vehicle body.

[0181] To ensure coordinate consistency, the adaptive steering boundary module uses the coordinate representation of the parking target to define the coordinates of the parking target. In the simulation engine, a specific parking space at the end of the parking process is obtained.

[0182] (3) Based on the coordinates of the parking target, kinematic constraints are applied to the high-level sub-target instructions through kinematic modeling to generate kinematically constrained sub-targets.

[0183] Based on the coordinates of the parking target, kinematic modeling is performed, and the formula is:

[0184] ;

[0185] in, This refers to the vehicle's wheelbase. The vehicle's heading angle. This is the distance between the rear axle of the vehicle and the target parking space.

[0186] To ensure the vehicle can complete effective parking, the adaptive steering boundary module generates kinematic constraint sub-objectives and dynamically adjusts the feasibility of the parking area, using the following formula:

[0187] ;

[0188] in, It is the set of kinematic constraints that a vehicle can achieve during parking. When the adaptive steering boundary value is less than or equal to zero, it means that the vehicle can successfully park in the target parking space.

[0189] (4) In the adaptive steering boundary module, the optimal shift boundary is calculated based on the vehicle's minimum turning radius and wheelbase to ensure that the sub-objectives satisfy the kinematically feasible set.

[0190] Based on the vehicle's minimum turning radius and vehicle wheelbase The formula for calculating the optimal shift boundary is:

[0191] Ensure that the sub-objectives satisfy the kinematic feasible set .

[0192] The adaptive steering boundary module calculates feasible regions, i.e., validated sub-objectives, to guide higher-level strategies in selecting the optimal sub-objectives and prevent the vehicle from entering infeasible regions.

[0193] like Figure 1 , Figure 5 As shown, in step S5, the verified sub-objective and joint state representation are input into the trained low-level policy network to generate continuous vehicle control actions.

[0194] S5-1. In the low-level policy network, a safe zone is defined during the vehicle parking process through a parking feasibility module based on kinematic constraints.

[0195] A kinematically constrained parking feasibility module is used to systematically delineate the safe zone during the parking process of autonomous vehicles. The module's phased optimization includes initial alignment, arc maneuvering, and final adjustment to improve adaptability to dynamic environments. For example... Figure 7 As shown, the parking feasibility module based on kinematic constraints controls the vehicle's trajectory during the parking process, ensuring that the vehicle can safely perform parking operations in three spatial areas.

[0196] Based on minimum turning radius and maximum feasible radius The parking feasibility module divides the feasible parking area into three sub-areas. ,in Corresponding area and ,area Used as an initial safety buffer zone for alignment before entering the parking space; area This area serves as a critical zone for perpendicular parking, responsible for adjusting the vehicle's body and heading; This final adjustment area is used to ensure that vehicles are parked precisely within the parking space.

[0197] The boundary of each sub-region is represented by the following formula:

[0198] ;

[0199] in, These are the coordinates of the target parking space. It is the minimum safety margin.

[0200] For the region The final adjustment zone, the vehicle boundary calculation formula is:

[0201] .

[0202] Each sub-region Defined as the set of all points within a region that satisfy the boundary conditions, and the feasible region within that set. The formula for the union of these subregions is:

[0203] ;

[0204] ;

[0205] in, and These respectively indicate that the area is in Minimum and maximum boundaries on the axis; and These respectively indicate that the area is in Minimum and maximum boundaries on the axis.

[0206] S5-2. Based on the safe area during vehicle parking, the verified sub-objectives and joint state representations are input into the Critic network of the low-level policy network to construct the joint state-sub-objective value function.

[0207] The corresponding Critic head estimates the joint state-sub-objective value function. This is to facilitate time-series credit allocation during low-level strategy optimization.

[0208] Joint state-sub-objective value function The formula is:

[0209] ;

[0210] in, Indicates at time step Select the sub-targets below; Indicates at time step The combined state includes throttle, steering, braking, and speed information; γ is the discount factor as above. In time step The immediate reward represents the return under the current state-sub-goal pair; This is the time step for the task to terminate. This indicates the expected operator.

[0211] S5-3. Based on the joint state-sub-objective value function, the verified sub-objective and joint state representations are input into the Actor network of the low-level policy network to generate sub-policy control vectors, which control continuous longitudinal and steering actions.

[0212] For low-level strategies, joint state representation and selected sub-targets Input is fed into the Actor branch, generating a control vector. The control vector is normalized using Tanh activation to control continuous longitudinal and steering movements.

[0213] Low-level policy modeling is based on Gaussian distribution. Sub-policies share network parameters but use different reward functions.

[0214] Specifically, the control vectors for the low-level policies are defined as follows:

[0215] ;

[0216] in, and These are the average actions of the forward search and reverse maneuver sub-strategies, respectively. Variance is used to control the flexibility and adaptability of the system during parking. This represents a sub-target of the vehicle's search for the target parking space; This represents the sub-target of parking.

[0217] S5-4. Based on the reward mechanism, a low-level policy optimization loss function is constructed using the near-end policy optimization algorithm, and the low-level policy network is trained and optimized.

[0218] In the optimization of low-level strategies, the goal is to adjust the control strategy by maximizing the expected return. The policy update is optimized by minimizing the policy loss function, and a pruning reward mechanism using the PPO algorithm is employed to stabilize the training process.

[0219] The process of constructing a low-level policy optimization loss function is as follows:

[0220] (1) Calculate the sampling ratio of low-level policy importance based on the low-level sub-policy control vector. The formula is:

[0221] ;

[0222] in, and These represent the low-level policy values ​​for the new and old policies, respectively.

[0223] (2) Calculate the low-level policy advantage function based on the joint state-sub-objective value function.

[0224] In the optimization of low-level strategies, low-level strategies Through the current reward and future rewards Discount value and current state value function The difference is calculated by introducing a discount factor. The impact of future rewards is taken into account, which helps to optimize decision-making strategies, enabling lower-level strategies to gradually approach the optimal control strategy.

[0225] The formula for calculating the advantage function of low-level strategies is:

[0226] ;

[0227] in, Rewards indicating environmental feedback; This represents the discount factor, used to account for future rewards; This indicates the low-level policy at the current moment. The value function; This indicates the lower-level strategy in the next time step. The value function of .

[0228] (3) Based on the sampling ratio of low-level policy importance and the low-level policy advantage function, construct the optimization objective of low-level policy.

[0229] ;

[0230] in, The sampling ratio for the importance of low-level strategies; This is the advantage function for low-level strategies. A cutoff threshold is set to ensure the stability of the training process; It is the expected value operation, which represents the cumulative loss over multiple time steps.

[0231] Low-level strategies are based on the current environment state. and selected sub-targets Execute continuous control actions.

[0232] (4) Provide reward guidance for low-level strategies.

[0233] Low-level reward function It consists of two parts: the forward sub-strategy reward. Reverse strategy rewards The strategy is dynamically adjusted based on sub-strategies, supporting multi-objective optimization, as shown in the formula:

[0234] .

[0235] 1) Rewards for the forward sub-strategy The formula is:

[0236] ;

[0237] in, This represents the first reward function, used to quantify the deviation between the forward search and the target parking space; This represents the collision reward function, used to penalize collisions. This represents a reward function that encourages autonomous vehicles to maintain vertical alignment with the target parking space. This represents the reward function based on smoothing operations.

[0238] A reward function that encourages autonomous vehicles to maintain vertical alignment with the target parking space. The formula is:

[0239] ;

[0240] in, Set it to be perpendicular to the direction of the target parking space. Indicates the current vehicle heading angle. In the direction of the target parking space, Control the rate at which rewards decay.

[0241] Reward function based on smoothing operation The formula is:

[0242] ;

[0243] in, For time steps Smooth steering angle, Time step A smooth steering angle of 1.

[0244] The function is designed as a linear trigonometric function to encourage autonomous vehicles to operate smoothly within the maximum angular velocity limit, and to penalize frequent heading oscillations, thereby promoting efficient forward search and reducing redundant actions.

[0245] 2) For the reversing strategy reward, the lower-level reward integrates the parking feasibility module constraint, and the formula is:

[0246] ;

[0247] in, This represents the second reward function, used to quantify the deviation between the forward search and the target parking space; This represents the second collision reward function, used to penalize collisions. This indicates the angular deviation between the vehicle and the target parking space; This represents the reward function based on the parking feasibility module.

[0248] Reward function based on parking feasibility module Defined as:

[0249] ;

[0250] in, This indicates the area occupied by autonomous vehicles. This represents the entire feasible parking area. The reward function encourages vehicle trajectories to be executed within the practically feasible safe area, thereby improving parking success rate and safety.

[0251] The adaptive parking model based on hierarchical reinforcement learning and kinematic constraints is trained and tested according to steps S1-S5.

[0252] Based on the reward mechanism, a hierarchical reinforcement learning Actor-Critic network framework is used to jointly train and optimize the adaptive parking model based on hierarchical reinforcement learning and kinematic constraints. The specific process is as follows:

[0253] (1) The bird's-eye view of the parking scene and the vehicle status information are processed by the perception module to generate a joint state representation. .

[0254] (2) Receive joint state representation through high-level policy network Output sub-target selection strategy and the corresponding state value .

[0255] (3) The lower-level policy network receives the joint state representation and selected sub-targets Output control vector Corresponding state-sub-target value .

[0256] (4) Construct the total reward through the high-level reward function and the low-level reward function respectively, and provide reward guidance for the high-level strategy and the low-level strategy.

[0257] The total reward formula is:

[0258] ;

[0259] in, This represents the high-level reward function. This represents the low-level reward function.

[0260] After the above training, a well-trained adaptive parking model based on hierarchical reinforcement learning and kinematic constraints is obtained, especially the well-trained high-level policy network and low-level policy network.

[0261] A pre-trained adaptive parking model based on hierarchical reinforcement learning and kinematic constraints was used to conduct comprehensive validation tests in both indoor and outdoor environments, such as... Figure 8 As shown.

[0262] This includes three typical vehicle types: small passenger cars (2.5m wheelbase, 4m minimum turning radius), mid-size SUVs (2.7m wheelbase, 5m minimum turning radius), and large sedans (3.0m wheelbase, 6m minimum turning radius). These vehicle types represent a diverse range of vehicle configurations from compact to full-size to evaluate the cross-platform compatibility and adaptive performance of the method in this embodiment.

[0263] The test scenario design includes two typical layouts: open parking lots and narrow roadside parking spaces. Dynamic obstacles (such as background vehicles) and static disturbance elements are introduced to simulate the complexity of real-world urban parking environments. This method achieves real-time adaptive optimization for different vehicle geometric characteristics by dynamically adjusting the parameters of the adaptive steering boundary module (minimum turning radius Rmin ranging from 3-6m) and the safety margin threshold w of the parking feasibility module, ensuring the kinematic feasibility of sub-objective generation and path execution.

[0264] like Figure 2As shown, in the specific parking process: ① The vehicle is in the forward search phase, exploring the environment through low-level strategy control, gradually approaching the target parking space; ② When the adaptive steering boundary module determines that the vehicle has moved to a suitable reversing posture, the high-level strategy switches the sub-target to preparing for parking, and the vehicle fine-tunes its posture within the P1 area defined by the parking feasibility module; ③ After confirming the target parking space, the high-level strategy formally issues the parking instruction, and the vehicle begins to reverse into the parking space. During this process, the system perceives the appearance of dynamic obstacles in real time, and through the rapid online adjustment of the low-level strategy and the area guidance of the parking feasibility module, it achieves coordinated operation of parking and obstacle avoidance; ④ The vehicle successfully parks in the target parking space, completing all tasks.

[0265] Following steps S1-S5, and through parameter adjustments, this embodiment achieves a highly efficient adaptive mechanism, achieving a parking success rate exceeding 96% in cross-vehicle generalization tests. This result fully validates the cross-platform generalization capability of the proposed framework, demonstrating its applicability to vertical parking tasks across various vehicle models and its ability to maintain high accuracy and robustness even under conditions of significant differences in vehicle size and maneuverability.

[0266] To further verify the robustness and practical deployment potential of this embodiment, targeted tests were conducted in extended dynamic real-world scenarios, including two high-constraint configurations: outdoor open roadside parking and parking in a narrow underground garage. Figure 8 (b) shows an outdoor vacant scene, i.e., an open scene simulating a spacious parking lot environment. The initial distance from the vehicle to the target parking space is 10m, and the speed of dynamic obstacles is controlled within <5km / h; Figure 8 (a) shows a narrow indoor scene and Figure 8 (c) shows an outdoor narrow scenario, simulating an underground parking garage layout with a parking space width of only 2.3m and an initial heading deviation exceeding 30°. Results show that in the open scenario, the parking success rate reaches 99.2%, and the path smoothness (heading change rate <10° / s) reaches 95%; in the narrow scenario, the parking success rate is 96.5%, the collision rate is <1%, and all tests achieve single-shot parking completion without mid-test reset. This embodiment highlights the crucial role of the three-stage adjustment mechanism (initial alignment, arc maneuvering, and final precise positioning) of the adaptive steering boundary and parking feasibility module in highly constrained environments, effectively improving the system's real-time adaptability and fault tolerance.

[0267] Example 2

[0268] The purpose of this embodiment is to provide an adaptive parking system based on hierarchical reinforcement learning and kinematic constraints, including:

[0269] The data acquisition module is used to acquire a bird's-eye view of the parking scene and vehicle status information;

[0270] The joint state representation acquisition module is used to extract and fuse features from the bird's-eye view environment representation and vehicle state information through the perception module to obtain a joint state representation.

[0271] The high-level policy module is used to input the joint state representation into the trained high-level policy network and generate a high-level sub-objective instruction.

[0272] The sub-target verification constraint module is used to verify the kinematic feasibility of high-level sub-target commands through the adaptive steering boundary module, and obtain the verified sub-target.

[0273] The low-level policy module is used to input the verified sub-objectives and joint state representations into the trained low-level policy network to generate continuous vehicle control actions.

[0274] During the training process, a hierarchical reinforcement learning framework is used for joint training optimization based on a reward mechanism.

[0275] The method steps in Example 1 are implemented by providing an adaptive parking system based on hierarchical reinforcement learning and kinematic constraints.

[0276] Example 3

[0277] The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0278] Example 4

[0279] The purpose of this embodiment is to provide a computer-readable storage medium.

[0280] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0281] Example 5

[0282] The purpose of this embodiment is to provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods and functions involved in any of the above embodiments.

[0283] The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0284] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0285] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An adaptive parking method based on hierarchical reinforcement learning and kinematic constraints, characterized in that, include: Obtain a bird's-eye view of the parking scene and vehicle status information; the vehicle status information includes at least the real-time coordinates of the rear axle center, the real-time heading angle, the vehicle wheelbase, the maximum front wheel steering angle, and the minimum turning radius. The perception module extracts and fuses features from the bird's-eye view environment representation and vehicle status information to obtain a joint state representation; The joint state representation is input into the trained high-level policy network to generate high-level sub-objective instructions; The kinematic feasibility of the high-level sub-target command is verified by the adaptive steering boundary module, and the verified sub-target is obtained. The verified sub-objectives and joint state representations are input into the trained low-level policy network to generate continuous vehicle control actions. The specific process is as follows: In the low-level policy network, a parking feasibility module based on kinematic constraints is used to delineate the safe zone during the vehicle parking process; Based on the safe area during vehicle parking, the verified sub-objectives and joint state representations are input into the Critic network of the low-level policy network to construct the joint state-sub-objective value function. Based on the joint state-sub-objective value function, the verified sub-objective and joint state representations are input into the Actor network of the low-level policy network to generate sub-policy control vectors, which control continuous longitudinal and steering actions. Based on the low-level reward mechanism, a low-level policy optimization loss function is constructed using the near-end policy optimization algorithm to train and optimize the low-level policy. The lower-level reward mechanism has the following formula: ； ； ； in, Indicates the reward for the forward sub-strategy; This indicates a reward for the reversing strategy; This represents the first reward function, used to quantify the deviation between the forward search and the target parking space; This represents the collision reward function, used to penalize collisions. This represents a reward function that encourages autonomous vehicles to maintain vertical alignment with the target parking space. This represents the reward function based on smoothing operations; This represents the second reward function, used to quantify the deviation between the forward search and the target parking space; This indicates the angular deviation between the vehicle and the target parking space; This represents the reward function based on the parking feasibility module; During the training process, based on the reward mechanism, the high-level policy network and the low-level policy network are jointly trained using a hierarchical reinforcement learning framework.

2. The adaptive parking method based on hierarchical reinforcement learning and kinematic constraints as described in claim 1, characterized in that, The process of obtaining a bird's-eye view representation of the parking scene is as follows: The sensor acquires 3D point cloud data containing information on obstacle location, vehicle orientation, lane boundaries, and road markings. Based on the vehicle's position and orientation, the 3D point cloud data is processed and mapped onto a 2D plane centered on the vehicle. The mapped data is integrated and encoded to generate a two-dimensional bird's-eye view representation of the parking scene.

3. The adaptive parking method based on hierarchical reinforcement learning and kinematic constraints as described in claim 1, characterized in that, The perception module extracts and fuses features from the bird's-eye view environment representation and vehicle status information. This perception module includes a first convolutional layer, four residual blocks, a global average pooling layer, a first fully connected layer, and a second fully connected layer. The specific process is as follows: The first feature map is obtained by extracting features from the bird's-eye view environment representation through the first convolutional layer; The first feature map is semantically enhanced by four residual blocks to obtain the enhanced feature map. The enhanced feature map is compressed and transformed by a global average pooling layer and a first fully connected layer to obtain the bird's-eye view feature vector. The vehicle state information is converted into an ontology measurement vector, which is then input into the second fully connected layer for feature extraction to obtain the ontology embedding vector. The feature vector of the bird's-eye view and the ontology embedding vector are concatenated to obtain the joint state representation.

4. The adaptive parking method based on hierarchical reinforcement learning and kinematic constraints as described in claim 1, characterized in that, The joint state representation is input into the trained high-level policy network to generate high-level sub-objective instructions. The specific process is as follows: By using the Actor network in the high-level policy network, the joint state representation is generated into a two-dimensional logits vector, which is then normalized to obtain the probability distribution of the high-level policy sub-objectives. Based on joint state representation, a state value function is constructed through the Critic network in the high-level policy network, and sub-objectives are selected according to the state value function. Based on the reward mechanism, a high-level policy optimization loss function is constructed using a near-end policy optimization algorithm to optimize the high-level policy.

5. The adaptive parking method based on hierarchical reinforcement learning and kinematic constraints as described in claim 1, characterized in that, The kinematic feasibility of high-level sub-target commands is verified using an adaptive steering boundary module. The specific process is as follows: In the adaptive steering boundary module, the vehicle's center position is determined based on vehicle state information; Determine the coordinates of the parking target based on the vehicle's center position; Based on the coordinates of the parking target, kinematic constraints are applied to the high-level sub-target commands through kinematic modeling to generate kinematically constrained sub-targets. The optimal shift boundary is calculated based on the vehicle's minimum turning radius and wheelbase to ensure that the sub-objectives satisfy the kinematically feasible set.

6. The adaptive parking method based on hierarchical reinforcement learning and kinematic constraints as described in claim 1, characterized in that, A low-level policy optimization loss function is constructed using a near-end policy optimization algorithm, and the low-level policy is optimized. The specific process is as follows: Based on the low-level sub-policy control vector, calculate the low-level policy importance sampling ratio; Calculate the low-level policy advantage function based on the joint state-sub-objective value function; Based on the sampling ratio of low-level policy importance and the low-level policy advantage function, an optimization objective for low-level policies is constructed, namely the low-level policy optimization loss function.

7. An adaptive parking system based on hierarchical reinforcement learning and kinematic constraints, characterized in that, Implementing the adaptive parking method based on hierarchical reinforcement learning and kinematic constraints as described in any one of claims 1-6, comprising: The data acquisition module is used to acquire a bird's-eye view of the parking scene and vehicle status information; The joint state representation acquisition module is used to extract and fuse features from the bird's-eye view environment representation and vehicle state information through the perception module to obtain a joint state representation. The high-level policy module is used to input the joint state representation into the trained high-level policy network and generate a high-level sub-objective instruction. The sub-target verification constraint module is used to verify the kinematic feasibility of high-level sub-target commands through the adaptive steering boundary module, and obtain the verified sub-target. The low-level policy module is used to input the verified sub-objectives and joint state representations into the trained low-level policy network to generate continuous vehicle control actions. During the training process, based on the reward mechanism, the high-level policy network and the low-level policy network are jointly trained using a hierarchical reinforcement learning framework.

Images