Parking path generation method and electronic equipment

Abstract

The invention provides a parking path generation method and electronic equipment, and relates to the field of auxiliary driving. According to the method, an auxiliary driving module of a vehicle determines k nodes of a parking path based on a first model according to starting position information and environment information of the vehicle, the k nodes comprise a first starting point and a first terminal point, and the first model is obtained based on reinforcement learning training; based on the k nodes, the starting position information of the vehicle and the environment information, a first parking path is generated through a mixed A * algorithm, the heuristic cost of the nth node is in positive correlation with the first distance of the nth node, and the first distance comprises the second distance and the third distance; the second distance is the distance from the nth node to the mth node meeting the condition in the k nodes, and the third distance is the sum of the distances between two adjacent nodes from the mth node to the first end point in the k nodes. By adopting the method, the rationality of the parking path can be improved.

Description

Technical Field

[0001] This application relates to the field of driver assistance technology, and in particular to a method for generating parking paths and an electronic device. Background Technology

[0002] In recent years, with the rapid advancement of artificial intelligence technology, its applications have gradually permeated all walks of life, bringing various conveniences to our lives. The rapid development of driver assistance systems, in particular, has triggered significant changes in people's travel patterns, lifestyles, and urban planning. Automatic Parking Assist (APA) systems use ultrasonic radar, cameras, and onboard computers to achieve environmental perception and parking path planning, automatically controlling the steering wheel, accelerator, and brakes to park the vehicle in a parking space, greatly reducing the driver's workload and the parking accident rate. Therefore, as an important function of automotive driver assistance, automatic parking assist is receiving increasing attention.

[0003] The hybrid A* algorithm is a commonly used method for parking path generation. Building upon the traditional A* algorithm, it introduces the vehicle's heading angle as a dimension, calculating the final parking path based on a cost function. However, parking paths generated using this method suffer from problems such as path complexity and a tendency to get stuck in dead ends. Summary of the Invention

[0004] In view of this, this application provides a parking path generation method and electronic device to improve the rationality of parking paths.

[0005] Firstly, this application provides a method for generating parking paths, which, exemplarily, can be applied to a vehicle's driver assistance module. The method includes the following processes:

[0006] Based on the first model, k nodes of the vehicle's parking path are determined according to the vehicle's starting position information and environmental information. The first node of these k nodes is the vehicle's first starting point, and the kth node is the vehicle's first ending point. The starting position information includes the vehicle's first starting point, and the environmental information includes the vehicle's target parking space. k is a positive integer greater than 2. The first model here is obtained based on reinforcement learning training.

[0007] Based on the starting position information of k nodes and the vehicle, as well as environmental information, a hybrid A* algorithm is used to generate the first parking path of the vehicle. The first parking path includes q nodes, the first of which is the first starting point. The heuristic cost of any nth node among these q nodes is positively correlated with the first distance of the nth node. The first distance of the nth node includes the second and third distances of the nth node. The second distance of the nth node is the distance between the nth node and the mth node among the k nodes that satisfies the first condition. The third distance of the nth node is the sum of the distances between any two adjacent nodes from the mth node to the first ending point among the k nodes. Here, q is a positive integer, n is a positive integer (n is less than or equal to q), and m is a positive integer (m is less than or equal to k).

[0008] The first condition mentioned above includes:

[0009] From k nodes, select r nodes whose distances to the first n nodes of q nodes are all greater than or equal to a set first distance threshold. From the r nodes, select the node with the shortest distance to the nth node as the mth node, where r is a positive integer less than or equal to k.

[0010] In a second aspect, this application provides an electronic device including a memory and a processor, the memory being coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, and one or more processors calling the computer instructions to cause the electronic device to perform the steps of the method as described in the first aspect.

[0011] Thirdly, this application provides a readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the steps of the method described in the first aspect.

[0012] Fourthly, this application provides a program product including instructions that, when executed on an electronic device, cause the electronic device to perform the steps of the method as described in the first aspect.

[0013] Fifthly, this application provides a chip including a processor coupled to a memory, the processor being configured to execute a computer program or instructions stored in the memory, such that the chip implements the steps of the method described in the first aspect.

[0014] In this embodiment, based on the first model, k nodes of the vehicle's parking path are determined according to the vehicle's starting position information and environmental information. These k nodes include a first starting point and a first ending point. When generating the first parking path using the hybrid A* algorithm, the heuristic cost of the nth node is positively correlated with a first distance. This first distance includes a second distance and a third distance. The second distance is the distance between the nth node and the mth node among the k nodes that meets the conditions. The third distance is the sum of the distances between any two adjacent nodes from the mth node to the first ending point among the k nodes. Since the first model is generated based on reinforcement learning training, the parking path represented by the k nodes determined by this first model is an optimized parking path that is less likely to get stuck in a dead end. Therefore, the heuristic cost guides the path search tree of the hybrid A* algorithm to grow towards the k nodes, ultimately reaching the target parking space, making the generated first parking path more reasonable.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0016] Figure 1A This is a schematic diagram of node expansion in the traditional A* algorithm;

[0017] Figure 1B This is a schematic diagram of node expansion in the hybrid A* algorithm;

[0018] Figure 1C A schematic diagram illustrating the generation of parking paths using the hybrid A* algorithm;

[0019] Figure 2 A schematic diagram of the process for generating parking paths provided in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the model training process provided in the embodiments of this application. Detailed Implementation

[0021] To ensure clarity and conciseness in the description of the following embodiments, the terminology used in the embodiments of this application will first be explained. It should be understood that this explanation is for the purpose of better understanding the embodiments of this application and does not necessarily constitute a limitation on the embodiments of this application.

[0022] Advanced Driver Assistance Systems (ADAS), also known as autonomous driving or intelligent driving, is a general term for technologies that use onboard sensors, cameras, radar, and other devices to perceive the environment and assist the driver in completing some driving tasks. It includes functions such as lane keeping, automatic parking, and adaptive cruise control.

[0023] Automatic Parking Assist (APA) is a sensor- and algorithm-based vehicle driver assistance system, belonging to the category of Advanced Driving Assistance Systems (ADAS). It uses ultrasonic radar, cameras, and onboard computers to achieve environmental perception and parking path planning, supporting parallel parking, perpendicular parking, and angled parking space recognition. By automatically controlling the steering wheel, accelerator, and brakes, it parks the vehicle into the parking space, greatly reducing the driver's workload and parking accident rate.

[0024] Reinforcement learning (RL) is a machine learning method that allows an agent to learn optimal policies through trial and error in its interactions with the environment. The agent performs actions in the environment and receives rewards based on the outcomes of those actions. At the heart of reinforcement learning is the value function, which estimates the expected cumulative reward achievable by starting from a given state and following a specific policy. These rewards guide the agent in adjusting its policy, enabling it to learn to distinguish which states and actions are more likely to yield high rewards, thus making better decisions.

[0025] Software-in-the-Loop (SiL) is a testing or training method that uses a simulation environment to execute and verify control algorithms or code without physical hardware. Engineers embed software into a virtual model simulating a real control system and then input it with inputs representing various operating conditions. This setup reduces reliance on physical hardware early on and more effectively identifies software logic flaws or performance limitations. The simplified process often accelerates feedback loops and shortens development cycles. In the field of driver assistance systems, SiL testing simulates various elements of real-vehicle testing in a simulation environment, enabling the algorithm to operate in a closed loop. This includes using simulation software to simulate various test elements, interacting with the algorithm through middleware to provide a runtime environment; using automated testing, evaluation, and management modules to automate testing; and using a scenario library for SiL testing that contains various test scenarios, fully covering various operating conditions of real vehicles.

[0026] Euclidean distance, also known as Euclidean distance, is one of the most common and intuitive distance metrics, originating from the concept of the straight-line distance between two points in geometry. It is a commonly used definition of distance, referring to the true distance between two points in multidimensional space, or the natural length of a vector, i.e., the distance from that point to the origin. In two-dimensional and three-dimensional space, Euclidean distance is the actual distance between two points. It is widely used in mathematics, physics, and computer science (such as machine learning, data mining, and image processing) to measure the "distance" between two points in space.

[0027] The Reeds-Shepp (RS) curve distance, also known as the RS distance, is the distance between two points along the RS curve. The RS curve is used for path planning, taking into account the vehicle's heading and forward / reverse motion, providing an optimal path from the starting point to the destination that conforms to the vehicle's kinematic constraints. In driver assistance systems, the RS curve is used to generate the optimal path for the vehicle within a limited space, which is particularly important for scenarios such as parking, obstacle avoidance, and navigation in complex environments.

[0028] Polar grid representation uses the pole as the center and angles (θ) and radii (r) to describe the location of data, which differs from the "horizontal and vertical axes" mode of the traditional Cartesian coordinate system. It is particularly suitable for displaying periodic and directional data, and can intuitively present the pattern of data changes with angle, as well as the numerical differences in different directions.

[0029] The Robot Operating System (ROS) is a middleware platform that provides frameworks, tools, and libraries for robot software development. It offers hardware abstraction, low-level device control, common services, message passing mechanisms, and software package management for robot applications, aiming to simplify the construction process of complex robot systems. ROS is widely used in fields such as assisted driving and unmanned aerial vehicles (UAVs).

[0030] Proximal Policy Optimization (PPO) is an important method in reinforcement learning, widely used for its stability and efficiency in complex tasks. PPO avoids instability caused by excessive parameter adjustments by limiting the magnitude of policy updates, while simultaneously improving sample utilization. Due to its simplicity, stability, and efficiency, PPO has become a mainstream algorithm in reinforcement learning. It is suitable for complex environments and large-scale tasks, such as language model optimization. Through pruning mechanisms and dominance functions, PPO achieves a good balance between performance and stability.

[0031] Generalized Advantage Estimation (GAE) is an efficient method for estimating the advantage function in reinforcement learning and is the core theoretical foundation of Policy Optimization (PPO). It solves the credit assignment problem in reinforcement learning by balancing bias and variance, i.e., how to accurately determine the contribution of historical actions to delayed rewards. GAE is based on the ideas of eligibility traces and temporal-difference learning with eligibility traces (TD-λ), employing a multi-step advantage exponential weighted averaging method to effectively integrate the advantage function into the loss function, providing a stable gradient signal for policy optimization. Compared to TD-λ, GAE is more suitable for modern policy gradient methods, driving the development of efficient reinforcement learning algorithms.

[0032] Hyperparameters are parameters that need to be manually set before the algorithm runs in machine learning or reinforcement learning. They control the model's behavior and performance, significantly impacting its training speed, convergence, capacity, and generalization ability. Unlike model parameters, which are learned automatically from training data (e.g., weights and biases in a neural network), hyperparameters need to be tuned through experience or trial-and-error methods. Hyperparameter selection is typically a trial-and-error process, combining experience and domain knowledge. To optimize model performance, methods such as grid search or random search can be used for hyperparameter tuning.

[0033] The Rectified Linear Unit (ReLU) activation function is a widely used non-linear activation function in deep learning. The ReLU function sets the less than 0 parts of the input to 0, while keeping the positive parts unchanged. This property makes it effective in mitigating the vanishing gradient problem in deep networks.

[0034] The hyperbolic tangent (Tanh) activation function is a classic activation function widely used in early neural networks and recurrent neural networks (RNNs). The Tanh activation function normalizes input values ​​to between -1 and 1, possesses central symmetry, and maps negative inputs to negative outputs, making it ideal for tasks requiring sensitivity to negative inputs.

[0035] The standard Mean Squared Error (MSE) function is a method used to measure the difference between predicted and actual values. It standardizes the MSE, making errors comparable across datasets of different sizes.

[0036] Policy entropy, a metric measuring the uncertainty of a model when choosing actions, is crucial in reinforcement learning, impacting its exploration capabilities and performance. Policy entropy represents the uncertainty of the probability distribution of different actions a policy chooses given a state. High entropy indicates greater uncertainty in action selection, meaning the model is exploring various possibilities; low entropy, on the other hand, suggests a more certain policy, potentially leading the model to stop exploring new solutions. Policy entropy plays a vital role in reinforcement learning, influencing the model's exploration capabilities and final performance. Understanding and controlling changes in policy entropy is essential for improving the performance of large models on complex tasks. Appropriate policy adjustments can effectively prevent entropy collapse, promoting continuous learning and optimization of the model.

[0037] The Hybrid A* Algorithm is a path planning algorithm that combines the traditional A* algorithm with continuous state-space motion planning methods to solve path planning problems for vehicles with nonholonomic constraints. Despite its complexity and high computational resource requirements, it holds great potential for applications in fields such as assisted driving vehicles and drones.

[0038] The following is combined Figure 1A , Figure 1B and Figure 1C This paper provides a detailed introduction to the hybrid A* algorithm.

[0039] like Figure 1A As shown, the extended child nodes of the traditional A* algorithm are the center points of the eight adjacent discrete grids of the current vehicle, thus failing to express the vehicle's orientation and position in continuous space.

[0040] like Figure 1B As shown, the hybrid A* algorithm introduces the vehicle's heading angle as a dimension. When expanding child nodes, it considers the vehicle's motion characteristics, generating trajectories that better match these characteristics. The typical approach to node expansion in the hybrid A* algorithm is to start from the current node, sample the vehicle's trajectory from both the forward and backward directions, taking into account a pre-set step size and the vehicle's maximum turning radius.

[0041] When expanding child nodes, the hybrid A* algorithm determines the expanded child nodes of the current node by calculating the cost. The total cost satisfies the following formula:

[0042] f(n) = g(n) + h(n);

[0043] Where f(n) is the total cost of the current nth node, g(n) is the actual cost of the current nth node, and h(n) is the heuristic cost of the current nth node.

[0044] Heuristic cost is used to guide the growth of the search tree in the hybrid A* algorithm toward the target parking space. The heuristic cost can take into account the vehicle's current heading angle and turning radius to ensure the vehicle maintains the correct attitude as it approaches the target parking space. Typically, the shortest distance between two points is calculated using an RS curve to obtain the closest distance from the current node to the target point, which is then used as the heuristic cost. The heuristic cost can also be calculated based on the shortest path distance from the current point to the target point, for example, using Euclidean distance.

[0045] The actual cost is the cumulative cost from the vehicle's starting point to the current nth node. It measures the actual consumption of the path that has been explored so far. The actual cost is usually the distance from the vehicle's starting point to the current nth node.

[0046] like Figure 1C As shown, the hybrid A* algorithm, when generating a parking path for a vehicle, starts from the vehicle's starting point, determines the vehicle's optional extended child nodes based on a pre-set step size and considering the vehicle's heading angle. By calculating the total cost, the child node with the lowest cost is selected as the extended child node of the current node, and then the process continues to extend to the next child node until the final target parking space is reached, forming the final parking trajectory.

[0047] However, when a vehicle enters a narrow area surrounded by obstacles (such as...) Figure 1C When encountering a dead end (as shown in the example), the heuristic cost h(n) still indicates "very close to the target parking space". Therefore, the hybrid A* algorithm will repeatedly explore this local trap, expanding a large number of nodes until the actual cost g(n) becomes very large, causing the total cost h(n) to exceed other branch paths, before "giving up" and backtracking. Therefore, the generated path is very complex and prone to getting stuck in dead ends.

[0048] In view of this, this application provides a parking path generation method and electronic device to improve the rationality of parking paths. The method provided in the embodiments of this application can be applied to the driver assistance module of a vehicle.

[0049] Next, combined Figure 2 and Figure 3 This application provides a detailed description of a parking path generation method based on its embodiments.

[0050] like Figure 2 As shown in the figure, this application proposes a method for generating parking paths, which can be used in the vehicle's driver assistance module. The method includes the following steps:

[0051] S101, based on the first model, determines k nodes of the parking path, including the start and end points, according to the vehicle's starting position information and environmental information.

[0052] Based on the first model, k nodes of the vehicle's parking path are determined according to the vehicle's starting position information and environmental information. The first node of these k nodes is the vehicle's first starting point, and the kth node is the vehicle's first ending point. The starting position information includes the vehicle's first starting point, and the environmental information includes the vehicle's target parking space. k is a positive integer greater than 2. The first model here is obtained based on reinforcement learning training.

[0053] The first model here is trained using reinforcement learning, which is based on a value function that estimates the expected cumulative reward achievable by starting from a given state and following a specific policy. These rewards guide the model to adjust its policy, allowing it to learn to distinguish which states and actions are more likely to yield high rewards, thus making better decisions. Understandably, the parking paths represented by the k nodes generated based on the first model are optimized and typically avoid dead ends.

[0054] S102, based on the starting position information of k nodes, the vehicle, and environmental information, the first parking path of the vehicle is generated using a hybrid A* algorithm.

[0055] Based on the starting position information of k nodes, the vehicle, and environmental information, a hybrid A* algorithm is used to generate the vehicle's first parking path. The first parking path includes q nodes, with the first node among the q nodes being the aforementioned first starting point. The heuristic cost of any nth node among these q nodes is positively correlated with the first distance of the nth node. The first distance of the nth node includes the second and third distances of the nth node. The second distance of the nth node is the distance between the nth node and the mth node among the k nodes that satisfies the first condition. The third distance of the nth node is the sum of the distances between any two adjacent nodes from the mth node to the first ending point among the k nodes. Here, q is a positive integer, n is a positive integer (n < q), and m is a positive integer (m < k).

[0056] The first condition mentioned above includes:

[0057] From k nodes, select r nodes whose distances to the first n nodes of q nodes are all greater than or equal to a set first distance threshold. From the r nodes, select the node with the shortest distance to the nth node as the mth node, where r is a positive integer less than or equal to k.

[0058] Using a hybrid A* algorithm, the search begins from the vehicle's initial starting point. Based on the heuristic cost, extended child nodes are determined, and the search continues until the target parking space is reached. When searching for extended child nodes from the initial starting point, the m-th node that satisfies the conditions is usually the second node among the k nodes. The heuristic cost of the initial starting point is positively correlated with the first distance, which includes the second distance from the initial starting point to the second node among the k nodes, and the third distance. The third distance is the sum of the distances between any two adjacent nodes from the second node to the first destination among the k nodes.

[0059] When searching for the extended child nodes of the nth node, the heuristic cost is positively correlated with the first distance, which includes the second distance and the third distance between the nth node and the mth node among the k nodes that meet the conditions. That is, among the k nodes, the sum of the distances between any two adjacent nodes from the mth node to the first endpoint.

[0060] When the first parking path has already passed some of the k nodes, we first select r nodes that the first parking path has not yet passed from the k nodes, and then select the m-th node that is closest to it to guide the growth of the hybrid A* algorithm's path search tree. Here, a first distance threshold is set; when the distance between nodes is greater than or equal to this first distance threshold, it is determined that the path has not yet been passed.

[0061] When all nodes except the first destination in the k nodes have been visited by the first parking path, the m-th node that meets the condition is the k-th node in the k nodes, which is the first destination. The second distance is the distance from the n-th node to the first destination, and the third distance is 0. This guides the path search tree to grow towards the first destination, and finally generates the complete first parking path.

[0062] Based on the above method, when using the hybrid A* algorithm to generate the first parking path, the path search tree of the hybrid A* algorithm is guided to grow towards k nodes, eventually reaching the target parking space.

[0063] Since the first model is generated based on reinforcement learning training, the parking path represented by the k nodes determined by the first model is an optimized parking path that is less likely to get stuck in a dead end. Therefore, the k nodes are used to guide the hybrid A* algorithm, making the generated first parking path more reasonable.

[0064] In some implementations, in step S102 above, the first distance of the nth node can be the sum of the weighted second and third distances of the nth node. This allows for consideration of different complex environments when determining the first distance, making the calculation result of the heuristic cost more reasonable. Alternatively, the first distance of the nth node can be equal to the sum of the second and third distances of the nth node; that is, the second and third distances are directly added to obtain the first distance, and the heuristic cost of the nth node is calculated accordingly. This method further simplifies the calculation and improves path search efficiency.

[0065] In some implementations, in step S102 above, the heuristic cost of the nth node is also positively correlated with the fourth distance of the nth node, which is the Euclidean distance or RS distance between the nth node and the target parking space. When calculating the fourth distance, the center point of the target parking space can be used, or the center point of the rear axle when the vehicle is parked in the target parking space can be used. This application does not limit the above calculation method.

[0066] Optionally, the heuristic cost, which is positively correlated with both the first and fourth distances to the nth node, satisfies the following formula:

[0067] h_total(n)=α×h_RL(n)+β×h_RS(n);

[0068] Where h_total(n) is the heuristic cost of the nth node, h_RL(n) is the first distance of the nth node, α is the weight coefficient of the first distance, h_RS(n) is the fourth distance of the nth node, β is the weight coefficient of the fourth distance, α and β are positive numbers, and the sum of α and β is 1.

[0069] The first distance guides the hybrid A* algorithm's path search tree to grow towards k nodes, while the fourth distance guides it towards the target parking space. It's understandable that when α is greater than β, the first distance has a larger weight in the heuristic cost, guiding the hybrid A* algorithm's path search tree to preferentially grow towards k nodes. When α is less than β, the fourth distance has a larger weight in the heuristic cost, guiding the hybrid A* algorithm's path search tree to preferentially grow towards the target parking space.

[0070] In some implementations, the fourth distances of the aforementioned q nodes can correspond to the same weight coefficient. Alternatively, the fourth distance of each of the aforementioned q nodes can correspond to a different weight coefficient, and the weight coefficient β_n of the fourth distance of the nth node among the aforementioned q nodes satisfies the following formula:

[0071] β_n = theta × (S_n / S_total);

[0072] Where theta is a hyperparameter and theta is a positive number, S_n is the path length of the first parking path from the first starting point to the nth node, and S_total is the sum of the distances between any two adjacent nodes among the k nodes.

[0073] Understandably, in the initial stage of the search, the value of S_n / S_total is small, the weight of the fourth distance is small, and the weight of the first distance is large, guiding the path search tree of the hybrid A* algorithm to grow preferentially towards the k nodes. However, as the exploration progresses, the value of S_n / S_total increases, the weight of the fourth distance becomes larger, and the weight of the first distance becomes smaller. At this point, the path search tree of the hybrid A* algorithm is guided to grow preferentially towards the target parking space. When the fourth distance uses the RS distance, the kinematic constraints of the vehicle are emphasized more.

[0074] In some implementations, in step S102 above, the nth node may correspond to both the heuristic cost and the actual cost. Optionally, the actual cost of the nth node is positively correlated with at least one of the following:

[0075] Based on the first parking path, the length of the path from the first starting point to the nth node;

[0076] Based on the first parking path, the cumulative change in heading angle of the path from the first starting point to the nth node;

[0077] Based on the first parking path, the number of times the driving direction is changed between the first starting point and the nth node.

[0078] It is understandable that when calculating the actual cost of the nth node, considering the length of the path from the first starting point to the nth node can yield a reasonable actual cost.

[0079] In the first parking path, the explored path contains heading angle information at each node. The cumulative change in heading angle is the sum of the absolute values ​​of the differences in heading angles between adjacent nodes. This cumulative change in heading angle represents the total amount of steering wheel adjustments required when driving through this path. Therefore, considering the cumulative change in heading angle is more reasonable when calculating the actual cost of the nth node, as it more accurately reflects the path's tortuosity. Similarly, considering the number of gear shifts and direction changes when calculating the actual cost of the nth node also makes the calculated actual cost more reasonable.

[0080] The environmental information in the above embodiments includes the vehicle's target parking space. When the environmental information also includes information about at least one obstacle, the actual cost of the nth node can also be negatively correlated with the minimum distance between the first n nodes and at least one obstacle. It is understood that the greater the distance between the first n explored nodes and the obstacle, the safer and more reliable the explored path is, thus reducing the value of the actual cost.

[0081] In some embodiments, the first model in step S101 above is obtained based on reinforcement learning training. The training method of the first model is as follows: Figure 3 As shown, this method can include a pre-training phase and a fine-tuning phase, specifically including the following steps:

[0082] S301, Determine the expert trajectory dataset;

[0083] S302, Expert Data Preprocessing;

[0084] S303, State-Action Pair Construction;

[0085] S304, Behavioral Cloning Pre-training;

[0086] S305, Model training initialization;

[0087] S306, Fine-tuning of reinforcement learning;

[0088] S307, collecting experience through environmental interaction;

[0089] S308, calculate the first reward;

[0090] S309, PPO is performed based on GAE;

[0091] S310, difficulty adjusted;

[0092] S311, performance evaluation;

[0093] S312, determine the convergence condition;

[0094] S313, Generate the first model.

[0095] In the above training method, firstly, the expert trajectory dataset is determined through step S301; then, the second model is pre-trained based on the expert trajectory dataset. This second model is used to generate key nodes of the parking path. The pre-training includes steps S302 to S304. After the pre-training is completed, the pre-trained second model is initialized through step S305 to generate the third model. The third model is then fine-tuned using the SiL training method, which includes steps S306 to S310. Finally, the performance evaluation in step S311 and the convergence condition judgment in step S312 determine whether the training of the third model is complete. Finally, the trained third model is used as the first model through step S313.

[0096] The first model trained through the above steps can generate key nodes of a reasonable parking trajectory based on the vehicle's starting position information and environmental information. When using the hybrid A* algorithm to generate the first parking path, it can quickly inspire the hybrid A* algorithm's path search tree to grow towards these key nodes, making the final generated first parking path more reasonable.

[0097] Each of the above steps is described in detail below.

[0098] For step S301 above, in order to better train the model, training data that meets at least one of the following conditions can be selected as the expert trajectory dataset for supervised learning to pre-train the second model:

[0099] The length of the parking path meets the set conditions;

[0100] The number of gear shifts and driving direction changes in the parking path meets the set conditions;

[0101] The cumulative change in the heading angle of the parking path meets the set conditions;

[0102] The minimum distance between the sampling point of the parking path and at least one obstacle satisfies the set conditions.

[0103] The aforementioned expert trajectory dataset is also known as the first training dataset. The following description mainly uses the expert trajectory dataset as an example.

[0104] The above parking path length meets the set conditions, which means that for the same vehicle starting position and environmental information, the training data with the shorter parking path length is selected.

[0105] The above parking path's number of gear shifts and driving direction changes meets the set conditions, meaning that for the same vehicle starting position and environmental information, the training data with fewer gear shifts and driving direction changes selected for the parking path is considered.

[0106] The cumulative change in heading angle of the above parking path meets the set conditions, which means that for the same vehicle starting position and environmental information, the training data with the smaller cumulative change in heading angle of the parking path is selected.

[0107] The minimum distance between the sampling points of the above parking path and at least one obstacle meets the set condition. This means that for the same vehicle starting position and environmental information, including information about at least one obstacle, the training data selected is the minimum distance between the sampling points of the parking path and at least one obstacle that is greater than or equal to the set obstacle safety distance threshold.

[0108] It is understandable that selecting training data that meets at least one of the above criteria for pre-training the second model can yield more reasonable training results.

[0109] Optionally, the training data that satisfies at least one of the above can be determined based on an expert system, or manually by an expert, or automatically determined by the test environment or simulation environment of the APA system through configuration parameters. This application does not limit the method of determining the above training data.

[0110] In step S302 above, the training data in the expert trajectory dataset is preprocessed, including extracting key nodes for each training data point.

[0111] The parking path of the Gth training data in the expert trajectory dataset includes H sampling nodes. The H sampling nodes include the second starting point and the second ending point of the parking path of the Gth training data. The key nodes corresponding to the Gth training data are determined from the H sampling nodes. The key nodes include at least one of the reversal point, curvature extreme point and obstacle avoidance key point, as well as the second starting point and the second ending point. G is a positive integer and H is a positive integer greater than 2.

[0112] The aforementioned reversal point can be the sampling node for gear shifting and changing driving direction in the parking path of the Gth training data among H sampling nodes. It can be understood that the aforementioned reversal point is the sampling node for gear shifting and changing driving direction in the parking path.

[0113] The aforementioned curvature extremum point can be any sampling node among H sampling nodes where the change in the heading angle of the parking path of the Gth training data is greater than or equal to a set threshold.

[0114] The aforementioned heading angle change value can correspond to curvature. Optionally, in some embodiments, the parking path can include multiple road segments, and the curvature extremum point can be the sampling node in each road segment where the heading angle change value is greater than or equal to a set threshold and the curvature is the largest. It can be seen that the aforementioned curvature extremum point is the point in the parking path, excluding the reversal point, where the heading change is the largest, i.e., the inflection point of the vehicle's turn.

[0115] The aforementioned obstacle avoidance key point can be the sampling node with the smallest distance from at least one obstacle among H sampling nodes.

[0116] Taking a training data point from the expert trajectory dataset as an example, this training data point includes a trajectory consisting of t path points. The i-th path point It contains information in five dimensions. Where x and y are path points. The coordinates of θ are the heading angle of the vehicle, kappa represents the curvature of the path at that point, dir represents the vehicle's orientation at that point, and the value of i ranges from 1 to t.

[0117] The k key points extracted from the training data can be used as the approximate direction of the trajectory. Understandably, different key point information can be extracted depending on the scenario. For example, in a simple parking scenario with many empty parking spaces, where a vehicle can park after one turn, three key points can be extracted: the second starting point of the parking path, the curvature extremum point, and the second ending point. These three key points can reflect the approximate direction of the trajectory.

[0118] If the scenario is complex, more key information can be extracted. For example, in a dead-end road scenario with a few empty parking spaces, the parking path is more complex, and vehicles need to make U-turns, turn, or avoid obstacles more often. Therefore, more reversal points, curvature extrema points, and / or obstacle avoidance key points can be extracted. Conversely, in a dead-end road scenario where the road is almost full of parked cars, the parking path is even more complex. Vehicles need to make more U-turns, turn, and / or avoid obstacles from the starting point to the end point. Therefore, even more reversal points, curvature extrema points, and / or obstacle avoidance key points can be extracted.

[0119] Taking five extracted key nodes as an example. Key point sequence. The i-th key point , where x_rel and y_rel represent relative coordinates, θ_rel represents relative heading angle, d_next represents suggested path length, i.e. distance from the current key point to the next key point, and i ranges from 1 to 5. As the second starting point, , and These can be the aforementioned reversal points, curvature extrema points, and obstacle avoidance key points, respectively. It is the second endpoint.

[0120] Optionally, adding the following constraints to the dimensions of these key points can better control the vehicle's kinematic constraints, enabling the pre-trained model to generate more reasonable key point sequences:

[0121] (x_rel, y_rel) ∈ [-10 meters, 10 meters];

[0122] θ_rel∈[-π,π];

[0123] d_next∈[0.5m,5m].

[0124] Optionally, the above keypoint sequence Action can also be evaluated from the following three dimensions to determine whether it is necessary to reselect keypoints:

[0125] 1. Reconstruction Error: Compare the path reconstructed from the key point sequence Action with the original trajectory to compare the reconstruction error.

[0126] 2. Control feasibility: Consider whether the key point sequence Action satisfies the vehicle kinematic constraints.

[0127] 3. Information retention: Consider whether the key point sequence Action retains important features of the original trajectory, such as avoiding obstacles and not getting stuck in dead ends.

[0128] In step S303 above, the state-action pair for reinforcement learning is constructed, which may specifically include the following steps.

[0129] Step 1: Construct the state representation S, that is, determine the input of reinforcement learning as S = [vehicle state, target parking space, obstacle features].

[0130] The obstacle features described above are represented using polar coordinate grids. This involves dividing the 360-degree space around the vehicle into eight sectors at 45-degree intervals. Then, the distance to the nearest obstacle within each sector and the average relative azimuth angle within each sector are calculated.

[0131] Specifically Where x and y are the vehicle's current coordinates, θ is the vehicle's current heading angle, kappa is the curvature of the vehicle's path at the current point, dir is the vehicle's current orientation, x_target and y_target are the coordinates of the target parking space, and θ_target is the heading angle of the target parking space. The distance to the obstacle in each of the eight sectors. It represents the location of obstacles in the eight sectors, and has a total of 24-dimensional state vectors.

[0132] Step 2: Construct the action representation A, that is, determine the output of reinforcement learning as A = keypoint sequence Action. Taking a keypoint sequence Action consisting of three keypoints as an example... Among them, the i-th key point The value of i ranges from 1 to 3, and each key point includes a vector with 4 dimensions.

[0133] Step 3: Determine the state-action mapping relationship using the following method.

[0134] Set the current sampling point In the section action Here, Δx_rel and Δy_rel are Compared to relative coordinates, This refers to the relative heading.

[0135] The calculation method is as follows:

[0136] The calculation of relative coordinate transformation (vehicle coordinate system) satisfies the following formula:

[0137] ;

[0138] .

[0139] The relative headings satisfy the following formula:

[0140] , where normalize_angle is an angle normalization function that maps any angle value to a standard interval.

[0141] The recommended path length should satisfy the following formula:

[0142] .

[0143] Through the above steps, the resulting state-action pairs, including the state representation, action representation, and mapping relationship, ultimately yield a set of state-action pairs. .

[0144] In step S304 above, the second model is pre-trained based on the expert trajectory dataset and the key nodes corresponding to each training data point therein to obtain the third model.

[0145] The aforementioned expert trajectory dataset and the information of the key nodes corresponding to each training data point have been constructed in step S303 and stored in the set D_expert.

[0146] The input layer for reinforcement learning is a 24-dimensional state vector, and the feature extraction layer used during training is a three-layer fully connected network as follows:

[0147] FC1:24→64, ReLU activated;

[0148] FC2:64→128, ReLU activated;

[0149] FC3:128→64, ReLU activated.

[0150] Taking a key node of 3 key points as an example, the output layer is a 12-dimensional action vector, including 3 key points, each key point includes 4-dimensional parameters, and the output layer operation is as follows:

[0151] Output: 64→12, using the Tanh activation function, normalized to [-1,1].

[0152] The pre-training process described above may also include the following processing:

[0153] 1. Inverse normalization:

[0154] Δx = 10·tanh_output[0];

[0155] Δy = 10·tanh_output[1];

[0156] Δθ=π·tanh_output[2];

[0157] d_next=2.75+2.25·tanh_output[3].

[0158] The above tanh_output is the value that is denormalized after being processed by the anh activation function back to the original value.

[0159] 2. Physical constraint check:

[0160] If d_next < 0.5, then d_next = 0.5; if d_next > 5.0, then d_next = 5.0.

[0161] During pre-training, the following loss function can also be designed:

[0162] Total loss: L_total=L_BC+λ_reg·L_reg+λ_consistency·L_consistency;

[0163] Among them, the behavioral cloning loss, i.e. the main loss L_BC=1 / N∑_{i=1}^N||π_θ(S_i)-A_i||²;

[0164] The regularization loss L_reg = ||θ||², which is the L2 weight decay, and λ_reg is the regularization weight coefficient;

[0165] The multi-step consistency loss is L_consistency=1 / N∑_{i=1}^N||π_θ(S_i)-π_θ(S_{i+1}')||², where λ_consistency is the consistency constraint weight coefficient, and S_{i+1}'=f(S_i,π_θ(S_i)). The simplified dynamic model is used to predict the next state.

[0166] The above pre-training can also use the following optimization strategies:

[0167] 1. Batch training: batch_size=64.

[0168] 2. Optimizer: Adam, initial learning rate lr=1e-3.

[0169] 3. Learning rate scheduling: The learning rate is reduced to 0.8 of the original rate every 20 epochs.

[0170] 4. Early stopping mechanism: If the validation set loss does not decrease for 10 consecutive epochs, the process will stop.

[0171] 5. Data augmentation: Apply small random perturbations to the state to increase robustness.

[0172] In step S305 above, after the pre-training of the second model is completed, the third model is obtained. The subsequent fine-tuning stage of reinforcement learning, including steps S306 to S310, involves training the third model.

[0173] In step S306 above, the following functions and methods are determined for training the third model.

[0174] The state transition function used in training is S_{t+1}=f(S_t,A_t), where S_t is the state and A_t is the action.

[0175] The following methods can be used during training:

[0176] 1. Using vehicle dynamics models, complex factors such as tire slippage and suspension effects can be ignored by using a simplified bicycle model.

[0177] 2. Collision detection: Detects whether a collision has occurred with an obstacle based on the distance characteristics of the obstacle.

[0178] 3. Progress calculation: The specific progress is calculated based on the relative position with respect to the target parking space.

[0179] In step S307 above, environmental interaction is performed to collect experience, that is, ROS is used to provide a simulated environment for model training.

[0180] ROS can provide a comprehensive simulation of the parking environment, including the inputs required for model training, namely the state representation in step S303 above, including the vehicle's current state, the target parking space, and obstacle information. Therefore, ROS can be used to conveniently simulate various parking environments, thereby enabling SiL training.

[0181] In step S308 above, a first reward is calculated to provide feedback to the third model.

[0182] Reinforcement learning is based on a value function, which estimates the expected cumulative reward obtainable from a given state and by following a specific policy, such as the first reward mentioned above. This first reward can guide the third model to adjust its policy, learning to distinguish which states and actions are more likely to yield high rewards, thereby making better decisions.

[0183] The primary rewards include sparse rewards, dense rewards, safety rewards, and / or comfort rewards.

[0184] Sparse rewards are positively correlated with successful parking in the target space, and negatively correlated with at least one of the following:

[0185] The number of times the vehicle collides with at least one obstacle during parking;

[0186] The difference between the parking time and the set time threshold.

[0187] The time taken for parking here minus the set time threshold can be understood as parking timeout.

[0188] Dense rewards are positively correlated with at least one of the following:

[0189] The distance between the end point of the parking path and the center point of the target parking space;

[0190] The environmental information also includes the deviation between the heading angle of the parking path at the end point and the parking direction of the target parking space when the parking direction information of the target parking space is obtained.

[0191] Safety rewards are positively correlated with the minimum distance between the sampling points of the parking path and at least one obstacle.

[0192] Comfort rewards are negatively correlated with at least one of the following:

[0193] The length of the parking path;

[0194] The cumulative change in heading angle during parking;

[0195] The number of times the vehicle shifts gears to change direction while parking.

[0196] Optionally, the first reward mentioned above satisfies the following formula:

[0197] ;

[0198] where \(R_{total}\) is the first reward, \(R_{sparse}\) is the sparse reward, is the weight coefficient of the sparse reward, \(R_{dense}\) is the dense reward, is the weight coefficient of the dense reward, \(R_{safety}\) is the safety reward, is the weight coefficient of the safety reward, \(R_{comfort}\) is the comfort reward, is the weight coefficient of the comfort reward.

[0199] In some embodiments, when the sparse reward is related to successfully parking in the target parking space, parking timeout, and the number of collisions with obstacles, the weight of successfully parking in the target parking space is the highest. For example, the reward value can be set to +1000, the weight of parking timeout is the lowest. For example, the reward value can be set to -200, and for one collision with an obstacle, the reward value can be set to -500.

[0200] Optionally, for the dense reward, both approaching the target parking space in terms of distance and heading will gradually bring rewards. However, the weight of approaching the end point in terms of heading is higher than that of approaching the end point in terms of distance. It can be understood that considering complex scenarios, when the target parking space needs to be reversed into, the vehicle needs to complete a U-turn first and then reverse for parking. And to complete the U-turn, it may be necessary to move away from the target parking space first. Therefore, a higher weight for heading will preferentially reward such situations.

[0201] Optionally, the safety reward is set in the following four gears according to the minimum distance \(d_{min}\) from the obstacle. The farther the distance from the obstacle, the greater the safety reward:

[0202] 1. When \(d_{min}>2.0\) meters, it is a safe distance and the safety reward is 0.5.

[0203] 2. When \(1.0\) meter \(<d_{min}\leq2.0\) meters, the safety reward is 0.0.

[0204] 3. When \(0.5\) meter \(<d_{min}\leq1.0\) meters, the safety reward is -1.0.

[0205] 4. When \(d_{min}\leq0.5\) meters, it is a dangerous distance and the safety reward is -5.0.

[0206] Optionally, when the comfort reward is related to the length of the parking path, the cumulative change value of the heading angle, and the number of gear shifts, the comfort reward satisfies the following formula:

[0207] \(R_{comfort}=-0.5\cdot|\Delta yaw|-0.2\cdot|\Delta dist|-2.0\cdot gear\_change\)

[0208] Where Δyaw is the change in steering wheel angle, i.e. the cumulative change in heading angle, Δdist is the parking path length, and gear_change is the number of gear changes.

[0209] In step S309 above, the third model is fed back using GAE based on the first reward obtained in step S308.

[0210] GAE (Generative Advantage Estimation) is an efficient method in reinforcement learning for estimating the advantage function. By balancing bias and variance, it solves the credit assignment problem in reinforcement learning, namely, how to accurately determine the contribution of historical actions to delayed rewards. Employing a multi-step advantage exponential weighted averaging method, it effectively integrates the advantage function into the loss function, providing a stable gradient signal for policy optimization and making reinforcement learning more efficient.

[0211] In some implementations, a Policy Optimization (PPO) can be performed based on the first reward and GAE to provide feedback to the third model. PPO is an important method in the field of reinforcement learning. By limiting the policy update magnitude, it avoids instability caused by excessive parameter adjustments, while improving sample utilization, achieving a good balance between performance and stability.

[0212] The specific methods for performing a PPO include the following:

[0213] 1. The dominance function is estimated as follows:

[0214] δ_t=R_t+γ·V_φ(S_{t+1})-V_φ(S_t);

[0215] Â_t=∑_{l=0}^{Tt-1}(γ·λ)^l·δ_{t+l};

[0216] Wherein, the discount factor γ=0.99, the GAE parameter λ=0.95, and V_φ is the state value estimated by the Critic network.

[0217] 2. The policy loss function is as follows:

[0218] ;

[0219] Among them, r_t(θ)=π_θ(A_t|S_t) / π_θ_old(A_t|S_t), PPO clipping parameter ε=0.2.

[0220] 3. The total loss function is as follows:

[0221] ;

[0222] The components are as follows:

[0223] L_CLIP(θ) is the loss of the PPO policy;

[0224] Value function loss ;

[0225] Behavioral cloning loss ;

[0226] Policy Entropy .

[0227] The weighting coefficient is, , , .

[0228] 4. The adaptive behavior cloning weights satisfy the following formula:

[0229] λ(t)=λ_init·exp(-t / τ)+λ_min;

[0230] in:

[0231] The initial behavior clone weight λ_init=0.5;

[0232] Attenuation constant τ = 50000;

[0233] The minimum behavior cloning weight λ_min=0.1 is used to maintain expert knowledge.

[0234] In step S310 above, multiple difficulty levels can be set, and the third model can be trained sequentially from low to high, specifically including:

[0235] For each of the 'a' difficulty levels, a corresponding success rate threshold and a first reward threshold are set. The third model is trained starting from the lowest difficulty level. When the parking success rate of L consecutive training sessions is greater than or equal to the success rate threshold corresponding to the difficulty level, and the average first reward of L consecutive training sessions is greater than or equal to the first reward threshold corresponding to the difficulty level, the model is trained at a higher difficulty level. Here, 'a' is a positive integer greater than 1, and 'L' is a positive integer greater than 1.

[0236] In some implementations, the following four difficulty levels can be defined:

[0237] Level 1: Simple perpendicular parking space, no obstacles, starting position is close.

[0238] Level 2: Parallel and perpendicular parking spaces, with a few static obstacles.

[0239] Level 3: Perpendicular parking spaces, dead-end road scenarios with few surrounding obstacles.

[0240] Level 4: Complex scene with multiple obstacles and a difficult starting position.

[0241] When the parking success rate is greater than 0.85 after L consecutive training sessions, and the average of the first reward after L consecutive training sessions is greater than or equal to the first reward threshold corresponding to the difficulty level, the training will proceed to a higher difficulty level.

[0242] In step S311 above, the training results of the reinforcement learning fine-tuning stage in steps S306 to S310 are evaluated to provide a basis for judging the convergence condition in the subsequent step S312.

[0243] The third model is trained for b evaluation periods. Each evaluation period includes W training iterations of the third model, where W is a positive integer greater than or equal to a, and b is a positive integer greater than 1. For each evaluation period, the average parking success rate, the average first reward, and the policy entropy are calculated.

[0244] In some implementations, the evaluation metrics can be set according to the following formula:

[0245] Success rate: P_success = N_success / N_total;

[0246] Average reward: R_avg=(∑R_i) / N_episodes;

[0247] Average number of steps: Steps_avg=(∑steps_i) / N_episodes;

[0248] Safety margin: Safety_margin=min_{t,s}d_s(t);

[0249] Strategy similarity: Expert_similarity=1-MSE(π_θ,π_expert), where MSE is the standard mean squared error function.

[0250] Optionally, for the four difficulty levels in step S310, the online testing metric is to test 100 random scenarios for each difficulty level, with the following success rate target for each difficulty level:

[0251] Level 1: Simple scenario, success rate target >95%;

[0252] Level 2: Medium-level scenario, target success rate >85%;

[0253] Level 3: Difficult scenarios, success rate target >70%;

[0254] Level 4: Extreme scenario, success rate target >50%;

[0255] In step S312 above, based on the performance evaluation results in step S311, it is determined whether the convergence condition is met. If the convergence condition is not met, the process proceeds to step S306 to continue training the third model. If the convergence condition is met, the process proceeds to the subsequent step S313.

[0256] Training of the third model based on difficulty levels a is considered complete when the following convergence conditions are met simultaneously:

[0257] The change in the average parking success rate over c consecutive evaluation periods is less than or equal to a set threshold, where c is a positive integer greater than 1 and c is less than or equal to b.

[0258] The change in the average first reward over the most recent d evaluation periods is less than or equal to a set threshold, where d is a positive integer greater than 1 and d is less than or equal to b.

[0259] For each of the a difficulty levels, the success rate of the most recent e training iterations of the third model is greater than or equal to the success rate threshold set for that difficulty level, where e is a positive integer.

[0260] The policy entropy range of the most recent f evaluation periods is within the set entropy threshold range, where f is a positive integer greater than 1 and f is less than or equal to b.

[0261] In some implementations, training of the third model based on difficulty levels a is considered complete when the following conditions are met simultaneously:

[0262] The change in parking success rate over 10 consecutive assessment periods is less than 1%;

[0263] The change in the average first reward over the most recent 5 evaluation periods is less than 5;

[0264] The success rate for all difficulty levels is greater than or equal to the success rate threshold corresponding to that level.

[0265] The strategy entropy for the most recent 5 evaluation periods is within the range of 0.1-0.3.

[0266] In step S313 above, after step S312 has determined that the training of the third model based on a difficulty level has been completed, the model trained on the third model will be used as the first model.

[0267] Using the above method, pre-training based on an expert trajectory dataset makes the pre-training results more reasonable. Based on the first reward, GAE (Guided Algorithm for Reinforcement Learning) is used to provide feedback to the model, making the reinforcement learning training process more efficient. Through training at multiple difficulty levels and using convergence conditions to determine training completion, a more effective first model is obtained. This first model can generate key nodes for a reasonable parking trajectory based on the vehicle's starting position and environmental information. These key nodes effectively preserve the important features of the parking trajectory. Subsequently, when using the hybrid A* algorithm to generate parking paths, the hybrid A* algorithm's path search tree can be quickly inspired to grow towards these key nodes, making the final generated first parking path more reasonable.

[0268] This application also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor. The memory is used to store computer program code, which includes computer instructions. One or more processors call the computer instructions to cause the electronic device to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0269] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0270] The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. In some examples, the readable storage medium may be a non-transient readable storage medium. This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments, achieving the same technical effects. To avoid repetition, it will not be described again here.

[0271] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here. It should be understood that the chip mentioned in this application embodiment can also be called a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0272] In summary, the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0273] In the embodiments of this application, the term "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, X and / or Y can represent: X alone, X and Y simultaneously, or Y alone, where X and Y can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0274] Unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, sequence, priority, or importance of multiple objects. Furthermore, the terms "comprising" and "having" in the embodiments, claims, and drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules and may also include steps or modules not listed.

[0275] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

Claims

1. A method for generating a parking path, characterized in that, An assisted driving module applied to a vehicle, the method comprising: Based on the first model, k nodes of the parking path of the vehicle are determined according to the starting position information and environmental information of the vehicle. The first node of the k nodes is the first starting point of the vehicle, and the kth node is the first ending point of the vehicle. The starting position information includes the first starting point of the vehicle, and the environmental information includes the target parking space of the vehicle. k is a positive integer greater than 2. The first model is obtained based on reinforcement learning training. Based on the k nodes, the vehicle's starting position information, and the environmental information, a hybrid A* algorithm is used to generate the vehicle's first parking path. The first parking path includes q nodes, the first node among the q nodes is the first starting point, and the heuristic cost of any nth node among the q nodes is positively correlated with the first distance of the nth node. The first distance of the nth node includes the second distance and the third distance of the nth node. The second distance of the nth node is the distance between the nth node and the mth node among the k nodes that satisfies the first condition. The third distance of the nth node is the sum of the distances between any two adjacent nodes among the k nodes from the mth node to the first ending point. Here, q is a positive integer, n is a positive integer (n < q), and m is a positive integer (m < k). The first condition includes: From the k nodes, select r nodes whose distances to the first n nodes of the q nodes are all greater than or equal to a set first distance threshold. From the r nodes, select the node with the shortest distance to the nth node as the mth node, where r is a positive integer less than or equal to k.

2. The method according to claim 1, characterized in that, The first distance of the nth node is equal to the sum of the second and third distances of the nth node.

3. The method according to claim 1 or 2, characterized in that, The heuristic cost of the nth node is also positively correlated with the fourth distance of the nth node, which is the Euclidean distance or Ritz-Shepp distance between the nth node and the target parking space. The heuristic cost of the nth node satisfies the following formula: h_total(n)=α×h_RL(n)+β×h_RS(n); Where h_total(n) is the heuristic cost of the nth node, h_RL(n) is the first distance of the nth node, α is the weight coefficient of the first distance, h_RS(n) is the fourth distance of the nth node, β is the weight coefficient of the fourth distance, α and β are positive numbers, and the sum of α and β is 1.

4. The method according to claim 3, characterized in that, The fourth distance of each of the q nodes corresponds to a weight coefficient, and the weight coefficient β_n of the fourth distance of the nth node satisfies the following formula: β_n = theta × (S_n / S_total); Where theta is a hyperparameter and theta is a positive number, S_n is the path length of the first parking path from the first starting point to the nth node, and S_total is the sum of the distances between any two adjacent nodes among the k nodes.

5. The method according to claim 1 or 2, characterized in that, The actual cost of the nth node is positively correlated with at least one of the following: Based on the first parking path, the length of the path from the first starting point to the nth node; Based on the first parking path, the cumulative change in heading angle of the path from the first starting point to the nth node; Based on the first parking path, the number of times the driving direction is changed from the first starting point to the nth node; And / or, The environmental information also includes information about at least one obstacle, and the actual cost of the nth node is negatively correlated with the minimum distance between the first n nodes of the q nodes and the at least one obstacle.

6. The method according to claim 1 or 2, characterized in that, The environmental information also includes information about at least one obstacle. The first model is obtained based on reinforcement learning training and includes: The training data that meets at least one of the following conditions will be used as the first training dataset: The length of the parking path meets the set conditions; The number of gear shifts and driving direction changes in the parking path meets the set conditions; The cumulative change in the heading angle of the parking path meets the set conditions; The minimum distance between the sampling point of the parking path and the at least one obstacle satisfies the set condition; Extract key nodes from each training data point in the first training dataset; The second model is pre-trained based on the first training dataset and the key nodes corresponding to each training data point in the first training dataset to obtain the third model. The second model is used to generate the key nodes of the parking path. The third model is trained using the Software-in-the-Loop (SiL) training method to obtain the first model.

7. The method according to claim 6, characterized in that, The step of extracting key nodes for each training data point in the first training dataset includes: The parking path of the Gth training data in the first training dataset includes H sampling nodes. The H sampling nodes include the second starting point and the second ending point of the parking path of the Gth training data. The key nodes corresponding to the Gth training data are determined from the H sampling nodes. The key nodes include at least one of the reversal point, curvature extreme point and obstacle avoidance key point, as well as the second starting point and the second ending point. G is a positive integer and H is a positive integer greater than 2. The reversal point is the sampling node in the H sampling nodes where the parking path of the Gth training data changes gears and switching driving directions; The curvature extremum point is the sampling node among the H sampling nodes where the change value of the parking path heading angle of the Gth training data is greater than or equal to a set threshold. The obstacle avoidance key point is the sampling node with the smallest distance from the at least one obstacle among the H sampling nodes.

8. The method according to claim 6, characterized in that, The method of training the third model using Software-in-the-Loop (SiL) to obtain the first model includes: Based on the first reward, the third model is fed back using generalized advantage estimation (GAE), whereby the first reward includes sparse reward, dense reward, safety reward, and / or comfort reward. For each of the 'a' difficulty levels, a corresponding success rate threshold and a first reward threshold are set. The third model is trained starting from the lowest difficulty level. When the parking success rate of L consecutive training sessions is greater than or equal to the success rate threshold corresponding to the difficulty level, and the average of the first rewards of the L consecutive training sessions is greater than or equal to the first reward threshold corresponding to the difficulty level, the model is trained at a higher difficulty level. Here, 'a' is a positive integer greater than 1, and 'L' is a positive integer greater than 1. Once it is determined that the training of the third model based on the a difficulty levels has been completed, the model trained on the third model is used as the first model; The sparse reward is positively correlated with successfully parking in the target parking space, and the sparse reward is negatively correlated with at least one of the following: The number of times the vehicle collides with at least one obstacle during parking; The difference between the parking time and the set time threshold; The dense reward is positively correlated with at least one of the following: The distance between the end point of the parking path and the center point of the target parking space; The environmental information also includes the deviation between the heading angle of the parking path at the end point and the parking direction of the target parking space when the parking direction information of the target parking space is obtained. The safety reward is positively correlated with the minimum distance between the sampling points of the parking path and the at least one obstacle; The comfort reward is negatively correlated with at least one of the following: The length of the parking path; The cumulative change in heading angle during parking; The number of times the vehicle shifts gears to change direction while parking.

9. The method according to claim 8, characterized in that, The determination that the training of the third model based on the a difficulty levels has been completed includes: The third model is trained for b evaluation periods, and each evaluation period includes W training cycles of the third model, where W is a positive integer greater than or equal to a, and b is a positive integer greater than 1. For each evaluation period, calculate the average parking success rate, the average first reward, and the policy entropy. The training of the third model based on the aforementioned a difficulty levels is considered complete when the following convergence condition is met: The change in the average parking success rate over c consecutive evaluation periods is less than or equal to a set threshold, where c is a positive integer greater than 1 and c is less than or equal to b. The change in the average first reward over the most recent d evaluation periods is less than or equal to a set threshold, where d is a positive integer greater than 1 and d is less than or equal to b. For each of the a difficulty levels, the success rate of the most recent e training iterations of the third model is greater than or equal to the success rate threshold set for that difficulty level, where e is a positive integer. The policy entropy range of the most recent f evaluation periods is within the set entropy threshold range, where f is a positive integer greater than 1 and f is less than or equal to b.

10. An electronic device, characterized in that, Including memory and processor; The memory is coupled to the processor and is used to store computer program code, the computer program code including computer instructions, wherein one or more of the processors invoke the computer instructions to cause the electronic device to perform the parking path generation method according to any one of claims 1 to 9.

Images