Vehicle trajectory planning method for integrated driving and parking, intelligent device and storage medium

Abstract

This application relates to the field of intelligent driving technology, specifically providing a vehicle trajectory planning method, intelligent device, and storage medium for integrated driving and parking, aiming to solve the problem of how to obtain the globally optimal vehicle driving trajectory for integrated driving and parking and achieve seamless switching between cruising and parking. To this end, the method of this application includes acquiring scene representation information of the vehicle driving scenario, which includes environmental representation information, task representation information, and state representation information of the vehicle and target objects at multiple consecutive time points. The task representation information includes task input information and category identifiers, where the task is either cruising or parking. A trajectory planning network is used to perform spatiotemporal joint attention encoding on the scene representation information, and the encoding result is decoded to obtain the driving trajectory. Based on this method, seamless connection is achieved when switching between cruising and parking tasks, ensuring that the driving trajectory obtained during either cruising or parking is the globally optimal result for the entire integrated driving and parking task.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, specifically providing a vehicle trajectory planning method, intelligent device, and storage medium for integrated driving and parking. Background Technology

[0002] Currently, in conventional intelligent driving technology, the vehicle's cruise planning task and parking planning task are usually implemented by two different modules (or planners), which are algorithmically separate from each other, meaning that their planning modes are completely different.

[0003] When a vehicle transitions from driving on the road to parking (or vice versa), a change in planning mode is required. Since cruise planning and parking planning have completely different planning modes, the transition may be disjointed, affecting vehicle safety and stability. For example, the vehicle trajectory may be discontinuous and the control may be uneven during the switch, potentially leading to sudden braking, sharp turns, or replanning jitter. Furthermore, because cruise planning and parking planning tasks are algorithmically separate, the vehicle trajectories obtained from these two tasks are only locally optimal results for their respective tasks. The complete trajectory formed by stitching together these two trajectories may not represent the globally optimal result considering the overall driving and parking process.

[0004] Accordingly, a new technical solution is needed in this field to solve the above problems. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies, this application is proposed to solve or at least partially solve the following technical problem: to provide a vehicle trajectory method for integrated driving and parking, which can obtain the globally optimal vehicle driving trajectory from the global perspective of integrated driving and parking, and can achieve seamless connection when switching between cruise planning and parking planning.

[0006] In a first aspect, this application provides a vehicle trajectory planning method for integrated driving and parking, the method comprising:

[0007] Acquire scene representation information of the vehicle driving scenario. The scene representation information includes environmental representation information, task representation information, vehicle state representation information, and target state representation information at multiple consecutive time points. The multiple consecutive time points include the current time and multiple consecutive historical time points before it. The task representation information includes task input information and category identifier. The task is a cruise task or parking task in the driving and parking integrated task. The vehicle is the current vehicle. The target is other objects in the environment where the current vehicle is located, excluding the current vehicle.

[0008] A trajectory planning network is used to perform spatiotemporal joint attention encoding on the scene representation information, and the encoding result is decoded to obtain the current vehicle's driving trajectory.

[0009] The driving trajectory includes trajectory point information at multiple future moments, and the trajectory point information includes at least the position and speed of the trajectory point; the trajectory planning network is obtained through supervised training based on samples of the scene representation information.

[0010] In one technical solution of the above vehicle trajectory planning method, the spatiotemporal joint attention coding includes temporal dimension attention coding and spatial dimension attention coding;

[0011] The temporal attention encoding is as follows: self-attention processing is performed on the state representation information of each entity in the scene representation information, wherein the entity includes the vehicle and the target body;

[0012] The spatial dimension attention encoding is as follows: cross-attention processing is performed on the vehicle's state representation information and other information in the scene representation information, wherein the other information includes the environment representation information, the task representation information and the state representation information of all target bodies.

[0013] In one technical solution of the above vehicle trajectory planning method, the spatiotemporal joint attention encoding of the scene representation information includes: sequentially performing the time-dimensional attention encoding and the spatial-dimensional attention encoding on the scene representation information.

[0014] In one technical solution of the above vehicle trajectory planning method, the task representation information is obtained in the following way:

[0015] The input information and category identifier of the task are encoded separately, and the encoding results of the input information and category identifier are fused to obtain the task representation information.

[0016] In one technical solution of the above vehicle trajectory planning method, the input information for the cruise task includes a cruise reference line; the input information for the parking task includes a parking target point.

[0017] In one technical solution of the above vehicle trajectory planning method, the trajectory planning network is further trained in the following way:

[0018] A joint loss function is determined for training the trajectory planning network, wherein the joint loss function is a loss function obtained by weighting and summing multiple sub-loss functions;

[0019] The trajectory planning network is trained in a supervised manner using the joint loss function and samples of the scene representation information.

[0020] Among them, the multiple sub-loss functions correspond one-to-one with multiple different network training objectives;

[0021] The plurality of sub-loss functions include at least a horizontal-vertical coupling loss function. The network training objective corresponding to the horizontal-vertical coupling loss function is to minimize the deviation between the driving trajectory output by the trajectory planning network and the optimal trajectory. The optimal trajectory is the optimal trajectory obtained by trajectory optimization with the coupling relationship between lateral acceleration, longitudinal velocity and trajectory curvature as constraints.

[0022] In one technical solution of the above vehicle trajectory planning method, the optimal trajectory is the optimal trajectory obtained by using the iLQR trajectory optimization algorithm and the coupling relationship as a constraint condition.

[0023] And / or, the trajectory point information in the driving trajectory also includes the curvature and heading angle of the trajectory points.

[0024] In one technical solution of the above vehicle trajectory planning method, the sample of the scene representation information further includes the true intent information of the vehicle, and the plurality of sub-loss functions further include at least one of the following loss functions:

[0025] The first loss function corresponds to the network training objective of minimizing the deviation between the driving trajectory output by the trajectory planning network and the true trajectory, where the true trajectory is the actual driving trajectory labeled by the sample.

[0026] The second loss function corresponds to the network training objective of making the driving trajectory output by the trajectory planning network satisfy the preset dynamic constraints.

[0027] The third loss function corresponds to the network training objective of making the driving trajectory output by the trajectory planning network meet the preset safety constraints.

[0028] The fourth loss function corresponds to the network training objective of making the driving trajectory output by the trajectory planning network meet the preset comfort constraints.

[0029] The fifth loss function corresponds to the network training objective of minimizing the deviation between the predicted intent information output by the trajectory planning network and the true intent information.

[0030] In a second aspect, a smart device is provided, the smart device including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the technical solutions provided in the first aspect.

[0031] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and executed by a processor to perform the method described in any of the technical solutions provided in the first aspect above.

[0032] The above-described technical solutions of this application have at least one or more of the following beneficial effects:

[0033] In one technical solution for implementing the vehicle trajectory planning method for integrated driving and parking provided in this application, the method may include the following steps:

[0034] Scene representation information of the vehicle driving scenario is obtained. This scene representation information can include environmental representation information, task representation information, vehicle state representation information, and target state representation information at multiple consecutive time points. The multiple consecutive time points include the current time and multiple consecutive historical time points preceding it. The task representation information includes task input information and category identifiers. The task is either a cruise task or a parking task in a driving-parking integrated task. The vehicle is the current vehicle, and the target is any object other than the current vehicle in its environment. A trajectory planning network is used to perform spatiotemporal joint attention encoding on the scene representation information, and the encoding result is then decoded to obtain the current vehicle's driving trajectory. The driving trajectory can include trajectory point information at multiple future time points, and the trajectory point information includes at least the position and velocity of the trajectory point. The trajectory planning network is obtained through supervised training based on samples of the scene representation information.

[0035] Based on the above implementation scheme, the cruise and parking tasks can be combined into a single integrated driving and parking task. A trajectory planning network is used to plan the trajectory for this integrated task, essentially considering the overall (or global) picture of the integrated driving and parking process and planning the vehicle's trajectory during cruise and parking. In other words, the trajectory obtained in both the cruise and parking tasks represents the globally optimal result for the entire integrated driving and parking task, overcoming the limitation of existing technologies that can only obtain locally optimal results for cruise and parking tasks. Furthermore, when switching between cruise and parking tasks, only the input information and category identifier in the task representation information need to be changed; the form of the driving trajectory output by the trajectory planning network remains unchanged. This ensures that the driving trajectory remains continuous and effective (or executable) during task switching, and the vehicle will not experience sudden braking / sharp turning / replanning jitter while traveling along this trajectory, thus achieving seamless connection of vehicle driving control when switching between cruise and parking tasks. Attached Figure Description

[0036] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Wherein:

[0037] Figure 1 This is a schematic flowchart of the main steps of a vehicle trajectory planning method according to an embodiment of this application;

[0038] Figure 2 This is a schematic flowchart illustrating the main steps of obtaining task representation information according to an embodiment of this application;

[0039] Figure 3 This is a schematic flowchart illustrating the main steps of training a trajectory planning network according to an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of the main structure of an intelligent driving control system according to an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of the main structure of a smart device according to an embodiment of this application. Detailed Implementation

[0042] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0043] The relevant user personal information that may be involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and includes personal information that users actively provide or that is generated as a result of using the product / service, as well as personal information obtained with user authorization.

[0044] The personal information processed in this application will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. This application will treat the user's personal information and its processing with the utmost diligence.

[0045] This application attaches great importance to the security of users' personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect users' information and prevent unauthorized access, disclosure, use, modification, damage or loss of personal information.

[0046] The following describes an embodiment of the vehicle trajectory planning method provided in this application. The application scenarios of this method may include low-speed vehicle driving scenarios.

[0047] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a vehicle trajectory planning method according to an embodiment of this application. Figure 1 As shown, the vehicle trajectory planning method in this application embodiment mainly includes the following steps S101 to S102.

[0048] Step S101: Obtain scene representation information of the vehicle driving scenario.

[0049] Scene representation information can include environmental representation information (Env tokens), task representation information (Task tokens), vehicle state representation information (Ego tokens), and target state representation information (Agent tokens) across multiple consecutive time points (including the current time point and multiple consecutive historical time points prior to it). In other words, scene representation information can include four temporal sequences: the first sequence consists of environmental representation information across multiple consecutive time points; the second sequence consists of task representation information across multiple consecutive time points; the third sequence consists of vehicle state representation information across multiple consecutive time points; and the fourth sequence consists of target state representation information across multiple consecutive time points. The consecutive time points in these four sequences are identical.

[0050] The vehicle is the vehicle that needs to be tracked (hereinafter referred to as the current vehicle), and the target is any other object in the environment of the current vehicle other than the current vehicle. This other object may include obstacles and other traffic participants (such as vehicles, pedestrians, etc.).

[0051] The following explains several types of information in scene representation information.

[0052] (1) Environmental characterization information may include lane lines, road boundaries, parking spaces, static obstacles and other environmental information.

[0053] (2) Task representation information may include task input information and category identifier, where the task is either a cruise task or a parking task within the integrated driving and parking task. The task category identifier enables the trajectory planning network to determine which task to execute. Task input information can be understood as the information that needs to be input into the trajectory planning network to execute the task. In some implementations, the input information for a cruise task may include a cruise reference line. When the vehicle is in cruise mode, the vehicle's trajectory needs to conform to the cruise reference line and not deviate from it. The input information for a parking task may include a goal-point parking, which can be understood as the point where the vehicle needs to park and come to a complete stop. The goal-point parking can be determined based on the parking space to be parked. In addition, the attribute information of the goal-point parking may include not only location information but also the vehicle's attitude at the goal-point parking and the boundary information of the parking space to be parked.

[0054] (3) The state representation information of the vehicle can be represented as: , Indicates the current moment. This can be understood as being caused by the current moment. and at the present moment Previous consecutive A sequence consisting of vehicle state representation information at each time step. The vehicle state representation information at each time step in this sequence can contain... Information about this subset , , , , , , , , These represent the longitudinal coordinate, lateral coordinate, longitudinal velocity, longitudinal acceleration, longitudinal jerk, heading angle, yaw rate, trajectory curvature, and rate of change of trajectory curvature, respectively.

[0055] (4) The state representation information of the target body can be represented as: , Indicates the current moment. and These represent the target's serial number and total number, respectively. This can be understood as being caused by the current moment. and at the present moment Previous consecutive A sequence of target state representation information at each time step. The target state representation information at each time step in this sequence can include information such as the target's relative pose, velocity, and category.

[0056] In some implementations, each piece of information in the scene representation information can be represented in the form of the following equation (1):

[0057] (1)

[0058] The meanings of the parameters in formula (1) are as follows: The first element in the scene representation information This information in the first The final numerical feature vector at each time step. Indicates the first This information in the first The original numerical feature vector at each time step. Indicates the first The type of information is embedded in the vector. Indicates the first The time embedding vector at each time step Indicates the sign of vector addition. This refers to the Multi-Layer Perceptron.

[0059] Step S102: Use a trajectory planning network to perform spatiotemporal joint attention encoding on the scene representation information, and perform trajectory decoding on the encoding result to obtain the current vehicle's driving trajectory.

[0060] Trajectory planning networks can be obtained through supervised training based on samples of scene representation information.

[0061] The spatiotemporal joint attention encoding and driving trajectory are explained below.

[0062] 1. Explanation of spatiotemporal joint attention encoding.

[0063] Spatiotemporal joint attention coding can be understood as an encoding method that uses an attention mechanism to jointly encode scene representation information from both temporal and spatial dimensions. The encoding result can simultaneously reflect high-dimensional features of the encoded information in both the temporal and spatial dimensions. The encoded information in the temporal dimension can at least reflect the temporal changes of various types of information within the scene representation information, while the encoded information in the spatial dimension can at least reflect the interaction relationships among these types of information. Based on spatiotemporal joint attention coding, the spatiotemporal consistency of dynamic interactions between the vehicle and environmental information and target objects can be improved. This facilitates obtaining a safer, more stable, and reliable driving trajectory when decoding the trajectory based on the encoding results of spatiotemporal joint attention coding.

[0064] The technical effects of the spatiotemporal joint attention encoding provided in this application will be explained below in conjunction with existing technologies. Specifically, in existing technologies, vehicle trajectory planning methods often separate the motion prediction of the target body, historical information processing, and trajectory planning, which leads to the following disadvantages: (1) insufficient utilization of historical information when planning the trajectory, sensitivity to short-term noise, and easy to cause frequent repetitive planning; (2) slow response to changes in the motion of the target body, resulting in unstable decision-making (such as hesitation, rushing, repeated probing, etc.) when planning the trajectory; (3) easy to make errors in densely interactive areas such as parking entrances / narrow passages. The method provided in this application jointly encodes scene representation information from both temporal and spatial dimensions, which can be understood as jointly modeling the temporal changes and interaction relationships of various types of information in the scene representation information within the same feature representation space. This is equivalent to simultaneously realizing the motion prediction of the target body, historical information processing, and trajectory planning within the same feature representation space. By decoding the trajectory of the encoding result of the spatiotemporal joint attention encoding, the vehicle's driving trajectory can be obtained. Therefore, the method provided in this application does not have the above-mentioned disadvantages of existing technologies.

[0065] (2) Describe the driving trajectory.

[0066] The driving trajectory can include trajectory point information at multiple future time points, and the trajectory point information includes at least the position and velocity of the trajectory point. The position of the trajectory point can be understood as the result of the vehicle's lateral planning, which includes the lateral and longitudinal coordinates of the trajectory point. Connecting the trajectory points at all time points based on their positions forms the vehicle's spatial path. The velocity of the trajectory point can be understood as the result of the vehicle's longitudinal planning, specifically referring to the vehicle's longitudinal velocity at the trajectory point. Therefore, it can be determined that the method provided in this application actually couples lateral and longitudinal planning into a single trajectory planning network, which can plan a driving trajectory that simultaneously includes lateral and longitudinal motion information.

[0067] The technical effects of the horizontal and vertical coupling planning method provided in this application will be explained below in conjunction with the existing technology. Specifically, in the existing technology, the vehicle trajectory planning method mainly adopts a layered strategy of horizontal and vertical decoupling, that is, first generating a spatial path (i.e., horizontal planning), and then planning the speed curve along the horizontal trajectory (i.e., vertical planning). Since the horizontal planning and vertical planning are independent of each other and lack global coordination, it often leads to a mismatch between trajectory and speed, resulting in trajectory and speed conflicts, especially in complex and dynamic driving scenarios where it is difficult to plan the optimal and safe driving trajectory. For example, the driving trajectory planned using the above layered strategy may have the following disadvantages: (1) The geometric constraints of the horizontal trajectory (such as trajectory curvature, orientation, etc.) meet the requirements of the vehicle's drivable area, but the horizontal trajectory combined with the vertical speed will increase the safety risk of vehicle driving and reduce the comfort of vehicle driving; (2) For parking / low-speed narrow scenarios, the driving trajectory planned using the above layered strategy may cause the vehicle to make unreasonable driving actions (or decisions); (3) It is easy to have a driving trajectory that looks reasonable, but the vehicle shakes when driving according to the driving trajectory.

[0068] The method provided in this application couples lateral and vertical planning into the same trajectory planning network, eliminating the hierarchical strategy of lateral and vertical decoupling and thus avoiding the drawbacks of that strategy. In any driving scenario, simply inputting the scenario representation information of the vehicle driving scenario into the trajectory planning network for processing will output a safe, reliable, and stable driving trajectory.

[0069] In some implementations, the trajectory point information may also include the curvature and heading angle of the trajectory points, which can also be understood as the lateral planning result of the vehicle. In this implementation, the driving trajectory can be represented by the following formula (2):

[0070] (2)

[0071] The meanings of the parameters in formula (2) are as follows: Indicates the driving trajectory. , , , , These represent the driving trajectory at the th... The longitudinal coordinates, lateral coordinates, heading angle, longitudinal velocity, and curvature of the trajectory point at a future moment. This represents the total number of future moments included in the driving trajectory.

[0072] In some implementations... , Absolute coordinates can be used, which can be understood as the actual coordinates of the trajectory points. Additionally, , Relative coordinates can also be used. Relative coordinates can be understood as the change between the actual coordinates of the trajectory point at the current moment and the actual coordinates at the previous moment.

[0073] Based on the methods described in steps S101 to S102 above, the cruise and parking tasks can be combined into a single integrated driving and parking task. A trajectory planning network is used to plan the trajectory for this integrated task, essentially considering the overall driving and parking process and planning the vehicle's trajectory during cruise and parking. In other words, the driving trajectory obtained in both the cruise and parking tasks is the globally optimal result for the entire integrated driving and parking task. Furthermore, the form of the driving trajectory output by the trajectory planning network does not change when switching tasks, ensuring that the driving trajectory remains continuous and effective (or executable) during task switching. The vehicle will not experience sudden braking / sharp turning / replanning jitter while traveling along this trajectory, thus achieving seamless connection of vehicle driving control when switching between cruise and parking tasks.

[0074] The following describes an embodiment of the vehicle trajectory planning method provided in this application, specifically the method for obtaining task representation information in step S101 above.

[0075] In some embodiments according to this application, it is possible to... Figure 2 The following steps S1011 to S1012 are shown to obtain task representation information:

[0076] Step S1011: Encode the task's input information and category identifier respectively.

[0077] Step S1012: The encoding results of the input information and the category identifier are fused to obtain the task representation information. The task representation information can be represented by the following formula (3):

[0078] (3)

[0079] The meanings of the parameters in formula (3) are as follows: Represents task representation information. This indicates the task input information. express The encoding result, Encoded information indicating the task category identifier.

[0080] Based on the method described in steps S1011 to S1012 above, the relevant information of two different tasks, cruise and parking, can be represented in the same information format. When switching from the cruise task to the parking task, there will be no problem of task connection failure due to different information formats, which is conducive to achieving seamless task switching.

[0081] The following describes an embodiment of the vehicle trajectory planning method provided in this application, specifically the spatiotemporal joint attention encoding in step S102 above.

[0082] In some embodiments of this application, spatiotemporal joint attention encoding may include temporal attention encoding and spatial attention encoding.

[0083] Temporal attention encoding can be achieved by performing self-attention processing on the state representation information of each entity in the scene representation information. Entities include the vehicle itself and target objects, where target objects are other objects within the current vehicle's environment besides the vehicle. Taking a single entity as an example, self-attention processing can be performed on its state representation information, and the result can be used as the temporal attention encoding result. The temporal attention encoding result can reflect the entity's motion state, allowing the trajectory planning network to determine whether the entity is accelerating towards or away from the vehicle.

[0084] Spatial-dimensional attention encoding can be achieved by performing cross-attention processing on the vehicle's state representation information and other information within the scene representation information. This other information includes environmental representation information, task representation information, and the state representation information of all target entities. The spatial-dimensional attention encoding result can reflect the interaction relationships between the vehicle and environmental information and target entities, which can be represented by the importance of environmental information and target entities to the vehicle's trajectory planning. For example, based on the spatial-dimensional attention encoding result, the trajectory planning network can determine which environmental information and target entities are most important for the vehicle's trajectory planning at the current moment, and at which moments each environmental information and target entity is most important for the vehicle's trajectory planning. This allows for greater utilization of this important environmental information and target entities to determine driving decisions, and trajectory planning based on these driving decisions.

[0085] Combining the encoding results of temporal and spatial attention encoding yields the encoding result of spatiotemporal joint attention encoding. Based on the encoding result of spatiotemporal joint attention encoding, the trajectory planning network can simultaneously combine the encoding results of temporal and spatial attention encoding to determine safe and reliable driving decisions, and then perform trajectory planning based on these decisions. For example, the trajectory planning network determines the driving decision by decoding the spatiotemporal joint attention encoding result as "target A has been moving towards the parking space in the past two frames, and the vehicle needs to slow down in advance and adjust the timing of parking." Based on this driving decision, the network plans the vehicle's trajectory, thus avoiding collisions with target A and ensuring the vehicle's safety when controlling the vehicle's movement according to this trajectory.

[0086] This embodiment utilizes the same trajectory planning network to complete temporal and spatial attention encoding, enabling the trajectory planning network to simultaneously absorb the historical motion states of each target in the vehicle's environment, as well as the interaction relationships between the vehicle and the environment and the target. This allows the network to quickly plan a safe and reliable driving trajectory using the historical motion states and interaction relationships, avoiding hesitation, repeated attempts, and frequent replanning.

[0087] In some implementations, when performing spatiotemporal joint attention encoding on scene representation information, the scene representation information can be sequentially encoded in the temporal dimension and then in the spatial dimension, i.e., a decomposed two-level attention encoding is adopted.

[0088] Specifically, as described above, the encoding result of temporal attention encoding includes the self-attention processing result of each entity. When performing spatial attention encoding, the self-attention processing results of all entities, environmental representation information, and task representation information can be used as input data. Then, the query vector is obtained based on the self-attention processing result of the vehicle in the input data, and the key vector and value vector are obtained based on the remaining input data. Finally, attention calculation is performed based on the query, key, and value vectors to obtain the encoding result of spatial attention encoding. This encoding result is also the encoding result of the entire spatiotemporal joint attention encoding.

[0089] In some implementations, the scene representation information can be simultaneously encoded with temporal and spatial attention, i.e., an integrated attention encoding method can be used.

[0090] The following describes an embodiment of the vehicle trajectory planning method provided in this application, specifically the training method for the trajectory planning network.

[0091] In some embodiments according to this application, it is possible to... Figure 3 The following steps S201 to S202 are shown to train the trajectory planning network.

[0092] Step S201: Determine the joint loss function used to train the trajectory planning network. The joint loss function is a loss function obtained by weighting and summing multiple sub-loss functions.

[0093] Multiple sub-loss functions correspond one-to-one with multiple different network training objectives.

[0094] In this embodiment, the multiple sub-loss functions may include at least a horizontal-vertical coupling loss function. The network training objective corresponding to the horizontal-vertical coupling loss function is to minimize the deviation between the driving trajectory output by the trajectory planning network and the optimal trajectory. Among them, the optimal trajectory is the optimal trajectory obtained by trajectory optimization with the coupling relationship between lateral acceleration, longitudinal velocity and trajectory curvature as the constraint condition. This coupling relationship can be shown in the following equation (4):

[0095] (4)

[0096] The meanings of the parameters in formula (4) are as follows: Indicates lateral acceleration. Indicates longitudinal velocity. This represents the curvature of the travel trajectory.

[0097] In this embodiment, a conventional trajectory optimization algorithm can be used to obtain the optimal trajectory by using the above-mentioned coupling relationship as a constraint. In some implementations, the iLQR (iterative Linear Quadratic Regulator) trajectory optimization algorithm can be used. iLQR is a trajectory optimization algorithm based on dynamic programming, used to solve the optimal control problem of nonlinear, non-convex systems. It gradually approximates the optimal solution by iteratively calculating the linear quadratic controller and the state trajectory. Based on this, the horizontal and vertical coupling loss function in this embodiment can be expressed as follows (5):

[0098] (5)

[0099] The meanings of the parameters in formula (5) are as follows: Represents the horizontal and vertical coupling loss function The calculated loss value, This represents the driving trajectory output by the trajectory planning network. This indicates that the iLQR trajectory optimization algorithm was used to obtain the optimal trajectory.

[0100] By using the aforementioned horizontal-vertical coupling loss function to train the trajectory planning network, the network can learn to perform coupled planning for both horizontal and vertical directions. Based on this capability, the trajectory planning network can encode the spatiotemporal joint attention result of scene representation information and simultaneously decode the horizontal and vertical motion information.

[0101] Step S202: Employ a joint loss function and conduct supervised training on the trajectory planning network based on samples of scene representation information.

[0102] The joint loss function calculates the loss value by weighting and summing the loss values ​​of all sub-loss functions. During supervised training, the network parameters of the trajectory planning network are updated based on the joint loss function. This can also be understood as simultaneously updating the network parameters based on the network training objectives corresponding to all sub-loss functions. Therefore, after completing training using the joint loss function, all the aforementioned network training objectives are simultaneously achieved.

[0103] Based on the methods described in steps S201 to S202 above, the trajectory planning network can be trained by integrating multiple network training objectives, so that the trained trajectory planning network can generate safer, more reliable and stable driving trajectories.

[0104] The following describes an embodiment of the vehicle trajectory planning method provided in this application, specifically the joint loss function used during trajectory planning network training.

[0105] In some embodiments according to this application, the sample of scene representation information may further include the vehicle's true intent information, which can also be understood as the vehicle's planning decision, such as detouring, yielding, or following. The joint loss function may also include at least one of the first to fifth loss functions, which are described below.

[0106] (1) First loss function The corresponding network training objective is to minimize the deviation between the driving trajectory output by the trajectory planning network and the ground truth trajectory, where the ground truth trajectory is the actual driving trajectory labeled with sample representation information of the scene. Training the trajectory planning network based on this first loss function can make the driving trajectory predicted by the trajectory planning network closer to the ground truth trajectory. For example, the position, heading, speed, and other information of the driving trajectory are closer to the ground truth trajectory.

[0107] (2) Second loss function The corresponding network training objective is to ensure that the driving trajectory output by the trajectory planning network meets preset dynamic constraints. This embodiment does not specifically limit the content of the dynamic constraints. For example, in some implementations, dynamic constraints may include adjacent points satisfying vehicle-vehicle model approximation, curvature continuity, etc.

[0108] (3) Third loss function The corresponding network training objective is to ensure that the driving trajectory output by the trajectory planning network meets preset safety constraints. This embodiment does not specifically limit the content of these safety constraints. For example, in some implementations, safety constraints may include ensuring the distance between the vehicle and static / dynamic obstacles is within a safe range, to avoid collisions with obstacles while the vehicle is traveling along the predicted trajectory.

[0109] (4) Fourth loss function The corresponding network training objective is to ensure that the driving trajectory output by the trajectory planning network meets preset comfort constraints. This embodiment does not specifically limit the content of these comfort constraints. For example, in some implementations, the comfort constraints may include the rate of change of the trajectory curvature within a first preset range and the trajectory acceleration within a second preset range, to make the vehicle more stable when traveling along the predicted trajectory.

[0110] (5) Fifth loss function The corresponding network training objective is to minimize the deviation between the predicted intent information output by the trajectory planning network and the ground truth intent information. Training the trajectory planning network based on this fifth loss function ensures that the predicted vehicle intent (Ego intent) aligns with the ground truth intent information. For example, in a narrow oncoming traffic scenario, the ground truth intent information labeled in the scene representation samples is "yield to the right" (i.e., telling the trajectory planning network what to do). After training the trajectory planning network based on the fifth loss function and using this sample, if the scene representation information of the narrow oncoming traffic scenario is input into the trajectory planning network, the network can predict the vehicle intent to yield to the right (i.e., the trained trajectory planning network knows what it should do).

[0111] The above five loss functions and the horizontal and vertical coupling loss function described in the foregoing embodiments. After weighted summation, the joint loss function is shown in equation (6) below:

[0112] (6)

[0113] The meanings of the parameters in formula (6) are as follows: Denotes the joint loss function. , , , , These represent the weights of the second, third, fourth, horizontal and vertical coupling loss functions, and the fifth loss function, respectively, with the weight of the first loss function being 1.

[0114] By combining the horizontal and vertical coupling loss function with the five loss functions mentioned above and training the common trajectory planning network, problems caused by the decoupling of horizontal and vertical planning, such as "the path looks stable but the speed is dizzying" or "the speed is smooth but the path rubs against the pillars", can be avoided.

[0115] The following is in conjunction with the appendix Figure 4 This paper describes the training and application of the trajectory planning network in this application.

[0116] In some embodiments according to this application, the vehicle is equipped with sensors such as cameras, radar, GNSS (Global Navigation Satellite System), IMU (Inertial Measurement Unit), and odometers to collect vehicle status information (such as speed, acceleration, turning angle, etc.) and environmental information of the vehicle's surroundings.

[0117] The vehicle's intelligent driving control system can be deployed on the vehicle's domain controller / computing platform. For example... Figure 4 As shown, the intelligent driving control system may include the following functional modules: data acquisition module, perception fusion module, map / geometric constraint module, dual-modal task input adaptation module, trajectory planning network, and control interface module.

[0118] The data acquisition module can acquire the sensor data collected by the aforementioned sensors.

[0119] The perception fusion module can fuse and perceive data from different sensors to determine the environmental information of the vehicle's environment. This environmental information can include information such as static obstacles, dynamic obstacles, and the vehicle's drivable area.

[0120] The map / geometric constraint module can provide information such as lane centerlines, lane boundaries, road / parking lot topology, geometric information of static obstacles, and parking space structure (such as parking space boundaries and pose).

[0121] The dual-modal task input adaptation module can provide input information for cruise and parking tasks. The input information for cruise tasks can include cruise reference lines, while the input information for parking tasks can include the parking target point.

[0122] Trajectory planning networks can perform spatiotemporal joint attention encoding on scene representation information and decode the encoding results to obtain the vehicle's driving trajectory.

[0123] The control interface module sends the driving trajectory obtained from the trajectory planning network to the vehicle's trajectory tracking controller, which can then control the vehicle's driving based on the trajectory. It's important to note that the same control interface module is used to send the driving trajectory to the vehicle's trajectory tracking controller for both cruise and parking tasks. This further avoids anomalies such as jitter / replanning during the transition between cruise and parking tasks, achieving seamless task transition. In some implementations, the system can update the data from the data acquisition module, perception fusion module, map / geometric constraint module, and bimodal task input adaptation module at a fixed cycle (e.g., 10Hz). When switching from cruise to parking, only the input information and category identifier need to be changed through the bimodal task input adaptation module; the form of the driving trajectory generated by the trajectory planning network remains unchanged, thus achieving seamless switching between cruise and parking.

[0124] In this embodiment, the trajectory planning network can also be trained offline using the aforementioned intelligent driving control system. During offline training, the scene representation information is preset sample information. This sample information includes not only environmental representation information, task representation information, vehicle state representation information, and target state representation information, but also the vehicle's ground truth intent information. When decoding the encoding result of the spatiotemporal joint attention encoding, the trajectory planning network can not only decode the trajectory but also decode the vehicle's intent to obtain predicted intent information. Furthermore, in this embodiment, a network training loss function module can be set within the system. This module can calculate the loss value of the joint loss function based on the driving trajectory and predicted intent information output by the trajectory planning network, according to the joint loss function described in the aforementioned embodiments. Then, the parameters of the trajectory planning network are updated based on the loss value to optimize the trajectory planning network.

[0125] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of this application, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in this application and therefore will also fall within the protection scope of this application.

[0126] Those skilled in the art will understand that all or part of the processes in the method of the above-described embodiment can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0127] Another aspect of this application provides a computer-readable storage medium.

[0128] In one embodiment of a computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program that performs the vehicle trajectory planning method for integrated parking and driving according to the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0129] Another aspect of this application provides a smart device.

[0130] In one embodiment of a smart device according to this application, the smart device may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the above embodiments. The smart device described in this application may include driving equipment, smart vehicles, robots, and other devices. See appendix. Figure 5 , Figure 5 The example illustrates a memory and processor connected via a bus communication connection.

[0131] In some embodiments of this application, the smart device may further include at least one sensor for sensing information. The sensor is communicatively connected to any type of processor mentioned in this application. Optionally, the smart device may further include an autonomous driving system for guiding the smart device to drive autonomously or assisting in driving. The processor communicates with the sensor and / or the autonomous driving system to perform the methods described in any of the above embodiments.

[0132] In the description of this application, "processor" can include hardware, software, or a combination of both. A processor can be a central processing unit, microprocessor, graphics processor, digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor can be implemented in software, in hardware, or a combination of both. Computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B.

[0133] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A vehicle trajectory planning method for integrated driving and parking, characterized in that, The method includes: Acquire scene representation information of the vehicle driving scenario. The scene representation information includes environmental representation information, task representation information, vehicle state representation information, and target state representation information at multiple consecutive time points. The multiple consecutive time points include the current time and multiple consecutive historical time points before it. The task representation information includes task input information and category identifier. The task is a cruise task or parking task in the driving and parking integrated task. The vehicle is the current vehicle. The target is other objects in the environment where the current vehicle is located, excluding the current vehicle. A trajectory planning network is used to perform spatiotemporal joint attention encoding on the scene representation information, and the encoding result is decoded to obtain the current vehicle's driving trajectory. The driving trajectory includes trajectory point information at multiple future moments, and the trajectory point information includes at least the position and speed of the trajectory point; the trajectory planning network is obtained through supervised training based on samples of the scene representation information. The spatiotemporal joint attention encoding includes temporal attention encoding and spatial attention encoding; the temporal attention encoding is to perform self-attention processing on the state representation information of each entity in the scene representation information, and the entities include the vehicle and the target; the spatial attention encoding is to perform cross-attention processing on the state representation information of the vehicle in the scene representation information and other information, and the other information includes the environmental representation information, the task representation information and the state representation information of all target entities.

2. The method according to claim 1, characterized in that, The spatiotemporal joint attention encoding of the scene representation information includes: sequentially performing the temporal dimension attention encoding and the spatial dimension attention encoding on the scene representation information.

3. The method according to claim 1, characterized in that, The task representation information is obtained through the following methods: The input information and category identifier of the task are encoded separately, and the encoding results of the input information and category identifier are fused to obtain the task representation information.

4. The method according to claim 1 or 3, characterized in that, The input information for the cruise mission includes the cruise reference line; The input information for the parking task includes the parking target point.

5. The method according to claim 1, characterized in that, The trajectory planning network was also trained in the following way: A joint loss function is determined for training the trajectory planning network, wherein the joint loss function is a loss function obtained by weighting and summing multiple sub-loss functions; The trajectory planning network is trained in a supervised manner using the joint loss function and samples of the scene representation information. Among them, the multiple sub-loss functions correspond one-to-one with multiple different network training objectives; The plurality of sub-loss functions include at least a horizontal-vertical coupling loss function. The network training objective corresponding to the horizontal-vertical coupling loss function is to minimize the deviation between the driving trajectory output by the trajectory planning network and the optimal trajectory. The optimal trajectory is the optimal trajectory obtained by trajectory optimization with the coupling relationship between lateral acceleration, longitudinal velocity and trajectory curvature as constraints.

6. The method according to claim 5, characterized in that, The optimal trajectory is the optimal trajectory obtained by using the iLQR trajectory optimization algorithm and the coupling relationship as a constraint condition. And / or, the trajectory point information in the driving trajectory also includes the curvature and heading angle of the trajectory points.

7. The method according to claim 5, characterized in that, The scene representation information sample also includes the vehicle's ground truth intent information, and the plurality of sub-loss functions further include at least one of the following loss functions: The first loss function corresponds to the network training objective of minimizing the deviation between the driving trajectory output by the trajectory planning network and the true trajectory, where the true trajectory is the actual driving trajectory labeled by the sample. The second loss function corresponds to the network training objective of making the driving trajectory output by the trajectory planning network satisfy the preset dynamic constraints. The third loss function corresponds to the network training objective of making the driving trajectory output by the trajectory planning network meet the preset safety constraints. The fourth loss function corresponds to the network training objective of making the driving trajectory output by the trajectory planning network meet the preset comfort constraints. The fifth loss function corresponds to the network training objective of minimizing the deviation between the predicted intent information output by the trajectory planning network and the true intent information.

8. A smart device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores a computer program, which, when executed by the at least one processor, implements the vehicle trajectory planning method for integrated driving and parking as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the vehicle trajectory planning method for parking and driving integration as described in any one of claims 1 to 7.

Images