Trajectory optimization method and device, and perceptual model training method and device

Abstract

The invention discloses a trajectory optimization method and device and a perceptual model training method and device. The trajectory optimization method comprises the following steps: acquiring multi-view image data acquired by a vehicle at the current moment and current vehicle state information; on the basis of the multi-view image data, corridor prediction and trajectory prediction are carried out through a pre-trained perception model, and predicted safe corridor information and predicted trajectory information of the vehicle within a predicted duration are obtained; and based on the predicted safety corridor information, the predicted trajectory information, the current vehicle state information and trajectory optimization conditions, optimizing the predicted trajectory information to obtain optimized predicted trajectory information. By adopting the method and the device, the track can be optimized through the sensing model on the premise of meeting the safety constraint and the vehicle dynamics limitation, so that the collision risk is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular, to a trajectory optimization method and device, as well as a training method and device for a perception model. Background Art

[0002] In recent years, end-to-end intelligent driving technology has developed rapidly. An end-to-end perception model can directly generate a predicted trajectory based on multi-view image data collected by a vehicle. However, since the process of generating a predicted trajectory by the end-to-end perception model lacks explicit safety constraints and compliance with physical laws, the avoidance effect of the predicted trajectory generated by the end-to-end perception model for road boundaries (such as curbs) and other traffic participants is limited, and its average collision rate is still relatively high.

[0003] Therefore, there is an urgent need for a method capable of optimizing the predicted trajectory generated by the end-to-end perception model. Summary of the Invention

[0004] To solve the above technical problems, the present disclosure provides a trajectory optimization method and device, as well as a training method and device for a perception model, so as to optimize the trajectory through the perception model on the premise of meeting safety constraints and vehicle dynamics limitations, thereby reducing the collision risk.

[0005] In a first aspect of the present disclosure, a trajectory optimization method is provided, including:

[0006] Obtaining multi-view image data collected by the host vehicle at the current moment and the current vehicle state information;

[0007] Based on the multi-view image data, performing corridor prediction and trajectory prediction through a pre-trained perception model to obtain the predicted safe corridor information and predicted trajectory information of the host vehicle within the prediction duration;

[0008] Based on the predicted safe corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization conditions, optimizing the predicted trajectory information to obtain optimized predicted trajectory information.

[0009] In a second aspect of the present disclosure, a training method for a perception model is provided, including:

[0010] Obtaining multi-view image data corresponding to a sample vehicle;

[0011] Based on the multi-view image data, performing corridor prediction and trajectory prediction through an initial perception model to obtain the predicted safe corridor information and predicted trajectory information of the sample vehicle within the prediction duration;

[0012] Based on the predicted safety corridor information and a preset target loss function, optimize the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

[0013] In a third aspect of the present disclosure, a trajectory optimization device is provided, including:

[0014] A data acquisition module, configured to acquire multi-view image data and current vehicle state information collected by the host vehicle at the current moment;

[0015] A first perception and prediction module, configured to perform corridor prediction and trajectory prediction through a pre-trained perception model based on the multi-view image data, to obtain predicted safety corridor information and predicted trajectory information of the host vehicle within a prediction duration;

[0016] A trajectory optimization module, configured to optimize the predicted trajectory information based on the predicted safety corridor information, the predicted trajectory information, the current vehicle state information, and trajectory optimization conditions, to obtain optimized predicted trajectory information.

[0017] In a fourth aspect of the present disclosure, a training device for a perception model is provided, including:

[0018] A data acquisition module, configured to acquire multi-view image data corresponding to a sample vehicle;

[0019] A perception and prediction module, configured to perform corridor prediction and trajectory prediction through an initial perception model based on the multi-view image data, to obtain predicted safety corridor information and predicted trajectory information of the sample vehicle within a prediction duration;

[0020] A first parameter optimization module, configured to optimize the corridor prediction parameters in the initial perception model based on the predicted safety corridor information and a preset target loss function, to obtain a perception model with optimized parameters.

[0021] In a fifth aspect of the present disclosure, a computer-readable storage medium is provided, where the storage medium stores a computer program, and the computer program is used to execute the trajectory optimization method provided in the first aspect above, or the training method for the perception model provided in the second aspect above.

[0022] In a sixth aspect of the present disclosure, an electronic device is provided, where the electronic device includes: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the trajectory optimization method provided in the first aspect above, or the training method for the perception model provided in the second aspect above.

[0023] A seventh aspect embodiment of the present disclosure provides a computer program product that, when executed by an instruction processor in the computer program product, executes the trajectory optimization method provided in the first aspect of the present disclosure, or the training method of the perception model provided in the second aspect above.

[0024] In an embodiment of the present disclosure, an electronic device acquires multi-view image data and current vehicle state information collected by the vehicle itself at the current moment. Then, based on the multi-view image data, the electronic device performs corridor prediction and trajectory prediction through a pre-trained perception model to obtain predicted safe corridor information and predicted trajectory information of the vehicle itself within the prediction duration. After that, the electronic device optimizes the predicted trajectory information based on the predicted safe corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization conditions to obtain optimized predicted trajectory information. In this way, by optimizing the predicted trajectory information through the trajectory optimization conditions, the optimized predicted trajectory information can conform to safety constraints and follow vehicle dynamics limitations, thereby reducing the collision risk. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flowchart of a trajectory optimization method provided by an exemplary embodiment of the present disclosure.

[0026] Figure 2 is a structural diagram of a perception model provided by an exemplary embodiment of the present disclosure.

[0027] Figure 3 is a flowchart of a trajectory optimization method provided by another exemplary embodiment of the present disclosure.

[0028] Figure 4 is a flowchart of a trajectory optimization method provided by another exemplary embodiment of the present disclosure.

[0029] Figure 5 is a flowchart of a trajectory optimization method provided by another exemplary embodiment of the present disclosure.

[0030] Figure 6 is a flowchart of a trajectory optimization method provided by another exemplary embodiment of the present disclosure.

[0031] Figure 7 is a flowchart of a training method of a perception model provided by an exemplary embodiment of the present disclosure.

[0032] Figure 8 is a training diagram of a perception model provided by an exemplary embodiment of the present disclosure.

[0033] Figure 9 is a structural diagram of a trajectory optimization device provided by an exemplary embodiment of the present disclosure.

[0034] Figure 10It is a schematic structural diagram of a training device for a perception model provided by an exemplary embodiment of the present disclosure.

[0035] Figure 11 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed implementation manners

[0036] To explain the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all embodiments. It should be understood that the present disclosure is not limited by the exemplary embodiments.

[0037] It should be noted that: unless otherwise specifically stated, the relative arrangements, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0038] Application overview

[0039] In recent years, end-to-end intelligent driving technology has developed rapidly. The end-to-end perception model can directly generate predicted trajectory information based on multi-view image data collected by the vehicle. The end-to-end perception model directly maps the input (multi-view image data) to the output (predicted trajectory information) only through a data-driven manner, focusing more on the mapping relationship between the multi-view image data and the predicted trajectory information, without explicitly incorporating safety constraints such as road boundaries and traffic rules, nor considering vehicle dynamics limitations.

[0040] On the one hand, since the end-to-end perception model does not explicitly incorporate safety constraints such as road boundaries and traffic rules, the safety distance between the vehicle and the road boundary and other traffic participants in the predicted trajectory information generated by it is too close, resulting in an easy collision in an emergency. On the other hand, since the end-to-end perception model does not consider vehicle dynamics limitations (such as steering angle and acceleration limit), the predicted trajectory information generated by it exceeds the vehicle's own dynamics limitations, resulting in an easy out-of-control of the vehicle in an emergency and increasing the collision risk.

[0041] To solve the above problems, the present disclosure provides a trajectory optimization method to optimize the trajectory through the end-to-end perception model on the premise of meeting safety constraints and dynamics limitations, thereby reducing the collision risk.

[0042] In the embodiments of the present disclosure, an electronic device obtains multi-view image data and current vehicle state information collected by the host vehicle at the current moment. Then, based on the multi-view image data, the electronic device performs corridor prediction and trajectory prediction through a pre-trained perception model to obtain predicted safe corridor information and predicted trajectory information of the host vehicle within the prediction duration. After that, the electronic device optimizes the predicted trajectory information based on the predicted safe corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization conditions, to obtain the optimized predicted trajectory information. In this way, the predicted trajectory information is optimized through the trajectory optimization conditions, so that the optimized predicted trajectory information can meet safety constraints and follow vehicle dynamics limitations, thereby reducing the collision risk.

[0043] Exemplary Method

[0044] Figure 1 is a schematic flowchart of a trajectory optimization method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, such as Figure 1 shown, and includes the following steps:

[0045] Step 101, obtain multi-view image data and current vehicle state information collected by the host vehicle at the current moment.

[0046] Exemplarily, during the driving of the host vehicle, the host vehicle can collect multi-view image data at the current moment through multi-view image sensors installed on the vehicle (such as front-view image sensors, rear-view image sensors, side-view image sensors, and surround-view image sensors, etc.). At the same time, the vehicle can also collect current vehicle state information through positioning sensors, inertial sensors, wheel speed sensors, steering wheel angle sensors, etc. installed on the vehicle. Among them, the vehicle state information can at least include vehicle position information, vehicle speed information, and vehicle heading information.

[0047] Step 102, based on the multi-view image data, perform corridor prediction and trajectory prediction through a pre-trained perception model to obtain predicted safe corridor information and predicted trajectory information of the host vehicle within the prediction duration.

[0048] Exemplarily, after the electronic device obtains the multi-view image data collected by the host vehicle at the current moment, it can further input the multi-view image data into the pre-trained perception model. The training process of the perception model will be introduced in detail later and will not be elaborated here. Figure 2 is a schematic structural diagram of a perception model provided by an exemplary embodiment of the present disclosure. As Figure 2As shown in the figure, the perception model 200 includes a feature map generation module 210, an obstacle perception module 220, a corridor prediction module 230, and a trajectory prediction module 240. Among them, the feature map generation module 210 is configured to receive multi-view image data collected by a vehicle, perform feature extraction, view transformation, and feature aggregation on the multi-view image data to obtain a bird's-eye view feature map. The obstacle perception module 220 is configured to identify dynamic and static obstacles in the driving scenario where the vehicle is located based on the bird's-eye view feature map, and predict the obstacle perception information of the identified dynamic and static obstacles within the prediction duration. The corridor prediction module 230 is configured to perform corridor prediction based on the bird's-eye view feature map and output the predicted safe corridor information of the vehicle within the prediction duration. The trajectory prediction module 240 is configured to perform trajectory prediction based on the bird's-eye view feature map and output the predicted trajectory information of the vehicle within the prediction duration. The feature map generation module 210, the obstacle perception module 220, the corridor prediction module 230, and the trajectory prediction module 240 in the perception model 200 may adopt a CNN (Convolutional Neural Networks), or may adopt a Transformer neural network model, which is not limited in the embodiments of the present disclosure.

[0049] First, the feature map generation module 210 in the perception model 200 can perform feature extraction, view transformation, and feature aggregation on the multi-view image data to obtain a bird's-eye view feature map. Among them, for the image data of each view, the feature map generation module 210 can first perform feature extraction processing on the image data of this view to obtain the feature map of this view. After the feature map generation module 210 obtains the feature maps of each view, for each view, the feature map generation module 210 can determine the mapping relationship from this view to the bird's-eye view based on the internal and external parameters of the image sensor of this view, and based on the mapping relationship from this view to the bird's-eye view, perform view transformation on the feature map of this view to obtain the bird's-eye view feature map of this view. After the feature map generation module 210 obtains the bird's-eye view feature maps of each view, it can further perform feature fusion processing on the bird's-eye view feature maps of each view to obtain the bird's-eye view feature map corresponding to the bird's-eye view.

[0050] Then, the corridor prediction module 230 and the trajectory prediction module 240 in the perception model 200 can respectively perform corridor prediction and trajectory prediction based on the bird's-eye feature map, and output the predicted safe corridor information and predicted trajectory information of the ego vehicle within the prediction duration. The prediction duration is a preset duration after the current moment, and the prediction duration can include multiple moments, and the time interval between any two adjacent moments is the same. For example, if the prediction duration is 3 s and the time interval △t between adjacent moments is 0.5 s, the prediction duration includes 6 moments. The predicted safe corridor information includes collision-free regions corresponding to multiple moments, and the collision-free region can be represented as (x, y, θ, l, w). Among them, x and y respectively represent the abscissa and ordinate of the center point of the collision-free region, θ represents the deflection angle of the collision-free region, l represents the length of the collision-free region, and w represents the width of the collision-free region. The predicted trajectory information can include trajectory points corresponding to multiple moments.

[0051] Step 103: Optimize the predicted trajectory information based on the predicted safe corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization condition to obtain the optimized predicted trajectory information.

[0052] Exemplarily, the electronic device can pre-store the trajectory optimization condition. The trajectory optimization condition is used to optimize the predicted trajectory information output by the perception model to ensure that the optimized predicted trajectory information can meet the safety constraints (i.e., the predicted safe corridor information), follow the vehicle dynamics limitations, improve the smoothness and comfort of vehicle control, and ensure that the optimized predicted trajectory information is as close as possible to the predicted trajectory information output by the perception model. Therefore, after the electronic device determines the predicted safe corridor information and the predicted trajectory information, it can further determine, based on the trajectory optimization condition, that on the basis of constraining the ego vehicle within the predicted safe corridor information, and based on the current vehicle state information, the trajectory points at each moment in the optimized predicted trajectory information are as close as possible to the trajectory points at each moment in the predicted trajectory information, so as to optimize the predicted trajectory information and obtain the optimized predicted trajectory information.

[0053] In the embodiments of the present disclosure, the electronic device acquires the multi-view image data and the current vehicle state information collected by the ego vehicle at the current moment. Then, the electronic device performs corridor prediction and trajectory prediction through a pre-trained perception model based on the multi-view image data to obtain the predicted safe corridor information and the predicted trajectory information of the ego vehicle within the prediction duration. After that, the electronic device optimizes the predicted trajectory information based on the predicted safe corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization condition to obtain the optimized predicted trajectory information. In this way, the predicted trajectory information is optimized through the trajectory optimization condition, so that the optimized predicted trajectory information can meet the safety constraints and follow the vehicle dynamics limitations, thereby reducing the collision risk.

[0054] Such asFigure 3 As shown Figure 1 Based on the above-described embodiments, step 103 may include the following steps:

[0055] Step 1031: Determine trajectory optimization conditions based on the predicted safety corridor information and the predicted trajectory information.

[0056] Exemplarily, after the electronic device obtains the predicted safety corridor information and the predicted trajectory information, it may further determine trajectory optimization conditions based on the predicted safety corridor information and the predicted trajectory information. Among them, the trajectory optimization conditions can be used to optimize the predicted trajectory information output by the perception model to ensure that the optimized predicted trajectory information can meet the safety constraints and that the optimized predicted trajectory information is as close as possible to the predicted trajectory information output by the perception model. The trajectory optimization conditions can also be used to optimize the predicted trajectory information output by the perception model to ensure that the optimized predicted trajectory information can meet the safety constraints and follow the vehicle dynamics limitations, improve the smoothness and comfort of vehicle control, and ensure that the optimized predicted trajectory information is as close as possible to the predicted trajectory information output by the perception model. Among them, the process of the electronic device determining the trajectory optimization conditions based on the predicted safety corridor information and the predicted trajectory information will be introduced in detail later and will not be elaborated here. That the optimized predicted trajectory information is as close as possible to the predicted trajectory information output by the perception model means that, under the condition that the optimized predicted trajectory information can meet the safety constraints and follow the vehicle dynamics limitations, the trajectory trend of the optimized predicted trajectory information is the same as that of the predicted trajectory information, the deviation of the trajectory points at the same moment is as small as possible, and the change trend of the speed over time is as similar as possible.

[0057] Step 1032: Determine the target vehicle state information and the target control signal of the host vehicle within the prediction duration based on the current vehicle state information and the trajectory optimization conditions.

[0058] Exemplarily, after the electronic device determines the trajectory optimization conditions, it may determine the target vehicle state information and target control signals corresponding to each moment within the prediction duration based on the current vehicle state information and the trajectory optimization conditions. Among them, the target vehicle state information and the target control signals corresponding to each moment within the prediction duration conform to the vehicle dynamics model. The target control signals corresponding to each moment within the prediction duration are the control signals required for the host vehicle to gradually reach the trajectory points corresponding to the prediction trajectory information at each moment starting from the current vehicle state information under the condition of meeting the trajectory optimization conditions. Among them, the control signals at least include acceleration and steering angle. The target vehicle state information corresponding to each moment within the prediction duration is the vehicle state information that the host vehicle gradually reaches at each moment based on the manipulation of the target control signals corresponding to each moment starting from the current vehicle state information. The process by which the electronic device determines the target vehicle state information and target control signals of the host vehicle within the prediction duration based on the current vehicle state information and the trajectory optimization conditions will be introduced in detail later and will not be elaborated here.

[0059] Step 1033: Optimize the prediction trajectory information based on the target vehicle state information and target control signals of the host vehicle within the prediction duration to obtain the optimized prediction trajectory information.

[0060] Exemplarily, after the electronic device determines the target vehicle state information and target control signals of the host vehicle within the prediction duration, for each moment within the prediction duration, the electronic device may update the trajectory point corresponding to the prediction trajectory information at this moment based on the target vehicle state information and target control signals corresponding to this moment, so as to optimize the prediction trajectory information and obtain the optimized prediction trajectory information.

[0061] In the embodiments of the present disclosure, based on the prediction safety corridor information and the prediction trajectory information, the trajectory optimization conditions are determined, and the prediction trajectory information output by the perception model is optimized based on the trajectory optimization conditions, so that the optimized prediction trajectory information can meet the safety constraints and follow the vehicle dynamics limitations, improving the smoothness and comfort of vehicle control, and making the optimized prediction trajectory information as close as possible to the prediction trajectory information output by the perception model.

[0062] Based on the above Figure 3 On the basis of the shown embodiments, the trajectory optimization conditions determined in step 1031 can be various. The embodiments of the present disclosure provide two feasible trajectory optimization conditions, which are specifically as follows:

[0063] Trajectory optimization condition 1: Based on the predicted safety corridor information and the predicted trajectory information, a trajectory optimization condition is constructed with the condition that the target vehicle state information of the host vehicle within the prediction duration is within the predicted safety corridor information, and the trajectory tracking error between the target vehicle state information and the predicted trajectory information of the host vehicle within the prediction duration is minimized as the control cost.

[0064] Exemplarily, for each moment within the prediction duration, the electronic device constrains the target vehicle state information of the host vehicle corresponding to that moment within the collision-free area corresponding to the predicted safety corridor information at that moment, so that the optimized predicted trajectory information can meet the safety constraints. At the same time, the electronic device minimizes the total trajectory tracking error between the target vehicle state information of the host vehicle corresponding to each moment and the trajectory points corresponding to the predicted trajectory information at each moment, so that the optimized predicted trajectory information is close to the predicted trajectory information before optimization. This trajectory optimization condition is as shown in formula (1).

[0065]

[0066] Among them, x t represents the target vehicle state information corresponding to the t-th moment within the prediction duration, C t represents the collision-free area corresponding to the predicted safety corridor information at the t-th moment within the prediction duration, ξ t represents the trajectory point corresponding to the t-th moment within the prediction duration of the predicted trajectory information, represents the trajectory tracking error between the target vehicle state information corresponding to the (t + 1)-th moment and the trajectory point within the prediction duration, N represents the number of moments within the prediction duration, represents the trajectory tracking cost corresponding to the (t + 1)-th moment within the prediction duration, x t+1 = Ax t + Bu t x1 = x init x init represents the current vehicle state information, u t represents the target control signal corresponding to the t-th moment within the prediction duration, and A and B are respectively the state transition matrix and the control input matrix in the vehicle motion model.

[0067] Regarding Trajectory optimization condition 1, as Figure 4 shown, based on the above Figure 3 shown embodiment, step 1032 may include the following steps:

[0068] Step 401: For each first moment in the prediction duration, based on the current vehicle state information and the vehicle motion model, determine the target vehicle state information corresponding to the first moment. Among them, the target vehicle state information corresponding to the first moment is determined by the current vehicle state information and the target control signal corresponding to the second moment in the prediction duration; the prediction duration is a preset duration after the current moment, and the second moment is all moments before the first moment.

[0069] Exemplarily, the driving of the vehicle conforms to the vehicle motion model. In the vehicle motion model, the vehicle state information corresponding to the vehicle at the t + 1 moment can be determined by the vehicle state information corresponding to the vehicle at the t moment and the control signal, that is, x t+1 = Ax t + Bu t . Among them, x t+1 is the vehicle state information corresponding to the vehicle at the t + 1 moment, x t and u t are respectively the vehicle state information corresponding to the vehicle at the t moment and the control signal, and A and B are respectively the state transition matrix and the control input matrix in the vehicle motion model. Therefore, based on the vehicle motion model, for each first moment within the prediction duration, the electronic device can determine the target vehicle state information corresponding to the first moment based on the current vehicle state information and the target control signal corresponding to the second moment within the prediction duration. Among them, the second moment is all moments before the first moment. For example, assume the first moment is the t5 moment, then the second moment is from the t1 moment to the t4 moment; assume the first moment is the t7 moment, then the second moment is from the t1 moment to the t6 moment. Specifically, the target vehicle state information x1 = x init , x init is the current vehicle state information, the target vehicle state information x2 = Ax1 + Bu1 = Ax init + Bu1 for the vehicle at the t2 moment, the target vehicle state information x3 = Ax2 + Bu2 = A(Ax init + Bu1)+ Bu2 for the vehicle at the t3 moment, the target vehicle state information x4 = Ax3 + Bu3 = A(A(Ax init + Bu1)+ Bu2)+ Bu3 for the vehicle at the t4 moment, and so on.

[0070] Step 402: Based on the target vehicle state information corresponding to the first moment and the trajectory point corresponding to the prediction trajectory information at the first moment, determine the trajectory tracking error corresponding to the first moment.

[0071] Exemplarily, referring to formula (1), after the electronic device determines the target vehicle state information corresponding to each first moment within the prediction duration, it can further determine the difference between the target vehicle state information corresponding to the first moment and the trajectory point of the prediction trajectory information corresponding to the first moment as the trajectory tracking error corresponding to the first moment. For example, the trajectory tracking error corresponding to the t-th moment where x t represents the target vehicle state information corresponding to the t-th moment, and ξ t represents the trajectory point of the prediction trajectory information corresponding to the t-th moment.

[0072] Step 403: Determine the trajectory tracking error corresponding to the first moment as the control cost corresponding to the first moment.

[0073] Exemplarily, referring to formula (1), in order to make the optimized prediction trajectory information close to the prediction trajectory information before optimization, after the electronic device determines the trajectory tracking error corresponding to the first moment, it can determine the trajectory tracking error corresponding to the first moment as the control cost corresponding to the first moment. Among them, the trajectory tracking error corresponding to the first moment characterizes the gap between the target vehicle state information corresponding to the first moment and the trajectory point of the prediction trajectory information corresponding to the first moment. The smaller the trajectory tracking error, the closer the optimized prediction trajectory information is to the prediction trajectory information before optimization. Therefore, determining the trajectory tracking error corresponding to the first moment as the control cost corresponding to the first moment can ensure that the optimized prediction trajectory information is close to the prediction trajectory information before optimization.

[0074] Step 404: When the target vehicle state information corresponding to each first moment is within the predicted safety corridor information, in response to the sum of the control costs corresponding to each first moment being the smallest, determine the target vehicle state information and the target control signal corresponding to each first moment.

[0075] Exemplarily, referring to formula (1), in order to make the optimized prediction trajectory information meet the safety constraints and the optimized prediction trajectory information be close to the prediction trajectory information before optimization, the electronic device constrains the target vehicle state information corresponding to each first moment within the collision-free area corresponding to the predicted safety corridor information at each first moment (i.e., s.t. x t ∈C t , where x t represents the target vehicle state information corresponding to the t-th moment, and C t represents the collision-free area of the predicted safety corridor information corresponding to the t-th moment). Then, the electronic device can gradually adjust the target control signal corresponding to each first moment to make the sum of the control costs corresponding to each first moment the smallest (i.e., where N represents the number of moments in the prediction duration, represents the control cost corresponding to the (t + 1)-th moment in the prediction duration). When the sum of the control costs corresponding to each first moment is minimized, the electronic device determines the target control signals corresponding to each first moment (i.e., u1 to u N ), and further determines the target vehicle state information corresponding to each first moment based on the current vehicle state information and the target control signals corresponding to each first moment.

[0076] In the embodiments of the present disclosure, for each moment in the prediction duration, the electronic device constrains the target vehicle state information of the host vehicle corresponding to that moment within the collision-free region corresponding to the prediction safety corridor information at that moment, so that the optimized prediction trajectory information can meet the safety constraints. At the same time, the electronic device minimizes the total trajectory tracking error between the target vehicle state information of the host vehicle corresponding to each moment and the trajectory points corresponding to the prediction trajectory information at each moment, so that the optimized prediction trajectory information is close to the prediction trajectory information before optimization.

[0077] Trajectory optimization condition two: Based on the prediction safety corridor information, the prediction trajectory information, and a preset control signal range, a trajectory optimization condition is constructed with the constraint that the target vehicle state information of the host vehicle within the prediction duration is within the prediction safety corridor information and the target control signal of the host vehicle within the prediction duration is within the control signal range, and with the sum of the trajectory tracking error between the target vehicle state information of the host vehicle within the prediction duration and the prediction trajectory information and the target control signal of the host vehicle within the prediction duration being minimized as the control cost.

[0078] Exemplarily, for each moment within the prediction duration, the electronic device constrains the target vehicle state information of the host vehicle corresponding to that moment within the collision-free region corresponding to the prediction safety corridor information at that moment, so that the optimized prediction trajectory information can meet the safety constraints, and constrains the target control signal of the host vehicle corresponding to that moment within the control signal range, so that the optimized prediction trajectory information follows the vehicle dynamics limitations, improving the smoothness and comfort of vehicle handling. At the same time, the electronic device minimizes the sum of the total trajectory tracking error between the target vehicle state information of the host vehicle corresponding to each moment and the trajectory points corresponding to the prediction trajectory information at each moment and the target control signal of the host vehicle corresponding to each moment, so that the optimized prediction trajectory information is close to the prediction trajectory information before optimization on the basis of following the vehicle dynamics limitations. This trajectory optimization condition two is shown in formula (2).

[0079]

[0080] where x t represents the target vehicle state information corresponding to the t-th moment in the prediction duration, u t represents the target control signal corresponding to the t-th moment in the prediction duration, C trepresents the collision-free area corresponding to the t-th moment in the prediction duration of the predicted safety corridor information, Q and R are symmetric positive definite weight matrices, u min represents the minimum control signal in the control signal range, u max represents the maximum control signal in the control signal range, ξ t represents the trajectory point corresponding to the t-th moment in the prediction duration of the predicted trajectory information, represents the trajectory tracking error between the target vehicle state information and the trajectory point corresponding to the (t + 1)-th moment in the prediction duration, N represents the number of moments in the prediction duration, represents the trajectory tracking cost corresponding to the (t + 1)-th moment in the prediction duration, represents the vehicle control cost corresponding to the t-th moment in the prediction duration, x t+1 = Ax t + Bu t , x1 = x init , x init represents the current vehicle state information, A and B are the state transition matrix and the control input matrix in the vehicle motion model respectively.

[0081] For trajectory optimization condition two, as Figure 5 shown, based on the above Figure 3 shown embodiment, for trajectory optimization condition two, step 1032 may include the following steps:

[0082] Step 501, for each first moment in the prediction duration, based on the current vehicle state information and the vehicle motion model, determine the target vehicle state information corresponding to the first moment. Among them, the target vehicle state information corresponding to the first moment is determined by the current vehicle state information and the target control signal corresponding to the second moment in the prediction duration; the prediction duration is a preset duration after the current moment, and the second moment is all moments before the first moment.

[0083] Exemplarily, the driving of the vehicle conforms to the vehicle motion model. In the vehicle motion model, the vehicle state information corresponding to the vehicle at the (t + 1)-th moment can be determined by the vehicle state information corresponding to the vehicle at the t-th moment and the control signal, that is, x t+1 = Ax t + Bu t . Among them, x t+1 is the vehicle state information corresponding to the vehicle at the (t + 1)-th moment, x t and u tare the vehicle state information and control signal corresponding to the vehicle at time t, respectively, and A and B are the state transition matrix and control input matrix in the vehicle motion model, respectively. Therefore, based on the vehicle motion model, for each first moment in the prediction duration, the electronic device can determine the target vehicle state information corresponding to the first moment based on the current vehicle state information and the target control signal corresponding to the second moment in the prediction duration. Wherein, the second moment is all moments before the first moment. For example, assuming the first moment is t5 moment, then the second moment is from t1 moment to t4 moment; assuming the first moment is t7 moment, then the second moment is from t1 moment to t6 moment. Specifically, the target vehicle state information x1 corresponding to the vehicle at t1 moment = x init , x init is the current vehicle state information, and the target vehicle state information x2 corresponding to the vehicle at t2 moment = Ax1 + Bu1 = Ax init + Bu1, the target vehicle state information x3 corresponding to the vehicle at t3 moment = Ax2 + Bu2 = A(Ax init + Bu1) + Bu2, the target vehicle state information x4 corresponding to the vehicle at t4 moment = Ax3 + Bu3 = A(A(Ax init + Bu1) + Bu2) + Bu3, and so on.

[0084] Step 502: Determine the trajectory tracking error corresponding to the first moment based on the target vehicle state information corresponding to the first moment and the trajectory point corresponding to the prediction trajectory information at the first moment.

[0085] Exemplarily, referring to formula (2), after the electronic device determines the target vehicle state information corresponding to each first moment in the prediction duration, it can further determine the difference between the target vehicle state information corresponding to the first moment and the trajectory point corresponding to the prediction trajectory information at the first moment as the trajectory tracking error corresponding to the first moment. For example, the trajectory tracking error corresponding to the t-th moment Wherein, x t represents the target vehicle state information corresponding to the t-th moment, and ξ t represents the trajectory point corresponding to the prediction trajectory information at the t-th moment.

[0086] Step 503: Determine the control cost corresponding to the first moment as the weighted sum of the trajectory tracking error corresponding to the first moment and the target control signal corresponding to the third moment in the prediction duration. Wherein, the third moment is the moment before the first moment, and the second moment includes the third moment.

[0087] Exemplarily, referring to Equation (2), in order to make the optimized predicted trajectory information close to the predicted trajectory information before optimization while following the vehicle dynamics constraints, after the electronic device determines the trajectory tracking error corresponding to the first moment, it can perform a weighted sum of the trajectory tracking error corresponding to the first moment and the target control signal corresponding to the third moment, and determine it as the control cost corresponding to the first moment. Among them, the third moment is the previous moment of the first moment, and the second moment includes the third moment. Specifically, the control cost corresponding to the (t + 1)-th moment is the weighted sum of the trajectory tracking error corresponding to the (t + 1)-th moment and the target control signal corresponding to the t-th moment. That is where Q and R are symmetric positive definite weight matrices, represents the trajectory tracking cost corresponding to the (t + 1)-th moment, represents the vehicle control cost corresponding to the t-th moment.

[0088] Step 504, when the target vehicle state information corresponding to each first moment is within the predicted safety corridor information and the target control signal corresponding to each first moment is within the control signal range, in response to the sum of the control costs corresponding to each first moment being the smallest, determine the target vehicle state information and the target control signal corresponding to each first moment.

[0089] Exemplarily, in order to make the optimized predicted trajectory information comply with safety constraints and follow the vehicle dynamics constraints, and the optimized predicted trajectory information is close to the predicted trajectory information before optimization, the electronic device constrains the target vehicle state information corresponding to each first moment within the collision-free region corresponding to the predicted safety corridor information at each first moment (i.e., s.t. x t ∈C t where x t represents the target vehicle state information corresponding to the t-th moment, and C t represents the collision-free region corresponding to the predicted safety corridor information at the t-th moment), and constrains the target control signal corresponding to each first moment within the control signal range (i.e., u t ∈[u min , u max , where u t represents the target control signal corresponding to the t-th moment in the prediction duration, u min represents the minimum control signal in the control signal range, and u max represents the maximum control signal in the control signal range). Then, the electronic device can gradually adjust the target control signal corresponding to each first moment to make the sum of the control costs corresponding to each first moment the smallest (i.e., where N represents the number of moments in the prediction duration, indicating the control cost corresponding to the (t + 1)-th moment in the prediction duration). When the sum of the control costs corresponding to each first moment is minimized, the electronic device determines the target control signals corresponding to each first moment (i.e., u1 to u N ), and further determines the target vehicle state information corresponding to each first moment based on the current vehicle state information and the target control signals corresponding to each first moment.

[0090] In the embodiments of the present disclosure, for each moment in the prediction duration, the electronic device constrains the target vehicle state information of the host vehicle corresponding to this moment within the collision-free area corresponding to the prediction safety corridor information at this moment, and constrains the target control signal of the host vehicle corresponding to this moment within the control signal range, so that the optimized prediction trajectory information can meet the safety constraints and follow the vehicle dynamics limitations, improving the smoothness and comfort of vehicle handling. At the same time, the electronic device minimizes the sum of the total trajectory tracking error between the target vehicle state information of the host vehicle corresponding to each moment and the trajectory points corresponding to the prediction trajectory information at each moment and the target control signals of the host vehicle corresponding to each moment, so that the optimized prediction trajectory information is close to the prediction trajectory information before optimization on the basis of following the vehicle dynamics limitations.

[0091] As Figure 6 shown, the electronic device also performs the following steps:

[0092] Step 601, based on the multi-view image data, perform obstacle prediction through a pre-trained perception model to obtain the predicted positions of each obstacle in the intelligent driving scenario where the host vehicle is located within the prediction duration.

[0093] Exemplarily, after the electronic device obtains the multi-view image data collected by the host vehicle at the current moment, it can further input the multi-view image data into the pre-trained perception model. First, the perception model can perform feature extraction and feature aggregation on the multi-view image data along the BEV direction to obtain a bird's-eye view feature map. Then, the obstacle perception module 220 in the perception model can perform obstacle prediction based on the bird's-eye view feature map to obtain the predicted positions of each obstacle in the intelligent driving scenario where the host vehicle is located at each moment within the prediction duration. Among them, the obstacles include dynamic obstacles (such as motor vehicles, non-motor vehicles, pedestrians, etc.) and / or static obstacles (road edges, street lights, etc.).

[0094] Step 602, in response to the presence of an obstacle in the prediction safety corridor information, optimize the prediction safety corridor information based on the target vehicle state information of the host vehicle within the prediction duration, the predicted positions of each obstacle within the prediction duration, and the maximum empty matrix algorithm to obtain the optimized prediction safety corridor information.

[0095] Exemplarily, after the electronic device obtains the predicted positions corresponding to each moment of each obstacle within the prediction duration, for each moment within the prediction duration, the electronic device may determine whether the predicted position corresponding to any obstacle at this moment is located in the obstacle-free area corresponding to the predicted safety corridor information at this moment. If the predicted position corresponding to any obstacle at this moment is not located in the obstacle-free area corresponding to the predicted safety corridor information at this moment, it indicates that there is no collision risk for the host vehicle at this moment. If the predicted position corresponding to any obstacle at this moment is located in the obstacle-free area corresponding to the predicted safety corridor information at this moment, it indicates that there is a collision risk for the host vehicle at this moment. Correspondingly, the electronic device may re-determine the collision-free area corresponding to the predicted safety corridor information at this moment based on the vehicle position corresponding to the host vehicle at this moment, the predicted positions corresponding to each obstacle at this moment, and the maximum empty matrix algorithm. Among them, based on the maximum empty matrix algorithm, the collision-free area re-determined for the predicted safety corridor information at this moment includes the vehicle position corresponding to the host vehicle at this moment and does not include the predicted position corresponding to any obstacle at this moment.

[0096] It should be noted that step 601 and step 602 may be executed after step 102 and before step 103; or may be executed after step 103, which is not limited in the embodiments of the present disclosure. The embodiments of the present disclosure preferably execute step 601 and step 602 after step 102 and before step 103. At this time, the optimized predicted safety corridor information is safer and has a lower collision risk compared with the predicted safety corridor information before optimization. Subsequently, the electronic device optimizes the predicted trajectory information based on the optimized predicted safety corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization condition, which can further reduce the collision risk of the optimized predicted trajectory information.

[0097] In the embodiments of the present disclosure, based on multi-view image data, the electronic device performs obstacle prediction through a pre-trained perception model to obtain the predicted positions of each obstacle in the intelligent driving scenario where the host vehicle is located within the prediction duration. When there are obstacles in the predicted safety corridor information, the electronic device optimizes the predicted safety corridor information based on the target vehicle state information of the host vehicle within the prediction duration, the predicted positions of each obstacle within the prediction duration, and the maximum empty matrix algorithm, and obtains the optimized predicted safety corridor information. In this way, the electronic device can optimize the predicted safety corridor information in real time, thereby reducing the collision risk.

[0098] Figure 7 is a schematic flowchart of a method for training a perception model provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, such as Figure 7 shown, and includes the following steps:

[0099] Step 701, obtain multi-view image data corresponding to the sample vehicle.

[0100] Exemplarily, when the electronic device trains the initial perception model, the electronic device can obtain the multi-view image data corresponding to the sample vehicle from the training dataset. Among them, the multi-view image data corresponding to the sample vehicle can be the multi-view image data collected by a multi-view image sensor (such as a front-view image sensor, a rear-view image sensor, a side-view image sensor, and a surround-view image sensor, etc.) installed on the vehicle during the driving process of the sample vehicle. Among them, the training dataset can include the multi-view image data corresponding to the sample vehicle at each moment, the vehicle position of the sample vehicle at each moment, and the geometric bounding boxes of each obstacle in the intelligent driving scenario where the sample vehicle is located at each moment. The training dataset can sample the nuScenes dataset or other datasets, which is not limited in the embodiments of the present disclosure.

[0101] Step 702: Based on the multi-view image data, perform corridor prediction and trajectory prediction through the initial perception model to obtain the predicted safe corridor information and predicted trajectory information of the sample vehicle within the prediction duration.

[0102] Exemplarily, after the electronic device obtains the multi-view image data corresponding to the sample vehicle, it can further input the multi-view image data into the initial perception model. First, the initial perception model can perform feature extraction, view transformation, and feature aggregation on the multi-view image data to obtain a bird's-eye view feature map. Then, the initial perception model can perform corridor prediction and trajectory prediction based on the bird's-eye view feature map, and output the predicted safe corridor information and predicted trajectory information of the sample vehicle within the prediction duration. The prediction duration can include multiple moments, and the time interval between any two adjacent moments is the same. For example, if the prediction duration is 3s and the time interval △t between adjacent moments is 0.5s, then the prediction duration includes 6 moments. The predicted safe corridor information includes collision-free regions corresponding to multiple moments, and the collision-free region can be represented as (x, y, θ, l, w). Among them, x and y respectively represent the abscissa and ordinate of the center point of the collision-free region, θ represents the deviation angle of the collision-free region, l represents the length of the collision-free region, and w represents the width of the collision-free region. The predicted trajectory information can include trajectory points corresponding to multiple moments.

[0103] Step 703: Based on the predicted safe corridor information and a preset target loss function, optimize the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

[0104] Exemplarily, the electronic device may pre-store a target loss function for optimizing the corridor prediction parameters in the initial perception model. The target loss function may include one or more of a corridor loss function, a safety loss function, and an area loss function. Among them, the corridor loss function is used to optimize the corridor prediction parameters in the initial perception model based on the regional error between the predicted safety corridor information and the reference safety corridor information; the safety loss function is used to optimize the corridor prediction parameters in the initial perception model based on the distance between the predicted safety corridor information and the obstacle points; the area loss function is used to optimize the corridor prediction parameters in the initial perception model based on the area of the collision-free region in the predicted safety corridor information. After the electronic device obtains the predicted safety corridor information, it may optimize the corridor prediction parameters in the initial perception model based on the predicted safety corridor information and the preset target loss function to obtain a perception model with optimized parameters.

[0105] In the embodiments of the present disclosure, the electronic device optimizes the corridor prediction parameters in the initial perception model based on the predicted safety corridor information and the preset target loss function to obtain a perception model with optimized parameters. In this way, the predicted safety corridor information output by the perception model with optimized parameters is more accurate and reliable.

[0106] For the target loss function being the corridor loss function, based on the above Figure 7 shown embodiments, step 703 may include the following steps:

[0107] Step 1, obtain the reference safety corridor information of the sample vehicle within the prediction duration.

[0108] Exemplarily, the reference safety corridor information is the reference safety corridor information determined by the electronic device based on the vehicle positions of the sample vehicle at each moment within the prediction duration, the obstacle points of the obstacles at each moment within the prediction duration, and the maximum empty matrix algorithm. Similar to the predicted safety corridor information, the reference safety corridor information also includes collision-free regions corresponding to multiple moments, and the collision-free region can be expressed as (x, y, θ, l, w). Among them, x and y respectively represent the abscissa and ordinate of the center point of the collision-free region, θ represents the deflection angle of the collision-free region, l represents the length of the collision-free region, and w represents the width of the collision-free region.

[0109] Step 2, for each moment within the prediction duration, determine the regional error corresponding to the moment based on the first collision-free region corresponding to the moment in the predicted safety corridor information and the second collision-free region corresponding to the moment in the reference safety corridor information.

[0110] Exemplarily, after the electronic device obtains the reference safety corridor information of the sample vehicle within the prediction duration, for each moment within the prediction duration, the electronic device may calculate the L1 distance (i.e., Manhattan distance) between the first collision-free area corresponding to the predicted safety corridor information at this moment and the second collision-free area corresponding to the reference safety corridor information at this moment as the regional error between the predicted safety corridor information and the reference safety corridor information at this moment. For example, a corridor loss function is constructed. Among them, L cor represents the loss value of the corridor loss function, and C represents the predicted safety corridor information. represents the reference safety corridor information. It should be noted that the electronic device may also calculate the regional error between the predicted safety corridor information and the reference safety corridor information at this moment in other ways, and the embodiments of the present disclosure do not make limitations.

[0111] Step 3: Based on the regional errors corresponding to each moment, determine the loss value of the corridor loss function.

[0112] Exemplarily, after the electronic device determines the regional error corresponding to each moment, based on the corridor loss function, it further determines the sum value of the regional errors corresponding to each moment as the loss value of the corridor loss function. The smaller the loss value of the corridor loss function, the smaller the total regional error between the predicted safety corridor information and the reference safety corridor information, and further indicates that the predicted safety corridor information and the reference safety corridor information are more consistent.

[0113] Step 4: Based on the loss value of the corridor loss function, perform parameter optimization on the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

[0114] Exemplarily, after the electronic device determines the loss value of the corridor loss function, it may further perform parameter optimization on the corridor prediction parameters in the initial perception model based on the loss value of the corridor loss function to obtain a perception model with optimized parameters. Subsequently, the electronic device may input the bird's-eye view feature map into the perception model with optimized parameters. The perception model with optimized parameters may perform corridor prediction based on this bird's-eye view feature map and output the predicted safety corridor information of the sample vehicle within the prediction duration. Then, the electronic device may determine the loss value of the corridor loss function based on the predicted safety corridor information and the reference safety corridor information output by the perception model with optimized parameters, and perform further parameter optimization on the corridor prediction parameters in the perception model with optimized parameters based on the loss value of the corridor loss function until the loss value of the corridor loss function reaches the minimum value, or the number of parameter optimization times reaches the preset optimization times threshold.

[0115] In the embodiments of the present disclosure, the corridor prediction parameters in the initial perception model are optimized through a corridor loss function to obtain a perception model with optimized parameters. In this way, the predicted safety corridor information output by the perception model with optimized parameters can be as consistent as possible with the reference safety corridor information, thereby improving the accuracy of the corridor prediction of the perception model.

[0116] Based on the above embodiments, the steps for the electronic device to obtain the reference safety corridor information of the sample vehicle within the prediction duration may include the following:

[0117] Step 1: Obtain the bounding boxes corresponding to the obstacles in the intelligent driving scenario where the sample vehicle is located.

[0118] Exemplarily, when the electronic device trains the initial perception model, the electronic device may obtain the geometric bounding boxes corresponding to each obstacle in the intelligent driving scenario where the sample vehicle is located at each moment within the prediction duration from the training dataset. Among them, the obstacles may include dynamic obstacles (such as motor vehicles, non-motor vehicles, pedestrians, etc.) and static obstacles (road edges, street lights, etc.).

[0119] Step 2: Determine obstacle points based on the bounding boxes corresponding to the obstacles and a preset obstacle point sampling interval.

[0120] Exemplarily, after the electronic device obtains the geometric bounding boxes corresponding to each obstacle at each moment, for the geometric bounding box corresponding to each obstacle at each moment, the electronic device may perform obstacle point sampling on the geometric bounding box corresponding to the obstacle at that moment according to the preset obstacle point sampling interval to obtain the obstacle points corresponding to the obstacle at that moment. The obstacle points can be represented as (x, y, t). Among them, t represents the moment corresponding to the obstacle point, and x and y represent the abscissa and ordinate corresponding to the obstacle point at moment t.

[0121] It should be noted that the electronic device may also preset different obstacle point sampling intervals and sampling methods based on the obstacle types of different obstacles, which are not limited in the embodiments of the present disclosure. For example, for vehicles, the obstacle point sampling interval is smaller, and the sampling method is uniform sampling; for road edges, the obstacle point sampling interval is larger, and the sampling method is uniform sampling; for pedestrians, the obstacle point sampling interval is smaller, and the sampling method is non-uniform sampling.

[0122] Step 3: For each moment within the prediction duration, based on the position information of the sample vehicle at the moment, the obstacle points, and the maximum empty matrix algorithm, determine the second collision-free region corresponding to the reference safety corridor information of the sample vehicle at the moment, and obtain the reference safety corridor information of the sample vehicle within the prediction duration.

[0123] Exemplarily, after the electronic device determines the obstacle points corresponding to each obstacle at each moment, for each moment within the prediction duration, the electronic device may, based on the position information of the sample vehicle, the coordinate information of the obstacle points, and the maximum empty matrix algorithm at that moment, determine the second collision-free region corresponding to the reference safety corridor information of the sample vehicle at that moment. Among them, the second collision-free region corresponding to that moment includes the vehicle position corresponding to the sample vehicle at that moment and does not include any obstacle point at that moment. After the electronic device determines the second collision-free regions corresponding to the reference safety corridor information at each moment, it may further generate the reference safety corridor information of the sample vehicle within the prediction duration.

[0124] In the embodiments of the present disclosure, the electronic device samples obstacle points for the geometric bounding box corresponding to the obstacle based on a preset obstacle point sampling interval, and further generates the second collision-free region of the reference safety corridor information based on the obstacle points, thereby ensuring that the second collision-free region of the generated reference safety corridor information is more accurate.

[0125] For the target loss function being the safety loss function, based on the above Figure 7 shown embodiments, step 703 may include the following steps:

[0126] Step 1, for each moment within the prediction duration, among the obstacle points corresponding to the moment, determine the target obstacle points that are within the first collision-free region corresponding to the prediction safety corridor information at the moment.

[0127] Exemplarily, for each moment within the prediction duration, the electronic device may, among the obstacle points corresponding to the moment, based on the position information of the obstacle points, determine the target obstacle points that are within the first collision-free region corresponding to the prediction safety corridor information at the moment.

[0128] Step 2, based on the target obstacle points corresponding to the moment, determine the shortest distances of the target obstacle points relative to the sides of the first collision-free region corresponding to the prediction safety corridor information at the moment.

[0129] Exemplarily, after the electronic device determines the target obstacle points that are within the first collision-free region corresponding to the prediction safety corridor information at that moment, for each target obstacle point corresponding to that moment, it may further determine the shortest distance of the target obstacle point relative to the sides of the first collision-free region corresponding to the prediction safety corridor information at that moment, that is, the relative distance of the side closest to the target obstacle point in the first collision-free region corresponding to that moment.

[0130] Step 3, determine the maximum value among the shortest distances corresponding to the target obstacle points corresponding to the moment as the maximum distance value corresponding to the moment.

[0131] Exemplarily, after the electronic device determines the shortest distances of the respective target obstacle points corresponding to this moment, it can further determine the maximum value among the shortest distances corresponding to the respective target obstacle points as the maximum distance value corresponding to this moment. For example, construct a safety loss function as where L safe represents the loss value of the safety loss function, N represents the number of moments within the prediction duration, represents the i-th target obstacle point within the first collision-free area of the predicted safety corridor information at the t-th moment, C t represents the first collision-free area of the predicted safety corridor information at the t-th moment, and D represents the distance function.

[0132] Step Four, based on the maximum distance values corresponding to each moment, determine the loss value of the safety loss function.

[0133] Exemplarily, after the electronic device determines the maximum distance values corresponding to each moment, based on the safety loss function, it further determines the sum value of the maximum distance values corresponding to each moment as the loss value of the safety loss function. Among them, the smaller the loss value of the safety loss function, the smaller the distance of each obstacle point from the edge of the first collision-free area of the predicted safety corridor information, and further indicates that the safety of the predicted safety corridor information is higher.

[0134] Step Five, based on the loss value of the safety loss function, perform parameter optimization on the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

[0135] Exemplarily, after the electronic device determines the loss value of the safety loss function, it can further perform parameter optimization on the corridor prediction parameters in the initial perception model based on the loss value of the safety loss function to obtain a perception model with optimized parameters. Subsequently, the electronic device can input the bird's-eye view feature map into the perception model with optimized parameters. The perception model with optimized parameters can perform corridor prediction based on this bird's-eye view feature map and output the predicted safety corridor information of the sample vehicle within the prediction duration. Then, the electronic device can determine the loss value of the safety loss function based on the predicted safety corridor information output by the perception model with optimized parameters and the obstacle points corresponding to each moment, and perform further parameter optimization on the corridor prediction parameters in the perception model with optimized parameters based on the loss value of the safety loss function until the loss value of the safety loss function reaches the minimum value, or the number of parameter optimization times reaches the preset optimization times threshold.

[0136] In the embodiments of the present disclosure, parameter optimization is performed on the corridor prediction parameters in the initial perception model through the safety loss function to obtain a perception model with optimized parameters. In this way, the collision-free area of the predicted safety corridor information output by the perception model with optimized parameters does not contain obstacle points, thereby improving the reliability of the corridor prediction of the perception model.

[0137] For the target loss function being the area loss function, based on the above Figure 7 shown embodiments, step 703 may include the following steps:

[0138] Step 1, for each moment within the prediction duration, based on the width, length, and a preset scale parameter of the first collision-free region corresponding to the moment in the predicted safety corridor information, determine the area exponential decay value of the first collision-free region corresponding to the moment in the predicted safety corridor information.

[0139] Exemplarily, for each moment within the prediction duration, the electronic device may determine the area exponential decay value of the first collision-free region corresponding to the moment in the predicted safety corridor information based on the width, length, and a preset scale parameter of the first collision-free region corresponding to the moment in the predicted safety corridor information. For example, construct the area loss function as where L area represents the loss value of the area loss function, N represents the number of moments within the prediction duration, and α represents the scale parameter. w t represents the width of the first collision-free region of the predicted safety corridor information at the t-th moment, and l t represents the length of the first collision-free region of the predicted safety corridor information at the t-th moment.

[0140] Step 2, based on the area exponential decay values of the first collision-free regions corresponding to each moment in the predicted safety corridor information, determine the loss value of the area loss function.

[0141] Exemplarily, after the electronic device determines the area exponential decay values of the first collision-free regions corresponding to each moment in the predicted safety corridor information, based on the area loss function, further determine the sum value of the area exponential decay values of the first collision-free regions corresponding to each moment in the predicted safety corridor information as the loss value of the area loss function. Among them, the smaller the loss value of the area loss function, the larger the area of the collision-free region of the predicted safety corridor information.

[0142] Step 3, based on the loss value of the area loss function, perform parameter optimization on the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

[0143] Exemplarily, after the electronic device determines the loss value of the area loss function, it can further optimize the corridor prediction parameters in the initial perception model based on the loss value of the area loss function to obtain a perception model with optimized parameters. Subsequently, the electronic device can input the bird's-eye view feature map into the perception model with optimized parameters. The perception model with optimized parameters can perform corridor prediction based on the bird's-eye view feature map and output the predicted safe corridor information of the sample vehicle within the prediction duration. Then, the electronic device can determine the loss value of the area loss function based on the width, length, and preset scale parameter of the first collision-free area at each moment of the predicted safe corridor information output by the perception model with optimized parameters, and further optimize the corridor prediction parameters in the perception model with optimized parameters based on the loss value of the area loss function until the loss value of the area loss function reaches the minimum value, or the number of parameter optimization times reaches the preset optimization times threshold.

[0144] In the embodiments of the present disclosure, the corridor prediction parameters in the initial perception model are optimized by the area loss function to obtain a perception model with optimized parameters. In this way, the area of the predicted safe corridor information output by the perception model with optimized parameters is as large as possible, thereby improving the reliability of the corridor prediction of the perception model.

[0145] In the above Figure 7 Based on the above-described embodiments, the electronic device further performs the following steps:

[0146] Based on the predicted safe corridor information, the predicted trajectory information, and the trajectory optimization condition, optimize the corridor prediction parameters and the trajectory prediction parameters in the perception model with optimized parameters to obtain a trained perception model.

[0147] Exemplarily, during the training process of the perception model, the electronic device can also determine the KKT condition based on the trajectory optimization condition. Then, the electronic device can perform implicit differentiation on the KKT condition to obtain a gradient function. After that, the electronic device can input the optimization result obtained by optimizing the trajectory through the predicted safe corridor information, the predicted trajectory information, and the trajectory optimization condition (i.e., the target vehicle state information and the target control signal of the sample vehicle within the prediction duration) into the gradient function to obtain a gradient value. Finally, the electronic device can optimize the corridor prediction parameters and the trajectory prediction parameters in the perception model with optimized parameters in an iterative manner based on the gradient value until a trained perception model is obtained.

[0148] Based on the above embodiments, the electronic device optimizes the corridor prediction parameters and the trajectory prediction parameters in the perception model with optimized parameters based on the predicted safe corridor information, the predicted trajectory information, and the trajectory optimization condition to obtain a trained perception model, which may include the following steps:

[0149] Step 1: Determine trajectory optimization conditions based on predicted safety corridor information and predicted trajectory information.

[0150] Exemplarily, after the electronic device obtains the predicted safety corridor information and the predicted trajectory information, it can further determine the trajectory optimization conditions based on the predicted safety corridor information and the predicted trajectory information. The process of the electronic device determining the trajectory optimization conditions based on the predicted safety corridor information and the predicted trajectory information is similar to step 1031 above and will not be elaborated here.

[0151] Step 2: Determine the target vehicle state information and target control signal of the sample vehicle within the prediction duration based on the initial vehicle state information of the sample vehicle and the trajectory optimization conditions.

[0152] Exemplarily, after the electronic device determines the trajectory optimization conditions, it can determine the target vehicle state information and target control signal corresponding to each moment of the host vehicle within the prediction duration based on the current vehicle state information and the trajectory optimization conditions. The process of the electronic device determining the target vehicle state information and target control signal of the sample vehicle within the prediction duration based on the initial vehicle state information of the sample vehicle and the trajectory optimization conditions is similar to step 1032 and will not be elaborated here.

[0153] Step 3: Determine the gradient value of the perception model with optimized parameters based on the trajectory optimization conditions, the target vehicle state information and target control signal of the sample vehicle within the prediction duration.

[0154] Exemplarily, after the electronic device determines the trajectory optimization conditions, it can determine the Lagrangian function based on the trajectory optimization conditions. Then, the electronic device can determine the KKT conditions based on the Lagrangian function. Among them, the KKT conditions include the primal feasibility condition, the dual feasibility condition, and the complementary slackness condition. After that, the electronic device can perform implicit differentiation on the KKT conditions to determine the gradient function. Finally, the electronic device can determine the gradient value of the perception model with optimized parameters based on the target vehicle state information and target control signal of the sample vehicle within the prediction duration and the gradient function.

[0155] Step 4: Based on the gradient value, perform parameter optimization on the corridor prediction parameters and trajectory prediction parameters in the perception model with optimized parameters respectively to obtain the trained perception model.

[0156] Exemplarily, after the electronic device determines the gradient value of the perception model with optimized parameters, it can perform parameter optimization on the corridor prediction parameters and trajectory prediction parameters in the perception model with optimized parameters in an iterative manner based on the gradient value until the trained perception model is obtained.

[0157] In an embodiment of the present disclosure, an electronic device determines the KKT conditions based on trajectory optimization conditions, performs implicit differentiation on the KKT conditions to obtain a gradient function. Then, the electronic device may input the optimization result obtained by trajectory optimization through predicted safety corridor information, predicted trajectory information, and trajectory optimization conditions into the gradient function to obtain a gradient value, and based on the gradient value, perform parameter optimization on the corridor prediction parameter and the trajectory prediction parameter in the perception model with optimized parameters in an iterative manner until a trained perception model is obtained. In this way, by performing implicit differentiation on the KKT conditions of the trajectory optimization conditions, the gradient value is backpropagated from the optimization result to the perception model, so as to perform parameter optimization on the corridor prediction parameter and the trajectory prediction parameter in the perception model with optimized parameters, thereby improving the accuracy and reliability of the predicted safety corridor information and the predicted trajectory information output by the trained perception model.

[0158] Figure 8 It is a training schematic diagram of a perception model provided by an exemplary embodiment of the present disclosure. As Figure 8As shown in the figure, first, the electronic device obtains a training data set. The training data set includes multi-view image data corresponding to a sample vehicle at each moment, the vehicle position of the sample vehicle at each moment, and the geometric bounding boxes of each obstacle in the intelligent driving scenario where the sample vehicle is located at each moment. Then, the electronic device inputs the multi-view image data into the initial perception model 200. Correspondingly, the feature map generation model 210 in the initial perception model 200 can perform feature extraction, view transformation, and feature aggregation on the multi-view image data to obtain a bird's-eye view feature map. Based on this bird's-eye view feature map, corridor prediction and trajectory prediction are performed through the corridor prediction module 230 and the trajectory prediction module 240, and the predicted safe corridor information and predicted trajectory information of the sample vehicle within the prediction duration are output. At the same time, for the geometric bounding box corresponding to each obstacle in the training data set at each moment, the electronic device can perform obstacle point sampling on the geometric bounding box corresponding to the obstacle at this moment according to a preset obstacle point sampling interval to obtain the obstacle points corresponding to the obstacle at this moment, and obtain the set of obstacle points corresponding to each obstacle at each moment. After that, the electronic device can determine the reference safe corridor information of the sample vehicle within the prediction duration based on the position information of the sample vehicle at each moment, the coordinate information of the obstacle points at each moment, and the maximum empty matrix algorithm. Then, the electronic device can optimize the corridor prediction parameters in the initial perception model based on the predicted safe corridor information, the reference safe corridor information, and the corridor loss function to obtain a perception model with optimized parameters. The electronic device can also optimize the corridor prediction parameters in the initial perception model based on the first collision-free area of the predicted safe corridor information at each moment, the obstacle points at each moment, and the safety loss function to obtain a perception model with optimized parameters. The electronic device can also optimize the corridor prediction parameters in the initial perception model based on the length, width, and area loss functions of the first collision-free area of the predicted safe corridor information at each moment to obtain a perception model with optimized parameters. Finally, the electronic device can optimize the corridor prediction parameters and trajectory prediction parameters in the perception model with optimized parameters based on the predicted safe corridor information, the predicted trajectory information, the initial vehicle state information of the sample vehicle, and the trajectory optimization conditions to obtain a trained perception model.

[0159] Exemplary device

[0160] Figure 9 is a schematic structural diagram of a trajectory optimization device provided by an exemplary embodiment of the present disclosure. As Figure 9 shown, the trajectory optimization device 900 includes a data acquisition module 910, a first perception prediction module 920, and a trajectory optimization module 930.

[0161] The data acquisition module 910 is configured to acquire multi-view image data collected by the vehicle itself at the current moment and the current vehicle state information;

[0162] The first perception prediction module 920 is configured to perform corridor prediction and trajectory prediction based on multi-view image data through a pre-trained perception model, and obtain predicted safety corridor information and predicted trajectory information of the host vehicle within the prediction duration;

[0163] The trajectory optimization module 930 is configured to optimize the predicted trajectory information based on the predicted safety corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization conditions, and obtain the optimized predicted trajectory information.

[0164] In some embodiments, the trajectory optimization module 930 includes:

[0165] The condition determination unit is configured to determine trajectory optimization conditions based on the predicted safety corridor information and the predicted trajectory information;

[0166] The information determination unit is configured to determine the target vehicle state information and the target control signal of the host vehicle within the prediction duration based on the current vehicle state information and the trajectory optimization conditions;

[0167] The trajectory optimization unit is configured to optimize the predicted trajectory information based on the target vehicle state information and the target control signal of the host vehicle within the prediction duration, and obtain the optimized predicted trajectory information.

[0168] In some embodiments, the condition determination unit is specifically configured to:

[0169] Based on the predicted safety corridor information and the predicted trajectory information, construct trajectory optimization conditions with the constraint that the target vehicle state information of the host vehicle within the prediction duration is within the predicted safety corridor information, and with the trajectory tracking error between the target vehicle state information of the host vehicle within the prediction duration and the predicted trajectory information as the minimum control cost.

[0170] In some embodiments, the condition determination unit is specifically configured to:

[0171] Based on the predicted safety corridor information, the predicted trajectory information, and a preset control signal range, construct trajectory optimization conditions with the constraint that the target vehicle state information of the host vehicle within the prediction duration is within the predicted safety corridor information and the target control signal of the host vehicle within the prediction duration is within the control signal range, and with the trajectory tracking error between the target vehicle state information of the host vehicle within the prediction duration and the predicted trajectory information and the target control signal of the host vehicle within the prediction duration as the minimum control cost.

[0172] In some embodiments, the information determination unit is specifically configured to:

[0173] For each first moment in the prediction duration, based on the current vehicle state information and the vehicle motion model, determine the target vehicle state information corresponding to the first moment; the target vehicle state information corresponding to the first moment is determined by the current vehicle state information and the target control signal corresponding to the second moment in the prediction duration; the prediction duration is a preset duration after the current moment, and the second moment is all the moments before the first moment;

[0174] Based on the target vehicle state information corresponding to the first moment and the trajectory point corresponding to the prediction trajectory information at the first moment, determine the trajectory tracking error corresponding to the first moment;

[0175] Determine the trajectory tracking error corresponding to the first moment as the control cost corresponding to the first moment;

[0176] When the target vehicle state information corresponding to each first moment is within the predicted safety corridor information, in response to the sum of the control costs corresponding to each first moment being the smallest, determine the target vehicle state information and the target control signal corresponding to each first moment.

[0177] In some embodiments, the information determination unit is specifically configured to:

[0178] For each first moment in the prediction duration, based on the current vehicle state information and the vehicle motion model, determine the target vehicle state information corresponding to the first moment; the target vehicle state information corresponding to the first moment is determined by the current vehicle state information and the target control signal corresponding to the second moment in the prediction duration; the prediction duration is a preset duration after the current moment, and the second moment is all the moments before the first moment;

[0179] Based on the target vehicle state information corresponding to the first moment and the trajectory point corresponding to the prediction trajectory information at the first moment, determine the trajectory tracking error corresponding to the first moment;

[0180] Determine the weighted sum of the trajectory tracking error corresponding to the first moment and the target control signal corresponding to the third moment in the prediction duration as the control cost corresponding to the first moment; the third moment is the moment before the first moment, and the second moment includes the third moment;

[0181] When the target vehicle state information corresponding to each first moment is within the predicted safety corridor information and the target control signal corresponding to each first moment is within the control signal range, in response to the sum of the control costs corresponding to each first moment being the smallest, determine the target vehicle state information and the target control signal corresponding to each first moment.

[0182] In some embodiments, it further includes:

[0183] The second perception prediction module is used to predict obstacles based on multi-view image data through a pre-trained perception model, and obtain the predicted positions of obstacles in the intelligent driving scenario where the vehicle is located within the prediction duration.

[0184] The corridor optimization module is used to, in response to the presence of obstacles in the predicted safety corridor information, optimize the predicted safety corridor information based on the target vehicle state information of the vehicle within the prediction duration, the predicted positions of obstacles within the prediction duration, and the maximum empty matrix algorithm, to obtain the optimized predicted safety corridor information.

[0185] For the beneficial technical effects corresponding to the exemplary embodiments of this device, reference can be made to the corresponding beneficial technical effects in the above exemplary method section, which will not be elaborated here.

[0186] Figure 10 It is a schematic structural diagram of a training device for a perception model provided by an exemplary embodiment of the present disclosure. As Figure 10 shown, the training device 1000 of the perception model includes a data acquisition module 1010, a perception prediction module 1020, and a first parameter optimization module 1030.

[0187] The data acquisition module 1010 is used to acquire multi-view image data corresponding to a sample vehicle.

[0188] The perception prediction module 1020 is used to perform corridor prediction and trajectory prediction based on the multi-view image data through an initial perception model, and obtain the predicted safety corridor information and predicted trajectory information of the sample vehicle within the prediction duration.

[0189] The first parameter optimization module 1030 is used to optimize the corridor prediction parameters in the initial perception model based on the predicted safety corridor information and a preset target loss function, to obtain a perception model with optimized parameters.

[0190] In some embodiments, the target loss function is a corridor loss function. The first parameter optimization module 1030 includes:

[0191] An information acquisition unit is used to acquire the reference safety corridor information of the sample vehicle within the prediction duration.

[0192] An error determination unit is used to, for each moment within the prediction duration, determine the regional error corresponding to the moment based on the first collision-free region corresponding to the predicted safety corridor information at the moment and the second collision-free region corresponding to the reference safety corridor information at the moment.

[0193] A first loss determination unit is used to determine the loss value of the corridor loss function based on the regional errors corresponding to each moment.

[0194] A second parameter optimization unit, configured to optimize the corridor prediction parameters in the initial perception model based on the loss value of the corridor loss function, so as to obtain a perception model with optimized parameters.

[0195] In some embodiments, the information acquisition unit is specifically configured to:

[0196] Obtain the bounding box corresponding to the obstacle in the intelligent driving scenario where the sample vehicle is located;

[0197] Determine obstacle points based on the bounding box corresponding to the obstacle and a preset obstacle point sampling interval;

[0198] For each moment within the prediction duration, based on the position information of the sample vehicle at the moment, the obstacle points, and the maximum empty matrix algorithm, determine the second collision-free region corresponding to the moment of the reference safety corridor information of the sample vehicle, so as to obtain the reference safety corridor information of the sample vehicle within the prediction duration.

[0199] In some embodiments, the target loss function is a safety loss function. The first parameter optimization module 1030 includes:

[0200] An obstacle point determination unit, configured to determine, for each moment within the prediction duration, among the obstacle points corresponding to the moment, the target obstacle points that are within the first collision-free region corresponding to the moment of the prediction safety corridor information;

[0201] A first distance determination unit, configured to determine the shortest distance from each target obstacle point to the edge of the first collision-free region corresponding to the moment of the prediction safety corridor information based on the target obstacle points corresponding to the moment;

[0202] A second distance determination unit, configured to determine the maximum value among the shortest distances corresponding to the target obstacle points at the moment as the maximum distance value corresponding to the moment;

[0203] A second loss determination unit, configured to determine the loss value of the safety loss function based on the maximum distance values corresponding to each moment;

[0204] A second parameter optimization unit, configured to optimize the corridor prediction parameters in the initial perception model based on the loss value of the safety loss function, so as to obtain a perception model with optimized parameters.

[0205] In some embodiments, the target loss function is an area loss function. The first parameter optimization module 1030 includes:

[0206] An attenuation determination unit, configured to determine, for each moment within the prediction duration, the area exponential attenuation value of the first collision-free region corresponding to the moment of the prediction safety corridor information based on the width and length of the first collision-free region corresponding to the moment of the prediction safety corridor information and a preset scale parameter;

[0207] A third loss determination unit, configured to determine a loss value of an area loss function based on an area exponential decay value of a first collision-free area corresponding to predicted safety corridor information at each moment;

[0208] A third parameter optimization unit, configured to optimize corridor prediction parameters in an initial perception model based on the loss value of the area loss function, and obtain a perception model with optimized parameters.

[0209] In some embodiments, it further includes:

[0210] A second parameter optimization module, configured to optimize corridor prediction parameters and trajectory prediction parameters in the perception model with optimized parameters based on predicted safety corridor information, predicted trajectory information, and trajectory optimization conditions, and obtain a trained perception model.

[0211] In some embodiments, the second parameter optimization module includes:

[0212] A condition determination unit, configured to determine trajectory optimization conditions based on predicted safety corridor information and predicted trajectory information;

[0213] An information determination unit, configured to determine target vehicle state information and target control signals of a sample vehicle within a prediction duration based on initial vehicle state information of the sample vehicle and the trajectory optimization conditions;

[0214] A gradient determination unit, configured to determine a gradient value of the perception model with optimized parameters based on the trajectory optimization conditions, target vehicle state information, and target control signals of the sample vehicle within the prediction duration;

[0215] A parameter optimization unit, configured to optimize corridor prediction parameters and trajectory prediction parameters in the perception model with optimized parameters respectively based on the gradient value, and obtain a trained perception model.

[0216] For the beneficial technical effects corresponding to the exemplary embodiments of this device, reference may be made to the corresponding beneficial technical effects in the above exemplary method section, which will not be elaborated herein.

[0217] Exemplary electronic device

[0218] Figure 11 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure, including at least one processor 11 and a memory 12.

[0219] The processor 11 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0220] The memory 12 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run one or more computer program instructions to implement the trajectory optimization method and the training method of the perception model in the various embodiments of the present disclosure above, and / or other desired functions.

[0221] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0222] The input device 13 may further include, for example, a keyboard, a mouse, and the like.

[0223] The output device 14 may output various information to the outside, which may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, and the like.

[0224] Of course, for simplicity, Figure 11 only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application scenarios, the electronic device 10 may further include any other appropriate components.

[0225] Exemplary computer program products and computer-readable storage media

[0226] In addition to the above methods and devices, the embodiments of the present disclosure may further provide a computer program product, including computer program instructions, and when the computer program instructions are run by a processor, the processor is caused to execute the steps in the trajectory optimization method or the training method of the perception model in the various embodiments of the present disclosure described in the above "Exemplary Method" section.

[0227] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0228] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium storing computer program instructions, which, when run by a processor, cause the processor to execute the steps in the trajectory optimization method or the training method of the perception model of various embodiments of the present disclosure described in the above "Exemplary Method" section.

[0229] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium includes, for example but not limited to, a system, apparatus, or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0230] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that they are essential for each embodiment of the present disclosure. In addition, the above-described specific details are only for the purposes of illustration and easy understanding, rather than limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0231] Those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure also intends to include these modifications and variations.

Claims

1. A trajectory optimization method, comprising: Obtaining multi-view image data collected by the host vehicle at the current moment and current vehicle state information; Based on the multi-view image data, performing corridor prediction and trajectory prediction through a pre-trained perception model to obtain predicted safe corridor information and predicted trajectory information of the host vehicle within a prediction duration; Based on the predicted safe corridor information, the predicted trajectory information, the current vehicle state information, and trajectory optimization conditions, optimizing the predicted trajectory information to obtain optimized predicted trajectory information.

2. The method according to claim 1, wherein, The optimizing the predicted trajectory information based on the predicted safe corridor information, the predicted trajectory information, the current vehicle state information, and trajectory optimization conditions to obtain optimized predicted trajectory information includes: Determining trajectory optimization conditions based on the predicted safe corridor information and the predicted trajectory information; Determining target vehicle state information and a target control signal of the host vehicle within the prediction duration based on the current vehicle state information and the trajectory optimization conditions; Based on the target vehicle state information and the target control signal of the host vehicle within the prediction duration, optimizing the predicted trajectory information to obtain optimized predicted trajectory information.

3. The method according to claim 2, wherein The determining trajectory optimization conditions based on the predicted safe corridor information and the predicted trajectory information includes: Based on the predicted safe corridor information and the predicted trajectory information, constructing a trajectory optimization condition with the target vehicle state information of the host vehicle within the prediction duration being within the predicted safe corridor information as a constraint condition and the trajectory tracking error between the target vehicle state information of the host vehicle within the prediction duration and the predicted trajectory information being the minimization of the control cost.

4. The method according to claim 2, wherein The determining trajectory optimization conditions based on the predicted safe corridor information and the predicted trajectory information includes: Based on the predicted safe corridor information, the predicted trajectory information, and a preset control signal range, constructing a trajectory optimization condition with the target vehicle state information of the host vehicle within the prediction duration being within the predicted safe corridor information and the target control signal of the host vehicle within the prediction duration being within the control signal range as constraint conditions and the trajectory tracking error between the target vehicle state information of the host vehicle within the prediction duration and the predicted trajectory information and the target control signal of the host vehicle within the prediction duration being the minimization of the control cost.

5. The method according to claim 3, wherein, The determining the target vehicle state information and the target control signal of the host vehicle within the prediction duration based on the current vehicle state information and the trajectory optimization conditions includes: For each first moment in the prediction duration, determining the target vehicle state information corresponding to the first moment based on the current vehicle state information and a vehicle motion model; the target vehicle state information corresponding to the first moment is determined by the current vehicle state information and the target control signal corresponding to a second moment in the prediction duration; the prediction duration is a preset duration after the current moment, and the second moment is all moments before the first moment; Determine the trajectory tracking error corresponding to the first moment based on the target vehicle state information corresponding to the first moment and the trajectory point corresponding to the prediction trajectory information at the first moment; Determine the control cost corresponding to the first moment by using the trajectory tracking error corresponding to the first moment; When the target vehicle state information corresponding to each of the first moments is within the predicted safety corridor information, in response to the sum of the control costs corresponding to each of the first moments being minimized, determine the target vehicle state information and the target control signal corresponding to each of the first moments.

6. The method according to claim 4, wherein, The determining of the target vehicle state information and the target control signal of the host vehicle within the prediction duration based on the current vehicle state information and the trajectory optimization condition includes: For each first moment within the prediction duration, determine the target vehicle state information corresponding to the first moment based on the current vehicle state information and the vehicle motion model; the target vehicle state information corresponding to the first moment is determined by the current vehicle state information and the target control signal corresponding to the second moment within the prediction duration; the prediction duration is a preset duration after the current moment, and the second moment is all moments before the first moment; Determine the trajectory tracking error corresponding to the first moment based on the target vehicle state information corresponding to the first moment and the trajectory point corresponding to the prediction trajectory information at the first moment; Determine the control cost corresponding to the first moment by using the weighted sum of the trajectory tracking error corresponding to the first moment and the target control signal corresponding to the third moment within the prediction duration; the third moment is the moment before the first moment, and the second moment includes the third moment; When the target vehicle state information corresponding to each of the first moments is within the predicted safety corridor information and the target control signal corresponding to each of the first moments is within the control signal range, in response to the sum of the control costs corresponding to each of the first moments being minimized, determine the target vehicle state information and the target control signal corresponding to each of the first moments.

7. The method according to claim 1, further comprising: Based on the multi-view image data, perform obstacle prediction through the pre-trained perception model to obtain the predicted positions of each obstacle in the intelligent driving scenario where the host vehicle is located within the prediction duration; In response to the presence of an obstacle within the predicted safety corridor information, optimize the predicted safety corridor information based on the target vehicle state information of the host vehicle within the prediction duration, the predicted positions of each obstacle within the prediction duration, and the maximum empty matrix algorithm to obtain the optimized predicted safety corridor information.

8. A training method for a perception model, comprising: Obtain the multi-view image data corresponding to the sample vehicle; Based on the multi-view image data, perform corridor prediction and trajectory prediction through the initial perception model to obtain the predicted safety corridor information and the predicted trajectory information of the sample vehicle within the prediction duration; Based on the predicted safety corridor information and a preset target loss function, optimize the corridor prediction parameters in the initial perception model to obtain the perception model with optimized parameters.

9. The method according to claim 8, wherein The target loss function is a corridor loss function. Based on the predicted safety corridor information and the preset target loss function, optimizing the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters includes: Obtain the reference safety corridor information of the sample vehicle within the prediction duration; For each moment within the prediction duration, based on the first collision-free region corresponding to the predicted safety corridor information at the moment and the second collision-free region corresponding to the reference safety corridor information at the moment, determine the regional error corresponding to the moment; Based on the regional errors corresponding to each moment, determine the loss value of the corridor loss function; Based on the loss value of the corridor loss function, optimize the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

10. The method according to claim 9, wherein The obtaining the reference safety corridor information of the sample vehicle within the prediction duration includes: Obtain the bounding boxes corresponding to the obstacles in the intelligent driving scenario where the sample vehicle is located; Based on the bounding boxes corresponding to the obstacles and the preset obstacle point sampling interval, determine the obstacle points; For each moment within the prediction duration, based on the position information of the sample vehicle at the moment, the obstacle points, and the maximum empty matrix algorithm, determine the second collision-free region corresponding to the reference safety corridor information of the sample vehicle at the moment, and obtain the reference safety corridor information of the sample vehicle within the prediction duration.

11. The method according to claim 8, wherein The target loss function is a safety loss function. Based on the predicted safety corridor information and the preset target loss function, optimizing the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters includes: For each moment within the prediction duration, among the obstacle points corresponding to the moment, determine the target obstacle points located within the first collision-free region corresponding to the predicted safety corridor information at the moment; Based on the target obstacle points corresponding to the moment, determine the shortest distances from the target obstacle points to the sides of the first collision-free region corresponding to the predicted safety corridor information at the moment; Determine the maximum value among the shortest distances corresponding to the target obstacle points corresponding to the moment as the maximum distance value corresponding to the moment; Based on the maximum distance values corresponding to each moment, determine the loss value of the safety loss function; Based on the loss value of the safety loss function, optimize the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

12. The method according to claim 8, wherein, The target loss function is an area loss function. Based on the predicted safety corridor information and the preset target loss function, optimizing the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters includes: For each moment within the prediction duration, based on the width and length of the first collision-free region corresponding to the predicted safety corridor information at the moment and the preset scale parameter, determine the area exponential decay value of the first collision-free region corresponding to the predicted safety corridor information at the moment; Determine the loss value of the area loss function based on the exponentially decaying value of the area of the first collision-free region corresponding to the predicted safety corridor information at each of the moments; Based on the loss value of the area loss function, optimize the corridor prediction parameters in the initial perception model to obtain a perception model with optimized parameters.

13. The method according to claim 8, wherein the method further comprises: Based on the predicted safety corridor information, the predicted trajectory information, and the trajectory optimization condition, optimize the corridor prediction parameters and the trajectory prediction parameters in the perception model with optimized parameters to obtain a trained perception model.

14. The method according to claim 13, wherein, The step of, based on the predicted safety corridor information, the predicted trajectory information, and the trajectory optimization condition, optimizing the corridor prediction parameters and the trajectory prediction parameters in the perception model with optimized parameters to obtain a trained perception model, comprises: Determine the trajectory optimization condition based on the predicted safety corridor information and the predicted trajectory information; Determine the target vehicle state information and the target control signal of the sample vehicle within the prediction duration based on the initial vehicle state information of the sample vehicle and the trajectory optimization condition; Determine the gradient value of the perception model with optimized parameters based on the trajectory optimization condition, the target vehicle state information, and the target control signal of the sample vehicle within the prediction duration; Based on the gradient value, optimize the corridor prediction parameters and the trajectory prediction parameters in the perception model with optimized parameters respectively to obtain a trained perception model.

15. A trajectory optimization device, comprising: A data acquisition module, configured to acquire multi-view image data and current vehicle state information collected by the vehicle itself at the current moment; A first perception and prediction module, configured to perform corridor prediction and trajectory prediction through a pre-trained perception model based on the multi-view image data to obtain the predicted safety corridor information and the predicted trajectory information of the vehicle itself within the prediction duration; A trajectory optimization module, configured to optimize the predicted trajectory information based on the predicted safety corridor information, the predicted trajectory information, the current vehicle state information, and the trajectory optimization condition to obtain optimized predicted trajectory information.

16. A training device for a perception model, comprising: A data acquisition module, configured to acquire multi-view image data corresponding to a sample vehicle; A perception and prediction module, configured to perform corridor prediction and trajectory prediction through an initial perception model based on the multi-view image data to obtain the predicted safety corridor information and the predicted trajectory information of the sample vehicle within the prediction duration; A first parameter optimization module, configured to optimize the corridor prediction parameters in the initial perception model based on the predicted safety corridor information and a preset target loss function to obtain a perception model with optimized parameters.

17. A computer-readable storage medium storing a computer program for executing the trajectory optimization method according to any one of claims 1-7 above, or the training method of the perception model according to any one of claims 8-14 above.

18. An electronic device, comprising: A processor; A memory for storing the processor-executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the trajectory optimization method according to any one of claims 1-7 above, or the training method of the perception model according to any one of claims 8-14 above.

Images