Control method and apparatus for vehicle, electronic device, and storage medium

Abstract

To provide a control method for a vehicle, an electronic device, and a storage medium.SOLUTION: The control method for a vehicle includes the steps of: acquiring a first image captured by a camera with a preset viewing angle mounted on the vehicle; transforming the first image into a second image in bird's-eye view; determining, based on the second image, a first predicted traveling trajectory of the vehicle in a local coordinate system; correcting the first predicted traveling trajectory on the basis of observation information corresponding to the first image to obtain a second predicted traveling trajectory after correction; and controlling a traveling state of the vehicle on the basis of the second predicted traveling trajectory. Embodiments of the present disclosure can significantly improve accuracy and effectiveness of a recommended traveling trajectory and thus improve vehicle traveling safety.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to computer vision technology, and in particular to a vehicle control method and device, an electronic device, and a storage medium. [Background technology]

[0002] In technical fields such as driver assistance, visual sensing means typically include technology for sensing a recommended driving path (which may also be called a driveline) from a forward perspective, and related technologies tend to determine the recommended driving path by heavily relying on detection of lane markings. However, in situations where road markings are severely worn, road markings are severely obstructed by traffic congestion, and intersections and the number of lanes are complex, detection of lane markings is likely to be missing or unstable, which can lead to problems such as an inability to determine the recommended driving path or low accuracy of the recommended driving path, and further to a decrease in vehicle safety. Summary of the Invention [Problem to be solved by the invention]

[0003] To solve the above technical problems, embodiments of the present disclosure provide a vehicle control method and device, an electronic device, and a storage medium for improving the accuracy of a recommended driving path, thereby improving the safety of vehicle driving. [Means for solving the problem]

[0004] A vehicle control method according to a first aspect of the present disclosure includes the steps of acquiring a first image collected by a camera of a vehicle with a preset viewpoint, converting the first image into a second image with a bird's-eye viewpoint, determining a first predicted driving trajectory in a local coordinate system of the vehicle based on the second image, correcting the first predicted driving trajectory based on observation information corresponding to the first image and acquiring a second predicted driving trajectory after the correction, and controlling the driving state of the vehicle based on the second predicted driving trajectory.

[0005] A vehicle control device according to a second aspect of the present disclosure includes an acquisition module for acquiring a first image collected by a camera at a preset viewpoint of the vehicle, a first processing module for converting the first image into a second image from a bird's-eye view, a second processing module for determining a first predicted driving trajectory in a local coordinate system of the vehicle based on the second image, a third processing module for correcting the first predicted driving trajectory based on observation information corresponding to the first image and acquiring a corrected second predicted driving trajectory, and a fourth processing module for controlling the driving state of the vehicle based on the second predicted driving trajectory.

[0006] A computer-readable storage medium according to a third aspect of the present disclosure stores a computer program for executing the vehicle control method described in any one of the embodiments of the present disclosure.

[0007] An electronic device according to a fourth aspect of the present disclosure includes a processor and a memory for storing instructions executable by the processor, and the processor is used to read and execute the executable instructions from the memory to realize a vehicle control method described in any one of the embodiments of the present disclosure.

[0008] A fifth aspect of the present disclosure provides a computer program product, wherein instructions in the computer program product, when executed by a processor, perform the vehicle control method according to any one of the embodiments of the present disclosure. [Effects of the Invention]

[0009] According to the vehicle control method and device, electronic device, and storage medium of the embodiments of the present disclosure, a first image collected by a camera with a preset viewpoint is converted into a bird's-eye view (i.e., a second image with a bird's-eye view), and the bird's-eye view can more effectively represent the overall situation of the environment around the vehicle. Therefore, an effective first predicted driving trajectory can be obtained based on the bird's-eye view. Furthermore, the obtained first predicted driving trajectory can be corrected based on structured observation information in the first image that can represent the true direction of the road, such as lane markings, road curbs, and the driving trajectories of other vehicles around the vehicle, thereby significantly improving the accuracy and effectiveness of the recommended driving trajectory and thereby improving the safety of vehicle driving. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is an exemplary application scenario of a vehicle control method according to the present disclosure. [Figure 2] 1 is a flowchart of a vehicle control method according to an exemplary embodiment of the present disclosure. [Figure 3] 10 is a flowchart of a vehicle control method according to another exemplary embodiment of the present disclosure. [Figure 4] 10 is a flowchart of a vehicle control method according to yet another exemplary embodiment of the present disclosure. [Figure 5] 10 is a flowchart of a vehicle control method according to yet another exemplary embodiment of the present disclosure. [Figure 6] 1 is a flowchart for determining a recommended driving path according to one exemplary embodiment of the present disclosure. [Figure 7] FIG. 2 is a schematic diagram of determining a first predicted driving trajectory according to one exemplary embodiment of the present disclosure. [Figure 8] FIG. 1 is a schematic diagram of obtaining a predicted driving trajectory to be corrected by multi-frame fusion according to an exemplary embodiment of the present disclosure. [Figure 9] 10 is a flowchart for determining a predicted traveling trajectory to be corrected according to an exemplary embodiment of the present disclosure. [Figure 10]10 is a flowchart of a flow for correcting a predicted traveling trajectory according to an exemplary embodiment of the present disclosure. [Figure 11] FIG. 2 is a schematic diagram of observation information according to one exemplary embodiment of the present disclosure. [Figure 12] 10 is a flowchart illustrating a process for determining a second predicted driving trajectory based on a restriction point according to an exemplary embodiment of the present disclosure. [Figure 13] FIG. 2 is a schematic diagram of a segmented cubic curve according to one exemplary embodiment of the present disclosure. [Figure 14] 1 is a structural schematic diagram of a control device for a vehicle according to one exemplary embodiment of the present disclosure. [Figure 15] FIG. 2 is a structural schematic diagram of a control device for a vehicle according to another exemplary embodiment of the present disclosure. [Figure 16] FIG. 10 is a structural schematic diagram of a control device for a vehicle according to yet another exemplary embodiment of the present disclosure. [Figure 17] 1 is a structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] In order to explain the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the drawings, and it should be clearly understood that the described embodiments are only some embodiments of the present disclosure, and not all embodiments, and the present disclosure is not limited to the exemplary embodiments.

[0012] It should be noted that the relative arrangement of components and steps, formulas and numerical values ​​described in these examples do not limit the scope of the present disclosure unless specifically stated otherwise.

[0013] Summary of the Disclosure In the process of realizing the present disclosure, the inventors discovered that in technical fields such as driver assistance, visual sensing means typically include technology for detecting a recommended driving path (also referred to as a driveline) from a forward perspective, and related technologies often rely heavily on the detection of lane markings to determine a recommended driving path. However, in scenes where road markings are severely worn, road markings are significantly obscured by traffic congestion, and intersections and the number of lanes are complex, detection of lane markings is likely to be missing or unstable, which can lead to problems such as an inability to determine a recommended driving path or low accuracy of the recommended driving path, and further to reduced vehicle safety. For example, in an intersection scene, lane markings on both sides of the intersection are typically extended and the extended lane markings on both sides are matched. If the match is successful, virtual lane markings are generated as lane boundaries for the vehicle based on the successfully matched extensions on both sides, and a recommended driving path is obtained and used for subsequent decision-making and control. However, the lane marking extension means has a high probability of false matching at intersections with large spans or offset intersections, while it is less effective at detecting lane markings on the opposite side of the intersection that are far from the vehicle, making it difficult to successfully match, and it is not possible to obtain an effective recommended driving trajectory.

[0014] Illustrative Overview FIG. 1 illustrates an exemplary application scenario of the vehicle control method according to the present disclosure. As illustrated in FIG. 1, a vehicle (i.e., a host vehicle) 11 may be equipped with one or more cameras 12 with different viewpoints. While the vehicle 11 is traveling, the cameras 12 with different viewpoints may collect images of the environment surrounding the vehicle. Using the vehicle control method according to the present disclosure, a first image collected by the camera 12 with a preset viewpoint (e.g., a forward viewpoint) of the vehicle 11 can be acquired. The first image can be converted into a second image with a bird's-eye view (abbreviated as BEV, and may also be referred to as a bird's-eye view viewpoint). Based on the second image, a first predicted traveling trajectory can be determined in the vehicle's local coordinate system (e.g., the xoy coordinate system in the figure). The first predicted traveling trajectory is corrected based on observation information corresponding to the first image, a corrected second predicted traveling trajectory is acquired, and the traveling state of the vehicle 11 is controlled based on the second predicted traveling trajectory. Because the second image is a bird's-eye view, the bird's-eye view can more effectively represent the overall situation of the environment surrounding the host vehicle 11, so an effective first predicted driving trajectory can be obtained based on the bird's-eye view. Furthermore, the obtained first predicted driving trajectory is corrected based on structured observation information in the first image that can represent the true direction of the road, such as lane markings 13, road curbs 14, and the driving trajectories of other vehicles 15 around the host vehicle 11. The corrected second predicted driving trajectory is then obtained and used as a recommended driving trajectory to control the driving state of the vehicle 11, significantly improving the accuracy and effectiveness of the recommended driving trajectory and thereby improving the driving safety of the vehicle 11. As shown in FIG. 1 , 17 denotes a crosswalk and 16 denotes an intersection. If the span of the intersection 16 is large, the second image from a bird's-eye view can more effectively represent the overall situation within the predetermined viewing angle range ahead of the host vehicle 11, so an effective first predicted driving trajectory can be obtained. Furthermore, the first predicted driving trajectory is corrected by combining it with observation information such as lane markings 13 ahead of the vehicle 11, road curbs 14, and the driving trajectories of other vehicles 15, to obtain an accurate and effective second predicted driving trajectory that does not rely on the detection of lane markings. Therefore, even if lane markings beyond the intersection 16 cannot be detected or the detection effect of lane markings is low, an accurate and effective recommended driving trajectory can still be obtained.

[0015] Exemplary Methods 2 is a flowchart of a vehicle control method according to an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, specifically, for example, an in-vehicle computing platform. As shown in FIG. 2, the method according to the embodiment of the present disclosure can include the following steps 201 to 205.

[0016] In step 201, a first image is acquired, which is collected by a camera at a preset viewpoint of the vehicle.

[0017] Here, the preset viewpoint may be a forward viewpoint or another viewpoint covering the viewing angle range of the forward viewpoint, and the other viewpoint may include, for example, at least one of a left front viewpoint and a right front viewpoint. The camera of the preset viewpoint may be a monocular camera, a binocular camera, etc. Accordingly, the first image may be a monocular image or a binocular image. In order to reduce hardware costs, the camera of the preset viewpoint may be a monocular camera, and the first image is a monocular image.

[0018] In step 202, the first image is transformed into a second image with a bird's-eye view.

[0019] Here, the second image from a bird's-eye view corresponds to an image obtained by observing the overall environment within the viewing angle range of a preset viewpoint from a bird's-eye view, and can better reflect the overall situation of the environment in front of the vehicle.

[0020] In some alternative embodiments, the conversion from a preset viewpoint to a bird's-eye view can be implemented in any suitable manner. For example, the first image can be converted into the second image with a bird's-eye view by inverse perspective mapping (IPM). The inverse perspective mapping can be implemented based on dynamically estimated camera extrinsic parameters, which can be acquired by an extrinsic parameter estimation module in the vehicle. The embodiments of the present disclosure are not limited thereto. For example, the first image can be converted into the second image with a bird's-eye view by a neural network model. The specific conversion method is not limited thereto.

[0021] In step 203, a first predicted driving trajectory in the local coordinate system of the vehicle is determined based on the second image.

[0022] Here, the local coordinate system may be a coordinate system whose origin is a preset position on the host vehicle (for example, the center of the rear axle of the host vehicle). The origin and coordinate axis directions of the local coordinate system change with changes in the position and attitude of the host vehicle relative to a global coordinate system (for example, a world coordinate system or a host vehicle coordinate system at the initial position of the vehicle).

[0023] In some alternative embodiments, the second image can be used to predict the driving trajectory from a bird's-eye view, and the predicted driving trajectory from the bird's-eye view can then be transformed into the vehicle's local coordinate system to obtain the predicted driving trajectory in the local coordinate system, i.e., the first predicted driving trajectory.

[0024] In some alternative embodiments, the second image may be converted into a local coordinate system, a third image may be acquired in the local coordinate system, and a travel trajectory of the vehicle in the local coordinate system may be predicted based on the third image to obtain a first predicted travel trajectory. The first predicted travel trajectory may be the predicted travel trajectory to be corrected.

[0025] In some alternative embodiments, the driving trajectory may be predicted based on any applicable neural network model to obtain the first predicted driving trajectory. The neural network model may include, for example, a series model based on a convolutional neural network (CNN), a series model based on a recurrent neural network (RNN), or a series model based on a long short-term memory network (LSTM). The series model based on a convolutional neural network may be, for example, a model based on a transformer and its series, or a model based on a residual network (Resnet) and its series.

[0026] In step 204, the first predicted traveling trajectory is corrected based on the observation information corresponding to the first image, and a corrected second predicted traveling trajectory is obtained.

[0027] Here, the observation information corresponding to the first image may include road observation information in a local coordinate system acquired by sensing and observation information of other surrounding vehicles. The road observation information may include at least one of observations such as lane marking curves and road curb curves acquired based on target detection, and lane marking point sequences and road curb point sequences acquired based on semantic segmentation. The observation information of other surrounding vehicles may include observations of other vehicle driving trajectory point sequences (which may be abbreviated as surrounding vehicle trajectory point sequences). These road observations may reflect, to some extent, the true directional change tendency of the road in the area ahead of the host vehicle. Therefore, the first predicted driving trajectory may be corrected based on at least one of these observations of the observation information to obtain a corrected second predicted driving trajectory. This allows the second predicted driving trajectory to better match the true directional change tendency of the road, thereby improving the accuracy and effectiveness of the predicted driving trajectory. In addition, model predictions cannot necessarily fully guarantee safety, so the first predicted driving trajectory may collide with a curb on the road, deviate from the main road, and be misguided into an oncoming lane or a non-motor vehicle lane. By making corrections through observation information corresponding to the first image, these situations can be effectively identified and the safety of the predicted driving trajectory can be improved.

[0028] In step 205, the vehicle's running state is controlled based on the second predicted running path.

[0029] Here, after the second predicted driving trajectory is obtained, the second predicted driving trajectory can be used as a recommended driving trajectory for vehicle planning and control to control the driving state of the vehicle. Specifically, based on the second predicted driving trajectory being set as the future driving trajectory of the vehicle, vehicle control commands can be generated according to the second predicted driving trajectory. The control commands can include lateral control commands and longitudinal control commands for the vehicle, and the lateral and longitudinal driving states of the vehicle can be controlled by the control commands. Specific applications of the second predicted driving trajectory in subsequent decision-making and control are not limited.

[0030] The vehicle control method of this embodiment converts a first image collected by a camera with a preset viewpoint into a bird's-eye view (i.e., a second image from a bird's-eye view), which can more effectively represent the overall situation of the environment around the vehicle, and an effective first predicted driving trajectory can be obtained based on the bird's-eye view.Furthermore, the obtained first predicted driving trajectory can be corrected based on structured observation information in the first image that can represent the true direction of the road, such as lane markings, road curbs, and the driving trajectories of other vehicles around the vehicle, thereby significantly improving the accuracy and effectiveness of the recommended driving trajectory and thereby improving the safety of vehicle driving.

[0031] FIG. 3 is a flowchart of a vehicle control method according to another exemplary embodiment of the present disclosure.

[0032] In some alternative embodiments, as shown in FIG. 3, in the embodiment shown in FIG. 2, step 203 of determining a first predicted driving trajectory in the vehicle's local coordinate system based on the second image may include the following steps 2031 to 2033.

[0033] In step 2031, the second image is processed based on the driving trajectory keypoint detection model to obtain a first sequence of driving trajectory keypoints from a bird's-eye view.

[0034] Here, the driving trajectory keypoint detection model may be a series model based on a convolutional neural network, a series model based on a regression neural network, or a series model based on a long short-term memory network. The second image is processed using the driving trajectory keypoint detection model to obtain a first sequence of driving trajectory keypoints at a bird's-eye view. The first sequence of driving trajectory keypoints may include a set of driving trajectory points at a bird's-eye view obtained by model prediction. The driving trajectory keypoint detection model may be a pre-trained model.

[0035] In some alternative embodiments, the driving trajectory keypoint detection model can use a keypoint detection network architecture based on a stack-hourglass. The detected and acquired first driving trajectory keypoint sequence is a two-dimensional pixel coordinate point sequence in a bird's-eye view.

[0036] In some alternative embodiments, the training data for the driving trajectory keypoint detection model may include a bird's-eye view acquired by collecting images from a vehicle, lane marking truth values ​​corresponding to the bird's-eye view, navigation line data in a high-precision map, and a selected vehicle movement trajectory that matches the change trend of the road.

[0037] In step 2032, the first sequence of driving trajectory key points is transformed from a bird's-eye view into a local coordinate system to obtain a second sequence of driving trajectory key points in the local coordinate system.

[0038] Here, based on the transformation relationship between the coordinate system corresponding to the bird's-eye view and the local coordinate system of the vehicle, the first sequence of driving trajectory key points can be transformed from the bird's-eye view to the local coordinate system to obtain a second sequence of driving trajectory key points in the local coordinate system. The transformation relationship between the coordinate system corresponding to the bird's-eye view and the local coordinate system of the vehicle can be obtained based on the external parameters and internal parameters of the observation camera (camera with a preset viewpoint) corresponding to the bird's-eye view.

[0039] In step 2033, a first predicted driving trajectory is determined based on the second driving trajectory keypoint sequence.

[0040] Here, a curve fitting can be performed on the second driving trajectory keypoint sequence to obtain a first predicted driving trajectory.

[0041] In some alternative embodiments, the second driving trajectory keypoint sequence may be further subjected to post-processing to improve the effectiveness of the driving trajectory keypoint sequence. The first predicted driving trajectory may be determined based on the post-processed driving trajectory keypoint sequence. The post-processing may include at least one of processes such as elimination of abnormal points and non-maximum suppression.

[0042] In this embodiment, the second image is a bird's-eye view image, which can better reflect the overall situation of the environment within the viewing angle range of the predetermined viewpoint of the vehicle. Furthermore, the second image is processed using a driving trajectory keypoint detection model to detect and obtain a valid driving trajectory keypoint sequence from the second image, which can further improve the accuracy and validity of the first predicted driving trajectory.

[0043] In some alternative embodiments, step 2033 of determining a first predicted driving trajectory based on the second driving trajectory keypoint sequence may include the steps of performing abnormal point removal and non-maximum value suppression processing on the second driving trajectory keypoint sequence to obtain a processed third driving trajectory keypoint sequence, and performing curve fitting on the third driving trajectory keypoint sequence to obtain the first predicted driving trajectory.

[0044] Here, the removal of abnormal points may refer to the removal of outlying points. Non-Maximum Suppression (NMS) may refer to the extraction of the most representative point from multiple repeatedly predicted points and the ignoring of other repeatedly predicted points. The repeatedly predicted points may refer to multiple points at different horizontal distances detected at the same vertical distance by the driving trajectory keypoint detection model during the keypoint detection process.

[0045] In some alternative embodiments, when the driving trajectory keypoint detection model detects and obtains the first driving trajectory keypoint sequence, it can further output a confidence level for each keypoint in the first driving trajectory keypoint sequence. Abnormal points can be removed based on the confidence level of each keypoint and a confidence level threshold. Specifically, keypoints with confidence levels lower than the confidence level threshold can be removed as abnormal points.

[0046] In some alternative embodiments, the order of the processes of removing abnormal points and suppressing non-maximum values ​​is not limited, and the order may be such that the abnormal points are removed first and then non-maximum value suppression is performed, or the order may be such that non-maximum value suppression is performed first and then the abnormal points are removed.

[0047] In some alternative embodiments, before the second driving trajectory keypoint sequence is subjected to the abnormal point removal and non-maximum suppression processes, the second driving trajectory keypoint sequence is further clustered to obtain a plurality of clusters, and an optimal cluster can be selected from the plurality of clusters as the second driving trajectory keypoint sequence, thereby further improving the accuracy and effectiveness of the driving trajectory keypoint sequence.The second driving trajectory keypoint sequence is further subjected to the abnormal point removal and non-maximum suppression processes to obtain a third driving trajectory keypoint sequence.

[0048] In some alternative embodiments, a cubic curve may be used to perform curve fitting on the third driving trajectory keypoint sequence to obtain the first predicted driving trajectory. Specifically, an arbitrary optimization method may be used based on the third driving trajectory keypoint sequence to find optimal cubic curve coefficients, and the first predicted driving trajectory may be obtained based on the optimal cubic curve coefficients. The optimization algorithm may include, for example, the least squares method and any other applicable optimization method.

[0049] In this embodiment, the accuracy and validity of the driving trajectory keypoint sequence can be improved by removing abnormal points and suppressing non-maximum values, thereby further improving the accuracy of the first predicted driving trajectory.

[0050] In some alternative embodiments, after step 2031 of processing the second image based on the driving trajectory keypoint detection model to obtain a first driving trajectory keypoint sequence from a bird's-eye view, the following steps may be performed: removing abnormal points and suppressing non-maximum values ​​from the first driving trajectory keypoint sequence to obtain a processed third driving trajectory keypoint sequence; transforming the third driving trajectory keypoint sequence from the bird's-eye view to a local coordinate system to obtain a second driving trajectory keypoint sequence in the local coordinate system; and performing curve fitting on the second driving trajectory keypoint sequence to obtain a first predicted driving trajectory.

[0051] In some alternative embodiments, after step 2031 of processing the second image based on the driving trajectory keypoint detection model to obtain a first driving trajectory keypoint sequence from a bird's-eye view, the following steps can be performed: removing abnormal points and suppressing non-maximum values ​​from the first driving trajectory keypoint sequence to obtain a processed third driving trajectory keypoint sequence; performing curve fitting on the third driving trajectory keypoint sequence to obtain a fitted trajectory; and transforming the fitted trajectory from the bird's-eye view to a local coordinate system to obtain a first predicted driving trajectory in the local coordinate system.

[0052] FIG. 4 is a flowchart of a vehicle control method according to yet another exemplary embodiment of the present disclosure.

[0053] In some alternative embodiments, as shown in FIG. 4, in the embodiment shown in FIG. 2, step 204 of correcting the first predicted driving trajectory based on observation information corresponding to the first image and obtaining a corrected second predicted driving trajectory may include the following steps 2041 to 2044.

[0054] In step 2041, a first historical predicted traveling locus is acquired.

[0055] Here, the first historical predicted traveling trajectory may be a corrected predicted traveling trajectory obtained by processing an image of a historical frame (the processing process may correspond to steps 201 to 204). The image of the historical frame may be an image acquired by a camera with a preset viewpoint at a time corresponding to the historical frame. In other words, the first historical predicted traveling trajectory is a second predicted traveling trajectory acquired in accordance with steps 201 to 204 based on the first image acquired at the historical frame time. In this case, the first historical predicted traveling trajectory is a traveling trajectory in the vehicle historical local coordinate system corresponding to the historical frame.

[0056] In some alternative embodiments, the first historical predicted driving trajectory can be obtained for one or more frames, or the first historical predicted driving trajectory can include one or more frames of historical predicted driving trajectory.

[0057] In some alternative embodiments, the number of frames of the historical predicted driving trajectory can be controlled by a sliding window. For example, the sliding window can be set to a preset number of frames, which includes the current frame and a first number of historical frames, thereby allowing the sliding window to always hold the first predicted driving trajectory of the current frame and the first historical predicted driving trajectory of the first number of historical frames. Each time a first predicted driving trajectory of a new frame is generated, the first historical predicted driving trajectory of the oldest historical frame slides out of the sliding window to ensure the timeliness of the included historical predicted driving trajectory.

[0058] In step 2042, the first historical predicted traveling path is transformed into a local coordinate system to obtain a second historical predicted traveling path.

[0059] Here, the local coordinate system is the local coordinate system of the current frame. That is, the first historical predicted driving trajectory is transformed from the historical local coordinate system of the historical frame to the local coordinate system of the current frame. For a first historical predicted driving trajectory of any historical frame, the first historical predicted driving trajectory can be transformed from the historical local coordinate system of the historical frame to the local coordinate system of the current frame based on the transformation relationship between the historical local coordinate system corresponding to the historical frame and the global coordinate system and the transformation relationship between the global coordinate system and the local coordinate system of the current frame, thereby obtaining a second historical predicted driving trajectory corresponding to the first historical predicted driving trajectory. The transformation relationship between the historical local coordinate system and the global coordinate system can be obtained during processing of the historical frame. The transformation relationship between the local coordinate system of the current frame and the global coordinate system can be obtained by positioning the vehicle based on vehicle chassis information of the current frame, which can include odometer information such as the vehicle's speed, acceleration, angular velocity, and position.

[0060] In step 2043, the predicted traveling locus to be corrected is determined based on the second historical predicted traveling locus and the first predicted traveling locus.

[0061] Here, the second historical predicted driving trajectory and the first predicted driving trajectory (the second historical predicted driving trajectory and the first predicted driving trajectory can be abbreviated as each predicted driving trajectory) can be integrated to determine the predicted driving trajectory to be corrected. Because the predicted driving trajectory is a single trajectory curve, there is often a vertical overlap between the predicted driving trajectories of multiple consecutive frames, and the prediction results of each of these overlapping portions can be reflected. Therefore, by integrating the predicted driving trajectories of multiple frames, a more reliable predicted driving trajectory to be corrected can be determined, thereby improving the accuracy and effectiveness of the predicted driving trajectory to be corrected.

[0062] In some alternative embodiments, the predicted driving trajectory to be corrected can be determined based on the lateral distribution of each predicted driving trajectory. For example, a target predicted driving trajectory that is concentrated among the predicted driving trajectories can be determined based on the lateral distribution, and the concentrated target predicted driving trajectories can be merged into a single predicted driving trajectory to be corrected.

[0063] In step 2044, the predicted traveling trajectory to be corrected is corrected based on the observation information corresponding to the first image, and a second predicted traveling trajectory is obtained.

[0064] Here, the operation of correcting the predicted traveling trajectory to be corrected based on the observation information is similar to the specific operation of correcting the first predicted traveling trajectory, and therefore a description thereof will be omitted here.

[0065] In this embodiment, there is often a vertical overlap between the predicted driving trajectories of multiple consecutive frames, and the prediction results for each of these overlapping portions can be reflected in each frame. Therefore, by integrating the historical predicted driving trajectories of multiple frames and the first predicted driving trajectory of the current frame, a more reliable predicted driving trajectory to be corrected can be determined, thereby improving the accuracy and effectiveness of the predicted driving trajectory to be corrected.

[0066] In some alternative embodiments, step 2043 of determining a predicted driving trajectory to be corrected based on the second historical predicted driving trajectory and the first predicted driving trajectory may include the steps of performing clustering on the second historical predicted driving trajectory and the first predicted driving trajectory to obtain a clustering result, the clustering result including at least one cluster; determining a target cluster that satisfies a predetermined condition from each cluster based on the clustering result; and fusing the predicted driving trajectories in the target cluster to obtain a predicted driving trajectory to be corrected.

[0067] Here, each predicted driving trajectory can be clustered using any feasible clustering method. The clustering method may be, for example, K-Means or another method. Clustering can be performed based on the horizontal distance between sampling points at multiple vertical distances on each predicted driving trajectory. The vertical distance may refer to the vertical distance between the sampling point and the host vehicle, and the horizontal distance may refer to the horizontal distance between the sampling point and the host vehicle. Since each predicted driving trajectory is a trajectory in the local coordinate system of the current frame, the vertical distance may be represented by the vertical coordinate of the sampling point, and the horizontal distance may be represented by the horizontal coordinate of the sampling point. The preset condition may include designating the cluster with the highest score as the target cluster. The cluster score can be obtained by scoring each cluster based on scoring rules, which may be set based on the aging, distribution concentration, number of trajectories, etc. of the predicted driving trajectories in the cluster. The scoring rules may include, for example, an aging weight rule, a distribution concentration calculation rule, and a comprehensive score calculation rule for aging, distribution concentration, and number of trajectories. The staleness weight rule may be set based on the principle that the closer to the current frame, the greater the staleness weight. For example, the staleness weight rule may be a linear function that decays over time. The weight of each predicted driving trajectory in the cluster is calculated based on this linear function, and the staleness score is obtained through weighting. The distribution concentration calculation rule may be set based on the horizontal distribution width of each predicted driving trajectory in the cluster at different vertical positions. The smaller the horizontal distribution width, the higher the distribution concentration and the higher the score. Based on this, the distribution concentration score of the cluster can be obtained. For the number of trajectories, a number score corresponding to each number can be set, and a score for the number of clusters can be obtained based on this. The overall score calculation rule for staleness, distribution concentration, and number of trajectories can include weighting coefficients corresponding to the staleness score, distribution concentration score, and number score, and further weights each score based on the weighting coefficient to obtain an overall score.The cluster with the highest overall score is designated as the target cluster. The predicted driving trajectories in the target cluster are merged into a single trajectory, which is designated as the predicted driving trajectory to be corrected.

[0068] In some alternative embodiments, the fusion method may be any feasible method. For example, the fusion method may calculate an average trajectory based on each predicted driving trajectory in the target cluster, and use the average trajectory as the predicted driving trajectory to be corrected. Alternatively, for example, multiple sampling points are sampled for each predicted driving trajectory based on multiple vertical coordinates. Fusion points corresponding to each vertical coordinate are determined based on the horizontal coordinates of each sampling point, and the predicted driving trajectory to be corrected is fitted based on each fusion point. For any one vertical coordinate, the average of the horizontal coordinates of each sampling point corresponding to that vertical coordinate (i.e., each sampling point has the same vertical coordinate) is calculated, and the horizontal coordinate of the fusion point corresponding to that vertical coordinate is obtained. The fusion point is then obtained based on the vertical coordinate and the horizontal coordinate of the fusion point. Alternatively, points that are out of sync with each sampling point corresponding to that vertical coordinate are removed, and the average is calculated to obtain the fusion point, thereby improving the accuracy and effectiveness of the fusion point and further improving the accuracy and effectiveness of the predicted driving trajectory to be corrected. Specific fusion methods are not limited.

[0069] In this embodiment, an optimal target cluster can be found from multiple clusters obtained by clustering according to preset conditions, and each predicted driving trajectory in the target cluster is more consistent with the future driving tendency of the current frame. Therefore, by fusing each predicted driving trajectory in the target cluster to obtain the predicted driving trajectory to be corrected, the accuracy and effectiveness of the predicted driving trajectory to be corrected can be improved.

[0070] FIG. 5 is a flowchart of a vehicle control method according to yet another exemplary embodiment of the present disclosure.

[0071] In some alternative embodiments, as shown in FIG. 5, in the embodiment shown in FIG. 2, step 204 of correcting the first predicted driving trajectory based on observation information corresponding to the first image and obtaining a corrected second predicted driving trajectory may include the following steps 204a to 204c.

[0072] In step 204a, a target observation amount for correcting the first predicted traveling trajectory is determined based on the observation information corresponding to the first image.

[0073] Here, the observation information can include at least one of observation quantities such as road observation information and observation information of other vehicles in the vicinity, so by setting a matching rule, it is possible to determine a target observation quantity for correcting the first predicted driving trajectory from the observation information.

[0074] In some alternative embodiments, the target observables may include one or more types of observables. For example, the target observables may include at least one of lane marking curves and road curb curves obtained by target detection, or the target observables may include at least one of lane marking curves and road curb curves, and at least one of a lane marking point sequence and a road curb point sequence obtained by semantic segmentation. Alternatively, the target observables may include at least one of lane marking curves and road curb curves, and a surrounding vehicle trajectory point sequence. Different observables may be used for different restrictions on the correction process. In practical applications, the effective effects and correction effects of different observables may be different. For example, the effective effects and correction effects of lane marking curves may be stronger than those of lane marking point sequences.

[0075] In step 204b, restriction points are determined based on the target observables.

[0076] Here, the restriction point may be a point on the centerline representing the main lane (i.e., the lane in which the vehicle is located). The restriction point can be determined based on the relationship between the target observation and the main lane centerline. For example, if the target observation is a lane marking curve, a road curb curve, a lane marking point sequence, or a road curb point sequence, the restriction point may be a point on the median line between the lane markings on both sides of the main lane. If only road curb observation information is available, the restriction point on the main lane can be determined based on the lateral distance between the curbs on both sides of the road and the vehicle. If only one lane marking is available, a point on the centerline of the main lane can be determined as the restriction point based on the lane width and the one lane marking. For example, the one lane marking may be translated laterally by half a lane toward the vehicle depending on the lane width to obtain the centerline of the main lane, and a point on the centerline may then be obtained as the restriction point. When the target observable is a peripheral vehicle trajectory point sequence, if it is determined that the peripheral vehicle trajectory point sequence does not have a lane change tendency, the peripheral vehicle trajectory point sequence can reflect the driving direction tendency of the lane centerline, and therefore, a restriction point on the main lane centerline can be determined based on the peripheral vehicle trajectory point sequence. The number of restriction points may be one or more. For example, the restriction point may be a point sequence consisting of multiple points, and this point sequence reflects the direction of the main lane centerline.

[0077] In step 204c, the first predicted driving trajectory is filtered based on the restriction points to obtain a second predicted driving trajectory.

[0078] Here, since the restriction points reflect the direction of the center line of the main lane, the first predicted driving trajectory is filtered based on the restriction points, and the second predicted driving trajectory after filtering is made closer to the direction of the restriction points. Any feasible filtering method can be used, for example, the filtering method may be an extended Kalman filter (abbreviated as EKF) or other filtering.

[0079] In this embodiment, since the observation amount of the observation information can reflect the true direction of the road, the correction target observation amount determined based on the observation information is used to determine the control point and serve as a control to filter the first predicted driving trajectory, so that the second predicted driving trajectory after filtering is more consistent with the true directional change tendency of the main lane, thereby improving the accuracy and effectiveness of the second predicted driving trajectory.

[0080] In some embodiments, step 204a of determining a target observable for correcting the first predicted driving trajectory based on observation information corresponding to the first image includes the steps of determining, based on the observation information, at least one of a lane marking curve and a road curb curve in a local coordinate system, a lane marking point sequence and a road curb point sequence, and a driving trajectory point sequence of a target vehicle that satisfies a first condition around the vehicle, and determining, as the target observable, at least one of a lane marking curve and a road curb curve in the local coordinate system, a lane marking point sequence and a road curb point sequence, and a driving trajectory point sequence of a target vehicle that satisfies the first condition around the vehicle.

[0081] The first condition may be set based on whether the driving trajectories of other vehicles around the host vehicle can reflect the true direction of the lane. For example, the first condition may include that the length of the driving trajectory point sequence of the other vehicles is greater than a length threshold, can be fitted using a cubic curve, and the trajectory point sequence does not exhibit any significant lane change behavior. This ensures that the driving trajectory point sequence of the target vehicle can accurately reflect the true direction of the lane, and can be used to correct the first predicted driving trajectory as a restriction on the predicted driving trajectory.

[0082] In this embodiment, an observation quantity that can reflect the true lane direction change tendency is used as a target observation quantity to regulate the correction of the first predicted driving trajectory, so that the corrected predicted driving trajectory is more consistent with the true lane direction change tendency, thereby improving the accuracy and effectiveness of the predicted driving trajectory.

[0083] In some alternative embodiments, step 204c of filtering the first predicted driving trajectory based on the restriction points and obtaining a second predicted driving trajectory may include the steps of: performing extended Kalman filtering on the first predicted driving trajectory based on the restriction points and obtaining a third predicted driving trajectory after filtering; determining a first sub-trajectory curve within a first distance range close to the vehicle and a second sub-trajectory curve within a second distance range far from the vehicle based on the third predicted driving trajectory; and determining the second predicted driving trajectory based on the first sub-trajectory curve and the second sub-trajectory curve.

[0084] Here, the correction is performed using extended Kalman filtering, gradually reducing the lateral deviation between the corrected predicted driving trajectory and the regulating point, and the predicted driving trajectory continuously approaches the regulating point to obtain a third filtered predicted driving trajectory. The first and second distance ranges can be set according to the longitudinal overlap distance between the front and rear frames and the need for a smooth connection. The first sub-trajectory curve within the first distance range can be referred to as the short-distance segment curve. The second sub-trajectory curve within the second distance range can be referred to as the long-distance segment curve. The segmentation purpose can include the following two aspects. On the one hand, the short-distance segment curve can complete the recursion of the front and rear frames based on the odometer, ensuring that the predicted driving trajectory of each frame basically corresponds to the same position in the global coordinate system, reducing the influence of observation fluctuations and avoiding obvious vibrations. On the other hand, the long-distance segment curve can timely adjust the extension direction and curve shape of the predicted driving trajectory based on the latest observations.

[0085] In some selectable examples, the first distance range may be a range of 0 to 20 meters in the longitudinal distance from the host vehicle. The second distance range may be a range where the longitudinal distance from the host vehicle is greater than 30 meters. An intermediate segment is used to complete a smooth connection between the short distance segment and the long distance segment. In practical applications, the specific boundary values ​​of the first distance range and the second distance range are not limited to the ranges in the above examples, and can be set according to the actual situation of the vehicle.

[0086] In some optional embodiments, after obtaining the first sub-trajectory curve and the second sub-trajectory curve, the first sub-trajectory curve can be modified, and the modified short-distance segment curve and long-distance segment curve can be smoothly connected to obtain a second predicted driving trajectory.

[0087] In this embodiment, by segmenting and displaying the filtered third predicted driving trajectory, the short-distance segment curve can complete the recursion of the previous and next frames based on the odometer, ensuring that the predicted driving trajectory of each frame basically corresponds to the same position in the global coordinate system, reducing the influence of observation fluctuations and avoiding obvious fluctuations, and the long-distance segment curve can timely adjust the extension direction and curve shape of the predicted driving trajectory based on the latest observations, so that the corrected predicted driving trajectory better reflects the true road extension trend and connection relationship, thereby greatly improving the accuracy and effectiveness of the predicted driving trajectory.

[0088] In some alternative embodiments, the step of determining the second predicted driving trajectory based on the first sub-trajectory curve and the second sub-trajectory curve may include the steps of determining a first connection point on the first sub-trajectory curve, modifying the first sub-trajectory curve based on the second sub-trajectory curve to obtain a modified third sub-trajectory curve, determining a second connection point on the second sub-trajectory curve, determining a connection curve that can connect the first connection point and the second connection point and meets a second condition, and determining the second predicted driving trajectory based on the second sub-trajectory curve, the third sub-trajectory curve, and the connection curve.

[0089] Here, the first connection point may be an end point on the first sub-trajectory curve that is far from the host vehicle, i.e., an end point on the first sub-trajectory curve that is close to the second sub-trajectory curve. For example, if the first distance range is from 0 to 20 meters, the first connection point may be a point on the first sub-trajectory curve that is 20 meters away from the host vehicle. The purpose of correcting the first sub-trajectory curve is to make the corrected third sub-trajectory curve (which may be referred to as the corrected short distance segment curve) tend to gradually match the long distance segment curve and reduce the positional and angular differences between the short distance segment curve and the long distance segment curve. Therefore, the short distance segment curve is corrected based on the long distance segment curve to obtain the corrected short distance segment curve.

[0090] In some alternative embodiments, a point satisfying a condition can be searched for as a second connection point from the second sub-trajectory curve according to a certain search rule. The search rule may include a search direction, a search step size, etc. The search direction may include directions such as close to far and far to close, where close and far indicate the proximity of the longitudinal distance to the host vehicle. The search step size may be a longitudinal distance step size. That is, after completing the search for one point, the longitudinal distance of the next search point is determined based on the longitudinal distance of the point and the longitudinal distance step size, and the point at this longitudinal distance on the second sub-trajectory curve is determined as the next search point based on the longitudinal distance. Whether the currently searched point is a second connection point can be determined based on a preset rule. The preset rule may include a rule for determining whether the relationship between the searched point and the first connection point satisfies a corresponding condition. This condition may include a condition for the difference (or deviation, which can also be called error) between the searched point and the first connection point to be satisfied. The difference between the searched point and the first connection point may include a position difference, an angle difference, etc. For example, this condition may include that the position difference is less than a first threshold and the angle difference is less than a second threshold. If it is determined that the currently searched point satisfies the corresponding condition, the search may be terminated, and the currently searched point may be set as the second connection point of the second sub-trajectory curve. Obtaining the second connection point through the search can ensure a smooth connection between the near segment curve and the far segment curve, and the obtained predicted driving trajectory can better support downstream planning and control.

[0091] In some alternative embodiments, the second condition may include a smooth connection condition. For example, the second condition may include that the curves are continuous at the two connection points, that the first derivatives of the curves are continuous at the two connection points, and that the change in the second derivatives of the curves at the two connection points is less than a threshold. The connection curve obtained based on the second condition may smoothly connect the corrected short-distance segment curves and long-distance segment curves, thereby obtaining a smooth and continuous second predicted driving trajectory.

[0092] In this embodiment, by modifying the short distance segment curve, the modified short distance segment curve tends to gradually match the long distance segment curve, reducing the positional difference and angle difference between the short distance segment curve and the long distance segment curve, making the short distance segment curve and the long distance segment curve match, ensuring the stability of the short distance segment curve, smoothly transmitting the road change trend sensed by the long distance segment to the short distance segment, avoiding the significant mismatch between the short distance segment and the long distance segment caused by the gradual increase in the difference between the short distance segment and the long distance segment, and improving the smoothness of the entire segmented curve.

[0093] In some alternative embodiments, the step of determining a second connection point on the second sub-trajectory curve includes the steps of searching for a sampling point on the second sub-trajectory curve based on a preset direction, and determining a target point on the third sub-trajectory curve corresponding to the current sampling point searched for, where the longitudinal distance of the target point to the vehicle is the same as the longitudinal distance of the current sampling point to the vehicle; determining a lateral position difference amount and an angle difference amount between the current sampling point and the target point; determining a relationship between the lateral position difference amount and the angle difference amount and a third condition; and determining the second connection point based on the relationship between the lateral position difference amount and the angle difference amount and the third condition.

[0094] Here, the preset direction can be set to a near-to-far or far-to-near direction according to actual needs. The current sampling point searched can be determined according to a search step size. The target point corresponding to the current sampling point on the third sub-trajectory curve is a point on the third sub-trajectory curve whose longitudinal distance is the same as that of the current sampling point, i.e., the longitudinal distance of the current sampling point relative to the vehicle is the same as that of the target point. The lateral position difference is the absolute value of the difference between the lateral coordinate of the current sampling point and the lateral coordinate of the target point. The angular difference may be the absolute value of the difference between the tangential angle at the current sampling point and the tangential angle at the target point. The third condition may include a condition that the lateral position difference and the longitudinal distance are satisfied, and a condition that the angular difference and the longitudinal distance are satisfied. The longitudinal distance in the third condition may be the longitudinal distance between the target point and the first connection point. The condition that the horizontal position difference amount and the vertical distance satisfy may be that the ratio of the horizontal position difference amount to the vertical distance is less than a first threshold. The condition that the angular difference amount and the vertical distance satisfy may be that the ratio of the angular difference amount to the vertical distance is less than a second threshold. The first and second thresholds are preset values. The relationship between the horizontal position difference amount and the angular difference amount and the third condition may include whether the horizontal position difference amount and the angular difference amount satisfy the third condition. If it is determined that the horizontal position difference amount and the angular difference amount satisfy the third condition, the search may be terminated and the current sampling point may be determined as the second connection point; if not, the search may continue to search for the next sampling point, and so on until the second connection point is found.

[0095] In this embodiment, the search for the second connection point is guided by the lateral position difference amount and the angle difference amount, and the difference between the searched second connection point and the corrected short-distance segment curve is reduced, thereby improving the smoothness of the connection between the corrected short-distance segment curve and the long-distance segment curve and the overall integrity of the predicted driving trajectory.

[0096] In some selectable embodiments, step 2044 of correcting the predicted driving trajectory to be corrected based on observation information corresponding to the first image and obtaining a second predicted driving trajectory may include the steps of determining a target observation quantity for correcting the predicted driving trajectory to be corrected based on the observation information corresponding to the first image, determining restriction points based on the target observation quantity, and filtering the predicted driving trajectory to be corrected based on the restriction points to obtain a second predicted driving trajectory. Here, the specific operations of each step can be referred to in the respective embodiments of correcting the first predicted driving trajectory described above, and therefore will not be described here.

[0097] In some selectable embodiments, Fig. 6 is a flowchart for determining a recommended driving path according to an exemplary embodiment of the present disclosure. As shown in Fig. 6, the first image of the preset viewpoint is a monocular forward viewpoint image, and the flow for determining a recommended driving path according to the embodiment of the present disclosure may include the following steps 301 to 308.

[0098] In step 301, a monocular forward view image is mapped to an IPM image based on a dynamic extrinsic parameter estimation matrix, and the IPM image is a second image of a bird's-eye view.

[0099] In step 302, the IPM image is processed through a CNN model to obtain a first sequence of bird's-eye view driving trajectory keypoints.

[0100] In step 303, the first sequence of driving trajectory key points is transformed into a local coordinate system to obtain a second sequence of driving trajectory key points in the local coordinate system.

[0101] In step 304, the second driving trajectory keypoint sequence is subjected to non-maximum value suppression and abnormal point removal processing, and then curve fitting is performed to obtain a first predicted driving trajectory.

[0102] In step 305, the first historical predicted driving trajectory of one frame or multiple frames is transformed into the local coordinate system of the current frame (i.e., the frame corresponding to the monocular forward viewpoint image) to obtain the second historical predicted driving trajectory in the local coordinate system.

[0103] In practical applications, step 305 and steps 301 to 304 may be performed in any order, and the second historical predicted driving trajectory in the local coordinate system may be obtained before step 306.

[0104] In step 306, the predicted driving trajectories of multiple frames of the first predicted driving trajectory and the second historical predicted driving trajectory are clustered, an optimal cluster is selected, and each predicted driving trajectory in the optimal cluster is merged to obtain the predicted driving trajectory to be corrected.

[0105] In step 307, forward-view sensing is performed based on the monocular forward view image and vehicle chassis information to obtain structured observation information such as lane markings, road curbs, and surrounding vehicle trajectories.

[0106] In actual applications, step 307 and steps 301 to 306 may be performed in any order, as long as the observation information is obtained before step 308.

[0107] In step 308, anomaly detection and curve correction are performed on the predicted driving trajectory to be corrected based on the observation information, and a second predicted driving trajectory is obtained.

[0108] The method of the embodiment of the present disclosure is based on a monocular forward view image, combined with vehicle chassis information, to recommend an accurate and effective driving trajectory for vehicle driving, and does not rely on high-cost sensors such as laser radar or binocular cameras, thereby achieving effective driving trajectory recommendation at low hardware costs.

[0109] 7 is a schematic diagram illustrating a process for determining a first predicted driving trajectory according to an exemplary embodiment of the present disclosure. As shown in FIG. 7, the CNN model outputs a first driving trajectory keypoint sequence 31. After that, the CNN model performs an abnormal point removal and non-maximum suppression process on the first driving trajectory keypoint sequence 31 in step 310 to obtain a processed third driving trajectory keypoint sequence 32. The CNN model then performs curve fitting on the third driving trajectory keypoint sequence 32 in step 320 to obtain a fitted trajectory 33. The CNN model then performs coordinate system transformation on the fitted trajectory 33 in step 330 to transform the fitted trajectory 33 from a bird's-eye view to a local coordinate system, thereby obtaining a first predicted driving trajectory 34 in the local coordinate system.

[0110] In some selectable embodiments, Fig. 8 is a schematic diagram of obtaining a predicted driving trajectory to be corrected by multi-frame fusion according to an exemplary embodiment of the present disclosure. As shown in Fig. 8, after obtaining a predicted driving trajectory 35 of multiple frames based on a first predicted driving trajectory of a current frame and a second historical predicted driving trajectory of a historical frame transformed into the local coordinate system of the current frame, the predicted driving trajectories 35 of the multiple frames can be fused. The multi-frame fusion flow can include the following steps 350 to 370.

[0111] In step 350, the predicted driving trajectories 35 of multiple frames are clustered based on the horizontal distance between sampling points at the same vertical coordinate in each predicted driving trajectory 35, to obtain multiple clusters 36.

[0112] In step 360, each cluster 36 is scored based on factors such as the age of the predicted driving trajectories 35 in each cluster 36, the distribution concentration, and the number of trajectories, and the optimal cluster (i.e., target cluster) 361 is selected according to the score of each cluster 36.

[0113] In step 370, the predicted driving trajectories 35 in the optimal cluster 361 are fused to obtain a fused trajectory 37, which is then used as the predicted driving trajectory 37 to be corrected.

[0114] 9 is a flowchart for determining a predicted driving trajectory to be corrected according to an exemplary embodiment of the present disclosure. As shown in FIG. 9, the flow for determining a predicted driving trajectory to be corrected may include the following steps 401 to 405.

[0115] In step 401, the historical predicted driving trajectories of multiple frames are projected onto the local coordinate system of the current frame to obtain the historical predicted driving trajectories of multiple frames in the local coordinate system.

[0116] In step 402, clustering is performed based on the first predicted driving trajectory in the local coordinate system of the current frame and the historical predicted driving trajectories of multiple frames, and the clusters formed by the historical frames are updated.

[0117] In step 403, the predicted driving trajectories of the history frames that exceed the sliding window range are removed in a timely manner to ensure the validity of the trajectories (i.e., predicted driving trajectories) participating in the clustering and the validity of the number of trajectories.

[0118] In step 404, the optimal cluster is selected based on the number of trajectories, the aging, the distribution concentration, etc. of the trajectories in each cluster.

[0119] In step 405, the trajectories in the optimal cluster are fused to obtain a fused trajectory, which is used as the predicted driving trajectory to be corrected.

[0120] In this embodiment, the predicted driving trajectory of the current frame participates in clustering, the clusters formed by the historical frames are updated, and historical frame trajectories that exceed the sliding window range are removed in a timely manner, thereby ensuring the timeliness of the trajectories participating in the clustering and the availability of the number of trajectories, and thereby further improving the accuracy of the predicted driving trajectory to be corrected.

[0121] In some alternative embodiments, Fig. 10 is a flowchart of a correction flow of a predicted driving trajectory according to an exemplary embodiment of the present disclosure. As shown in Fig. 10, the correction flow may include the following steps 411 to 413.

[0122] In step 411, a target observation for correction is matched based on the predicted driving trajectory of the object to be corrected and the observation information acquired by forward viewpoint sensing. The observation information may include lane marking curves and road curb curves acquired by road observation, a lane marking point sequence and road curb point sequence acquired by semantic segmentation, and a surrounding vehicle trajectory point sequence. The surrounding vehicles are other vehicles around the host vehicle.

[0123] In step 412, the target observables are utilized to generate correction points (ie, constraint points).

[0124] In step 413, the EKF is used in combination with the correction points to correct the predicted traveling trajectory to be corrected, and a corrected second predicted traveling trajectory is obtained.

[0125] In this embodiment, the predicted driving trajectory to be corrected is corrected by using the EKF to regulate correction points, and the correction points are generated based on observation information that can reflect the true road direction, such as lane marking curves, road curb curves, lane marking point sequences, road curb point sequences, and surrounding vehicle trajectory point sequences, so that the corrected second predicted driving trajectory can be made to more closely match the true road direction, thereby effectively improving the accuracy and effectiveness of the second predicted driving trajectory.

[0126] 11 is a schematic diagram of observation information according to one exemplary embodiment of the present disclosure. As shown in FIG. 11, the observation information acquired by the forward viewpoint sensing system can include at least one of observations, such as a lane marking curve 43 and a road curb curve 44 acquired by target detection, a lane marking point sequence 45 acquired by semantic segmentation, and a surrounding vehicle trajectory point sequence 46 acquired by obstacle detection. Furthermore, at least one of the acquired observations can be determined as a target observation for correcting the predicted driving trajectory to be corrected. The lane marking curves 43 and the road curb curves 44 can be represented as cubic curves. Based on the lane marking curves 43 and the road curb curves 44, correction points are generated that more accurately match the center line of the main lane (i.e., the lane where the host vehicle 11 is located) to more accurately reflect the position and directional change tendency of the main lane. These correction points are used to restrict the correction of the predicted driving trajectory to be corrected, or to restrict the correction of a portion of the predicted driving trajectory to be corrected that is close to the host vehicle 11, thereby avoiding a situation where the predicted driving trajectory is biased toward or intersects with a lane marking on one side. While it may be impossible to clearly detect the lane marking curves 43 and / or road curb curves 44 at a position far away from the host vehicle 11, a point sequence indicating the lane marking lines and / or road curbs, i.e., a lane marking point sequence 45 and a road curb point sequence (not shown), can be obtained by semantic segmentation processing. These point sequences are matched and verified to determine whether they are located on the left and right lane markings or the extensions of the left and right road curbs. If it is determined that these point sequences are located on the left and right lane markings or the extensions of the left and right road curbs, correction points can be generated based on these point sequences and used to correct the predicted driving trajectory to be corrected or to correct the curve of the distant part of the predicted driving trajectory to be corrected.In locations where road observations are significantly lacking, such as where traffic is dense, a portion of the surrounding vehicle trajectory point sequences 46 that satisfy a first condition can be screened to generate correction points and correct the predicted driving trajectory to be corrected. The first condition may be, for example, that the trajectory length is greater than a length threshold, that curve fitting (e.g., cubic curve fitting) is used, and that there is no obvious lane change behavior. The surrounding vehicle trajectory point sequences 46 that satisfy the first condition can reflect the directional change tendency of the road and can be used to correct the local curvature of the predicted driving trajectory to be corrected, making the corrected trajectory more consistent with the true direction of the road and improving the accuracy of the driving trajectory.

[0127] In some alternative embodiments, Fig. 12 is a flowchart illustrating a process for determining a second predicted driving trajectory based on a restriction point according to an exemplary embodiment of the present disclosure. As shown in Fig. 12, this process may include the following steps 421 to 426.

[0128] In step 421, an extended Kalman filtering (EKF) process is performed on the predicted driving trajectory to be corrected based on the regulation points, and a third predicted driving trajectory after filtering is obtained.

[0129] In step 422, based on the longitudinal distance from the vehicle, curve segmentation is performed on the third predicted driving trajectory to obtain a close-distance segment curve (i.e., a first sub-trajectory curve within a first distance range) and a long-distance segment curve (i.e., a second sub-trajectory curve within a second distance range).

[0130] In step 423, a connection point (ie, the first connection point) on the near segment curve is determined.

[0131] In step 424, the near segment curve is modified based on the far segment curve, so that the modified near segment curve (i.e., the third sub-trajectory curve) tends to gradually match the far segment curve. If the position of the first connection point changes after the modification, the modified connection point is set as the first connection point.

[0132] In step 425, a connection point (i.e., a second connection point) on the far segment curve is searched for from near to far on the far segment curve. Here, the preset direction is described as from near to far as an example, but this is not a limitation on the preset direction of the embodiment of the present disclosure.

[0133] In step 426, two connection points on the modified short distance segment curve and long distance segment curve are smoothly connected, so that the curves at the two connection points are continuous, the first derivatives are continuous, and the change in the second derivative is smaller than a threshold value. That is, a connection curve that can connect the first connection point and the second connection point and meets a second condition is determined. The first connection point and the second connection point are connected based on the connection curve to obtain a blended curve in the form of a segmented cubic curve, that is, a blended curve formed by the modified short distance segment curve, the connection curve, and the long distance segment curve is obtained, and this blended curve is used as a second predicted driving trajectory.

[0134] In this embodiment, the second predicted driving trajectory in the form of a smoothly connected segmented cubic curve can ensure that it accurately matches the center line of the main lane at different longitudinal distances.

[0135] In some alternative embodiments, FIG. 13 is a schematic diagram of a segmented cubic curve according to an exemplary embodiment of the present disclosure. As shown in FIG. 13, after determining a first connection point 611 on the modified near-distance segment curve 61, a second connection point 621 is searched for on the far-distance segment curve 62 from near to far. For the currently searched sampling point A (abbreviated as the current sampling point), a corresponding target point B is determined on the modified near-distance segment curve 61 based on the vertical distance d. The target point B is typically located on an extension line from the near-distance segment curve 61 to the far-distance segment curve 62. A position difference (i.e., a horizontal position difference) ΔP between the current sampling point A and the target point B is calculated based on the horizontal coordinates of the current sampling point A and the target point B. An angular difference Δa between the current sampling point A and the target point B is calculated based on the tangent direction l1 at the current sampling point A and the tangent direction l2 at the target point B. The relationship between the horizontal position difference ΔP and the angular difference Δa and the third condition is determined. Whether the current sampling point A can be the second connection point 621 is determined based on the relationship between the horizontal position difference ΔP and the angular difference Δa and a third condition. The third condition may include a condition that the horizontal position difference ΔP and the vertical distance d are satisfied, and a condition that the angular difference Δa and the vertical distance d are satisfied. The vertical distance d is the vertical distance between the target point B and the first connection point 611. The condition that the horizontal position difference ΔP and the vertical distance d are satisfied may be that a ratio between the horizontal position difference ΔP and the vertical distance d is less than a first threshold. The condition that the angular difference Δa and the vertical distance d are satisfied may be that a ratio between the angular difference Δa and the vertical distance d is less than a second threshold. The first threshold and the second threshold are preset values.If it is determined that the currently searched current sampling point A is the second connection point 621, the search can be terminated; if not, the next searched sampling point can be determined as the current sampling point A based on the longitudinal distance step size, and whether it is the second connection point 621 is determined according to the above process. By analogy, the search is terminated until the second connection point 621 is found. After the second connection point 621 is determined, the first connection point 611 and the second connection point 621 are smoothly connected to obtain the connection curve 63. A second predicted driving trajectory represented by a segmented cubic curve is obtained based on the modified short distance segment curve 61, connection curve 63, and long distance segment curve 62.

[0136] The method of the embodiment of the present disclosure is based on monocular forward perspective sensing and utilizes a combination of a neural network model and post-processing. The neural network model is used to comprehensively model the vehicle driving environment, effectively obtaining a reference predicted driving trajectory (i.e., the predicted driving trajectory to be corrected). This is then combined with road observation information and observation information of other surrounding vehicles to correct the reference predicted driving trajectory. The corrected predicted driving trajectory is more consistent with the true road direction in a complex road environment, better reflecting the extension trend and connection relationship of main lanes, providing a better road reference for planning and controlling following vehicles and improving the adaptability and robustness of vehicle driving scenes. This allows the vehicle to safely drive in a variety of different and complex scenes, and, for example, can guide the vehicle to smoothly transition from its current lane to a distant lane even in scenes without lane markings. Furthermore, the error accumulation problem caused by the lack of structured observation information can be mitigated. In addition, the method of the embodiment of the present disclosure uses segmented trajectories in combination with multi-frame fusion to ensure the frame-to-frame stability of the near-distance segment predicted trajectory and avoid lateral drift, and also uses long-distance segment trajectories to capture the extension and connection tendency of the main lane, so that the predicted driving trajectory can have high accuracy in both the near-distance segment and the long-distance segment.

[0137] The above-described embodiments of the present disclosure may be implemented independently or may be implemented in any combination if not inconsistent, and may be specifically set according to actual needs, and the present disclosure is not limited thereto.

[0138] Any one of the vehicle control methods according to the embodiments of the present disclosure may be executed by any device having appropriate data processing capabilities, including, but not limited to, a terminal device, a server, etc. Alternatively, any one of the vehicle control methods according to the embodiments of the present disclosure may be executed by a processor, for example, the processor executes any one of the vehicle control methods mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. Further description will be omitted below.

[0139] Exemplary Apparatus 14 is a structural schematic diagram of a vehicle control device according to an exemplary embodiment of the present disclosure. The device of this embodiment can be used to realize the corresponding method embodiment of the present disclosure. The device shown in FIG. 14 can include an acquisition module 51, a first processing module 52, a second processing module 53, a third processing module 54, and a fourth processing module 55.

[0140] The acquisition module 51 can be used to acquire a first image collected by a camera of a preset viewpoint of the vehicle.

[0141] A first processing module 52 can be used to convert the first image into a bird's-eye view second image.

[0142] A second processing module 53 can be used to determine a first predicted driving trajectory in a local coordinate system of the vehicle based on the second image.

[0143] The third processing module 54 can be used to correct the first predicted traveling trajectory based on observation information corresponding to the first image, and to obtain a corrected second predicted traveling trajectory.

[0144] The fourth processing module 55 can be used to control the driving state of the vehicle based on the second predicted driving trajectory.

[0145] FIG. 15 is a structural schematic diagram of a control device for a vehicle according to another exemplary embodiment of the present disclosure.

[0146] In some alternative embodiments, as shown in FIG. 15, the second processing module 53 includes a first processing unit 531, a second processing unit 532, and a third processing unit 533.

[0147] The first processing unit 531 can be used to process the second image based on the driving trajectory keypoint detection model to obtain a first driving trajectory keypoint sequence at a bird's-eye view.

[0148] The second processing unit 532 can be used to transform the first sequence of driving trajectory keypoints from a bird's-eye view into a local coordinate system, and obtain a second sequence of driving trajectory keypoints in the local coordinate system.

[0149] The third processing unit 533 may be used to determine a first predicted driving trajectory based on the second driving trajectory keypoint sequence.

[0150] In some alternative embodiments, the third processing unit 533 may be specifically used to perform abnormal point removal and non-maximum suppression processing on the second driving trajectory keypoint sequence to obtain a third driving trajectory keypoint sequence after post-processing, and perform curve fitting on the third driving trajectory keypoint sequence to obtain a first predicted driving trajectory.

[0151] In some alternative embodiments, as shown in FIG. 15, the third processing module 54 may include an acquisition unit 541, a conversion unit 542, a fourth processing unit 543 and a correction unit 544.

[0152] The obtaining unit 541 can be used to obtain a first historical predicted driving trajectory.

[0153] The transformation unit 542 may be used to transform the first historical predicted driving trajectory into a local coordinate system to obtain a second historical predicted driving trajectory.

[0154] The fourth processing unit 543 can be used to determine the predicted traveling trajectory to be corrected based on the second historical predicted traveling trajectory and the first predicted traveling trajectory.

[0155] The correction unit 544 can be used to correct the predicted traveling trajectory to be corrected based on the observation information corresponding to the first image, and obtain a second predicted traveling trajectory.

[0156] In some alternative embodiments, the fourth processing unit 543 can be specifically used to perform clustering on the second historical predicted driving trajectory and the first predicted driving trajectory, obtain a clustering result including at least one cluster, determine a target cluster that satisfies a predetermined condition from each cluster based on the clustering result, merge the predicted driving trajectories in the target cluster, and obtain a predicted driving trajectory to be corrected.

[0157] In some alternative embodiments, the correction unit 544 can be specifically used to determine a target observation quantity for correcting the predicted driving trajectory to be corrected based on observation information corresponding to the first image, determine a restriction point based on the target observation quantity, and filter the predicted driving trajectory to be corrected based on the restriction point to obtain a second predicted driving trajectory.

[0158] FIG. 16 is a structural schematic diagram of a control device for a vehicle according to yet another exemplary embodiment of the present disclosure.

[0159] In some alternative embodiments, as shown in FIG. 16, the third processing module 54 may include a first determining unit 54a, a second determining unit 54b, and a filtering unit 54c.

[0160] The first determination unit 54a can be used to determine a target observation quantity for correcting the first predicted traveling trajectory based on the observation information corresponding to the first image.

[0161] The second determination unit 54b can be used to determine restriction points based on the target observables.

[0162] The filtering unit 54c can be used to filter the first predicted driving trajectory based on the restriction points to obtain a second predicted driving trajectory.

[0163] In some alternative embodiments, the first determination unit 54a may be used to determine target observables for correcting the predicted driving trajectory to be corrected based on observation information corresponding to the first image. The second determination unit 54b may be used to determine restriction points based on the target observables. The filtering unit 54c may be used to filter the predicted driving trajectory to be corrected based on the restriction points to obtain a second predicted driving trajectory.

[0164] In some alternative embodiments, the first determination unit 54a can be specifically used to determine, based on the observation information, at least one of a lane marking curve and a road curb curve in a local coordinate system, a lane marking point sequence and a road curb point sequence, and a driving trajectory point sequence of a target vehicle that satisfies a first condition around the vehicle, and to determine at least one of a lane marking curve and a road curb curve in a local coordinate system, a lane marking point sequence and a road curb point sequence, and a driving trajectory point sequence of a target vehicle that satisfies a first condition around the vehicle as a target observation.

[0165] In some alternative embodiments, the filtering unit 54c can be specifically used to perform extended Kalman filtering on the first predicted driving trajectory based on the control points, obtain a third predicted driving trajectory after filtering, determine a first sub-trajectory curve within a first distance range close to the vehicle and a second sub-trajectory curve within a second distance range far from the vehicle based on the third predicted driving trajectory, and determine a second predicted driving trajectory based on the first sub-trajectory curve and the second sub-trajectory curve.

[0166] In some alternative embodiments, the filtering unit 54c can be specifically used to determine a first connection point on a first sub-trajectory curve, modify the first sub-trajectory curve based on the second sub-trajectory curve, obtain a modified third sub-trajectory curve, determine a second connection point on the second sub-trajectory curve, determine a connection curve that can connect the first connection point and the second connection point and meets a second condition, and determine a second predicted driving trajectory based on the second sub-trajectory curve, the third sub-trajectory curve, and the connection curve.

[0167] In some alternative embodiments, the filtering unit 54c may be specifically used to search for sampling points on the second sub-trajectory curve based on a preset direction, determine a target point on the third sub-trajectory curve corresponding to the current sampling point searched for, determine a lateral position difference and an angle difference between the current sampling point and the target point, determine a relationship between the lateral position difference and the angle difference and a third condition, and determine a second connection point based on the relationship between the lateral position difference and the angle difference and the third condition, where the longitudinal distance of the target point to the vehicle is the same as the longitudinal distance of the current sampling point to the vehicle.

[0168] The beneficial technical effects corresponding to the exemplary embodiment of the present device can be referred to the beneficial technical effects corresponding to the exemplary method part above, so the description thereof will be omitted here.

[0169] Exemplary Electronic Devices FIG. 17 is a structural diagram of an electronic device according to an embodiment of the present disclosure, where an electronic device 90 includes at least one processor 91 and a memory 92.

[0170] The processor 91 may be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and may control other components of the electronic device 90 to perform desired functions.

[0171] The memory 92 may include one or more computer program products, which may include various types of computer-readable storage media, such as, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored in the computer-readable storage media, and the processor 91 may execute the one or more computer program instructions to implement the methods and / or other desired functions of each embodiment of the present disclosure.

[0172] In one example, electronic device 90 may further include input devices 93 and output devices 94, with these components connected to one another via a bus system and / or other form of connection (not shown).

[0173] The input device 93 may further include, for example, a keyboard, a mouse, and the like.

[0174] The output device 94 can output various types of information to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected to these.

[0175] 17 shows only some of the components related to the present disclosure in the electronic device 90, and omits components such as buses, input / output interfaces, etc. The electronic device 90 may further include any other appropriate components depending on the specific application.

[0176] Exemplary Computer Program Products and Computer-Readable Storage Media In addition to the above methods and apparatus, embodiments of the present disclosure may further provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods of various embodiments of the present disclosure described in the "Example Methods" section above.

[0177] The computer program product may have program code for carrying out operations of embodiments of the present disclosure written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may execute entirely on a user's computing device, partially on a user's device, as separate software packages, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0178] Additionally, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods of various embodiments of the present disclosure described in the "Exemplary Methods" section above.

[0179] The computer-readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0180] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the benefits, advantages, effects, etc. mentioned in the present disclosure are not limited but merely illustrative, and these benefits, advantages, effects, etc. do not necessarily exist in each embodiment of the present disclosure. Furthermore, the specific details disclosed above are not limited but merely serve to serve as examples and to facilitate understanding, and the above details do not necessarily limit the present disclosure to be realized by the above specific details.

[0181] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure also intends to include these modifications and variations.

Claims

1. acquiring a first image collected by a camera of a vehicle at a preset viewpoint; converting the first image into a bird's-eye view second image; determining a first predicted driving trajectory in a local coordinate system of the vehicle based on the second image; correcting the first predicted traveling trajectory based on observation information corresponding to the first image, and obtaining a corrected second predicted traveling trajectory; and controlling a traveling state of the vehicle based on the second predicted traveling trajectory.

2. The step of determining a first predicted traveling trajectory in a local coordinate system of the vehicle based on the second image includes: processing the second image based on a driving trajectory keypoint detection model to obtain a first sequence of driving trajectory keypoints at the bird's-eye view; Transforming the first sequence of driving trajectory key points from the bird's-eye view to the local coordinate system to obtain a second sequence of driving trajectory key points in the local coordinate system; and determining the first predicted driving trajectory based on the second driving trajectory key point sequence.

3. The step of determining the first predicted driving trajectory based on the second driving trajectory key point sequence includes: performing an abnormal point removal and non-maximum value suppression process on the second driving trajectory keypoint sequence to obtain a processed third driving trajectory keypoint sequence; and performing curve fitting on the third sequence of driving trajectory key points to obtain the first predicted driving trajectory.

4. The step of correcting the first predicted traveling trajectory based on observation information corresponding to the first image and obtaining a corrected second predicted traveling trajectory includes: acquiring a first historical predicted traveling locus; Transforming the first historical predicted traveling trajectory into the local coordinate system to obtain a second historical predicted traveling trajectory; determining a predicted traveling locus to be corrected based on the second historical predicted traveling locus and the first predicted traveling locus; 2. The vehicle control method according to claim 1, further comprising: correcting the predicted traveling trajectory of the object to be corrected based on the observation information corresponding to the first image, and acquiring the second predicted traveling trajectory.

5. determining a predicted traveling locus to be corrected based on the second historical predicted traveling locus and the first predicted traveling locus, performing clustering on the second historical predicted traveling trajectory and the first predicted traveling trajectory to obtain a clustering result including at least one cluster; determining a target cluster that satisfies a predetermined condition from each of the clusters based on the clustering result; The vehicle control method according to claim 4 , further comprising: fusing the predicted traveling trajectories in the target cluster to obtain the predicted traveling trajectory to be corrected.

6. The step of correcting the first predicted traveling trajectory based on observation information corresponding to the first image and obtaining a corrected second predicted traveling trajectory includes: determining a target observation amount for correcting the first predicted traveling trajectory based on the observation information corresponding to the first image; determining restriction points based on the target observables; The vehicle control method according to claim 1 , further comprising: filtering the first predicted traveling trajectory based on the restriction points to obtain the second predicted traveling trajectory.

7. determining a target observation amount for correcting the first predicted traveling trajectory based on the observation information corresponding to the first image, determining, based on the observation information, at least one of a lane marking curve and a road curb curve in the local coordinate system, a lane marking point sequence and a road curb point sequence, and a driving trajectory point sequence of a target vehicle that satisfies a first condition around the vehicle; and determining, as the target observable, at least one of a lane marking curve and a road curb curve in the local coordinate system, a lane marking point sequence and a road curb point sequence, and a driving trajectory point sequence of a target vehicle around the vehicle that satisfies a first condition.

8. The step of filtering the first predicted traveling trajectory based on the restriction points and acquiring the second predicted traveling trajectory includes: performing extended Kalman filtering on the first predicted traveling trajectory based on the restriction points to obtain a third predicted traveling trajectory after filtering; determining a first sub-trajectory curve within a first distance range close to the vehicle and a second sub-trajectory curve within a second distance range far from the vehicle based on the third predicted driving trajectory; The vehicle control method according to claim 6 , further comprising: determining the second predicted traveling trajectory based on the first sub-trajectory curve and the second sub-trajectory curve.

9. The step of determining the second predicted traveling trajectory based on the first sub-trajectory curve and the second sub-trajectory curve includes: determining a first connection point on the first sub-trajectory curve; modifying the first sub-trajectory curve based on the second sub-trajectory curve to obtain a modified third sub-trajectory curve; determining a second connection point on the second sub-trajectory curve; determining a connection curve that can connect the first connection point and the second connection point and meets a second condition; The vehicle control method according to claim 8 , further comprising: determining the second predicted traveling trajectory based on the second sub-trajectory curve, the third sub-trajectory curve, and the connecting curve.

10. The step of determining a second connection point on the second sub-trajectory curve includes: searching for a sampling point on the second sub-trajectory curve based on a preset direction, and determining a target point on the third sub-trajectory curve corresponding to the current sampling point, where the longitudinal distance of the target point relative to the vehicle is the same as the longitudinal distance of the current sampling point relative to the vehicle; determining a lateral positional difference and an angular difference between the current sampling point and the target point; determining a relationship between the lateral position difference amount and the angular difference amount and a third condition; The vehicle control method according to claim 9, further comprising: determining the second connection point based on a relationship between the lateral position difference amount and the angle difference amount and a third condition.

11. an acquisition module for acquiring a first image collected by a camera of a predetermined viewpoint of the vehicle; a first processing module for converting the first image into a bird's-eye view second image; a second processing module for determining a first predicted driving trajectory in a local coordinate system of the vehicle based on the second image; a third processing module for correcting the first predicted traveling trajectory based on observation information corresponding to the first image and obtaining a corrected second predicted traveling trajectory; a fourth processing module for controlling a traveling state of the vehicle based on the second predicted traveling trajectory.

12. A computer-readable storage medium storing a computer program for executing the vehicle control method according to any one of claims 1 to 10.

13. a processor; a memory for storing instructions executable by the processor; The electronic device is used to implement the vehicle control method according to any one of claims 1 to 10, wherein the processor reads and executes the executable instructions from the memory.

Images