A smart vehicle lane-changing decision-making system and method based on BEV perspective and digital twin.

Abstract

This invention discloses an intelligent vehicle lane-changing decision-making system and method based on BEV perspective and digital twin. The system includes an intelligent vehicle terminal and an edge cloud. The intelligent vehicle terminal includes an intelligent vehicle perception module and an intelligent vehicle motion control module. The edge cloud includes a BEV module, a lane-changing decision-making module, and a verification module. This invention obtains the BEV perspective during the intelligent vehicle's driving process through BEV perception technology. From this perspective, considering the intelligent vehicle's position, the number of surrounding vehicles, and relative distance factors, the most suitable lane-changing direction of the intelligent vehicle is obtained. A lane-changing trajectory planning algorithm is used to plan the set of lane-changing trajectories required for the intelligent vehicle to travel in the lane-changing direction. The lane-changing trajectory set is then tested in parallel using digital twin technology to obtain the optimal lane-changing decision scheme that meets the safety requirements of intelligent vehicle lane-changing driving.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicle lane change decision technology, and particularly relates to an intelligent vehicle lane change decision system and method based on BEV perspective and digital twin. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence, multi-sensor fusion, and image processing, intelligent vehicles are beginning to shift from partially autonomous driving to highly autonomous or fully autonomous driving. The lane-changing decision module is an important component of the intelligent vehicle system. This module needs to select the optimal lane-changing direction and plan a safe lane-changing trajectory based on the actual driving environment of the intelligent vehicle. Then, the motion control module controls the intelligent vehicle's drive system, steering system, and braking system to complete the relevant lane-changing actions.

[0003] In lane-changing decisions, rationally selecting the lane-changing direction and planning the lane-changing trajectory of an intelligent vehicle can ensure the safety and comfort of its driving, avoiding dangerous situations such as rollovers and collisions. However, obtaining a safe and feasible lane-changing decision-making scheme depends on the accurate environmental perception results of the intelligent vehicle. When the intelligent vehicle is perceiving its environment, due to issues such as the installation position and angle of the onboard sensors, as well as the sensor's own detection range, different sensors extract different environmental features. For example, the front camera sensor can only acquire the features of the front view image in the same direction as the intelligent vehicle's travel. Using the inverse perspective transformation algorithm to transform the perspective of the front view image will result in serious information loss. In actual driving, when the intelligent vehicle is following closely behind another vehicle, the camera sensor's field of view may be obstructed, leading to incomplete perception information. At the same time, traditional lane-changing decision-making methods, such as game theory, hierarchical reinforcement learning, or probabilistic output models, lack verification of the safety and feasibility of the lane-changing decision-making scheme. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide an intelligent vehicle lane-changing decision-making system and method based on a BEV perspective and digital twin, thereby solving the problems of insufficient consideration of the environmental characteristics surrounding the intelligent vehicle, low lane-changing decision efficiency, and lack of safety and feasibility verification of lane-changing decision results in existing technologies. The present invention obtains the BEV perspective during the intelligent vehicle's driving process through BEV perception technology. From this perspective, it comprehensively considers the intelligent vehicle's position, the number of surrounding vehicles, and relative distance factors to obtain the most suitable lane-changing direction for the intelligent vehicle. A lane-changing trajectory planning algorithm is used to plan the set of lane-changing trajectories required for the intelligent vehicle to travel in the lane-changing direction. Furthermore, the lane-changing trajectory set is tested in parallel using digital twin technology to obtain the optimal lane-changing decision scheme that satisfies the safety requirements of intelligent vehicle lane-changing driving.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The present invention provides an intelligent vehicle lane-changing decision-making system based on BEV perspective and digital twin, comprising: intelligent vehicle terminal and edge cloud;

[0007] The intelligent vehicle terminal includes: an intelligent vehicle perception module and an intelligent vehicle motion control module;

[0008] The intelligent vehicle perception module acquires driving environment data through multiple sensors and transmits the driving environment data to the edge cloud;

[0009] The intelligent vehicle motion control module is used to execute the optimal lane-changing decision scheme, and it includes: a drive-by-wire unit, a steering-by-wire unit, and a brake-by-wire unit;

[0010] The drive-by-wire unit adopts a distributed four-wheel independent drive system to provide the driving force required for the intelligent vehicle to drive.

[0011] The steer-by-wire unit is used to provide the steering torque required for the intelligent vehicle to turn and drive.

[0012] The brake-by-wire unit adopts a hydraulic brake-by-wire system to provide the braking force required when the intelligent vehicle brakes.

[0013] The edge cloud includes: a BEV module, a lane change decision module, and a verification module;

[0014] The BEV module includes: a perception data storage unit, a perception data processing unit, and a BEV perspective generation unit; the perception data storage unit is used to store driving environment data acquired by the intelligent vehicle perception module; the perception data processing unit is used to process the driving environment data acquired by the intelligent vehicle perception module; the BEV perspective generation unit is an end-to-end BEV neural network, with the input being the driving environment data processed by the perception data processing module, and the output being the BEV perspective of the intelligent vehicle at the current moment;

[0015] The lane-changing decision module includes: a lane-changing direction planning unit and a lane-changing trajectory planning unit; the lane-changing direction planning unit is used to determine the lane-changing direction of the intelligent vehicle from the perspective of the BEV; the lane-changing trajectory planning unit is used to plan the set of lane-changing trajectories required for the intelligent vehicle to change lanes.

[0016] The verification module is used to test and obtain an optimal lane-changing decision scheme that meets safety requirements, and then transmits the optimal lane-changing decision scheme to the intelligent vehicle motion control module. It includes a twin data storage unit, a digital twin unit, and a simulation testing unit. The twin data storage unit stores intelligent vehicle data, static environment data, and dynamic environment data. The digital twin unit uses the data stored in the twin data storage unit to construct a digital twin driving scenario. The simulation testing unit is used to obtain the optimal lane-changing decision scheme through parallel testing.

[0017] Furthermore, the intelligent vehicle perception module includes: a first camera sensor, a second camera sensor, a third camera sensor, a fourth camera sensor, and a lidar sensor; the first camera sensor, the second camera sensor, the third camera sensor, and the fourth camera sensor are respectively installed at the front end, rear end, left end, and right end of the top of the intelligent vehicle, and are used to acquire RGB images of the surrounding environment during the intelligent vehicle's driving process; the lidar sensor is installed on the top of the intelligent vehicle and is used to acquire 3D point cloud data of vehicles around the intelligent vehicle.

[0018] Furthermore, the first camera sensor, the second camera sensor, the third camera sensor, and the fourth camera sensor are all monocular vision sensors.

[0019] Furthermore, the front end, rear end, left end, and right end refer to the front, rear, left, and right relative to the driving direction of the intelligent vehicle.

[0020] Furthermore, the driving environment data includes: RGB image data acquired by each camera sensor and 3D point cloud data acquired by LiDAR.

[0021] Furthermore, the intelligent vehicle data includes: intelligent vehicle system structure data, intelligent vehicle dynamic model parameters, and kinematic model parameters; the static environment data includes road surface adhesion coefficient and road surface material; the dynamic environment data includes: the intelligent vehicle's drivable area, the number and location of lane lines, the intelligent vehicle's absolute speed, the geometric shape data of surrounding vehicles, and the absolute speed and relative position information of surrounding vehicles.

[0022] The present invention provides an intelligent vehicle lane-changing decision-making method based on BEV perspective and digital twin technology. Based on the above system, the steps are as follows:

[0023] Step 1): When the intelligent vehicle needs to change lanes during driving, it uses multiple sensors installed on the vehicle to perceive the driving environment around the intelligent vehicle in real time and transmits the driving environment data acquired by the multiple sensors to the edge cloud.

[0024] Step 2): The driving environment data obtained in Step 1) is processed by a filtering algorithm to obtain the BEV features of the intelligent vehicle at the current time t-Δt and time t, and the BEV perspective containing information on vehicles around the intelligent vehicle and lane lines is obtained by a 3D target detection method.

[0025] Step 3): Based on the BEV perspective at time t, solve for the drivable area of ​​the intelligent vehicle, the number and position of lane lines, the position of the centroid of surrounding vehicles relative to the intelligent vehicle's moving coordinate system, the geometric data and number of surrounding vehicles, and calculate the absolute speed of surrounding vehicles and the intelligent vehicle from the BEV perspective at time t-Δt and time t. Store the solved data.

[0026] Step 4): Determine the location of the intelligent vehicle in the drivable area from the BEV perspective. When the left side of the intelligent vehicle is the left boundary of the drivable area, the intelligent vehicle chooses to change lanes to the right. When the right side of the intelligent vehicle is the right boundary of the drivable area, the intelligent vehicle chooses to change lanes to the left. When neither the left nor right side of the intelligent vehicle is a boundary of the drivable area, the direction of lane changing is determined by the lane changing conditions.

[0027] Step 5): Based on the lane-changing direction of the intelligent vehicle determined in Step 4), the lane-changing trajectory set of the intelligent vehicle is planned using a lane-changing trajectory planning algorithm;

[0028] Step 6): Based on the intelligent vehicle data, static environment data, and dynamic environment data at time t, construct a digital twin intelligent vehicle, a digital twin static environment, and a digital twin dynamic environment on the edge cloud, and place the three into the same digital twin virtual space to complete the initialization of the digital twin driving scenario. In the digital twin driving scenario, control the digital twin intelligent vehicle to perform parallel testing on the lane-changing trajectory set planned in Step 5), and determine whether there is an optimal lane-changing decision scheme in the lane-changing trajectory set. If there is, proceed to Step 7); otherwise, proceed to Step 5.

[0029] Step 7): Convert the data on the changes in driving force, steering angle and braking force over time corresponding to the optimal lane-changing decision scheme into executable electrical signals, and transmit them to the drive-by-wire unit, steering-by-wire unit and braking-by-wire unit respectively, so as to complete the execution task of the optimal lane-changing decision scheme of the intelligent vehicle.

[0030] Furthermore, the filtering algorithms in step 2) include: RGB image filtering algorithms and 3D point cloud data filtering algorithms; the RGB image filtering algorithms include, but are not limited to: Gaussian filtering, bilateral filtering and guided filtering; the 3D point cloud data filtering algorithms include, but are not limited to: statistical filtering and voxel filtering.

[0031] Furthermore, the 3D object detection methods in step 2) include, but are not limited to, PointNet++ and VoxelNet.

[0032] Furthermore, in step 2), the surrounding vehicles detected by the 3D target detection method are represented by a 3D bounding box containing the length, width, and height information of the surrounding vehicles.

[0033] Furthermore, the specific steps for obtaining BEV features in step 2) are as follows:

[0034] 21) Calculate the outer product features of RGB images from multiple viewpoints;

[0035] The processed front-view RGB image, rear-view RGB image, left-view RGB image, and right-view RGB image are input into the first image feature extraction backbone, the second image feature extraction backbone, the third image feature extraction backbone, the fourth image feature extraction backbone, the first depth estimation network, the second depth estimation network, the third depth estimation network, and the fourth depth estimation network, respectively. This yields front-view image features, rear-view image features, left-view image features, right-view image features, front-view depth estimation features, rear-view depth estimation features, left-view depth estimation features, and right-view depth estimation features, respectively. Then, the image features at the same viewpoint are multiplied by the corresponding depth estimation features to obtain front-view outer product features, rear-view outer product features, left-view outer product features, and right-view outer product features containing depth information. The outer product calculation expression for each pixel in the outer product features is as follows:

[0036]

[0037] In the formula, A represents the cross product of pixels; η = [η0, η1, η2, η3, ... η d-1 ] T A probability vector representing the estimated depth value of a pixel; This represents the image feature vector corresponding to a pixel.

[0038] 22) Obtain the BEV space of the image;

[0039] A moving coordinate system is established with the centroid of the intelligent vehicle as the origin, and a three-dimensional mesh is established around the intelligent vehicle with the origin. Each pixel of the four extra-product features obtained in step 21) is projected into the mesh of the moving coordinate system through the camera intrinsic and extrinsic parameters. The features of each point in the mesh are summed to obtain the image BEV space fused from four perspectives.

[0040] 23) Fusion of image BEV features and point cloud BEV features;

[0041] The BEV space of the image obtained in step 22) is encoded to obtain image BEV feature F1, and the processed point cloud data is encoded to obtain point cloud BEV feature F2. The two features are fused to obtain BEV feature F3. The calculation expression for the fusion is as follows:

[0042] F3=σ(f avg (f cov ([F1,F2])))·f cov ([F1,F2])

[0043] In the formula, σ is the sigmoid function; f avg For global average pooling operation; f cov[F1, F2] represents the convolution operation; [F1, F2] represents the concatenation operation.

[0044] Furthermore, the first image feature extraction backbone, the second image feature extraction backbone, the third image feature extraction backbone, and the fourth image feature extraction backbone in step 21) are all of the same type of backbone, including but not limited to: ResNet34, ResNet50, and MobileNetV3.

[0045] Furthermore, the first depth estimation network, the second depth estimation network, the third depth estimation network, and the fourth depth estimation network in step 21) are of the same type, including but not limited to MonoDepth2 and Fast-Depth.

[0046] Furthermore, the moving coordinate system in step 22) is specifically as follows: with the center of mass of the intelligent vehicle as the origin, the positive x-axis is parallel to the ground and points in the direction of travel of the intelligent vehicle, the positive y-axis points to the left of the driver, and the positive z-axis is perpendicular to the ground and points upward.

[0047] Furthermore, in step 3), the drivable area is the area between the leftmost lane line and the rightmost lane line from the BEV's perspective, with the leftmost lane line being the left boundary of the drivable area and the rightmost lane line being the right boundary of the drivable area.

[0048] Furthermore, in step 3), the centroid of the surrounding vehicles is the geometric center of the 3D target detection method that represents the 3D bounding box of the surrounding vehicles.

[0049] Furthermore, the geometric data of the surrounding vehicles in step 3) are the length, width, and height data of the 3D frame corresponding to the surrounding vehicles of the intelligent vehicle in the BEV view.

[0050] Furthermore, the expression for calculating the absolute speed of surrounding vehicles or intelligent vehicles in step 3) is as follows:

[0051]

[0052] In the formula, v r P represents the absolute speed of surrounding vehicles or intelligent vehicles. t P represents the position of surrounding vehicles or intelligent vehicles in the BEV's view at time t; t-Δt This represents the position of surrounding vehicles or intelligent vehicles in the BEV's view at time t-Δt; Δt represents the time interval.

[0053] Furthermore, the lane-changing conditions in step 4) are specifically as follows:

[0054] Let Lane be the current driving lane of the intelligent vehicle.mid The left lane is Lane lef The right lane is Lane rig The intelligent vehicle changes lanes in direction D, and the number of vehicles in the left and right lanes obtained from the BEV's perspective at time t are n respectively. lef n rig Located in Lane lef The relative distances of the surrounding vehicles to the intelligent vehicle at time t are summed as s. lef Located in Lane rig The relative distances of the surrounding vehicles to the intelligent vehicle at time t are summed as s. rig The expression for determining the lane-changing direction D of the intelligent vehicle is:

[0055] D = f D (min(α1s lef +β1n lef ,α2s rig +β2n rig ))

[0056] In the formula, min() is the minimum value function; f D () represents the direction determination function; α1, β1, α2, β2 represent the weighting coefficients;

[0057] Where s lef and s rig The calculation expression is:

[0058]

[0059]

[0060] In the formula, x i y i Lane lef The x and y values ​​of the i-th surrounding vehicle relative to the coordinate system of movement; x j y j Lane rig The x and y values ​​of the j-th surrounding vehicle relative to the coordinate system of movement.

[0061] Furthermore, the direction determination function selects the lane with the smaller weighted sum of the relative distances and numbers of surrounding vehicles as the lane-changing direction for the intelligent vehicle.

[0062] Furthermore, the lane-changing trajectory planning algorithm in step 5) includes, but is not limited to, particle swarm optimization and fifth-order polynomial curve fitting algorithm.

[0063] Furthermore, the lane-changing trajectory set in step 5) consists of 10 lane-changing decision schemes. Each lane-changing decision scheme includes: the starting point of the lane-changing trajectory, the direction of the lane-changing trajectory, the ending point of the lane-changing trajectory, the parameters of the lane-changing trajectory curve, and the speed and acceleration of the intelligent vehicle in the x and y directions.

[0064] Furthermore, the parallel testing method in step 6) is as follows: 10 digital twin intelligent vehicles simultaneously start changing lanes from the starting points of 10 lane-changing trajectories until they reach the end point of the lane-changing trajectory, and record the changes in driving force, turning angle and braking force of the digital twin intelligent vehicles over time during the lane-changing process.

[0065] Furthermore, the criteria for determining the optimal lane-changing decision scheme in step 6) include: the digital twin intelligent vehicle does not collide with surrounding vehicles when executing the lane-changing trajectory scheme; the turning angle and longitudinal speed of the four drive wheels of the digital twin intelligent vehicle meet the threshold requirements; and the shortest lane-changing time t from the start to the end of the lane-changing trajectory for the digital twin intelligent vehicle. min .

[0066] Furthermore, the statement that the digital twin intelligent vehicle does not collide specifically means that, during the execution of the lane-changing trajectory plan, the digital twin intelligent vehicle has no spatial overlap with the 3D frames corresponding to the surrounding vehicles in the digital twin. The specific judgment expression is:

[0067] (x p 0,y p 0,z p 0)≠(x q i ,y q i ,z q i )

[0068] In the formula, (x p 0,y p 0,z p 0) represents the three-dimensional coordinates of any point on the body of the digital twin intelligent vehicle in the digital twin translation coordinate system; (x) q i ,y q i ,z q i ) represents the three-dimensional coordinates of any point on the 3D frame corresponding to the vehicles surrounding the i-th digital twin in the digital twin's moving coordinate system.

[0069] Furthermore, the threshold requirements for the rotation angle and longitudinal speed of the four drive wheels of the digital twin intelligent vehicle are as follows:

[0070]

[0071] In the formula, δ represents the actual rotation angle of any drive wheel during the digital twin intelligent vehicle simulation test; δ min δ max These represent the minimum and maximum threshold values ​​for the drive wheel rotation angle, respectively; v represents the longitudinal velocity of any drive wheel during the digital twin intelligent vehicle simulation test; v min v max These represent the minimum and maximum threshold values ​​for the longitudinal speed of the drive wheels, respectively.

[0072] The beneficial effects of this invention are:

[0073] 1. This invention integrates data from multiple camera sensors and LiDAR, and generates a BEV perspective with three-dimensional information during the intelligent vehicle's driving process through BEV perception technology. This allows the intelligent vehicle to obtain global information about its surrounding environment, solving the problem of loss of perception information caused by inverse perspective transformation or occlusion by other vehicles. This makes the intelligent vehicle more robust in planning lane-changing schemes in different driving environments.

[0074] 2. This invention fully considers the lane position of the intelligent vehicle and the influence of surrounding vehicles in the drivable area during the driving process. By combining edge cloud technology, it completes the tasks of determining the lane-changing direction and planning the lane-changing trajectory of the intelligent vehicle from a unified BEV perspective, thereby improving the efficiency of lane-changing decision-making.

[0075] 3. This invention constructs a virtual driving environment using digital twin technology, and combines the kinematic and dynamic models of the intelligent vehicle to construct a digital twin intelligent vehicle. Through parallel testing, it quickly identifies the optimal lane-changing decision scheme that may exist in the lane-changing trajectory set, ensuring that the lane-changing decision scheme planned by the intelligent vehicle meets the requirements of safety and feasibility.

[0076] 4. This invention provides a lane-changing decision-making system and method with theoretical significance and practical application value for the development of intelligent vehicles towards higher levels of autonomous driving. It has strong practicality and feasibility, and is conducive to promoting the development of intelligent vehicles in lane-changing decision-making. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of the system architecture of the present invention.

[0078] Figure 2 A diagram showing the layout of sensors for an intelligent vehicle.

[0079] Figure 3 This is a flowchart of the method of the present invention. Detailed Implementation

[0080] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0081] Reference Figure 1 As shown, the present invention provides an intelligent vehicle lane-changing decision-making system based on BEV perspective and digital twin, comprising: intelligent vehicle terminal and edge cloud;

[0082] The intelligent vehicle terminal includes: an intelligent vehicle perception module and an intelligent vehicle motion control module;

[0083] The intelligent vehicle perception module acquires driving environment data through multiple sensors and transmits the driving environment data to the edge cloud; it includes: a first camera sensor, a second camera sensor, a third camera sensor, a fourth camera sensor, and a lidar sensor; the first, second, third, and fourth camera sensors are respectively installed at the front, rear, left, and right ends of the intelligent vehicle's roof to acquire RGB images of the surrounding environment during the intelligent vehicle's operation; the lidar sensor is installed on the roof of the intelligent vehicle to acquire 3D point cloud data of vehicles around the intelligent vehicle;

[0084] Among them, the first camera sensor, the second camera sensor, the third camera sensor, and the fourth camera sensor are all monocular vision sensors;

[0085] Wherein, the front end, rear end, left end, and right end are front, rear, left, and right relative to the driving direction of the intelligent vehicle;

[0086] The driving environment data includes: RGB image data acquired by each camera sensor and 3D point cloud data acquired by LiDAR;

[0087] The intelligent vehicle motion control module is used to execute the optimal lane-changing decision scheme, and it includes: a drive-by-wire unit, a steering-by-wire unit, and a brake-by-wire unit;

[0088] The drive-by-wire unit adopts a distributed four-wheel independent drive system to provide the driving force required for the intelligent vehicle to drive; it uses four identical hub motors as the driving force.

[0089] The steer-by-wire unit is used to provide the steering torque required for the intelligent vehicle to turn and drive.

[0090] The brake-by-wire unit adopts a hydraulic brake-by-wire system to provide the braking force required when the intelligent vehicle brakes.

[0091] The edge cloud includes: a BEV module, a lane change decision module, and a verification module;

[0092] The BEV module includes: a perception data storage unit, a perception data processing unit, and a BEV perspective generation unit; the perception data storage unit is used to store driving environment data acquired by the intelligent vehicle perception module; the perception data processing unit is used to process the driving environment data acquired by the intelligent vehicle perception module; the BEV perspective generation unit is an end-to-end BEV neural network, with the input being the driving environment data processed by the perception data processing module, and the output being the BEV perspective of the intelligent vehicle at the current moment;

[0093] The lane-changing decision module includes: a lane-changing direction planning unit and a lane-changing trajectory planning unit; the lane-changing direction planning unit is used to determine the lane-changing direction of the intelligent vehicle from the perspective of the BEV; the lane-changing trajectory planning unit is used to plan the set of lane-changing trajectories required for the intelligent vehicle to change lanes.

[0094] The verification module is used to test and obtain an optimal lane-changing decision scheme that meets safety requirements, and then transmits the optimal lane-changing decision scheme to the intelligent vehicle motion control module. It includes a twin data storage unit, a digital twin unit, and a simulation testing unit. The twin data storage unit stores intelligent vehicle data, static environment data, and dynamic environment data. The digital twin unit uses the data stored in the twin data storage unit to construct a digital twin driving scenario. The simulation testing unit is used to obtain the optimal lane-changing decision scheme through parallel testing.

[0095] The intelligent vehicle data includes: intelligent vehicle system structure data, intelligent vehicle dynamic model parameters, and kinematic model parameters; the static environment data includes road surface adhesion coefficient and road surface material; the dynamic environment data includes: the intelligent vehicle's drivable area, the number and location of lane lines, the intelligent vehicle's absolute speed, the geometric shape data of surrounding vehicles, and the absolute speed and relative position information of surrounding vehicles.

[0096] Reference Figure 2 As shown, the first to fourth camera sensors are respectively installed at the front, rear, left and right ends of the top of the smart car to acquire RGB images of the surrounding environment during the smart car's driving process.

[0097] Reference Figure 2 As shown, the lidar sensor is installed on the top of the smart car to acquire 3D point cloud data of vehicles around the smart car.

[0098] Reference Figure 3 As shown, the present invention provides an intelligent vehicle lane-changing decision-making method based on BEV perspective and digital twin technology. Based on the above system, the steps are as follows:

[0099] Step 1): When the intelligent vehicle needs to change lanes during driving, it uses multiple sensors installed on the vehicle to perceive the driving environment around the intelligent vehicle in real time and transmits the driving environment data acquired by the multiple sensors to the edge cloud.

[0100] Step 2): The driving environment data obtained in Step 1) is processed by a filtering algorithm to obtain the BEV features of the intelligent vehicle at the current time t-Δt and time t, and the BEV perspective containing information on vehicles around the intelligent vehicle and lane lines is obtained by a 3D target detection method.

[0101] Step 3): Based on the BEV perspective at time t, solve for the drivable area of ​​the intelligent vehicle, the number and position of lane lines, the position of the centroid of surrounding vehicles relative to the intelligent vehicle's moving coordinate system, the geometric data and number of surrounding vehicles, and calculate the absolute speed of surrounding vehicles and the intelligent vehicle from the BEV perspective at time t-Δt and time t. Store the solved data.

[0102] Step 4): Determine the location of the intelligent vehicle in the drivable area from the BEV perspective. When the left side of the intelligent vehicle is the left boundary of the drivable area, the intelligent vehicle chooses to change lanes to the right. When the right side of the intelligent vehicle is the right boundary of the drivable area, the intelligent vehicle chooses to change lanes to the left. When neither the left nor right side of the intelligent vehicle is a boundary of the drivable area, the direction of lane changing is determined by the lane changing conditions.

[0103] Step 5): Based on the lane-changing direction of the intelligent vehicle determined in Step 4), the lane-changing trajectory set of the intelligent vehicle is planned using a lane-changing trajectory planning algorithm;

[0104] Step 6): Based on the intelligent vehicle data, static environment data, and dynamic environment data at time t, construct a digital twin intelligent vehicle, a digital twin static environment, and a digital twin dynamic environment on the edge cloud, and place the three into the same digital twin virtual space to complete the initialization of the digital twin driving scenario. In the digital twin driving scenario, control the digital twin intelligent vehicle to perform parallel testing on the lane-changing trajectory set planned in Step 5), and determine whether there is an optimal lane-changing decision scheme in the lane-changing trajectory set. If there is, proceed to Step 7); otherwise, proceed to Step 5.

[0105] Step 7): Convert the data on the changes in driving force, steering angle and braking force over time corresponding to the optimal lane-changing decision scheme into executable electrical signals, and transmit them to the drive-by-wire unit, steering-by-wire unit and braking-by-wire unit respectively, so as to complete the execution task of the optimal lane-changing decision scheme of the intelligent vehicle.

[0106] The expression for calculating the absolute speed of surrounding vehicles or intelligent vehicles in step 3) is as follows:

[0107]

[0108] In the formula, v r P represents the absolute speed of surrounding vehicles or intelligent vehicles. t P represents the position of surrounding vehicles or intelligent vehicles in the BEV's view at time t; t-Δt This indicates the position of surrounding vehicles or intelligent vehicles in the BEV's view at time t-Δt; Δt represents the interval time, specifically 0.04 seconds.

[0109] Specifically, the lane-changing conditions in step 4) are as follows:

[0110] Let Lane be the current driving lane of the intelligent vehicle. mid The left lane is Lane lef The right lane is Lane rig The intelligent vehicle changes lanes in direction D, and the number of vehicles in the left and right lanes obtained from the BEV's perspective at time t are n respectively. lef n rig Located in Lane lef The relative distances of the surrounding vehicles to the intelligent vehicle at time t are summed as s. lef Located in Lane rig The relative distances of the surrounding vehicles to the intelligent vehicle at time t are summed as s. rig The expression for determining the lane-changing direction D of the intelligent vehicle is:

[0111] D = f D (min(α1s lef +β1n lef ,α2s rig +β2n rig ))

[0112] In the formula, min() is the minimum value function; f D () represents the direction judgment function; α1, β1, α2, β2 represent weight coefficients, with specific values ​​of α1 = 0.1, β1 = 8, α2 = 0.1, β2 = 8.

[0113] The threshold requirements for the rotation angle and longitudinal speed of the four drive wheels of the digital twin intelligent vehicle are as follows:

[0114]

[0115] In the formula, δ represents the actual rotation angle of any drive wheel during the digital twin intelligent vehicle simulation test; δ min δ max These represent the minimum and maximum threshold values ​​for the drive wheel rotation angle, respectively, with a specific value of δ. min =0°, δ max =35°; v represents the longitudinal velocity of any drive wheel during the digital twin intelligent vehicle simulation test; vmin v max These represent the minimum and maximum threshold values ​​for the longitudinal speed of the drive wheels, respectively. min =0km / h, v max =60km / h.

[0116] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A smart vehicle lane-changing decision-making method based on BEV perspective and digital twin technology, characterized in that, The steps are as follows: Step 1): When the intelligent vehicle needs to change lanes during driving, it uses multiple sensors installed on the vehicle to perceive the driving environment around the intelligent vehicle in real time and transmits the driving environment data acquired by the multiple sensors to the edge cloud. Step 2): Process the driving environment data obtained in Step 1) using a filtering algorithm to obtain the current driving environment data of the intelligent vehicle. Time and The BEV characteristics at any time are obtained, and the BEV perspective, which includes information on vehicles around the intelligent vehicle and lane lines, is obtained through 3D object detection methods. Step 3): According to From a BEV perspective, this method solves for the drivable area of ​​the intelligent vehicle, the number and location of lane lines, the position of the centroid of surrounding vehicles relative to the intelligent vehicle's coordinate system, and the geometric data and number of surrounding vehicles. Time and The absolute speeds of surrounding vehicles and intelligent vehicles are calculated from the perspective of the BEV at any time, and the calculated data is stored. Step 4): Determine the location of the intelligent vehicle in the drivable area from the BEV perspective. When the left side of the intelligent vehicle is the left boundary of the drivable area, the intelligent vehicle chooses to change lanes to the right. When the right side of the intelligent vehicle is the right boundary of the drivable area, the intelligent vehicle chooses to change lanes to the left. When neither the left nor right side of the intelligent vehicle is a boundary of the drivable area, the direction of lane changing is determined by the lane changing conditions. Step 5): Based on the lane-changing direction of the intelligent vehicle determined in Step 4), the lane-changing trajectory set of the intelligent vehicle is planned using a lane-changing trajectory planning algorithm; Step 6): Based on intelligent vehicle data, static environment data, and Using real-time dynamic environmental data, a digital twin intelligent vehicle, a digital twin static environment, and a digital twin dynamic environment are constructed on the edge cloud. These three are then placed in the same virtual space of the digital twin to initialize the digital twin driving scenario. Within this scenario, the digital twin intelligent vehicle is controlled to perform parallel testing on the lane-changing trajectory set planned in step 5), and it is determined whether an optimal lane-changing decision scheme exists within the trajectory set. If it does, proceed to step 7); otherwise, proceed to step 5). Step 7): Convert the data on the changes in driving force, steering angle and braking force over time corresponding to the optimal lane-changing decision scheme into executable electrical signals, and transmit them to the drive-by-wire unit, steering-by-wire unit and braking-by-wire unit respectively, so as to complete the execution task of the optimal lane-changing decision scheme of the intelligent vehicle; The specific steps for obtaining BEV features in step 2) are as follows: 21) Calculate the outer product features of RGB images from multiple viewpoints; The processed front-view RGB image, rear-view RGB image, left-view RGB image, and right-view RGB image are input into the first image feature extraction backbone, the second image feature extraction backbone, the third image feature extraction backbone, the fourth image feature extraction backbone, the first depth estimation network, the second depth estimation network, the third depth estimation network, and the fourth depth estimation network, respectively. This yields front-view image features, rear-view image features, left-view image features, right-view image features, front-view depth estimation features, rear-view depth estimation features, left-view depth estimation features, and right-view depth estimation features, respectively. Then, the image features at the same viewpoint are multiplied by the corresponding depth estimation features to obtain front-view outer product features, rear-view outer product features, left-view outer product features, and right-view outer product features containing depth information. The outer product calculation expression for each pixel in the outer product features is as follows: ； In the formula, This represents the result of the outer product of pixels; A probability vector representing the estimated depth value of a pixel; This represents the image feature vector corresponding to a pixel. 22) Obtain the BEV space of the image; A moving coordinate system is established with the centroid of the intelligent vehicle as the origin, and a three-dimensional mesh is established around the intelligent vehicle with the origin. Each pixel of the four extra-product features obtained in step 21) is projected into the mesh of the moving coordinate system through the camera intrinsic and extrinsic parameters. The features of each point in the mesh are summed to obtain the image BEV space fused from four perspectives. 23) Fusion of image BEV features and point cloud BEV features; The BEV space of the image obtained in step 22) is encoded to obtain the BEV features of the image. The processed point cloud data is encoded to obtain the point cloud BEV features. By fusing the two features, the BEV feature is obtained. The calculation expression for fusion is: ； In the formula, It is the sigmoid function; This is a global average pooling operation; For convolution operations; [] indicates a splicing operation.

2. The intelligent vehicle lane-changing decision-making method based on BEV perspective and digital twin technology according to claim 1, characterized in that, The filtering algorithms in step 2) include: RGB image filtering algorithm and 3D point cloud data filtering algorithm; the RGB image filtering algorithm includes, but is not limited to: Gaussian filtering, bilateral filtering and guided filtering; the 3D point cloud data filtering algorithm includes, but is not limited to: statistical filtering and voxel filtering.

3. The intelligent vehicle lane-changing decision-making method based on BEV perspective and digital twin technology according to claim 1, characterized in that, The specific coordinate system used in step 22) is: with the intelligent vehicle's center of mass as the origin. The positive axis is parallel to the ground and points in the direction the intelligent vehicle is traveling. The positive direction of the axis points to the driver's left. The positive direction of the axis is perpendicular to the ground and points upwards.

4. The intelligent vehicle lane-changing decision-making method based on BEV perspective and digital twin technology according to claim 1, characterized in that, The expression for calculating the absolute speed of surrounding vehicles or intelligent vehicles in step 3) is as follows: ； In the formula, Indicates the absolute speed of surrounding vehicles or intelligent vehicles; express The position of surrounding vehicles or smart cars in the BEV's view at all times; express The position of surrounding vehicles or smart cars in the BEV's view at all times; Indicates the interval time.

5. The intelligent vehicle lane-changing decision-making method based on BEV perspective and digital twin technology according to claim 1, characterized in that, The lane-changing conditions in step 4) are specifically as follows: Let the current lane of the intelligent vehicle be The left lane is The right lane is The intelligent vehicle changes lanes in the following directions: ,pass The number of vehicles in the left and right lanes obtained from the BEV perspective at any given time are respectively , ,lie in The surrounding vehicles The relative distance between each moment and the intelligent vehicle is the sum of its values. ,lie in The surrounding vehicles The relative distance between each moment and the intelligent vehicle is the sum of its values. The intelligent vehicle changes lane direction The conditional expression is: ； In the formula, ( ) minimum value function; ( ) is the direction determination function; , , , Indicates the weighting coefficient; in and The calculation expression is: ； ； In the formula, , express The Middle The coordinate system of the surrounding vehicles relative to their movement. , value; , express The Middle The coordinate system of the surrounding vehicles relative to their movement. , value.

6. The intelligent vehicle lane-changing decision-making method based on BEV perspective and digital twin technology according to claim 1, characterized in that, The criteria for determining the optimal lane-changing decision scheme in step 6) include: the digital twin intelligent vehicle does not collide with surrounding vehicles when executing the lane-changing trajectory scheme; the turning angle and longitudinal speed of the four drive wheels of the digital twin intelligent vehicle meet the threshold requirements; and the digital twin intelligent vehicle has the shortest lane-changing time from the start to the end of the lane-changing trajectory. ; The digital twin intelligent vehicle not colliding specifically means that, during the execution of the lane-changing trajectory plan, the digital twin intelligent vehicle has no spatial overlap with the 3D frames corresponding to the surrounding vehicles in the digital twin. The specific judgment expression is: ； In the formula, This represents the three-dimensional coordinates of any point on the body of the digital twin intelligent vehicle in the digital twin's moving coordinate system. Indicates the first The three-dimensional coordinates of any point on the 3D frame corresponding to the vehicles surrounding the digital twin in the digital twin's moving coordinate system.

Images