A method for remote takeover of an autonomous vehicle

Abstract

The present invention belongs to the technical field of autonomous driving, and particularly relates to a method for remotely taking over an autonomous vehicle. First, obtain the current vehicle state information of the autonomous vehicle and the current traffic information of the environment where the autonomous vehicle is located; then, use the obtained current vehicle state information and current traffic information, as well as the constructed digital twin model, to match the current vehicle state information and current traffic information with historical information, and output the takeover requirement of the autonomous vehicle: if the takeover requirement is zero, it is taken over by the autonomous driving system; if the takeover requirement is weak, it is taken over by the digital twin platform; if the takeover requirement is strong, it is taken over by the remote driving platform. The present invention actively takes over before the autonomous driving system fails or malfunctions, and there is no problem of stopping and waiting in the middle caused by the traditional takeover method, which can ensure the continuity of the driving task, increase passenger comfort, improve traffic efficiency and improve safety.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and particularly relates to a method for remotely taking over an autonomous driving vehicle. Background Art

[0002] An autonomous driving vehicle is a vehicle integrated with intelligent sensors, an advanced in-vehicle operating system, and an autonomous driving system. It can obtain surrounding traffic and environmental information based on the output of the perception layer and then perform autonomous driving. Autonomous driving vehicles can improve traffic safety and traffic efficiency and are an important part of a smart city.

[0003] The safety issue is currently the biggest problem restricting the development of autonomous driving vehicles. To make autonomous driving vehicles gradually approach the essential and permanent safety goals, when an autonomous driving vehicle faces system failures, exceeds the designed operating domain, or other unexpected situations, and when it exceeds the control ability range of the vehicle terminal, it needs to be processed in a timely manner, and remote takeover and active intervention are carried out when necessary.

[0004] In the existing autonomous driving system, perception, planning, and decision-making are all carried out locally. This method has higher real-time performance, but in some extreme situations, the autonomous driving system will have problems that are difficult to handle or even system failures. Especially for the existing remote takeover requests of autonomous driving vehicles, they are also initiated by the autonomous driving system. When the autonomous driving system fails or the traffic conditions exceed its processing ability, the autonomous driving system will send a takeover request to the remote driving platform. However, at this time, the autonomous driving vehicle has entered a high-risk state, is extremely prone to accidents, and has extremely low safety. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for remotely taking over an autonomous driving vehicle to solve the problem of extremely low vehicle safety caused by initiating a remote takeover request for an autonomous driving vehicle only when the autonomous driving system fails or the traffic conditions exceed its processing ability.

[0006] To solve the above technical problems, the present invention provides a method for remotely taking over an autonomous driving vehicle, including the following steps:

[0007] 1) Obtain the current vehicle state information of the autonomous driving vehicle and the current traffic information of the environment where the autonomous driving vehicle is located;

[0008] 2) The digital twin platform utilizes the acquired current vehicle state information and current traffic information, as well as the constructed digital twin model, to match the current vehicle state information and current traffic information with historical information, so as to evaluate the current safety state of the autonomous vehicle and output the takeover requirements for the autonomous vehicle: If the takeover requirement is a zero takeover requirement, control the autonomous vehicle to be taken over by the autonomous driving system of the autonomous vehicle; if the takeover requirement is a weak takeover requirement, control the digital twin platform to take over the autonomous vehicle; if the takeover requirement is a strong takeover requirement, control the remote driving platform to take over the autonomous vehicle for manual takeover;

[0009] Among them, a zero takeover requirement indicates that the autonomous driving system is normal and within the designed operating domain of the autonomous driving system, a weak takeover requirement indicates that the autonomous driving system fails or exceeds the designed operating domain of the autonomous driving system, and a strong takeover requirement indicates that the digital twin platform cannot meet the vehicle safety requirements.

[0010] Its beneficial effects are as follows: The present invention is based on the vehicle state information and traffic information of the autonomous vehicle, and combines the constructed digital twin model to compare the current vehicle state information and current traffic information with historical information, so as to evaluate the safety state of the autonomous vehicle, predict possible dangers in advance, and allocate vehicle decision-making power for possible dangers. When the autonomous driving system can meet the safety requirements, directly use the autonomous driving system to take over the vehicle. When the autonomous driving system cannot meet the safety requirements, the digital twin platform takes over. Further, when the digital twin platform also cannot meet the safety requirements, the remote driving platform takes over, and active takeover is carried out before the autonomous driving system fails or malfunctions, and there is no problem of stopping and waiting in the middle caused by traditional takeover methods, which can ensure the continuity of the driving task, increase passenger comfort, improve traffic efficiency and improve safety.

[0011] Further, the following method is adopted to determine the takeover requirements of the autonomous vehicle:

[0012] a) Calculate the potential energy field, kinetic energy field and behavior field generated by each field source around the autonomous vehicle according to the following formula; the potential energy field is determined by the static road elements, the kinetic energy field is determined by the dynamic road elements, and the behavior field is determined by the state elements of the autonomous vehicle under driving behavior;

[0013] b) Perform a summation operation according to the potential energy field, kinetic energy field and behavior field generated by each field source to obtain the driving risk field of the autonomous vehicle;

[0014] c) Determine the inherent safety degree of the autonomous vehicle according to the driving risk field of the vehicle:

[0015]

[0016] where S j represents the intrinsic safety degree at time j, and i, j = 1, 2, 3 represent the autonomous driving system, digital twin platform, and manual takeover respectively; F j represents the field force received by the vehicle at the field source (x j , y j ); E j represents the driving risk field; M j represents the equivalent mass; R j represents the road condition influence factor; k2 represents a constant; v j represents the driving speed; θ j represents the angle between the speed direction and the displacement between two points (x i , y i ) and (x j , y j ); D rj represents the risk factor;

[0017] d) Determine the takeover requirement of the autonomous driving vehicle according to the intrinsic safety degree interval where the intrinsic safety degree of the autonomous driving vehicle is located; among them, one takeover requirement corresponds to one intrinsic safety degree interval.

[0018] Furthermore, the vehicle state information includes at least two of vehicle speed information, vehicle position information, steering wheel angle information, brake and accelerator pedal opening information, gear information, power battery SOC information, parking state information, and vehicle lane position information.

[0019] Its beneficial effect is that combining various vehicle state information for prediction can improve the prediction accuracy.

[0020] Furthermore, the traffic information includes at least two of road topology structure information, weather state information, obstacle information, and traffic signal information.

[0021] Its beneficial effect is that combining various traffic information for prediction can improve the prediction accuracy.

[0022] Furthermore, the digital twin model includes: an autonomous driving vehicle model, a traffic participant model, and an environmental condition model. The autonomous driving vehicle model is a model constructed based on the characteristics of the autonomous driving vehicle. The traffic participant model is a model constructed based on the characteristics of the remaining vehicles and pedestrians other than the autonomous driving vehicle to be predicted. The environmental condition model is a model constructed based on environmental information. And according to the autonomous driving requirements, an internal association relationship, a spatial relationship, and a constraint relationship are added between the autonomous driving vehicle model, the traffic participant model, and the environmental condition model to realize the mapping between the digital twin model and the actual traffic physical system.

[0023] Its beneficial effects are as follows: The digital twin model includes an autonomous vehicle model, a traffic participant model, and an environmental condition model, which has a high degree of matching with the actual physical world and effectively improves the prediction accuracy.

[0024] Furthermore, the autonomous vehicle model includes a geometric model, a physical model, a behavior model, and a rule model; the geometric model of the autonomous vehicle is constructed using the geometric feature parameter information of the autonomous vehicle; the physical model of the autonomous vehicle is obtained by mathematically formulating the meta-model based on the physical characteristics between the autonomous vehicle and the road surface and the physical parameters affecting the driving characteristics of the autonomous vehicle; the behavior model of the autonomous vehicle is used to demonstrate the behavior characteristics of the autonomous vehicle and the vehicle-road response mechanism; the rule model of the autonomous vehicle is used to describe the behavior rules and logics of the autonomous vehicle according to urban traffic passing rules and vehicle-road influence laws.

[0025] Its beneficial effects are as follows: The autonomous vehicle model is designed by combining four models, which can accurately map the urban traffic physical system, thus completing the construction of the vehicle-traffic digital twin.

[0026] Furthermore, the traffic participant model includes a geometric model, a behavior model, and a rule model; the geometric model of the traffic participant is constructed using the geometric feature parameter information of the traffic participant; the behavior model of the traffic participant is used to demonstrate the behavior characteristics of the traffic participant and the vehicle-road response mechanism; the rule model of the traffic participant is used to describe the behavior rules and logics of the traffic participant according to urban traffic passing rules and vehicle-road influence laws.

[0027] Its beneficial effects are as follows: The traffic participant model is designed by combining three models, which can accurately map the urban traffic physical system, thus completing the construction of the vehicle-traffic digital twin.

[0028] Furthermore, the environmental condition model includes a behavior model and a rule model; the behavior model includes a macro-micro following model based on the coupling of the front and rear vehicles in the same lane and a macroscopic traffic flow model based on the lane-to-lane coupling relationship.

[0029] The macro-micro following model is:

[0030]

[0031] In the formula, u(x,t) is the traffic flow speed at section x at time t, which is affected by the headway distance and speed difference with the first l vehicles in front, k represents the traffic density flow, u e is the desired speed, T l is the relaxation time for multiple leading vehicles, reflecting the sensitivity of the distance between the target vehicle and multiple leading vehicles, τ l is the disturbance propagation backward Δx lThe time required for the distance reflects the sensitivity of the speed difference between the target vehicle and multiple preceding vehicles;

[0032] The macroscopic traffic flow model is as follows:

[0033]

[0034] In the formula, m, n = 1, 2,..., m ≠ n, μ m = b0u′ e (k m , βk n ) < 0, k m and q m respectively represent the traffic density and flow of the m-th lane, k n represents the traffic density of the n-th lane, s mn represents the flow transfer rate from lane m to lane n, s nm represents the flow transfer rate from lane n to lane m, u m is the vehicle flow speed of the m-th lane, c m00 is the disturbance propagation speed of the m-th lane, β is the lane coupling coefficient, T m is the relaxation time of the m-th lane, u em represents the desired speed of the m-th lane, which is a function of k m and k n The function u′ em represents the first-order total derivative of u em with respect to k m and k n , x represents the spatial position, t represents the time, u′ e represents the first-order total derivative of u e with respect to k m and k n , b0 represents a non-negative constant, x m represents the position at x of the m-th lane.

[0035] Its beneficial effect is that the environmental condition model is designed by combining two models and can accurately map the urban traffic physical system, thus completing the construction of the vehicle-traffic digital twin.

[0036] Furthermore, the road traffic facility model includes a geometric model and a physical model; the geometric model of the road traffic facility model is constructed using the geometric feature parameter information of the road traffic facility.

[0037] Its beneficial effect is that the road traffic facility model is designed by combining two models and can accurately map the urban traffic physical system, thus completing the construction of the vehicle-traffic digital twin. Description of the Drawings

[0038] Figure 1 is the structural connection diagram of the remote takeover system for autonomous vehicles of the present invention;

[0039] Figure 2 is the flowchart of the method for the remote takeover system of autonomous vehicles of the present invention;

[0040] Figure 3 is the schematic diagram of the simplified model of four-wheel eight-degree-of-freedom vehicle dynamics. Specific Embodiments

[0041] This application can actively evaluate and predict the safety state of autonomous vehicles to determine the takeover entity of the control authority of autonomous vehicles. The takeover entity includes the autonomous driving system, digital twin platform, and remote driving platform of autonomous vehicles, so as to actively take over before the failure or malfunction of the autonomous driving system, and there is no problem of waiting for mid-course parking caused by traditional takeover methods, which can ensure the continuity of driving tasks, increase passenger comfort, improve traffic efficiency, and enhance safety.

[0042] To more clearly illustrate the technical solutions, objectives, and advantages of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0043] Embodiment of the method for remotely taking over an autonomous vehicle:

[0044] This embodiment provides a remote takeover system for autonomous vehicles, as Figure 1 shown, including a vehicle terminal, a digital twin platform, and a remote driving platform.

[0045] 1. Vehicle terminal.

[0046] The vehicle terminal is used to obtain the vehicle state information and traffic information of the autonomous vehicle, and control the autonomous vehicle according to the control instructions of the autonomous driving system, digital twin platform, or remote driving platform of the autonomous vehicle.

[0047] Devices such as sensors, radars, high-precision maps, and cameras are installed on the autonomous vehicle to obtain the vehicle state information of the autonomous vehicle. The vehicle state information here includes vehicle speed information, vehicle position information, steering wheel angle information, brake pedal opening information, gear information, power battery SOC information, parking state information, and vehicle lane position information. The traffic information includes road topology structure information, weather state information, obstacle information, and traffic signal information.

[0048] The autonomous vehicle is equipped with a communication module that communicates with both the digital twin platform and the remote driving platform. It can transmit the vehicle status information and traffic information of the autonomous vehicle to the digital twin platform and the remote driving platform in real time through the network environment, and is capable of receiving control instructions from the digital twin platform and the remote driving platform. The network environment here can be Bluetooth transmission, 4G network, 5G network, WLAN technology, etc. Of course, multiple transmission methods can also be used simultaneously to increase the reliability of data transmission.

[0049] 2. Digital twin platform.

[0050] The digital twin platform is used to generate a digital twin model of the autonomous vehicle and its traffic environment according to digital twin technology. The vehicle parameters, status, and traffic environment in the digital twin model are synchronized with the actual physical world in real time.

[0051] The digital twin platform will first construct an initial digital twin model through model construction, model assembly and fusion, model verification, and model correction of its main elements according to the designed operating conditions and operating domain of the autonomous vehicle on urban roads. The digital twin model includes: constructing a geometric model, a physical model, a behavior model, and a rule model for the autonomous vehicle; constructing a geometric model, a behavior model, and a rule model for traffic participants (including various motor vehicles, non-motor vehicles, and behaviors other than autonomous vehicles); constructing a geometric model and a physical model for roads and traffic infrastructure (including temporary facilities); constructing a behavior model and a rule model for environmental conditions such as weather and lighting. Among them, the geometric model is mainly a 3D CAD model; the physical model is the interaction between directly acting objects or the characteristics of the object itself; the rule model is some definite rules in the real physical world, such as traffic light rules, etc.

[0052] For the multiple models constructed for the autonomous vehicle, among them, the geometric model is the mapping of the appearance, size, structure, etc. of the autonomous vehicle in the digital twin body. Its construction method is: according to information such as the designed geometric feature parameters of the autonomous vehicle, use CAD software to construct its 3D geometric model, and further compress and reconstruct it based on 3D lightweight technology to achieve standardization and lightweight under the premise of ensuring model accuracy; the physical model is the mapping of parameters such as the mass, centroid position, and vehicle-road interaction characteristics of the autonomous vehicle in the digital twin body. Its construction method is: based on the physical characteristics between the autonomous vehicle and the road surface, as well as the basic physical parameters such as mass and centroid position that affect the driving characteristics of the vehicle, use Modelica (a multi-domain unified modeling language) to mathematically represent the meta-models in different domains and construct the physical model.

[0053] Such as Figure 3As shown in the figure, taking the vehicle-road coupling characteristics modeling of an eight-degree-of-freedom autonomous vehicle with four-wheel independent drive as an example, the construction process of the physical model in a single field is illustrated. In the figure, CG is the vehicle's center of mass, L f , L r represent the distances from the center of mass to the front and rear axles respectively, L is the wheelbase between the left and right wheels; δ is the front wheel steering angle given by the vehicle controller, F xfl , F xfr , F xrl , F xrr represent the longitudinal forces of the left front, right front, left rear, and right rear 4 wheels respectively, F yfl , F yfr , F yrl , F yrr represent the lateral forces of the 4 wheels respectively, α fl , α fr , α rl , α rr are the sideslip angles of the 4 wheels respectively.

[0054] The state equations for the lateral and yaw motions of the vehicle body are:

[0055]

[0056] In the formula, V x , V y are the longitudinal and lateral velocities of the vehicle respectively, γ is the yaw angular velocity, m is the total vehicle mass, I z is the vehicle's yaw moment of inertia.

[0057] Under extreme conditions, the lateral and longitudinal forces of the tire affect each other, and there is a coupled non-linear relationship between the tire lateral force and the slip ratio and sideslip angle. The composite slip LuGre tire model is adopted. When the vehicle is in a steady state, the lateral force of the tire is:

[0058]

[0059] In the formula, σ 0y is the tire lateral stiffness coefficient, σ 2y is the tire lateral viscous damping, k y is the lateral load distribution coefficient, μ is the road surface adhesion coefficient, g(v r ) is the Stribeck equation, R e is the tire rolling radius, ω is the tire rotation angular velocity, F n is the vertical load of the tire, v r = [v rx v ry T represents the vector matrix composed of the longitudinal and lateral slip amounts, where the lateral slip amount v ry = v xt α.​

[0060] Based on the relationship between the lateral slip amount v rx and the longitudinal slip amount v xy with parameters such as the vehicle steering wheel angle δ and the road adhesion coefficient μ, a coupling model between the vehicle, tire, and road surface is established, as shown in Equation (3), providing a theoretical basis for vehicle speed tracking and motion control.

[0061]

[0062] In the formula, K1, K2, and K3 are the coefficients of the tire resultant force, k y is the vehicle lateral load distribution coefficient, g tr and g br are the values of the Stribeck equation under driving and braking conditions, respectively.

[0063] The connection between the vehicle and the road is established through the tire. Under urban road driving conditions, the deformation of the road can be ignored relative to the tire. Therefore, based on the above vehicle-road coupling model, the longitudinal and lateral slip amounts of autonomous vehicles can be accurately predicted and controlled to ensure driving safety.

[0064] The construction method of the behavior model is as follows: Based on vehicle dynamics, kinematics, vehicle-road coupling mechanism, etc., using finite state machines, Markov chains, and ontology-based modeling methods, construct its behavior model to depict the behavior characteristics of autonomous vehicles and the vehicle-road response mechanism; taking the construction of the behavior model between various motor vehicles except autonomous vehicles as an example, illustrate the construction process of the behavior model.

[0065] Following the ideas from micro, macro, and the combination of macro and micro, considering the real-time states of adjacent and sub-adjacent vehicles, establish a macro-micro car-following model based on the coupling of front and rear vehicles in the same lane and a macroscopic traffic flow model based on the coupling relationship between lanes respectively.

[0066] Regarding the influence of multi-leader vehicle information (perceived by roadside units) on autonomous vehicles, introduce the weight coefficient of coupling information. Based on the vehicle car-following model, according to the micro-macro parameter correlation relationship adopted in traffic flow research, establish the following macroscopic car-following model affected by multi-vehicle coupling disturbances:

[0067]

[0068] In the formula, u(x,t) is the traffic flow speed on the x-section at time t, affected by the headway distance and speed difference with the first l vehicles in front. k represents the traffic density flow, u e is the desired speed, T l is the relaxation time of multi-leader vehicles, reflecting the sensitivity of the distance between the target vehicle and multi-leader vehicles, τ l is the backward propagation of the disturbance by Δx lThe time required for the distance reflects the sensitivity of the speed difference between the target vehicle and multiple leading vehicles.

[0069] For lane-changing and lane-to-lane coupling effects in multi-lane scenarios, based on the high-order continuous medium traffic flow model, a multi-lane coupling coefficient is introduced, and a multi-lane density-viscosity coupling model is proposed, as shown in Equation (5).

[0070]

[0071] where m, n = 1, 2, …, m ≠ n, μ m = b0u′ e (k m , βk n ) < 0, k m and q m represent the traffic density and flow of the m-th lane respectively, k n represents the traffic density of the n-th lane, s mn represents the flow transfer rate from lane m to lane n, s nm represents the flow transfer rate from lane n to lane m, u m is the vehicle flow speed of the m-th lane, c m00 is the disturbance propagation speed of the m-th lane, β is the lane-to-lane coupling coefficient, T m is the relaxation time of the m-th lane, u em represents the desired speed of the m-th lane, which is a function of k m and k n , u′ em represents the first-order total derivative of u em with respect to k m and k n , x represents the spatial position, t represents the time, u′ e represents the first-order total derivative of u e with respect to k m and k n , b0 represents a non-negative constant, x m represents the position at x in the m-th lane.

[0072] The above-established macro-micro following model and macroscopic traffic flow model, combined with the information of the leading vehicle's position and speed obtained by vehicle-end perception, and the traffic flow information obtained by roadside perception, provide a basis for speed and path planning in scenarios such as lane-changing and overtaking.

[0073] The construction method of the rule model is as follows: Based on urban traffic rules and vehicle-road influence laws, use decision trees, rough set theory, and neural network methods to complete rule extraction, and describe the operation rules and logical models of autonomous driving based on XML language, enabling the digital twin model of autonomous driving vehicles to have the capabilities of reasoning, judgment, evaluation, and prediction.

[0074] After the construction of each model of traffic participants, based on the constraints of getting off, roads, people, and the environment in urban traffic, the hierarchical relationship of the models is constructed and the assembly order of the models is clarified. For the needs of autonomous driving, the element models (such as autonomous vehicles, infrastructure such as roads, obstacles, traffic signs and signs, surrounding buildings, etc., and traffic participants such as pedestrians and non-motor vehicles) are associated and assembled by adding spatial relationships, constraint relationships, etc., and integrated into the system-level digital twin basic model. Based on the vehicle-road-person-environment coupling mechanism, it is assembled and integrated with models such as weather, lighting, and traffic signals into a dynamic model of the urban traffic system. Furthermore, the internal correlation relationships are added to the assembled models. By mapping action timing relationships, safe time intervals, traffic flows, information flows, etc. into the models, the urban traffic physical system is accurately mapped, and the construction of the vehicle-traffic digital twin is completed. Among them, the spatial relationship is such as the relative position relationship between the road and the vehicle; the constraint relationship is the relationship such as pedestrians can only walk on the sidewalk and motor vehicles can only drive on the motor vehicle lane; the safe time interval refers to the safe distance S between two vehicles divided by the speed V of the following vehicle, and its unit is seconds. When mapping to the digital twin model, only the corresponding parameters need to be returned to the digital twin model.

[0075] The vehicle-traffic digital twin combines the virtual and the real with the object environment, interacts in real time, continuously self-learns and updates, and gradually improves the model accuracy through data-driven iteration and optimization.

[0076] After the construction of each model of traffic participants, for the needs of autonomous driving, the spatial relationships such as the relative spatial positions and constraint relationships (such as pedestrians can only walk on the crosswalk and motor vehicles can only drive on the motor vehicle lane) between the element models (such as autonomous vehicles, infrastructure such as roads, obstacles, traffic signs and signs, surrounding buildings, etc., and traffic participants such as pedestrians and non-motor vehicles) measured by various sensors are associated and assembled, and integrated into the system-level digital twin basic model. Based on the vehicle-road-person-environment coupling mechanism, it is assembled and integrated with models such as weather, lighting, and traffic signals into a dynamic model of the urban traffic system. Furthermore, by mapping action timing relationships (such as the sequence of traffic lights: go straight first and then turn left, etc.), safe time intervals, traffic flows, etc. into the models, the urban traffic physical system is accurately mapped, and the construction of the vehicle-traffic digital twin is completed. The vehicle-traffic digital twin combines the virtual and the real with the object environment, interacts in real time, continuously self-learns and updates, and gradually improves the model accuracy through data-driven iteration and optimization.

[0077] The digital twin model can obtain vehicle status information and traffic information, and make the status information of autonomous vehicles and traffic environment information at the same time correspond one by one, so as to realize the recording of information throughout the life cycle of autonomous vehicles. Historical data can improve the accuracy of the digital twin platform, improve the accuracy of safety assessment and prediction of the entire life cycle of autonomous vehicles, and can also record extreme working conditions and reproduce and learn continuously, so as to continuously optimize the digital twin model.

[0078] Moreover, based on the current vehicle status information and current traffic information of the autonomous vehicle, as well as the constructed digital twin model, the digital twin platform can match the current vehicle status information and current traffic information with historical information to evaluate and predict the current safety status of the autonomous vehicle, allocate decision-making power for the autonomous vehicle, output the takeover requirements of the autonomous vehicle, and transfer the control authority of the autonomous vehicle to the corresponding takeover entity according to the output result.

[0079] When the control authority of the autonomous vehicle is transferred to the digital twin platform, control instructions can be sent to the autonomous vehicle, so that the autonomous vehicle works according to the control instructions of the digital twin platform and ignores the control instructions from the autonomous driving system and the remote driving platform.

[0080] 3. Remote driving platform.

[0081] The remote driving platform is provided with a display device. After receiving the autonomous vehicle and traffic information, the remote driving platform can display the vehicle status information and traffic information through the display device. The display form can be text, image, sound, and the display method can be a display, a virtual reality device.

[0082] When the remote driving platform receives the takeover requirement of the digital twin platform and the takeover requirement is that the cloud safety officer needs to take over, the remote driving platform reminds the cloud safety officer to make takeover preparations, and the reminder method can be sound, vibration, light, etc. The remote driving platform will convert the driving operations of the cloud safety officer into control instructions and send them down (including steering wheel angle, desired speed, desired acceleration, light control, pulling / releasing the parking brake, etc.) to the autonomous vehicle, so that the autonomous vehicle ignores the control instructions from the digital twin platform and the remote driving platform and executes the control instructions of the remote driving platform.

[0083] Moreover, the takeover signal of the cloud safety officer can be a set of specific signals, which can be sent by setting a special takeover button.

[0084] Based on the above-introduced remote takeover system for autonomous vehicles, a method for remotely taking over an autonomous vehicle according to the present invention can be realized. The overall process is as Figure 2 shown, and the following is a specific introduction.

[0085] Step 1: Obtain the current vehicle state information of the autonomous vehicle and the current traffic information of the environment where the autonomous vehicle is located through various sensors arranged on the autonomous vehicle, and send them to the digital twin platform and the remote driving platform in real time through the network environment.

[0086] The vehicle state information includes vehicle position information, destination information, vehicle speed information, steering wheel angle information, accelerator / brake pedal opening information, power battery SOC, etc. The traffic information includes road topology structure, weather conditions, number of obstacles, types of obstacles, positions and speeds of obstacles, traffic signal information, etc.

[0087] During the process of sending information, if the network environment is unstable or the network load is too high, then give priority to sending more important information to ensure the realization of basic remote driving functions. Here, the important information includes: vehicle position information, destination information, vehicle speed information, steering wheel angle information, accelerator / brake pedal opening information, and video information.

[0088] Step 2: The digital twin platform simulates, judges, and predicts the vehicle safety state based on the current vehicle state information, the current traffic information, and the constructed digital twin model. The specific simulation, judgment, and prediction methods are as follows:

[0089] First, unify the relationships between various risk factors in terms of time and space, and then quantify the risks. Introduce the field theory into the traffic system risk assessment, combine the constructed BTA model with the traffic risk situation deduction model, and comprehensively consider the influence of interference factors such as extreme weather and communication delay. Reconstruct the three types of risks (field strengths) generated by various elements (field sources) of the scene traffic on the vehicle: the "potential energy field" determined by static elements in the road environment, the "kinetic energy field" determined by moving objects on the road, and the "behavior field" determined by the characteristics of vehicle driving behavior. Finally, unify them into the driving risk field.

[0090] The "potential energy field" is determined by road static elements (stopped vehicles, isolation belts, roadblocks, traffic signs, etc.), and the influencing factors include object type, mass, environmental visibility, etc., represented by ; The "kinetic energy field" is determined by road dynamic elements (moving vehicles, pedestrians, animals, non-motor vehicles, etc.), and the influencing factors include field source type, mass, speed, acceleration, road surface adhesion coefficient (related to weather), road slope, etc., represented by ; The "behavior field" is determined by the vehicle state elements under driving behavior, represented by .

[0091] The field source i(x i ,y i ) at (x j ,yj ) The electric field strength generated at

[0092]

[0093] In the formula, M i is the equivalent mass, T i is the object type, m i is the object mass, a i , β k are undetermined constants, R i is the road condition influence factor, r ij =(x j -x i , y j -y i ) represents the displacement between two points, and k1, k2, G1 are constants to be calibrated; v i is the driving speed, θ i is the angle between the speed direction and r ij ; D ri is an undetermined risk factor, which is determined by the behavioral characteristics of different driving modes.

[0094] Construct a unified model of the driving risk field:

[0095]

[0096] In the formula: E j , are the resultant vector of the electric field strengths of the driving risk field, kinetic energy field, potential energy field, and behavior field at the position of vehicle j, respectively; and are the electric field vectors of the single kinetic energy field, potential energy field, and behavior field at the position of vehicle j, respectively.

[0097] Vehicles in the driving risk field will be affected by the field force, which is used to characterize the danger level of the current driving state of the vehicle. The influencing factors include the electric field strength of the driving risk field, the road conditions at the vehicle's location, the vehicle's own attributes, the motion state, and the vehicle's driving behavior characteristics, etc. During the driving process of the vehicle, the greater the field force it receives, the lower the safety level. To quantify the relationship between the field force and the safety level, the essential safety degree S j is introduced, and the specific expression is:

[0098]

[0099] In the formula, F j is the field force received by the vehicle at (x j , y j ).

[0100] The determination of the vehicle-cloud control right rules based on the intrinsic safety level and time is divided into two steps. One is the determination of the switching threshold, and the other is the switching logic.

[0101] The thresholds to be determined include: the upper bound of the intrinsic safety level for cloud automatic takeover The lower bound of the intrinsic safety level for cloud automatic takeover The upper bound of the intrinsic safety level for cloud safety officer takeover The lower bound of the intrinsic safety level for cloud safety officer takeover To avoid abnormal cyclic switching of the control right during the takeover process, the vehicle-cloud control right switching rules need to consider the low safety level interval The duration T P . The determination of the above thresholds and the low safety level duration needs to consider vehicle dynamics, kinematics, cloud control delay characteristics, and the resulting boundary of the vehicle-end and cloud-end behavior control capabilities, and be optimized and calibrated through subjective and objective evaluations.

[0102] The switching rule for the control right to switch from the i-th end to the j-th end at time k can be defined as:

[0103] E ij (k):=rules(S k ,T P ) (9)

[0104] In the formula, S k is the intrinsic safety level at time k, and i, j = 1, 2, 3, representing the vehicle-end controller, cloud-end controller, and cloud safety officer respectively. To avoid sudden changes in the vehicle driving state, the control right can only be switched step by step, specifically expressed as |j - i| = {0, 1}. When i = j, the control right remains unchanged.

[0105] According to the above rules, after the system of the autonomous vehicle fails or the driving conditions reach the critical designed operating domain, when the intrinsic safety level drops to a certain threshold, the vehicle-cloud control right will automatically switch, and through remote takeover and active intervention, the vehicle state will be quickly moved away from the ODD boundary to ensure driving safety.

[0106] The digital twin platform evaluates and predicts the vehicle safety state according to the above rules and outputs takeover requirements: if the takeover requirement is a zero takeover requirement, step three is executed; if the takeover requirement is a weak takeover requirement, step four is executed; if the takeover requirement is a strong takeover requirement, step five is executed.

[0107] In step three, at this time, the takeover requirement is a zero takeover requirement, indicating that the perception device and actuator of the autonomous vehicle are working normally, the autonomous driving system can work normally and can handle various traffic conditions to ensure the driving safety of the autonomous vehicle. Therefore, in this situation, the autonomous driving system of the autonomous vehicle continues to take over the vehicle and ignores the control instructions sent by the digital twin platform in step two.

[0108] Step 4: At this time, the takeover requirement is a weak takeover requirement, indicating that there may be two situations: ① The perception device and actuator of the autonomous vehicle are working properly and the autonomous driving system is working properly, but the autonomous driving system cannot handle the traffic conditions; ② The perception device and actuator of the autonomous vehicle are working properly, and the autonomous driving system is partially or completely ineffective and cannot handle the traffic conditions it faces. In summary, when the vehicle safety state is that the autonomous driving system fails or is close to the operating design domain, in the face of such a situation, the digital twin platform will automatically take over the autonomous vehicle and continue to send control instructions to the autonomous vehicle. After receiving the weak takeover requirement, the autonomous vehicle blocks the control instructions output by its own autonomous driving system and executes the control instructions generated in Step 2. During this period, the autonomous vehicle normally sends vehicle status information and traffic information to the digital twin platform.

[0109] Step 5: At this time, the takeover requirement is a strong takeover requirement, indicating that there may be two situations: ① The perception device and actuator of the autonomous vehicle are working properly and the autonomous driving system is working properly, but the autonomous driving system and the digital twin platform cannot handle the traffic conditions they face; ② The perception device and actuator of the autonomous vehicle are working properly, the autonomous driving system is partially or completely ineffective, and the autonomous driving system and the digital twin platform cannot handle the traffic conditions. In summary, when the digital twin platform also fails to meet the safety requirements, the remote driving platform will issue an alarm to remind the cloud safety officer to prepare to take over the autonomous vehicle. At this time, the cloud safety officer sends a control signal to the autonomous vehicle through the remote driving platform. When the autonomous vehicle receives the control signal from the remote driving platform, it starts to prepare to receive the control instructions from the remote driving platform.

[0110] Moreover, before the cloud safety officer takes over the autonomous vehicle, the vehicle is still taken over by the autonomous driving system until the takeover control signal from the cloud safety officer is received.

[0111] After the autonomous driving system receives the takeover signal from the safety officer and receives valid control instructions, the autonomous driving system transfers the vehicle control right to the remote driving platform, blocks its own decision-making output, and executes the control instructions received from the remote driving platform. During this period, the autonomous vehicle normally sends vehicle status information and traffic information to the digital twin platform.

[0112] To ensure the smooth handover of vehicle-cloud control rights and avoid drastic changes in vehicle states during the handover of control rights, which may affect driving safety, based on the concept of parallel control and digital twin models, a cloud-based reinforcement learning agent and a vehicle-side reinforcement learning agent are respectively constructed based on a decision-making module and an evaluation module. The motion state feedback during the driving process of an autonomous vehicle is used as the input of the reinforcement learning model. The cloud decision-making module of the reinforcement learning agent outputs the decision result as a according to the current state. The cloud evaluation module evaluates the decision result according to the state-action value function Q 1 (s 1 ,a 1 ), and thus calculates and obtains the new value function Q 2 (s 2 ,a 2 ) as the vehicle-side decision input. The cloud decision-making module selects the optimal decision instruction π according to the maximum evaluation function value, as shown in Equation (10). By minimizing the difference between the value functions of the cloud evaluation module and the vehicle-side evaluation module, as shown in Equation (11), the cloud decision-making gradually approaches the vehicle-side decision output, realizing a fast and smooth handover of control rights between the cloud and the vehicle side.

[0113]

[0114] In the formula, the value functions of the cloud evaluation module and the vehicle-side evaluation module are cumulative discounted rewards.

[0115]

[0116] When J(t) < ε, ε can be optimized and calibrated according to the needs of ride comfort control. The evaluation module outputs a trigger instruction to the vehicle-side decision-making module to complete the smooth handover of control rights from the vehicle side to the cloud.

[0117] After the control rights are successfully transferred, the cloud should actively intervene to reduce driving risks and hand back the control rights as soon as possible. The specific method is as follows:

[0118] Based on the vehicle-road coupling dynamics model, according to the road surface adhesion coefficient and the acceleration performance limit of the vehicle itself, with the current state of the vehicle as the initial position, calculate the limit position that can be reached on the road in each stage, and establish an objective function for the decision-making problem based on the intrinsic safety degree, as shown in Equation (12).

[0119]

[0120] In the formula, S k (x k ,dec k ) represents that the evolution scenario is in the x k state and executes the decision dec kThe essential safety level after, Saf(t) is the optimal index function and is an important basis for solving the optimal decision sequence. j(x j ,y j ) is the vehicle position coordinate, v i is the driving speed, r ij represents the distance vector between two points, θ i is the angle between the speed direction and r ij , a x , a y are the longitudinal and lateral accelerations respectively, μ is the road surface adhesion coefficient, a engine is the maximum acceleration of the vehicle itself, t k is the future time corresponding to stage k, x0 and v0 are the initial position and speed respectively, a lim and x lim are the acceleration limit and the limit position that can be reached within time t k .

[0121] The cloud active intervention process oriented to safety is the optimal solution process of Equation (12), and finally the optimal path and vehicle speed decision sequence are obtained through deep reinforcement learning.

[0122] In summary, compared with the traditional scheme in which the autonomous driving vehicle itself initiates a takeover request, the present invention can actively evaluate and predict the safety state of the autonomous driving vehicle, take over actively before the autonomous driving system fails or malfunctions, and the digital twin platform or the remote driving platform takes over the vehicle actively, without the problem of mid-way parking and waiting caused by the traditional takeover method, which can ensure the continuity of the driving task, increase passenger comfort, improve traffic efficiency and improve safety.

[0123] Finally, it should be noted that each embodiment is only used to illustrate the technical solution of the present invention, rather than limiting it; similarly, in the embodiments of the present application, various numbers are for the convenience of description and are not used to limit the scope of the embodiments of the present application, and the number sequence does not mean the execution sequence, and the execution sequence of each process should be based on its internal logical sequence. The above specific implementation manners further elaborate the purpose, technical solution and beneficial effects of the present application. It should be understood that the above is only the specific implementation manner of the present invention and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present application shall be included in the protection scope of the present application.

Claims

1. A method for remotely taking over an autonomous vehicle, characterized in that, The method includes the following steps: 1) Obtain the current vehicle state information of the autonomous vehicle and the current traffic information of the environment where the autonomous vehicle is located; 2) The digital twin platform uses the obtained current vehicle state information and current traffic information, as well as the constructed digital twin model, to evaluate the current safety state of the autonomous vehicle and output the takeover requirement of the autonomous vehicle: If the takeover requirement is a zero takeover requirement, control the autonomous driving system of the autonomous vehicle to take over the autonomous vehicle; if the takeover requirement is a weak takeover requirement, control the digital twin platform to take over the autonomous vehicle; if the takeover requirement is a strong takeover requirement, control the remote driving platform to take over the autonomous vehicle for manual takeover; Among them, a zero takeover requirement indicates that the autonomous driving system is normal and within the designed operation domain of the autonomous driving system, a weak takeover requirement indicates that the autonomous driving system fails or exceeds the designed operation domain of the autonomous driving system, and a strong takeover requirement indicates that the digital twin platform cannot meet the vehicle safety requirements; The following method is used to determine the takeover requirement of the autonomous vehicle: a) Calculate the potential energy field, kinetic energy field, and behavior field generated by each field source around the autonomous vehicle; the potential energy field is determined by road static elements, the kinetic energy field is determined by road dynamic elements, and the behavior field is determined by the state elements of the autonomous vehicle under driving behavior; b) Perform a summation operation based on the potential energy field, kinetic energy field, and behavior field generated by each field source to obtain the driving risk field of the autonomous vehicle; c) Determine the inherent safety degree of the autonomous vehicle according to the driving risk field of the vehicle; Where, S j represents the intrinsic safety degree at the j-th moment, where i, j = 1, 2, 3 represent the autonomous driving system, the digital twin platform, and manual takeover respectively; F j represents the field force received by the vehicle at the field source (x j , y j ); E j represents the driving risk field; M j represents the equivalent mass; R j represents the road condition influence factor; k2 represents a constant; v j represents the driving speed; θ j represents the angle between the speed direction and the displacement between two points (x i , y i ) and (x j , y j ); D rj represents the risk factor; d) Determine the takeover requirement of the autonomous vehicle according to the inherent safety degree interval where the inherent safety degree of the autonomous vehicle is located; among them, one takeover requirement corresponds to one inherent safety degree interval.

2. The method for remotely taking over an autonomous vehicle according to claim 1, wherein The vehicle state information includes at least two of vehicle speed information, vehicle position information, steering wheel angle information, brake and accelerator pedal opening information, gear information, power battery SOC information, parking state information, and vehicle lane position information.

3. The method for remotely taking over an autonomous vehicle according to claim 1, wherein The traffic information includes at least two of road topology structure information, weather state information, obstacle information, and traffic signal information.

4. The remote takeover method of an autonomous vehicle according to claim 1, wherein The digital twin model includes: an autonomous vehicle model, a traffic participant model, a road traffic facility model, and an environmental condition model. The autonomous vehicle model is a model constructed based on the characteristics of the autonomous vehicle. The traffic participant model is a model constructed based on the characteristics of other vehicles and pedestrians except the autonomous vehicle to be predicted. The road traffic facility model is a model constructed based on roads and traffic facilities. The environmental condition model is a model constructed based on environmental information. And according to the autonomous driving requirements, internal association relationships, spatial relationships, and constraint relationships are added between the autonomous vehicle model, the traffic participant model, the road traffic facility model, and the environmental condition model to realize the mapping between the digital twin model and the actual traffic physical system.

5. The method for remotely taking over an autonomous vehicle according to claim 4, wherein The described autonomous vehicle model includes a geometric model, a physical model, a behavior model, and a rule model; the geometric model of the autonomous vehicle model is constructed using the geometric feature parameter information of the autonomous vehicle; the physical model of the autonomous vehicle model is obtained by mathematically representing the meta-model based on the physical characteristics between the autonomous vehicle and the road surface and the physical parameters affecting the driving characteristics of the autonomous vehicle; the behavior model of the autonomous vehicle model is used to exhibit the behavior characteristics of the autonomous vehicle and the vehicle-road response mechanism; the rule model of the autonomous vehicle model is used to describe the behavior rules and logics of the autonomous vehicle according to urban traffic rules and vehicle-road influence laws.

6. The method for remotely taking over an autonomous vehicle according to claim 4, wherein The described traffic participant model includes a geometric model, a behavior model, and a rule model; the geometric model of the traffic participant model is constructed using the geometric feature parameter information of the traffic participant; the behavior model of the traffic participant model is used to exhibit the behavior characteristics of the traffic participant and the vehicle-road response mechanism; the rule model of the traffic participant model is used to describe the behavior rules and logics of the traffic participant according to urban traffic rules and vehicle-road influence laws.

7. The method for remotely taking over an autonomous vehicle according to claim 4, wherein The described environmental condition model includes a behavior model and a rule model, and the behavior model includes a macro-micro following model based on the coupling of the vehicle in front and behind in the same lane and a macroscopic traffic flow model based on the lane-to-lane coupling relationship. The macro-micro following model is: Wherein, \(u(x,t)\) is the traffic flow speed of section \(x\) at time \(t\), which is affected by the headway distance and speed difference with the first \(l\) vehicles ahead. \(k\) represents the traffic density flow, and \(u\) e is the desired speed, \(T\) l is the relaxation time of multiple vehicles ahead, reflecting the sensitivity of the distance between the target vehicle and multiple vehicles ahead. \(\tau\) l is the time required for the disturbance to propagate backward by \(\Delta x\) l distance, reflecting the sensitivity of the speed difference between the target vehicle and multiple vehicles ahead; The macroscopic traffic flow model is: where \(m,n = 1,2,\cdots,m\neq n\), \(\mu\) m \(=b_0u'\) e (\(k\) m ,\(\beta_k\) n ) \(\lt 0\), \(k\) m and \(q\) m respectively represent the traffic density and flow of the \(m\)-th lane, \(k\) n represents the traffic density of the \(n\)-th lane, \(s\) mn represents the flow transfer rate from lane \(m\) to lane \(n\), \(s\) nm represents the flow transfer rate from lane \(n\) to lane \(m\), \(u\) m is the vehicle flow speed of the \(m\)-th lane, \(c\) m00 is the disturbance propagation speed of the \(m\)-th lane, \(\beta\) is the lane - to - lane coupling coefficient, \(T\) m is the relaxation time of the \(m\)-th lane, \(u\) em represents the desired speed of the \(m\)-th lane, which is a function of \(k\) m and \(k\) n \(u'\) em represents the first - order total derivative of \(u\) em with respect to \(k\) m and \(k\) n , \(x\) represents the spatial position, \(t\) represents the time, \(u'\) e represents the first - order total derivative of \(u\) e with respect to \(k\) m and \(k\) n , \(b_0\) represents a non - negative constant, \(x\) m represents the position at \(x\) of the \(m\)-th lane.

8. The method for remotely taking over an autonomous vehicle according to claim 4, wherein The described road traffic facility model includes a geometric model and a physical model; the geometric model of the road traffic facility model is constructed using the geometric feature parameter information of the road traffic facility.

Images