Remote takeover operation method for self-driving vehicle, and system
By judging the type of driving scenario in an autonomous driving vehicle and selecting manual or automatic takeover process, and using cloud platform for data transmission and control, the problem of manual operation dependence in the existing technology is solved, and the accuracy and real-time nature of takeover operation are improved. It reduces labor costs and is suitable for the large-scale popularization of autonomous vehicles.
Patent Information
- Application Number
- PCT/CN2024/094844
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-23
- Filing Date
- 2024-05-23
- Publication Date
- 2024-11-28
AI Technical Summary
The existing remote takeover method of autonomous driving vehicles relies on manual operations, with high requirements and inability to achieve 24/7 supervision, resulting in insufficient accuracy of human resource investment and takeover operations.
By judging the current driving scenario type of the vehicle, it is decided to enter the manual takeover or automatic takeover process, and use the cloud platform to transmit data and send control instructions to realize the automatic takeover of the vehicle and reduce manual intervention.
It improves the accuracy and real-time nature of takeover operations, reduces labor costs, and realizes automated takeover of vehicles, which is suitable for the large-scale popularization of autonomous vehicles.
Smart Images

Figure CN2024094844_28112024_PF_FP_ABST
Abstract
Description
A method and system for remotely taking over operation of an autonomous driving vehicle Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and system for remotely taking over operation of an autonomous driving vehicle. Background Art
[0002] With the rapid development of intelligent autonomous driving, the number of vehicles with high-level autonomous driving (L3 and above) is increasing. To ensure safe operation without a human driver, a cloud-based control platform with remote takeover capabilities is essential. Existing remote takeover methods typically rely on manual control of the vehicle to a safe location using remote simulation equipment based on video information transmitted from the vehicle's cameras. This method places high demands on the remote operator and requires constant operator supervision, making it impossible to provide 24 / 7 remote control and operation.
[0003] Summary of the Invention
[0004] The purpose of the present invention is to provide a method and system for remote takeover operation of an autonomous driving vehicle, so as to realize automatic remote takeover of the vehicle under special driving conditions, reduce the human resource investment brought by pure manual takeover mode, and improve the accuracy of takeover operation.
[0005] To solve the above technical problems, the present invention provides a technical solution: a method for remotely taking over operation of an autonomous driving vehicle, comprising:
[0006] When the remote takeover conditions are met, receive the remote takeover request message sent by the vehicle and the current driving scenario type of the vehicle;
[0007] Determine whether to enter a manual takeover process or an automatic takeover process based on the current driving scenario type of the vehicle;
[0008] In the manual takeover process, a control command is generated by manually operating the vehicle-side simulator and sent to the vehicle;
[0009] In the automatic takeover process, the vehicle is positioned and a local driving path is planned, and the local driving path is converted into a control instruction and sent to the vehicle.
[0010] In a second aspect, the present invention provides a cloud platform system for implementing the method for remotely taking over the operation of the autonomous driving vehicle, comprising:
[0011] The autonomous driving cloud control platform is configured to receive a remote takeover request message from a vehicle and the vehicle's current driving scenario type when remote takeover conditions are met; and determine whether to enter a manual takeover process or an automatic takeover process based on the vehicle's current driving scenario type;
[0012] A manual takeover operation platform, which is used to generate control instructions by manually operating the vehicle-side simulator under the manual takeover process, and transmit the control instructions to the vehicle through the autonomous driving cloud control platform;
[0013] The automated takeover cloud platform is used to locate the vehicle and plan the local driving path under the automated takeover process, and the autonomous driving cloud control platform converts the local driving path into control instructions and sends them to the vehicle.
[0014] In a third aspect, the present invention further provides a vehicle-side remote takeover program module for implementing the method for remote takeover of the autonomous driving vehicle, comprising:
[0015] A takeover judgment module is used to judge whether the remote takeover conditions are met;
[0016] The takeover control module is used to send a remote takeover request message and the vehicle's current driving scenario type to the cloud platform system when the remote takeover conditions are met, and receive control instructions transmitted by the cloud platform system to control the vehicle's driving;
[0017] The data upload module is used to upload vehicle video surveillance in the manual takeover process and to upload vehicle sensor information and current driving scene type in the automatic takeover process.
[0018] The beneficial effects of the present invention are:
[0019] 1. The vehicle side only needs to trigger remote takeover through remote takeover conditions. After takeover, the vehicle side only needs to transmit data, and the rest is completed by the cloud, thereby realizing the separation of sensor information and control information, reducing the delay in data generation and transmission, and ensuring the real-time and accuracy of vehicle takeover control; after remote takeover, the cloud side not only includes manual takeover but also automatic takeover. Under automatic takeover, vehicle information is collected for positioning and path planning, which can replace manual takeover to a certain extent, reduce the labor cost of takeover work, and is conducive to the large-scale popularization of autonomous driving vehicles.
[0020] 2. The automated takeover cloud platform receives vehicle sensor information, establishes digital twin scenarios, and plans the vehicle's local driving path. The autonomous driving cloud control platform controls the vehicle, improving the accuracy of vehicle takeover control. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] FIG1 is a flow chart of a method for remotely taking over an operation of an autonomous driving vehicle according to an embodiment of the present invention.
[0022] FIG2 is a flow chart of data interaction between the vehicle side and the cloud side according to an embodiment of the present invention.
[0023] FIG3 is a schematic diagram of the data interaction structure between the vehicle side and the cloud side according to an embodiment of the present invention.
[0024] FIG4 is a schematic structural diagram of an automatic takeover cloud platform according to an embodiment of the present invention. DETAILED DESCRIPTION
[0025] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0026] Referring to FIG1 , the present invention provides a method for remotely taking over operation of an autonomous driving vehicle, comprising:
[0027] When the remote takeover conditions are met, receive the remote takeover request message sent by the vehicle and the current driving scenario type of the vehicle;
[0028] Determine whether to enter a manual takeover process or an automatic takeover process based on the current driving scenario type of the vehicle;
[0029] In the manual takeover process, a control command is generated by manually operating the vehicle-side simulator and sent to the vehicle;
[0030] In the automatic takeover process, the vehicle is positioned and a local driving path is planned, and the local driving path is converted into a control instruction and sent to the vehicle.
[0031] In a second aspect, the present invention provides a cloud platform system for implementing the method for remotely taking over the operation of the autonomous driving vehicle, comprising:
[0032] The autonomous driving cloud control platform is configured to receive a remote takeover request message from a vehicle and the vehicle's current driving scenario type when remote takeover conditions are met; and determine whether to enter a manual takeover process or an automatic takeover process based on the vehicle's current driving scenario type;
[0033] A manual takeover operation platform, which is used to generate control instructions by manually operating the vehicle-side simulator under the manual takeover process, and transmit the control instructions to the vehicle through the autonomous driving cloud control platform;
[0034] The automated takeover cloud platform is used to locate the vehicle and plan the local driving path under the automated takeover process, and the autonomous driving cloud control platform converts the local driving path into control instructions and sends them to the vehicle.
[0035] In a third aspect, the present invention further provides a vehicle-side remote takeover program module for implementing the method for remote takeover of the autonomous driving vehicle, comprising:
[0036] A takeover judgment module is used to judge whether the remote takeover conditions are met;
[0037] The takeover control module is used to send a remote takeover request message and the vehicle's current driving scenario type to the cloud platform system when the remote takeover conditions are met, and receive control instructions transmitted by the cloud platform system to control the vehicle's driving;
[0038] The data upload module is used to upload vehicle video surveillance in the manual takeover process and to upload vehicle sensor information and current driving scene type in the automatic takeover process.
[0039] Referring to Figure 2, a complete process of remote takeover of an autonomous vehicle includes the following steps:
[0040] S1. Determine whether the remote takeover conditions are met. If so, proceed to the next steps.
[0041] Specifically, the method for determining whether a vehicle meets the remote takeover conditions is as follows:
[0042] S101: If there is no drivable area for the vehicle within the duration T1, then proceed to S2; otherwise proceed to S102;
[0043] S102: If there is no drivable trajectory plan for the vehicle within the duration T2, then proceed to S2; otherwise, return to S101;
[0044] Where T1 and T2 are the preset time lengths;
[0045] This step is completed by the takeover judgment module on the vehicle side.
[0046] S2. Set the vehicle to a pending takeover state and send a remote takeover request message and the vehicle's current driving scenario type to the autonomous driving cloud control platform;
[0047] Specifically, when the vehicle is in the waiting-for-takeover state, the current driving scene type is compared with the preset driving scene types. If the current driving scene type matches one of the preset driving scene types, the current driving scene type is marked as a known driving scene, and the accompanying remote takeover request message includes the takeover reason. Otherwise, it is marked as an undefined driving scene, and the accompanying remote takeover request message includes the takeover reason and the current driving scene information.
[0048] The elements of the preset driving scene type include road type, driving status and takeover event. The types of the elements provided in this embodiment are as follows:
[0049] The preset driving scenarios include driving scenarios obtained by combining the above-mentioned different types of elements according to certain rules, for example, the straight-ahead traffic light at the intersection is invalid.
[0050] This step is completed by the takeover control module on the vehicle side.
[0051] S3: The autonomous driving cloud control platform determines whether to enter manual or automatic takeover based on the vehicle's current driving scenario. Manual takeover proceeds to S4, while automatic takeover proceeds to S5. The specific types of scenarios that enter manual takeover and automatic takeover can be freely configured. For example, all known driving scenarios enter automatic takeover, while undefined driving scenarios enter manual takeover. Alternatively, a subset of known driving scenarios with a high probability of automatic takeover are selected for automatic takeover, while other known and undefined driving scenarios enter manual takeover.
[0052] S4: The autonomous driving cloud control platform activates the manual takeover operation platform and sends a manual takeover message to the vehicle. The manual takeover operation platform manually controls the vehicle driving. After the manual takeover process is completed, the vehicle continues autonomous driving and returns to S1.
[0053] Specifically, during the manual takeover process, the vehicle pushes the video surveillance to the manual takeover operation platform. The manual takeover operation platform generates control instructions through manual operation of the vehicle-side simulator, and the control instructions are forwarded to the vehicle by the autonomous driving cloud control platform.
[0054] S5. The autonomous driving cloud control platform activates the automatic takeover cloud platform and sends an automatic takeover message to the vehicle. The vehicle then uploads the sensor information and the current driving scenario type to the automatic takeover cloud platform; then executes S6 to S9.
[0055] Specifically, after receiving the automatic takeover message, the vehicle enters the automatic takeover mode (in the automatic takeover mode, the vehicle indicates that it is in the automatic takeover mode in a certain way, such as turning on the double flash). At this time, the takeover control module forwards the automatic takeover message to the automatic driving main program module. The automatic driving main program module notifies the data upload module to upload the sensor information and the current driving scene type to the automatic takeover cloud platform.
[0056] S6. The automated takeover cloud platform locates the vehicle based on the positioning information in the vehicle sensor information and the pre-recorded static road map. It then constructs a digital twin scene based on the target object information around the vehicle in the vehicle sensor information, and plans a local driving path based on the current driving scene type, and sends the local driving path to the autonomous driving cloud control platform.
[0057] Specifically, the high-precision static road map pre-loaded into the automated takeover cloud platform is used to accurately locate the vehicle, ensuring the accuracy of the vehicle's position information. The vehicle's high-precision positioning information is then combined with information about surrounding objects (including object size, type, relative position to the host vehicle, and speed) and roadside traffic flow information to create a digital twin scenario. Different current driving scenarios correspond to different path planning rules, for example:
[0058] For driving scenarios involving straight-ahead construction on general roads, the strategy of expanding the drivable area is adopted, with the oncoming lane or adjacent solid white lane as the drivable area. This is combined with real-time traffic flow information to plan local driving routes for unmanned vehicles.
[0059] b. For driving scenarios with through traffic control on main roads (road closure ahead), global driving path planning is performed first, followed by selecting a suitable anchor point (the midpoint of the lane after the U-turn) as a reference point for local driving path planning, and then performing local driving path planning for the autonomous vehicle.
[0060] c. For the takeover scenario of through traffic control on the main road (road ahead is closed), global driving path planning is performed first, and then a suitable anchor point (the midpoint of the driving lane after the U-turn) is selected as the reference point for local driving path planning. Finally, local driving path planning is performed for the autonomous vehicle.
[0061] S7. The autonomous driving cloud control platform generates control instructions based on the local driving path and sends the control instructions to the vehicle.
[0062] S8. The vehicle drives according to the control instruction.
[0063] Specifically, while the vehicle is driving according to the control instructions, it feeds back control information to the autonomous driving cloud control platform. The autonomous driving cloud control platform then matches the local driving path with the control instructions. When there is an error between the control information and the local driving path, compensation adjustments are made to ensure the accuracy of the local driving path planning.
[0064] S9: During the automatic takeover process, the vehicle still performs remote takeover condition judgment in real time. When the remote takeover conditions are not met, the vehicle automatically drives again and returns to S1.
[0065] 3 and 4 , a system for remotely taking over an autonomous vehicle for implementing the method for remotely taking over an autonomous vehicle described above includes:
[0066] The vehicle-side automatic driving system includes an automatic driving main program module and a remote takeover program module, wherein the remote takeover program module includes a takeover control module, a takeover judgment module and a data upload module; the takeover judgment module is used to execute the remote takeover condition judgment in S1; the takeover control module is used to send a remote takeover request message and the current driving scene type of the vehicle to the automatic driving cloud control platform, and receive control instructions transmitted by the automatic driving cloud platform; the data upload module is used to transmit sensor information and the current driving scene type to the automatic takeover cloud platform during the automatic takeover process.
[0067] The cloud platform system includes an autonomous driving cloud control platform, a manual takeover operation platform, and an automated takeover cloud platform; the autonomous driving cloud control platform is used to determine the type of takeover process to be executed based on the remote takeover request message uploaded by the vehicle and the current driving scenario type of the vehicle, and activate the corresponding takeover platform to enable communication between the corresponding takeover platform and the vehicle; the manual takeover operation platform and the automated takeover cloud platform respectively execute the manual takeover process and the automatic takeover process when activated to take over the vehicle.
[0068] The manual takeover operation platform includes a vehicle-side simulator and a video display. The operator obtains real-time vehicle driving video through the video display and operates the vehicle-side simulator to generate control instructions.
[0069] The automated takeover cloud platform includes a positioning module, a digital twin module, a trajectory prediction module, a planning and decision-making module, and a path planning rule library; the positioning module stores a high-precision static map of the road surface, the digital twin module constructs a digital twin scene based on the vehicle's surrounding target object information and roadside traffic flow information in the input vehicle sensor information, the trajectory prediction module is used to predict the trajectory of targets around the vehicle, and the planning and decision-making module is used to select the corresponding path planning rules in the path planning rule library based on the current driving scenario type.
[0070] An autonomous driving vehicle adopts the method of remotely taking over operation of an autonomous driving vehicle as described above.
[0071] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for remotely taking over operation of an autonomous driving vehicle, characterized in that: include: When the remote takeover conditions are met, receiving the remote takeover request message sent by the vehicle and the current driving scenario type of the vehicle; According to the current driving scene type of the vehicle, determining whether to enter a manual takeover process or an automatic takeover process; In the manual takeover process, a control command is generated by manually operating the vehicle-side simulator and sent to the vehicle; In the automatic takeover process, the vehicle is positioned and a local driving path is planned, and the local driving path is converted into a control command and sent to the vehicle.
2. The method for remotely taking over operation of an autonomous driving vehicle according to claim 1, characterized in that: The remote takeover conditions specifically include: There is no drivable area for vehicles for T1 consecutive days, or There is no drivable trajectory planning for the vehicle within T2; The T1 and T2 are preset time lengths.
3. The method for remotely taking over operation of an autonomous driving vehicle according to claim 1, characterized in that: The current driving scene type of the vehicle is compared with a preset driving scene type. If the current driving scene type matches one of the preset driving scene types, the current driving scene type is determined to be a known driving scene, otherwise it is an undefined driving scene; When the current driving scene type is an undefined driving scene, the remote takeover request message also includes driving scene information of the current driving scene type.
4. The method for remotely taking over operation of an autonomous driving vehicle according to claim 3, characterized in that: The preset driving scene types include road type, driving status and content of takeover event.
5. The method for remotely taking over operation of an autonomous driving vehicle according to claim 1, characterized in that: The vehicle is positioned according to the sensor information sent by the vehicle in combination with a pre-recorded static road map.
6. The method for remotely taking over operation of an autonomous driving vehicle according to claim 1, characterized in that: According to the current driving scene type of the vehicle, a digital twin scene is constructed, and the local driving path is planned according to the current driving scene type of the vehicle.
7. The method for remotely taking over operation of an autonomous driving vehicle according to claim 6, characterized in that: When constructing the digital twin scenario, roadside traffic flow information is also used.
8. The method for remotely taking over operation of an autonomous driving vehicle according to claim 1, characterized in that: Under the manual takeover process, video surveillance of the vehicle is received.
9. The method for remotely taking over operation of an autonomous driving vehicle according to claim 1, characterized in that: In the automatic takeover process, the control information fed back by the vehicle is received in real time, the local driving path is matched with the control information for consistency, and compensation adjustment is performed when there is an error between the control information and the local driving path.
10. A cloud platform system for implementing the method for remotely taking over operation of an autonomous driving vehicle as described in any one of claims 1 to 9, characterized in that: include: The autonomous driving cloud control platform is used to receive the remote takeover request message sent by the vehicle and the current driving scenario type of the vehicle when the remote takeover conditions are met; According to the current driving scene type of the vehicle, determining whether to enter a manual takeover process or an automatic takeover process; A manual takeover operation platform is used to generate control instructions by manually operating the vehicle-side simulator under the manual takeover process, and send the control instructions to the vehicle through the autonomous driving cloud control platform; The automated takeover cloud platform is used to locate the vehicle and plan a local driving path under the automated takeover process, and the automated driving cloud control platform converts the location into control instructions and sends them to the vehicle.
11. The cloud platform system according to claim 10, characterized in that: The automatic takeover cloud platform locates the vehicle specifically based on the sensor information sent by the vehicle and in combination with a pre-recorded static road map.
12. The cloud platform system according to claim 10, characterized in that: The automated takeover cloud platform constructs a digital twin scene specifically based on the vehicle's sensor information, current driving scene type, and roadside traffic flow information.
13. The cloud platform system according to claim 12, characterized in that: The automated takeover cloud platform predicts the trajectories of objects around the vehicle based on the constructed digital twin scenario.
14. The cloud platform system according to claim 13, characterized in that: The automatic takeover cloud platform selects the corresponding path planning rules in the path planning rule library as the local driving path of the vehicle according to the trajectory prediction and the current driving scene type.
15. A vehicle-side remote takeover program module for implementing the method for remote takeover of an autonomous driving vehicle as claimed in any one of claims 1 to 9, characterized in that: include: A takeover judgment module is used to judge whether the remote takeover conditions are met; The takeover control module is used to send a remote takeover request message and the current driving scene type of the vehicle to the cloud platform system when the remote takeover conditions are met, and receive control instructions transmitted by the cloud platform system to control the driving of the vehicle; The data upload module is used to upload vehicle video surveillance in the manual takeover process, and to upload the vehicle's sensor information and current driving scene type in the automatic takeover process.
Citation Information
Patent Citations
Remote takeover method for autonomous vehicle
CN115593433A
Method and device for remote takeover operation of autonomous vehicle
CN116750010A
Automatic drive function control method, device, electronic apparatus and storage medium
JP2022000376A
Remote driving method, apparatus, and system, device, and medium
US20230030446A1
