A remote monitoring unmanned driving system

Through the remote monitoring of the monitoring platform and takeover platform of the unmanned driving system, combined with high-precision maps and roadside equipment information, the identification result consistency inspection, scenario risk assessment and ODD monitoring are carried out, and the takeover supervision queueing mechanism is adopted to solve the existing technology The problem of insufficient feasibility and effectiveness of vehicle takeover improves the efficiency and safety of multiple vehicle takeovers.

CN114590276BActive Publication Date: 2024-11-12TONGJI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210337054.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-31
Publication Date
2024-11-12
Estimated Expiration
2042-03-31

AI Technical Summary

Technical Problem

The remote monitoring system of existing networked autonomous driving vehicles has insufficient feasibility and effectiveness in environmental perception and takeover decisions, especially when multiple vehicles need to takeover at the same time, it is difficult for the backend technical supervisor to make quick and accurate choices, resulting in The effectiveness of takeover is reduced.

Method used

Design a remote monitoring unmanned driving system, including the field end and the backend, through the monitoring platform, the consistency inspection of identification results, scenario risk assessment, ODD monitoring and remote takeover strategy determination, and adopt the takeover supervision queue mechanism to give priority to driving difficulties For high-quality vehicles, use high-precision maps and roadside equipment information to evaluate driving difficulties, improving the feasibility and effectiveness of takeover.

Benefits of technology

Through the real-time interaction of the monitoring platform and the takeover supervision and queuing mechanism, the feasibility and effectiveness of vehicle takeover are enhanced, labor costs are reduced, and the backend technical supervisors are avoided in case of emergency situations. The safety of vehicle operation is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114590276B_ABST
    Figure CN114590276B_ABST
Patent Text Reader

Abstract

The present invention relates to a remote monitoring unmanned driving system, including a field end (including a vehicle end and a road end) and a backend (including a monitoring platform and a takeover platform) connected to each other, the vehicle end obtains the vehicle end recognition result and transmits it to the monitoring platform; the road end obtains the road end recognition result and transmits it to the monitoring platform; the monitoring platform is used to perform recognition result consistency verification, scene risk assessment, ODD monitoring, determine the remote takeover strategy, and perform driving difficulty assessment on the single vehicle takeover request; the takeover platform is used to generate takeover prompt information and data inspection prompt information, simulate the road environment outside the vehicle in real time, and realize the remote takeover of the vehicle end through interaction with the backend technical supervisor, and run the takeover supervision queuing mechanism. When multiple vehicles need to be taken over, the takeover platform queues the single vehicle takeover request according to the driving difficulty, and takes over the vehicle with high driving difficulty first. Compared with the prior art, the present invention can effectively improve the feasibility and effectiveness of the backend takeover of the vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a remote monitoring unmanned driving system. Background Art

[0002] With the development of intelligent driving technology, the safety requirements for intelligent driving of vehicles are getting higher and higher. At present, intelligent driving mainly includes single-vehicle autonomous driving and networked autonomous driving: single-vehicle intelligent autonomous driving mainly relies on the vehicle's own sensors for environmental perception, calculation and decision-making, and control execution. However, there are technical bottlenecks to varying degrees in environmental perception, calculation and decision-making, and control execution, which leads to various failure problems in the application process;

[0003] The networked autonomous driving, which organically links the "people-vehicle-road-cloud" traffic participation elements, can effectively solve some problems that single-vehicle autonomous driving cannot solve. Among them, remote monitoring is a feasible measure to ensure the safe operation of autonomous driving vehicles. At present, most remote monitoring of networked autonomous driving vehicles requires that autonomous driving vehicles must be driven in designated areas of public roads, must be equipped with background technical supervisors, and always be able to take over remotely at any time. Therefore, in order to ensure the safety of autonomous driving vehicles, background technical supervisors must be equipped to supervise the operation process of autonomous driving vehicles. When necessary, background technical supervisors need to take over the vehicle to ensure driving safety. However, in actual applications, since background technical supervisors need to manually identify and manually determine whether to take over the vehicle, the feasibility of takeover is poor. In addition, when there are multiple vehicles that need to be taken over at the same time, it will also make it difficult for acquired technical supervisors to make choices quickly and accurately, greatly reducing the effectiveness of takeover. Summary of the invention

[0004] The purpose of the present invention is to provide a remote monitoring unmanned driving system in order to overcome the defects of the above-mentioned prior art, so as to improve the feasibility and effectiveness of the back-end takeover of the vehicle.

[0005] The object of the present invention can be achieved by the following technical scheme: a remote monitoring unmanned driving system, comprising a field end and a background connected to each other, the field end comprises a vehicle end and a road end, the background comprises a monitoring platform and a takeover platform, the vehicle end is used to obtain the vehicle end recognition result and transmit it to the monitoring platform; the road end is used to obtain the road end recognition result and transmit it to the monitoring platform;

[0006] The monitoring platform is used to perform consistency verification of recognition results, scenario risk assessment, ODD (Operational Design Domain) monitoring, determine remote takeover strategies, and conduct driving difficulty assessment for single-vehicle takeover requests;

[0007] The takeover platform is used to generate takeover prompt information and data check prompt information; simulate the road environment outside the vehicle in real time and realize remote takeover of the vehicle through interaction with the background technical supervisor; run the takeover supervision queuing mechanism; when multiple vehicles need to be taken over, the takeover platform will queue the single vehicle takeover requests according to the driving difficulty, and give priority to taking over vehicles with high driving difficulty.

[0008] Furthermore, the monitoring platform specifically determines the remote takeover strategy based on the scenario risk assessment results and the ODD monitoring results. The scenario risk assessment results include high-risk scenarios and low-risk scenarios. The ODD monitoring results include the vehicle being outside the ODD range and the vehicle being within the ODD range.

[0009] Furthermore, the remote takeover strategy is specifically: when the vehicle is in a high-risk scenario, the vehicle executes an emergency risk avoidance strategy;

[0010] When the vehicle is in a low-risk scenario, if the vehicle is outside the ODD range, the monitoring platform will request the background technical supervisor to take over the vehicle. If the vehicle is within the ODD range, the on-board automatic driving system will control the vehicle to perform the driving task.

[0011] Furthermore, the monitoring platform specifically maps the vehicle-side recognition results and the road-side recognition results to a high-precision map for matching, and performs a consistency check on the recognition results by judging whether the vehicle-side recognition results are consistent with the road-side recognition results.

[0012] Furthermore, the monitoring platform specifically determines whether the vehicle-side recognition result is consistent with the road-side recognition result based on the type, motion state and target information of the dynamic and static targets.

[0013] Furthermore, the specific process of the monitoring platform performing scenario risk assessment is as follows:

[0014] If the vehicle-side recognition result is inconsistent with the road-side recognition result, the scene risk assessment is performed in combination with the vehicle-side recognition result and the road-side recognition result respectively. As long as either end is high risk, it is judged as high risk, and the monitoring platform sends an emergency risk avoidance instruction to the vehicle-side, and then marks this section of driving data as suspected high-risk scene data, and outputs a data inspection request; if the scene risk assessment results combined with the vehicle-side recognition result and the scene risk assessment combined with the road-side recognition result are both low risk, continue driving, and mark this section of driving data as scene data with abnormal recognition but low risk, and output a data inspection request;

[0015] If the vehicle-side recognition result is consistent with the road-side recognition result, the scene risk assessment will continue. If the scene risk assessment result shows high risk, the monitoring platform will send an emergency risk avoidance instruction to guide the vehicle to perform emergency risk avoidance. If the assessment result shows low risk, the vehicle will continue to drive.

[0016] Furthermore, the monitoring platform performs ODD monitoring based on the scenario risk assessment result. If the scenario risk assessment result is low risk, it further determines whether the vehicle has left the ODD range. If the vehicle has left the ODD range, the monitoring platform generates a single-vehicle takeover request, otherwise the on-board automatic driving system controls the vehicle to continue driving.

[0017] Furthermore, the takeover supervision queuing mechanism is specifically as follows: when two or more vehicles need to be taken over or inspected at the same time, the supervisor is prompted to perform the takeover task first; when two or more vehicles need to be taken over at the same time, the takeover platform will give priority to prompting the supervisor to take over the vehicle with higher driving difficulty based on the driving difficulty.

[0018] Furthermore, the monitoring platform specifically evaluates the driving difficulty based on the roadside equipment status, environmental conditions, road conditions and geographical conditions. The driving difficulty evaluation process is as follows: the vehicle transmits the location information to the monitoring background, and the monitoring background calculates the road condition difficulty score D based on the high-precision map combined with the location information. road and Geographic Difficulty Score D geography ;

[0019] Then, the local weather information is retrieved based on the location information, and the environmental condition difficulty score D is calculated in combination with the vehicle-side perception information. wnvironment ;

[0020] Finally by D road , D geography , D environment The driving difficulty is calculated together.

[0021] Furthermore, the calculation formula of the driving difficulty is:

[0022] D=w roadside (w road D road +w geography D geography +w environment D environment )

[0023] Among them, w roadside is the roadside coefficient. When there is roadside equipment outside the ODD and the equipment is normal, w roadside is a value less than 1. When there is no roadside equipment outside the ODD or the equipment fails, w roadside =1,w road is the road condition coefficient, w geography is the geographical condition coefficient, w environment is the environmental condition coefficient.

[0024] Compared with the prior art, the present invention sets up a field end including a vehicle end and a road end, and a background including a monitoring platform and a takeover platform. The field end is interactively connected with the background, and the road end perception and the vehicle end perception are connected in parallel. The monitoring platform is used to perform consistency verification on the road end recognition results and the vehicle end recognition results. Different response methods are adopted according to the consistency verification results, which can effectively improve the reliability of the recognition results.

[0025] In the present invention, the monitoring platform determines the remote takeover strategy through risk level and ODD limit, avoiding the situation that the background technical supervisor does not have time to take over when an emergency occurs suddenly, thereby enhancing the feasibility of takeover: when the vehicle is in a low-risk scenario, if the vehicle is outside the ODD range, the background monitoring platform requests the background technical supervisor to take over the vehicle, which can reduce the probability of emergency when the vehicle is driving; if the vehicle is within the ODD range, the on-board automatic driving system controls the vehicle to perform the driving task.

[0026] The present invention adopts a takeover supervision queuing mechanism and uses a monitoring platform to evaluate the driving difficulty. Through the takeover supervision queuing, one person can effectively supervise multiple vehicles at the same time, reducing labor costs. By evaluating the driving difficulty, the background technical supervisor is given priority to take over the vehicles with high driving difficulty, avoiding the situation where the supervisor has difficulty in making a quick choice when multiple vehicles need to be taken over at the same time, thereby enhancing the effectiveness of the takeover. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the system architecture of the present invention;

[0028] Figure 2 It is a schematic diagram of part of the operation process of the present invention;

[0029] Figure 3 This is a road collapse scene in the embodiment;

[0030] Figure 4 This is the scene where the lane line disappears in the embodiment. DETAILED DESCRIPTION

[0031] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] Example

[0033] like Figure 1 As shown, a remote monitoring unmanned driving system has two parts: a field end and a background end, wherein: the field end includes a vehicle end and a road end, and the background end includes a background monitoring platform and a background takeover platform.

[0034] The vehicle side is an unmanned vehicle that can transmit the vehicle-side recognition results (including vehicle perception information and vehicle status information) to the background monitoring platform, and the road side includes a variety of roadside perception devices, which are used to obtain the road-side recognition results (including environmental information) and transmit them to the background monitoring platform; the background monitoring platform performs recognition result consistency verification, scene risk assessment, ODD monitoring, determination of remote takeover strategy (including single-vehicle takeover request prediction) and driving difficulty assessment for single-vehicle takeover requests based on the vehicle-side / road-side recognition results (including vehicle-side perception information and roadside information) and high-precision map information sent by the field side, and transmits the single-vehicle takeover request and driving difficulty information to the background takeover platform. When the vehicle cannot avoid an accident through conventional driving strategies, the background monitoring platform will send an emergency avoidance instruction to the vehicle for avoidance operations; the background takeover platform is used to generate takeover prompt information and data inspection prompt information, simulate the road environment outside the vehicle in real time, and realize the takeover of the vehicle through interaction with the background technical supervisor, and implement the takeover supervision queuing mechanism: when there are multiple vehicles that need to be taken over, the takeover requests will be queued according to the driving difficulty, and vehicles with high driving difficulty will be taken over first.

[0035] In practical applications, roadside equipment includes but is not limited to visual cameras, millimeter-wave radars, ultrasonic radars, and lidars.

[0036] Figure 2 It is a schematic diagram of the application process of the present invention, including recognition result consistency check, scenario risk assessment, ODD monitoring and driving difficulty evaluation process; when the unmanned vehicle is driving on the road, the perception and decision planning module on the vehicle side will transmit the vehicle side perception information and vehicle status information to the background monitoring platform in real time, and the vehicle side perception information at least includes the type, motion state, target information, environment and road conditions of dynamic and static targets around the vehicle, and the vehicle status information at least includes the positioning information of the unmanned driving and the motion state of the unmanned vehicle; the roadside perception equipment on the road side will transmit the roadside information to the background monitoring platform in real time, and the roadside information at least includes the type, motion state, target information, environment and road condition information of dynamic and static targets on the road.

[0037] The backend monitoring platform performs a consistency check on the vehicle-side recognition results and the road-side recognition results. Specifically, the backend monitoring platform maps the recognition results to the high-precision map for matching, and determines whether the recognition results are consistent;

[0038] The road-side recognition result information is the information sent by the roadside equipment within a certain range around the unmanned vehicle (determined according to the design perception range of the unmanned vehicle).

[0039] The monitoring platform determines whether the vehicle-side / road-side recognition results are consistent based on the type, motion status, and target information of the dynamic and static targets;

[0040] When the recognition results are inconsistent, the scene risk assessment is performed in combination with the vehicle-side information and the road-side information respectively. As long as one end has a high risk, it is considered to be high risk; specifically, the inconsistent recognition results include one end not being recognized and both ends having recognition results and different recognition results. When the recognition results are inconsistent, the scene risk assessment is performed in combination with the vehicle-side information and the road-side information respectively. As long as one end has a high risk, it is considered to be high risk, and the background monitoring platform sends an emergency risk avoidance instruction to the vehicle, and then marks this section of driving data as suspected high-risk scene data, and outputs a data inspection request; if the scene risk assessments at both ends show low risk, continue driving, mark this section of driving data as recognition abnormal but low-risk scene data, and output a data inspection request;

[0041] Among them, high risk refers to the risk value corresponding to when the vehicle cannot avoid an accident through conventional driving strategies.

[0042] When the recognition results are consistent, the vehicle driving scene is evaluated for scene risk. If the evaluation result shows high risk, the background monitoring platform sends an emergency avoidance instruction to the vehicle. If the evaluation result shows low risk, ODD monitoring is performed.

[0043] Among them, ODD includes but is not limited to roadside equipment status restrictions, environmental condition restrictions, road condition restrictions and geographical condition restrictions;

[0044] For both inconsistent low-risk results and consistent low-risk results, ODD monitoring is performed to determine whether the vehicle has left the ODD. If so, a single-vehicle takeover request is generated. Otherwise, the on-board autonomous driving system controls the vehicle to continue driving.

[0045] In addition, the driving difficulty of the driving task corresponding to the single-vehicle takeover request is evaluated; specifically, the evaluation factors include the status of roadside equipment, environmental conditions, road conditions, and geographical conditions; considering that there may be no roadside equipment or roadside equipment failure outside the ODD, the driving difficulty calculation does not involve roadside information; during the evaluation, the vehicle transmits the location information to the background, and the background calculates the road condition difficulty score D based on the high-precision map combined with the location information road and Geographic Difficulty Score D geography , and retrieve local weather information based on location information, and combine the vehicle-side perception information to comprehensively calculate the environmental condition difficulty score D environment The specific calculation formula of the final driving difficulty D is as follows:

[0046] D=w roadside (w road D road +w geography D geography +w environment D environment )

[0047] Among them, w roadside is the roadside coefficient. When there is roadside equipment outside the ODD and the equipment is normal, w roadside is a value less than 1. When there is no roadside equipment outside the ODD or the equipment fails, e roadside =1,w road is the road condition coefficient, w geography is the geographical condition coefficient, w environment is the environmental condition coefficient.

[0048] When multiple vehicle takeover requests or data check requests are transmitted to the background takeover platform, the background technical supervisor will be prompted to perform the takeover task first; specifically, when two or more vehicles need to be taken over at the same time, the background takeover platform will give priority to prompting the supervisor to take over the vehicles with higher driving difficulty based on the driving difficulty.

[0049] In this embodiment, when the autonomous driving vehicle A travels to a road section where the vehicle-side perception distance is limited due to adverse weather conditions, the number of targets recognized by the vehicle-side is less than the number of targets recognized by the roadside equipment. After the vehicle-side and road-side information are sent to the background monitoring platform, the background monitoring platform first performs a consistency check on the recognition results. The result shows that the recognition results are inconsistent. Then, the scene risk assessment is performed in combination with the vehicle-side information and the road-side information. The scene risk assessment is low risk. Then, ODD monitoring is performed, which shows that the vehicle driving ODD limit is not exceeded. Therefore, this section of driving data is marked as scene data with abnormal recognition but low risk, and a data check request is output;

[0050] like Figure 3 As shown in the figure, the autonomous driving vehicle B is driving on a road section with good weather conditions. However, due to the limitation of the vehicle-side perception performance, vehicle B fails to recognize the road collapse ahead. However, at this time, the roadside equipment recognizes the road collapse. After the vehicle-side and road-side information are sent to the background monitoring platform, the background monitoring platform first performs a consistency check on the recognition results. The result shows that the recognition results are inconsistent. Then, the scene risk assessment is performed by combining the vehicle-side information and the road-side information respectively. Since the scene risk assessment result combined with the road-side information is high risk, an emergency avoidance instruction is sent to vehicle B, and this section of driving data is marked as suspected high-risk scene data, and a data check request is output;

[0051] like Figure 4 As shown in the figure, the autonomous driving vehicle C is driving on a road section with good weather conditions. After the vehicle-side and road-side information are sent to the background monitoring platform, the recognition result consistency check is first performed. Since no lane line is recognized, the recognition results are consistent. Then the scene risk assessment of the vehicle driving scene is performed, and it is assessed as low risk. Then ODD monitoring is performed. Since driving on a road section without lane line markings exceeds the ODD limit, a single vehicle takeover request is generated and sent to the background takeover platform, requesting the driver to take over;

[0052] When cars A, B and C enter the corresponding road section at the same time, the background monitoring platform will take over the supervision according to the takeover and supervision queuing mechanism, and the priority is: car C>car B>car A, that is, car C will be taken over first, then the supervisor will be reminded to check the driving data of car B, and finally the supervisor will be reminded to check the driving data of car A.

Claims

1. A remote monitoring unmanned driving system, It is characterized in that It includes a field end and a backend that are connected to each other. The field end includes a vehicle end and a road end. The backend includes a monitoring platform and a takeover platform. The vehicle end is used to obtain the vehicle end recognition result and transmit it to the monitoring platform. The road terminal is used to obtain the road terminal recognition result and transmit it to the monitoring platform; The monitoring platform is used to perform consistency verification of identification results, scenario risk assessment, ODD monitoring, determine remote takeover strategies, and conduct driving difficulty assessment for single vehicle takeover requests; The takeover platform is used to generate takeover prompt information and data check prompt information; simulate the road environment outside the vehicle in real time and realize remote takeover of the vehicle through interaction with the background technical supervisor; run the takeover supervision queuing mechanism; when multiple vehicles need to be taken over, the takeover platform will queue the single vehicle takeover requests according to the driving difficulty, and give priority to taking over vehicles with high driving difficulty; The specific process of the monitoring platform performing scenario risk assessment is as follows: If the vehicle-side recognition result is inconsistent with the road-side recognition result, the scene risk assessment is performed in combination with the vehicle-side recognition result and the road-side recognition result respectively. As long as either end is high risk, it is judged as high risk, and the monitoring platform sends an emergency risk avoidance instruction to the vehicle-side, and then marks this section of driving data as suspected high-risk scene data, and outputs a data inspection request; if the scene risk assessment results combined with the vehicle-side recognition result and the scene risk assessment combined with the road-side recognition result are both low risk, continue driving, and mark this section of driving data as scene data with abnormal recognition but low risk, and output a data inspection request; If the vehicle-side recognition result is consistent with the road-side recognition result, the scenario risk assessment will continue. If the scenario risk assessment result shows high risk, the monitoring platform will send an emergency risk avoidance instruction to guide the vehicle to take emergency risk avoidance. If the assessment result shows low risk, the vehicle will continue to drive. The takeover supervision queuing mechanism is specifically as follows: when more than two vehicles need to be taken over or inspected at the same time, the supervisor is prompted to perform the takeover task first; when more than two vehicles need to be taken over at the same time, the takeover platform will give priority to the supervisor to take over the vehicle with the highest driving difficulty based on the driving difficulty; The monitoring platform specifically evaluates the driving difficulty based on the roadside equipment status, environmental conditions, road conditions and geographical conditions. The driving difficulty evaluation process is as follows: the vehicle transmits the location information to the monitoring platform, and the monitoring platform calculates the road condition difficulty score D based on the high-precision map combined with the location information. road and Geographic Difficulty Score D geography ; Then, the local weather information is retrieved based on the location information, and the environmental condition difficulty score D is calculated in combination with the vehicle-side perception information. environment ; Finally by D road , D geography , D environment The driving difficulty is calculated together; The calculation formula of the driving difficulty is: D=w roadside (W road D road +w geography D geography +w enuironment D environment ) Among them, w roadside is the roadside coefficient. When there is roadside equipment outside the ODD and the roadside equipment is normal, w roadside is a value less than 1. When there is no roadside equipment outside the ODD or the roadside equipment fails, w roadside =1,w road is the road condition coefficient, w geography is the geographical condition coefficient, w environment is the environmental condition coefficient.

2. A remote monitoring unmanned driving system according to claim 1, It is characterized in that The monitoring platform specifically determines the remote takeover strategy based on the scenario risk assessment results and the ODD monitoring results. The scenario risk assessment results include high-risk scenarios and low-risk scenarios. The ODD monitoring results include the vehicle being outside the ODD range and the vehicle being within the ODD range.

3. A remote monitoring unmanned driving system according to claim 2, It is characterized in that The remote takeover strategy is specifically: when the vehicle is in a high-risk scenario, the vehicle executes an emergency risk avoidance strategy; When the vehicle is in a low-risk scenario, if the vehicle is outside the ODD range, the monitoring platform will request the background technical supervisor to take over the vehicle. If the vehicle is within the ODD range, the on-board automatic driving system will control the vehicle to perform the driving task.

4. A remote monitoring unmanned driving system according to claim 3, It is characterized in that The monitoring platform specifically maps the vehicle-side recognition results and the road-side recognition results to a high-precision map for matching, and performs a consistency check on the recognition results by judging whether the vehicle-side recognition results are consistent with the road-side recognition results.

5. A remote monitoring unmanned driving system according to claim 4, It is characterized in that The monitoring platform specifically determines whether the vehicle-side recognition result is consistent with the road-side recognition result based on the type, motion state and target information of the dynamic and static targets.

6. A remote monitoring unmanned driving system according to claim 3, It is characterized in that The monitoring platform specifically performs ODD monitoring based on the scenario risk assessment results. If the scenario risk assessment result is low risk, it further determines whether the vehicle has left the ODD range. If the vehicle has left the ODD range, the monitoring platform generates a single-vehicle takeover request, otherwise the on-board automatic driving system controls the vehicle to continue driving.