Supervision method and system for autonomous vehicle, operation supervision platform and medium

Abstract

The invention discloses a supervision method and system for an autonomous vehicle, an operation supervision platform and a medium, and relates to the technical field of communication. According to the implementation scheme, in response to monitoring that vehicle data associated with the autonomous vehicle is abnormal, a first video instruction is sent to the autonomous vehicle according to the abnormal moment of the vehicle data; receiving first target data sent by the autonomous vehicle in response to the first video instruction; and supervising the autonomous vehicle based on the first target data and the vehicle data. Therefore, when it is monitored that the vehicle data associated with the autonomous vehicle is abnormal, the operation monitoring platform can actively interact with the autonomous vehicle, and the autonomous vehicle reports the video data in the abnormal time period by issuing the video instruction to the autonomous vehicle. The dimension of the supervision data of the operation supervision platform can be enriched, the transmission time of the video data can be greatly reduced, and the data screening efficiency and analysis efficiency of the operation supervision platform are improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a supervision method, system, operation supervision platform and medium for an autonomous driving vehicle. Background Art

[0002] At present, the operation supervision platform (or cloud control platform) can supervise autonomous vehicles based on the operating status data actively reported by the autonomous vehicles. For example, the autonomous vehicles can be supervised based on conventional data items such as whether the vehicle is online, the vehicle's operating time, mileage, and driving location information.

[0003] However, the dimensions of the regulatory data of the above-mentioned operational regulatory platform are relatively single, which is not conducive to the rapid analysis of problems. Summary of the invention

[0004] The present application provides a supervision method, system, operation supervision platform and medium for an autonomous driving vehicle.

[0005] According to one aspect of the present application, a method for supervising an autonomous driving vehicle is provided, which is applied to an operation supervision platform, and the method includes: in response to monitoring an abnormality in vehicle data associated with the autonomous driving vehicle, sending a first video instruction to the autonomous driving vehicle according to the abnormal time of the vehicle data; wherein the first video instruction carries the abnormal time, a first duration parameter and a second duration parameter; receiving first target data sent by the autonomous driving vehicle in response to the first video instruction; wherein the first target data includes: first video data collected inside and / or outside the cabin of the autonomous driving vehicle within a first time period before the abnormal time, and second video data collected inside and / or outside the cabin within a second time period after the abnormal time; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter; based on the first target data and the vehicle data, supervising the autonomous driving vehicle.

[0006] As a possible implementation, in response to detecting that vehicle data associated with the autonomous driving vehicle is abnormal, sending a first video instruction to the autonomous driving vehicle according to the abnormal time of the vehicle data, including: in response to detecting that the vehicle data associated with the autonomous driving vehicle is abnormal, sending the first video instruction to a multi-mode vehicle-mounted domain controller in the autonomous driving vehicle based on a first communication channel;

[0007] Receiving first target data sent by the autonomous driving vehicle in response to a first video instruction includes: receiving the first target data sent by the multi-mode vehicle domain controller through a second communication channel.

[0008] As a possible implementation, the first communication channel is established by the operation supervision platform using the following steps: sending a first authentication request to the multi-mode vehicle-mounted domain controller; wherein the first authentication request carries a first security certificate of the operation supervision platform; in response to receiving a first authentication pass response sent by the multi-mode vehicle-mounted domain controller, establishing a first communication channel with the multi-mode vehicle-mounted domain controller; wherein the first authentication pass response is generated when the multi-mode vehicle-mounted domain controller authenticates the first security certificate and passes the authentication;

[0009] Correspondingly, the second communication channel is established by the multi-mode vehicle-mounted domain controller using the following steps: sending a second authentication request to the operation supervision platform; wherein the second authentication request carries the second security certificate of the multi-mode vehicle-mounted domain controller; in response to receiving a second authentication pass response sent by the operation supervision platform, establishing a second communication channel with the operation supervision platform; wherein the second authentication pass response is generated when the operation supervision platform authenticates the second security certificate and passes the authentication.

[0010] As a possible implementation method, the number of second communication channels is multiple, and the first target data is sent to the operation supervision platform by the multi-mode vehicle-mounted domain controller using the following steps: determine whether there is a first data record to be transmitted in the local storage file of the multi-mode vehicle-mounted domain controller; if the first data record exists in the storage file, transmit the first data record and the first target data in parallel through multiple second communication channels; if the first data record does not exist in the storage file, transmit the first target data in parallel through multiple second communication channels.

[0011] As a possible implementation method, the timing of writing the first data record to the storage file includes at least one of the following: the multi-mode vehicle-mounted domain controller or the autonomous driving vehicle loses power or has network communication abnormalities during the process of sending the first data record; the multi-mode vehicle-mounted domain controller does not receive a first confirmation response sent by the operation supervision platform within a set time period after sending the first data record; wherein the first confirmation response is used to indicate that the operation supervision platform has received the first data record; after receiving the first confirmation response, the multi-mode vehicle-mounted domain controller receives a write failure message sent by the operation supervision platform; wherein the write failure message is used to indicate that the operation supervision platform has failed to store the first data record.

[0012] As a possible implementation, the method further includes: in response to receiving a first data record, storing the first data record, and sending a second confirmation response to the multi-mode vehicle domain controller; wherein the second confirmation response is used to indicate that the operation supervision platform has received the first data record; in response to receiving the first target data, storing the first target data, and sending a third confirmation response to the multi-mode vehicle domain controller; wherein the third confirmation response is used to indicate that the operation supervision platform has received the first target data; wherein the storage path of the first data record is different from that of the first target data.

[0013] As a possible implementation method, the first data record is provided with first label information, and the first label information is used to mark the first data record as supplementary data. The vehicle data is provided with second label information, and the second label information is used to indicate that the vehicle data is abnormal data. The storage paths of the first data record, the abnormal vehicle data and the first target data are different. Based on the first target data and the vehicle data, the autonomous driving vehicle is supervised, including: querying the supplementary data with the first label information sent by the multi-mode vehicle domain controller from the stored data; querying the abnormal data with the second label information sent by the multi-mode vehicle domain controller from the stored data; and supervising the autonomous driving vehicle based on the supplementary data with the first label information, the abnormal data with the second label information and the first target data.

[0014] As a possible implementation method, after the multi-mode vehicle-mounted domain controller transmits the first data record and the first target data in parallel through multiple second communication channels, it is also used to perform the following operations: determine whether a second confirmation response is received within a set time period after the first data record is transmitted through multiple second communication channels; if the second confirmation response is not received, determine the first supplementary transmission reporting interval according to the number of supplementary transmissions of the first data record; after waiting for the first supplementary transmission reporting interval, re-transmit the first data record through the second communication channel; if the second confirmation response is still not received within the set time period after the retransmission of the first data record, amplify the first supplementary transmission reporting interval to obtain the second supplementary transmission reporting interval; after waiting for the second supplementary transmission reporting interval, transmit the first data record again through the second communication channel until the second confirmation response is received and the set conditions are met, the first data record in the storage file is deleted; wherein the set conditions include: the storage time of the first data record reaches the set time.

[0015] As a possible implementation, a multi-mode vehicle-mounted domain controller is used to receive and store video data and CAN data sent in real time by the autonomous driving vehicle, where the CAN data is the transmission data on the CAN bus of the autonomous driving vehicle;

[0016] The vehicle data associated with the autonomous vehicle includes: video data and / or CAN data sent in real time by a multi-mode vehicle domain controller;

[0017] Among them, CAN data includes at least one of the following: driving data; driving mode; brake system data; fault status data; tire pressure data; steering system data; fuel consumption data.

[0018] As a possible implementation method, the multi-mode vehicle-mounted domain controller adopts the following steps to send video data and / or CAN data to the sent operation supervision platform in real time: determine whether there is a second data record to be transmitted in the local storage file of the multi-mode vehicle-mounted domain controller; if the second data record exists in the storage file, send the second data record to the sent operation supervision platform, and send video data and / or CAN data; if the second data record does not exist in the storage file, send video data and / or CAN data to the sent operation supervision platform.

[0019] As a possible implementation method, the multi-mode vehicle domain controller has a multi-mode communication standard, and the communication standards include: EUHT; 4G; 5G; LTE-V.

[0020] As a possible implementation method, the vehicle data includes CAN data sent by the autonomous driving vehicle, and the vehicle data abnormality includes at least one of the following: CAN data indicates emergency braking of the autonomous driving vehicle; CAN data indicates that the driving speed of the autonomous driving vehicle drops to zero within a set time period, wherein the duration of the set time period is less than a first set threshold; CAN data indicates that the autonomous driving vehicle accelerates urgently; CAN data indicates that at least one system in the autonomous driving vehicle has failed; CAN data indicates that at least one component in the autonomous driving vehicle has failed; CAN data indicates a dashboard warning of the autonomous driving vehicle; CAN data indicates that the autonomous driving vehicle is taken over manually; the difference between the actual mileage of the autonomous driving vehicle indicated by the CAN data and the reference mileage counted by the odometer is greater than a set difference threshold; the transmission success rate of the CAN data is lower than a second set threshold.

[0021] As a possible implementation manner, the vehicle data includes video data sent by the autonomous driving vehicle, and the vehicle data abnormalities include: a transmission success rate of the video data is lower than a third set threshold.

[0022] As a possible implementation, the method also includes: in response to monitoring that the autonomous driving vehicle drives to the intersection, according to the arrival time of the autonomous driving vehicle at the intersection, sending a second video instruction to the roadside unit and the autonomous driving vehicle at the intersection; wherein the second video instruction carries the arrival time, the third duration parameter and the fourth duration parameter; receiving the second target data sent by the roadside unit in response to the second video instruction, and receiving the third target data sent by the autonomous driving vehicle in response to the second video instruction; supervising the autonomous driving vehicle according to the second target data and the third target data; wherein the second target data includes: the third video data collected at the intersection within a third time period before the arrival time, and the fourth video data collected at the intersection within a fourth time period after the arrival time; the duration of the third time period matches the third duration parameter, and the duration of the fourth time period matches the fourth duration parameter; wherein the third target data includes: the fifth video data collected from inside and / or outside the cabin within the third time period, and the sixth video data collected from inside and / or outside the cabin within the fourth time period.

[0023] According to another aspect of the present application, a supervision system for an autonomous driving vehicle is provided, characterized in that it includes:

[0024] Autonomous driving vehicles, used to send vehicle data to the operation supervision platform;

[0025] The operation supervision platform is used to receive vehicle data and, when the vehicle data is abnormal, send a first video instruction to the autonomous driving vehicle according to the abnormal time of the vehicle data; wherein the first video instruction carries the abnormal time, the first duration parameter and the second duration parameter;

[0026] The autonomous driving vehicle is further configured to send first target data to the operation supervision platform in response to the first video instruction; wherein the first target data includes: first video data collected from inside and / or outside the cockpit of the autonomous driving vehicle in a first time period before the abnormal moment, and second video data collected from inside and / or outside the cockpit in a second time period after the abnormal moment; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter;

[0027] The operation supervision platform is also used to supervise the autonomous driving vehicles based on the first target data and vehicle data.

[0028] As a possible implementation method, the operation supervision platform is specifically used to: send a first video instruction to a multi-mode vehicle-mounted domain controller in an autonomous driving vehicle based on a first communication channel; and receive first target data sent by the multi-mode vehicle-mounted domain controller through a second communication channel.

[0029] As a possible implementation, the operation supervision platform is further used to: send a first authentication request to the multi-mode vehicle-mounted domain controller; wherein the first authentication request carries a first security certificate of the operation supervision platform; in response to receiving a first authentication pass response sent by the multi-mode vehicle-mounted domain controller, establish a first communication channel with the multi-mode vehicle-mounted domain controller; wherein the first authentication pass response is generated when the multi-mode vehicle-mounted domain controller authenticates the first security certificate and passes the authentication;

[0030] The multi-mode vehicle-mounted domain controller is also used to: send a second authentication request to the operation supervision platform; wherein the second authentication request carries the second security certificate of the multi-mode vehicle-mounted domain controller; in response to receiving a second authentication pass response sent by the operation supervision platform, establish a second communication channel with the operation supervision platform; wherein the second authentication pass response is generated when the operation supervision platform authenticates the second security certificate and passes the authentication.

[0031] As a possible implementation method, the number of second communication channels is multiple, and the multi-mode vehicle-mounted domain controller is specifically used to: determine whether there is a first data record to be transmitted in the local storage file of the multi-mode vehicle-mounted domain controller; if the first data record exists in the storage file, the first data record and the first target data are transmitted in parallel through multiple second communication channels; if the first data record does not exist in the storage file, the first target data is transmitted in parallel through multiple second communication channels.

[0032] As a possible implementation method, the timing of writing the first data record to the storage file includes at least one of the following: the multi-mode vehicle-mounted domain controller or the autonomous driving vehicle loses power or the network communication is abnormal during the process of sending the first data record; the multi-mode vehicle-mounted domain controller does not receive a first confirmation response sent by the operation supervision platform within a set time period after sending the first data record, wherein the first confirmation response is used to indicate that the operation supervision platform has received the first data record; after receiving the first confirmation response, the multi-mode vehicle-mounted domain controller receives a write failure message sent by the operation supervision platform; wherein the write failure message is used to indicate that the operation supervision platform has failed to store the first data record.

[0033] As a possible implementation method, the operation supervision platform is also used to: in response to receiving a first data record, store the first data record, and send a second confirmation response to the multi-mode vehicle domain controller; wherein the second confirmation response is used to indicate that the operation supervision platform has received the first data record; in response to receiving the first target data, store the first target data, and send a third confirmation response to the multi-mode vehicle domain controller; wherein the third confirmation response is used to indicate that the operation supervision platform has received the first target data; wherein the storage path of the first data record is different from that of the first target data.

[0034] As a possible implementation method, the first data record is provided with first label information, and the first label information is used to mark the first data record as supplementary data; the vehicle data is provided with second label information, and the second label information is used to indicate that the vehicle data is abnormal data; the first data record, the abnormal vehicle data and the first target data have different storage paths in the operation supervision platform; the operation supervision platform is specifically used to: query the supplementary data with the first label information sent by the multi-mode vehicle domain controller from the stored data; query the abnormal data with the second label information sent by the multi-mode vehicle domain controller from the stored data; supervise the autonomous driving vehicle based on the supplementary data with the first label information, the abnormal data with the second label information and the first target data.

[0035] As a possible implementation method, the multi-mode vehicle-mounted domain controller is also used to: determine whether a second confirmation response is received within a set time period after the first data record is transmitted through multiple second communication channels; if the second confirmation response is not received, determine the first supplementary transmission reporting interval based on the number of supplementary transmissions of the first data record; after waiting for the first supplementary transmission reporting interval, re-transmit the first data record through the second communication channel; if the second confirmation response is still not received within the set time period after the re-transmission of the first data record, amplify the first supplementary transmission reporting interval to obtain a second supplementary transmission reporting interval; after waiting for the second supplementary transmission reporting interval, transmit the first data record again through the second communication channel until the second confirmation response is received and the set conditions are met, the first data record in the storage file is deleted; wherein the set conditions include: the storage time of the first data record reaches the set time.

[0036] As a possible implementation, a multi-mode vehicle-mounted domain controller is used to receive and store video data and CAN data sent in real time by the autonomous driving vehicle, where the CAN data is the transmission data on the CAN bus of the autonomous driving vehicle;

[0037] The vehicle data associated with the autonomous vehicle includes: video data and / or CAN data sent in real time by a multi-mode vehicle domain controller;

[0038] Among them, CAN data includes at least one of the following: driving data; driving mode; brake system data; fault status data; tire pressure data; steering system data; fuel consumption data.

[0039] As a possible implementation method, the multi-mode vehicle-mounted domain controller is specifically used to: determine whether there is a second data record to be transmitted in the local storage file of the multi-mode vehicle-mounted domain controller; if the second data record exists in the storage file, send the second data record to the sent operation supervision platform, and send video data and / or CAN data; if the second data record does not exist in the storage file, send video data and / or CAN data to the sent operation supervision platform.

[0040] As a possible implementation method, the multi-mode vehicle domain controller has a multi-mode communication standard, and the communication standards include: EUHT; 4G; 5G; LTE-V.

[0041] As a possible implementation method, the vehicle data includes CAN data sent by the autonomous driving vehicle, and the vehicle data abnormality includes at least one of the following: CAN data indicates emergency braking of the autonomous driving vehicle; CAN data indicates that the driving speed of the autonomous driving vehicle drops to zero within a set time period, wherein the duration of the set time period is less than a first set threshold; CAN data indicates that the autonomous driving vehicle accelerates urgently; CAN data indicates that at least one system in the autonomous driving vehicle has failed; CAN data indicates that at least one component in the autonomous driving vehicle has failed; CAN data indicates a dashboard warning of the autonomous driving vehicle; CAN data indicates that the autonomous driving vehicle is taken over manually; the difference between the actual mileage of the autonomous driving vehicle indicated by the CAN data and the reference mileage counted by the odometer is greater than a set difference threshold; the transmission success rate of the CAN data is lower than a second set threshold.

[0042] As a possible implementation manner, the vehicle data includes video data sent by the autonomous driving vehicle, and the vehicle data abnormalities include: a transmission success rate of the video data is lower than a third set threshold.

[0043] As a possible implementation method, the supervision system also includes: a roadside unit;

[0044] The operation supervision platform is further used to: in response to monitoring that the autonomous driving vehicle drives to the intersection, send a second video instruction to the roadside unit and the autonomous driving vehicle at the intersection according to the arrival time of the autonomous driving vehicle at the intersection, receive second target data sent by the roadside unit in response to the second video instruction and third target data sent by the autonomous driving vehicle in response to the second video instruction, and supervise the autonomous driving vehicle according to the second target data and the third target data;

[0045] The roadside unit is used to receive the second video instruction sent by the operation supervision platform, and send the second target data to the operation supervision platform in response to the second video instruction;

[0046] The autonomous driving vehicle is further used to: receive a second video instruction sent by the operation supervision platform, and send third target data to the operation supervision platform in response to the second video instruction;

[0047] The second video instruction carries the arrival time, the third duration parameter and the fourth duration parameter; the second target data includes: the third video data collected at the intersection within the third time period before the arrival time, and the fourth video data collected at the intersection within the fourth time period after the arrival time; the duration of the third time period matches the third duration parameter, and the duration of the fourth time period matches the fourth duration parameter; the third target data includes: the fifth video data collected inside and / or outside the cabin within the third time period, and the sixth video data collected inside and / or outside the cabin within the fourth time period.

[0048] According to another aspect of the present application, an operation supervision platform is provided, including a memory, a transceiver, and a processor;

[0049] A memory for storing computer programs; a transceiver for sending and receiving data under the control of a processor; and a processor for reading the computer program in the memory and executing any of the aforementioned methods for supervising an autonomous driving vehicle.

[0050] According to another aspect of the present application, a processor-readable storage medium is provided, which stores a computer program, and the computer program is used to enable a processor to execute a supervision method for any of the aforementioned autonomous driving vehicles.

[0051] According to another aspect of the present application, a computer program product is provided, which, when an instruction processor in the computer program product executes, performs a supervision method for any of the aforementioned autonomous driving vehicles.

[0052] The present application has the following technical effects: when the operation monitoring platform detects abnormal vehicle data associated with the autonomous driving vehicle, the operation monitoring platform can actively interact with the autonomous driving vehicle, and by issuing video instructions to the autonomous driving vehicle, allow the autonomous driving vehicle to report the video data within the abnormal period. This can not only enrich the dimensions of the supervision data of the operation supervision platform, but also, the autonomous driving vehicle only reports the video data within the abnormal period, which can greatly reduce the transmission time of the video data and improve the data screening efficiency and analysis efficiency of the operation supervision platform, thereby facilitating the rapid analysis of supervision data, control of the operating status of the autonomous driving vehicle, and rapid location of the cause of the abnormality.

[0053] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0055] Figure 1 It is a schematic diagram of the interaction process between the existing operation supervision platform and the autonomous driving vehicle;

[0056] Figure 2 It is a flowchart of a method for supervising an autonomous driving vehicle provided in an embodiment of the present application;

[0057] Figure 3 It is a flowchart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application;

[0058] Figure 4 It is a flowchart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application;

[0059] Figure 5 It is a flowchart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application;

[0060] Figure 6 It is a flowchart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application;

[0061] Figure 7 It is a schematic diagram of the process of the operation supervision platform providing the embodiment of the present application issuing video instructions and receiving video data;

[0062] Figure 8 It is a schematic diagram of the sending process of the data record to be supplemented provided in the embodiment of the present application;

[0063] Fig. 9 It is a schematic diagram of the interaction process between the autonomous driving vehicle, the multi-mode vehicle-mounted domain controller, the operation and supervision platform, and the smart intersection provided in the embodiment of the present application;

[0064] Fig.10 is a schematic diagram of the structure of a supervision system for an autonomous driving vehicle provided according to an embodiment of the present application;

[0065] Fig.11 It is a structural diagram of an operation supervision platform provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] That is, the term "and / or" in the embodiments of the present application describes the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0068] At present, the operation supervision platform (or cloud control platform) supervises the data of autonomous driving vehicles, mainly relying on the operating status data and video data reported by the autonomous driving vehicles to manage the autonomous driving vehicles. As an example, the interaction process between the operation supervision platform (or cloud control platform) and the autonomous driving vehicle can be as follows: Figure 1 As shown in the figure, the data reported by the autonomous vehicle to the operation supervision platform (or cloud control platform) are basically routine data such as whether the autonomous vehicle is online, operating time, mileage, location information, video data, etc. The operation supervision platform (or cloud control platform) issues instructions to the autonomous vehicle more in the V2N (Vehicle to Network) application scenario, and does not combine the abnormal CAN (Controller Area Network) data of the autonomous vehicle or the autonomous vehicle arriving at the intersection, requiring the upload of instruction video data. The dimensions of supervision data and test evaluation methods are relatively simple.

[0069] Figure 1 In the specification, OBU refers to the abbreviation of On board Unit, and LTE-V refers to the abbreviation of vehicle-to-everything communication technology (LTE (Long Term Evolution) -V2X (Vehicle-to-Everything)).

[0070] The above regulatory approach has at least the following disadvantages:

[0071] 1. The operation supervision platform (or cloud control platform) relies more on conventional data items such as whether the autonomous driving vehicle is online, operating time, mileage, location information, video data, etc. for supervision, and there is no downlink data interaction;

[0072] 2. When an autonomous vehicle is uploading regulatory data, if a network anomaly or the vehicle is powered off causes data upload failure, and there is no mechanism for data retransmission or retransmission, regulatory data is easily lost;

[0073] 3. When an autonomous driving vehicle reports abnormal CAN data, the operation supervision platform (or cloud control platform) only stores the records and does not require the autonomous driving vehicle to upload command video data. The supervision data collection is single, which is not conducive to rapid problem analysis.

[0074] In response to at least one of the above-mentioned problems, the present application provides a supervision method, system, operation supervision platform and medium for an autonomous driving vehicle.

[0075] The following describes the supervision method, apparatus, access network equipment and storage medium of the autonomous driving vehicle of this embodiment with reference to the accompanying drawings.

[0076] Figure 2 It is a flow chart of a method for supervising an autonomous driving vehicle provided in an embodiment of the present application.

[0077] The method for supervising an autonomous driving vehicle according to an embodiment of the present application may be applied to an operation supervision platform.

[0078] like Figure 2 As shown, the supervision method of the autonomous driving vehicle may include the following steps:

[0079] Step S201, in response to monitoring an abnormality in vehicle data associated with the autonomous driving vehicle, sending a first video instruction to the autonomous driving vehicle according to the abnormal moment of the vehicle data; wherein the first video instruction carries the abnormal moment, a first duration parameter and a second duration parameter.

[0080] Among them, the autonomous driving vehicle can be any vehicle with an autonomous driving function.

[0081] The vehicle data includes but is not limited to video data and / or CAN data.

[0082] The first duration parameter is a preset duration parameter. For example, the value of the first duration parameter may be 15s (seconds), 16s, etc.

[0083] The second duration parameter is also a preset duration parameter. For example, the value of the second duration parameter may be 14s, 15s, etc.

[0084] It should be noted that in the embodiments of the present application, there is no restriction on the size relationship between the first duration parameter and the second duration parameter. For example, the value of the first duration parameter may be equal to the value of the second duration parameter, or the value of the first duration parameter may be greater than the value of the second duration parameter, or the value of the first duration parameter may be less than the value of the second duration parameter.

[0085] In an embodiment of the present application, the operation supervision platform can receive vehicle data (including but not limited to video data and / or CAN data) sent by the autonomous driving vehicle, and monitor the vehicle data to determine whether the vehicle data is abnormal. In the case of abnormal vehicle data, a video instruction (referred to as the first video instruction in this application) can be sent to the autonomous driving vehicle according to the abnormal moment of the vehicle data. The first video instruction carries the abnormal moment, the first duration parameter and the second duration parameter.

[0086] Step S202, receiving first target data sent by the autonomous driving vehicle in response to the first video instruction; wherein the first target data includes: first video data collected inside and / or outside the cabin of the autonomous driving vehicle during a first time period before the abnormal moment, and second video data collected inside and / or outside the cabin during a second time period after the abnormal moment.

[0087] In an embodiment of the present application, a camera may be provided inside the autonomous driving vehicle, through which the interior of the cockpit of the autonomous driving vehicle may be photographed (referred to as the internal camera for short), and a camera may also be provided outside the autonomous driving vehicle, through which the exterior of the cockpit of the autonomous driving vehicle may be photographed (referred to as the external camera for short).

[0088] In an embodiment of the present application, when the autonomous driving vehicle receives a first video instruction, it can respond to the first video instruction to obtain first video data collected by an internal camera inside the cabin of the autonomous driving vehicle during a first time period before the abnormal moment, and / or obtain first video data collected by an external camera outside the cabin of the autonomous driving vehicle during a first time period before the abnormal moment.

[0089] In an embodiment of the present application, the autonomous driving vehicle may also obtain second video data collected by an internal camera inside the cabin during a second time period after the abnormal moment, and / or obtain second video data collected by an external camera outside the cabin during a second time period after the abnormal moment.

[0090] Therefore, in this application, the autonomous driving vehicle can generate first target data based on the first video data and / or the second video data, and send the first target data to the operation supervision platform. The duration of the first time period matches or is consistent with the first duration parameter, and the duration of the second time period matches or is consistent with the second duration parameter.

[0091] Step S203: Supervise the autonomous driving vehicle based on the first target data and the vehicle data.

[0092] In an embodiment of the present application, after receiving the first target data sent by the autonomous driving vehicle, the operation supervision platform can supervise the autonomous driving vehicle based on the first target data and abnormal vehicle data.

[0093] For example, regulatory testing and evaluation can be carried out based on the first target data and abnormal vehicle data, and the autonomous driving algorithms in autonomous vehicles can be improved.

[0094] For example, based on the first target data and abnormal vehicle data, the vehicle operation status of the autonomous driving vehicle can be quickly screened, analyzed and controlled.

[0095] The method for supervising an autonomous driving vehicle of an embodiment of the present application, in response to detecting abnormal vehicle data associated with the autonomous driving vehicle, sends a first video instruction to the autonomous driving vehicle according to the abnormal time of the vehicle data; receives the first target data sent by the autonomous driving vehicle in response to the first video instruction; and supervises the autonomous driving vehicle based on the first target data and the vehicle data. Thus, when the operation monitoring platform detects abnormal vehicle data associated with the autonomous driving vehicle, the operation monitoring platform can actively interact with the autonomous driving vehicle, and by sending video instructions to the autonomous driving vehicle, the autonomous driving vehicle can report the video data within the abnormal period, which can not only enrich the dimension of the supervision data of the operation supervision platform, but also the autonomous driving vehicle only reports the video data within the abnormal period, which can greatly reduce the transmission time of the video data, improve the data screening efficiency and analysis efficiency of the operation supervision platform, and thus facilitate the rapid analysis of supervision data, control of the operating status of the autonomous driving vehicle, and rapid location of the cause of the abnormality.

[0096] In order to clearly illustrate the above embodiments, the present application also proposes a supervision method for an autonomous driving vehicle.

[0097] Figure 3 It is a flow chart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application.

[0098] like Figure 3 As shown, the supervision method of the autonomous driving vehicle can be applied to the operation supervision platform, including the following steps:

[0099] Step S301, in response to monitoring an abnormality in vehicle data associated with the autonomous driving vehicle, sending a first video instruction to a multi-mode vehicle-mounted domain controller in the autonomous driving vehicle based on a first communication channel.

[0100] In an embodiment of the present application, the autonomous driving vehicle can communicate or interact with the operation supervision platform through a multi-mode vehicle-mounted domain controller, wherein the multi-mode vehicle-mounted domain controller is used to receive and store video data and CAN data sent by the autonomous driving vehicle in real time, wherein the CAN data is the transmission data on the CAN bus of the autonomous driving vehicle.

[0101] Among them, the multi-mode vehicle-mounted domain controller can have a multi-mode communication standard, wherein the communication standard includes but is not limited to: EUHT (Enhanced Ultra High Throughput); 4G; 5G; LTE-V.

[0102] In an embodiment of the present application, the vehicle data associated with the autonomous driving vehicle may include video data and / or CAN data sent in real time by a multi-mode vehicle domain controller.

[0103] In any embodiment of the present application, in order to ensure safe driving of the autonomous driving vehicle, the CAN data may include but is not limited to at least one of the following:

[0104] The first item is driving data, such as driving speed, driving position, etc.

[0105] The second item is driving mode.

[0106] The third item is brake system data.

[0107] Among them, the brake system data may include: brake disc temperature (since the rising speed and maximum temperature of the brake disc temperature will affect the performance and effect of braking, the brake disc temperature can be recorded and analyzed), brake pedal force (since the size of the brake pedal force will affect the effect of braking, the brake pedal force can be recorded), etc.

[0108] The fourth item is fault status data.

[0109] Among them, fault status data may include: fault code (when an autonomous driving vehicle fails, the on-board equipment will record the fault code. These fault codes can help technicians quickly determine the cause of the failure of the autonomous driving vehicle), fault phenomenon (when an autonomous driving vehicle fails, there may be some obvious phenomena, such as the fault light on the dashboard lighting up, the vehicle cannot start, etc., therefore, these fault phenomena can be recorded), etc.

[0110] Fifth item: tire pressure data.

[0111] Among them, the tire pressure data may include: tire pressure value (the tire pressure value is the basic data in the tire pressure data, including the tire inflation pressure, the maximum inflation pressure of the tire and the rated air pressure of the tire, etc.), tire temperature (since the tire temperature will affect the performance and service life of the tire, the tire temperature can be recorded), etc.

[0112] Item 6: steering system data.

[0113] Among them, the steering system data may include: steering wheel torque (steering wheel torque is one of the basic data in the steering system, indicating the magnitude of the torque acting on the steering wheel at a certain steering angle), steering wheel speed (steering wheel speed indicates the speed at which the steering wheel rotates, which can help analyze the driver's operating behavior and the handling performance of the autonomous driving vehicle), steering gear ratio (steering gear ratio indicates the transmission ratio between the steering gear and the steering shaft, which can help analyze the handling performance and steering sensitivity of the autonomous driving vehicle), steering gear angular transmission ratio (steering gear angular transmission ratio indicates the angular transmission ratio between the steering gear input shaft and the output shaft, which can help analyze the handling performance and steering sensitivity of the autonomous driving vehicle), steering gear oil pressure (steering gear oil pressure indicates the pressure of the oil inside the steering gear, which can help analyze the performance and working status of the steering gear), steering gear speed (steering gear speed indicates the rotation speed of the steering gear, which can help analyze the driver's operating behavior and the handling performance of the autonomous driving vehicle), steering line pressure (steering line pressure indicates the pressure of the oil in the steering line, which can help analyze the sealing and oil circulation of the steering line), steering torque sensor signal (steering torque sensor The sensor is used to detect the torque applied by the driver to the steering wheel. The steering torque sensor signal can help analyze the driver's operating behavior and the handling performance of the autonomous vehicle), the steering angle sensor signal (the steering angle sensor is used to detect the rotation angle of the steering wheel. The steering angle sensor signal can help analyze the driver's operating behavior and the handling performance of the autonomous vehicle), the yaw rate sensor signal (the yaw rate sensor is used to detect the rotation speed of the autonomous vehicle around its longitudinal axis. The yaw rate sensor signal can help analyze the handling performance and driving stability of the autonomous vehicle), the lateral acceleration sensor signal (the lateral acceleration sensor is used to detect the lateral acceleration of the autonomous vehicle. The lateral acceleration sensor signal can help analyze the handling performance and driving stability of the autonomous vehicle), the brake pressure sensor signal (the brake pressure sensor is used to detect the pressure on the brake pedal. The brake pressure sensor signal can help analyze the performance and working status of the brake system), etc., wheel alignment parameters (wheel alignment parameters include toe-in, camber angle, caster angle and other parameters. These parameters can affect the handling performance and driving stability of the autonomous vehicle).

[0114] Item 7: fuel consumption data.

[0115] Among them, fuel consumption data may include: fuel consumption (fuel consumption indicates the amount of fuel consumed by an autonomous driving vehicle over a period of time, which can help analyze the economic performance and fuel efficiency of the autonomous driving vehicle), etc.

[0116] In an embodiment of the present application, when the operation supervision platform detects abnormal vehicle data associated with the autonomous vehicle, a first video instruction can be sent to the multi-mode vehicle domain controller in the autonomous vehicle based on the first communication channel. The first video instruction carries the abnormal time of the vehicle data, the first duration parameter, and the second duration parameter.

[0117] In any embodiment of the present application, the first communication channel may be established by the operation supervision platform using the following steps:

[0118] 1. The operation supervision platform can use the existing communication method to send a first authentication request to the multi-mode vehicle-mounted domain controller; wherein the first authentication request carries the first security certificate of the operation supervision platform.

[0119] 2. After receiving the first authentication request, the multi-mode vehicle domain controller can respond to the first authentication request, authenticate the first security certificate, and determine whether the first security certificate passes the authentication. If the first security certificate passes the authentication, the multi-mode vehicle domain controller can send a first authentication pass response to the operation supervision platform. Correspondingly, when the operation supervision platform receives the first authentication pass response sent by the multi-mode vehicle domain controller, it can establish a first communication channel with the multi-mode vehicle domain controller.

[0120] If the first security certificate fails to pass the authentication, the multi-mode vehicle-mounted domain controller may send a first authentication failure response to the operation supervision platform to refuse the operation supervision platform from establishing a first communication channel with the multi-mode vehicle-mounted domain controller.

[0121] Step S302: receiving first target data sent by the multi-mode vehicle-mounted domain controller through the second communication channel.

[0122] Among them, the first target data includes: first video data collected inside and / or outside the cabin of the autonomous driving vehicle within a first time period before the abnormal moment, and second video data collected inside and / or outside the cabin within a second time period after the abnormal moment; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter.

[0123] In an embodiment of the present application, when the multi-mode vehicle-mounted domain controller receives the first video instruction, it can respond to the first video instruction and send the first target data to the operation supervision platform through the second communication channel.

[0124] In any embodiment of the present application, the second communication channel may be established by the multi-mode vehicle-mounted domain controller using the following steps:

[0125] 1. The multi-mode vehicle-mounted domain controller can use the existing communication method to send a second authentication request to the operation supervision platform; wherein the second authentication request carries the second security certificate of the multi-mode vehicle-mounted domain controller.

[0126] 2. When receiving the second authentication request, the operation supervision platform can respond to the second authentication request, authenticate the second authentication certificate, and determine whether the second security certificate passes the authentication. If the second security certificate passes the authentication, the operation supervision platform can send a second authentication pass response to the multi-mode vehicle domain controller. Correspondingly, when the multi-mode vehicle domain controller receives the second authentication pass response sent by the operation supervision platform, it can establish a second communication channel with the operation supervision platform.

[0127] If the second security certificate fails to pass the authentication, the operation supervision platform may send a second authentication failure response to the multi-mode vehicle domain controller to refuse the multi-mode vehicle domain controller from establishing a second communication channel with the operation supervision platform.

[0128] In summary, before establishing a communication channel, two-way authentication of the operation supervision platform and the multi-mode vehicle domain controller can ensure that both communication nodes are authentic, rather than forged or impersonated, to prevent man-in-the-middle attacks, prevent illegal access, and improve communication security. In addition, through two-way authentication, it can be ensured that each message is one-time and cannot be replayed, and communication delays can be reduced, thereby improving communication efficiency.

[0129] Step S303: Supervise the autonomous driving vehicle based on the first target data and the vehicle data.

[0130] For the explanation of step S303, please refer to the relevant description in any embodiment of the present application, which will not be repeated here.

[0131] In the supervision method of the autonomous driving vehicle in the embodiment of the present application, the operation supervision platform and the multi-mode vehicle-mounted domain controller use different communication channels to send data, which can reduce communication delays and improve communication efficiency.

[0132] In order to clearly illustrate any of the above embodiments of the present application, the present application also proposes a method for supervising an autonomous driving vehicle.

[0133] Figure 4 It is a flow chart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application.

[0134] like Figure 4As shown, the supervision method of the autonomous driving vehicle can be applied to the operation supervision platform, including the following steps:

[0135] Step S401, in response to monitoring an abnormality in vehicle data associated with the autonomous driving vehicle, sending a first video instruction to a multi-mode vehicle-mounted domain controller in the autonomous driving vehicle based on a first communication channel.

[0136] Step S402: receiving first target data sent by the multi-mode vehicle-mounted domain controller through multiple second communication channels.

[0137] The explanation of steps S401 to S402 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0138] In any embodiment of the present application, the multi-mode vehicle-mounted domain controller may adopt the following steps to send the first target data to the operation supervision platform through multiple second communication channels:

[0139] 1. Determine whether there is a first data record to be retransmitted in the local storage file of the multi-mode vehicle-mounted domain controller.

[0140] The timing of writing the first data record to be supplemented into the storage file includes at least one of the following:

[0141] The first item is that the multi-mode vehicle domain controller or the autonomous driving vehicle loses power or has network communication abnormalities during the process of sending the first data record.

[0142] The second item is that the multi-mode vehicle-mounted domain controller does not receive a first confirmation response sent by the operation supervision platform within a set time period after sending the first data record; wherein the first confirmation response is used to indicate that the operation supervision platform has received the first data record.

[0143] Among them, the duration of the set time period is pre-set. For example, if the duration of the marked set time period is N, the multi-mode vehicle-mounted domain controller can determine whether it has received a first confirmation response sent by the operation supervision platform within N seconds after sending the first data record. If so, it indicates that the first data record is sent successfully. At this time, there is no need to use the first data record as data to be supplemented, that is, there is no need to resend the first data record to the operation supervision platform; if not, it indicates that the first data record has failed to be sent. At this time, the first data record can be used as data to be supplemented, that is, the first data record needs to be resent to the operation supervision platform.

[0144] The third item is that after receiving the first confirmation response, the multi-mode vehicle-mounted domain controller receives a write failure message sent by the operation supervision platform; wherein the write failure message is used to indicate that the operation supervision platform has failed to store the first data record.

[0145] That is to say, even if the multi-mode vehicle-mounted domain controller successfully sends the first data record to the operation supervision platform, but the operation supervision platform fails to store the first data record, the multi-mode vehicle-mounted domain controller needs to resend the first data record to the operation supervision platform.

[0146] 2. If the first data record exists in the storage file, the first data record and the first target data are transmitted in parallel through multiple second communication channels.

[0147] 3. If the first data record does not exist in the storage file, the first target data is transmitted in parallel through multiple second communication channels.

[0148] In summary, when the first data record fails to be sent, or the first data record is sent successfully but the operation supervision platform fails to store it, the first data record is resent to the operation supervision platform based on the data supplement or retransmission mechanism, which can avoid the loss of supervision data and ensure the integrity of the supervision data of the autonomous driving vehicle.

[0149] In any embodiment of the present application, after receiving the first data record sent by the multi-mode vehicle-mounted domain controller, the operation supervision platform can store the first data record and send a second confirmation response to the multi-mode vehicle-mounted domain controller; wherein the second confirmation response is used to indicate that the operation supervision platform has received the first data record.

[0150] Similarly, after receiving the first target data sent by the multi-mode vehicle domain controller, the operation supervision platform can store the first target data and send a third confirmation response to the multi-mode vehicle domain controller; wherein the third confirmation response is used to indicate that the operation supervision platform has received the first target data.

[0151] The storage paths of the first data record and the first target data are different. In other words, the storage paths of the supplementary data and the normally transmitted data are different, which is conducive to rapid data search, screening, analysis and control of the operating status of the autonomous driving vehicle.

[0152] In any embodiment of the present application, after the multi-mode vehicle-mounted domain controller transmits the first data record and the first target data to the operation supervision platform in parallel through multiple second communication channels, it may also perform the following operations:

[0153] 1. The multi-mode vehicle-mounted domain controller may determine whether a second confirmation response is received within a set period of time after the first data record is transmitted through multiple second communication channels. If not, step 2 is executed.

[0154] The duration of the set time period is pre-set, for example, the duration of the set time period may be 2 seconds, 3 seconds, 4 seconds, etc.

[0155] If so, it indicates that the first data record is sent successfully. At this time, the first data record in the storage file can be deleted if the set conditions are met.

[0156] The set conditions include: the storage time of the first data record reaches a set time (such as 12 hours, 24 hours, etc.).

[0157] 2. If the second confirmation response is not received, it indicates that the first data record fails to be sent. At this time, the first supplementary transmission reporting interval can be determined according to the number of supplementary transmissions of the first data record.

[0158] For example, the following formula may be used to determine the first supplementary transmission reporting interval according to the number of supplementary transmissions of the first data record: N*T, where N is the number of supplementary transmissions and T is a set duration, for example, T may be 1 second.

[0159] 3. After waiting for the first supplementary transmission reporting interval, retransmit the first data record through the second communication channel.

[0160] 4. If a second confirmation response is received within a set period of time after the first data record is retransmitted, the first data record in the storage file is deleted if a set condition is met; wherein the set condition includes: the storage time of the first data record reaches a set time.

[0161] 5. If the second confirmation response is still not received within the set time period after the first data record is retransmitted, the first supplementary transmission reporting interval is enlarged to obtain a second supplementary transmission reporting interval.

[0162] For example, the time interval between the N+1th supplementary transmission of the first data record and the Nth supplementary transmission of the first data record is (N+1)*T.

[0163] 6. After waiting for the second supplementary transmission reporting interval, transmit the first data record again through the second communication channel until the second confirmation response is received and the set conditions are met, delete the first data record in the storage file; wherein the set conditions include: the storage time of the first data record reaches the set time.

[0164] In summary, after the first data record is resent to the operation supervision platform based on the data retransmission or retransmission mechanism, the first data record will be deleted only when a confirmation response sent by the operation supervision platform is received and the storage time of the first data record reaches the set time. This can further avoid the loss of supervision data and ensure the integrity of the supervision data of the autonomous driving vehicle.

[0165] In any embodiment of the present application, the first data record of supplementary transmission may be provided with first label information, and the first label information is used to mark the first data record as supplementary transmission data. The abnormal vehicle data may be provided with second label information, and the second label information is used to indicate that the vehicle data is abnormal data. The storage paths of the first data record, the abnormal vehicle data and the first target data are different.

[0166] In other words, abnormal data and retransmitted data have special labels and storage paths, which are conducive to rapid data screening, analysis, and control of the operating status of autonomous driving vehicles.

[0167] Step S403, querying the stored data for the supplementary data having the first tag information sent by the multi-mode vehicle-mounted domain controller.

[0168] In an embodiment of the present application, the operation supervision platform can query the supplementary data with the first tag information sent by the multi-mode vehicle-mounted domain controller in the autonomous driving vehicle from the stored data.

[0169] Step S404, querying the stored data for abnormal data having the second tag information sent by the multi-mode vehicle-mounted domain controller.

[0170] In an embodiment of the present application, the operation supervision platform can query the abnormal data with second label information sent by the multi-mode vehicle domain controller in the autonomous driving vehicle from the stored data.

[0171] Step S405: Supervise the autonomous driving vehicle based on the supplementary data with the first label information, the abnormal data with the second label information, and the first target data.

[0172] In an embodiment of the present application, the operation supervision platform can supervise the autonomous driving vehicle based on the supplementary data with first label information, abnormal data with second label information and first target data sent by the multi-mode vehicle-mounted domain controller in the autonomous driving vehicle.

[0173] In the method for supervising an autonomous driving vehicle of the embodiment of the present application, the multi-mode vehicle-mounted domain controller can establish multiple second communication channels for communicating with the operation supervision platform, which can further improve communication efficiency and reduce communication delay. In addition, the operation supervision platform supervises the autonomous driving vehicle based on the abnormal data and retransmission data of the autonomous driving vehicle, which is conducive to rapid data screening, analysis and control of the operating status of the autonomous driving vehicle.

[0174] In order to clearly illustrate any of the above embodiments of the present application, the present application also proposes a method for supervising an autonomous driving vehicle.

[0175] Figure 5It is a flow chart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application.

[0176] like Figure 5 As shown, the supervision method of the autonomous driving vehicle can be applied to the operation supervision platform, including the following steps:

[0177] Step S501, in response to monitoring an abnormality in vehicle data associated with the autonomous driving vehicle, sending a first video instruction to a multi-mode vehicle-mounted domain controller in the autonomous driving vehicle based on a first communication channel; wherein the vehicle data includes video data and / or CAN data sent in real time by the multi-mode vehicle-mounted domain controller.

[0178] The first video instruction carries the abnormal moment of the vehicle data, the first duration parameter and the second duration parameter.

[0179] For the explanation of step S501, please refer to the relevant description in any embodiment of the present application, which will not be repeated here.

[0180] In an embodiment of the present application, the multi-mode vehicle-mounted domain controller can adopt the following steps to send video data and / or CAN data to the operation supervision platform in real time:

[0181] 1. Determine whether there is a second data record to be supplemented in the local storage file of the multi-mode vehicle-mounted domain controller.

[0182] The timing of writing the second data record into the storage file includes at least one of the following:

[0183] First, the multi-mode vehicle domain controller or the autonomous driving vehicle loses power or has network communication abnormalities during the process of sending the second data record.

[0184] The second item is that the multi-mode vehicle-mounted domain controller does not receive a fourth confirmation response sent by the operation supervision platform within a set time period after sending the second data record; wherein the fourth confirmation response is used to indicate that the operation supervision platform has received the second data record.

[0185] The third item is that after receiving the fourth confirmation response, the multi-mode vehicle domain controller receives a write failure message sent by the operation supervision platform; wherein the write failure message is used to indicate that the operation supervision platform has failed to store the second data record.

[0186] 2. If there is a second data record in the storage file, the second data record can be sent to the operation supervision platform through the second communication channel, and the video data and / or CAN data can be sent.

[0187] 3. If the second data record does not exist in the storage file, the video data and / or CAN data may be sent to the operation supervision platform via the second communication channel.

[0188] In summary, during the process of the multi-mode vehicle domain controller uploading regulatory data, if a network anomaly or vehicle power failure occurs, resulting in a failure in uploading the regulatory data, the multi-mode vehicle domain controller automatically stores the failed uploaded data locally, and after the link is restored to normal or the vehicle is powered on, based on the data retransmission or retransmission mechanism, it actively triggers the sending of the failed uploaded data (i.e., retransmission data) to the operation and supervision platform. After waiting for confirmation and feedback from the operation and supervision platform, the multi-mode vehicle domain controller deletes the locally stored retransmission data, which can effectively ensure the integrity of the regulatory data of the autonomous driving vehicle.

[0189] Step S502: receiving first target data sent by the multi-mode vehicle-mounted domain controller through the second communication channel.

[0190] Among them, the first target data includes: first video data collected inside and / or outside the cabin of the autonomous driving vehicle within a first time period before the abnormal moment, and second video data collected inside and / or outside the cabin within a second time period after the abnormal moment; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter.

[0191] Step S503: Supervise the autonomous driving vehicle based on the first target data and the vehicle data.

[0192] The explanation of steps S502 to S503 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0193] In the supervision method of the autonomous driving vehicle in the embodiment of the present application, the operation supervision platform can store the video data and / or CAN data sent in real time by the multi-mode vehicle-mounted domain controller in the autonomous driving vehicle, which can improve the richness of the supervision data, effectively enhance the supervision of the autonomous driving vehicle, and help reduce the probability of traffic accidents.

[0194] In order to clearly illustrate any of the above embodiments of the present application, the present application also proposes a method for supervising an autonomous driving vehicle.

[0195] Figure 6 It is a flow chart of another method for supervising an autonomous driving vehicle provided in an embodiment of the present application.

[0196] like Figure 6 As shown, the supervision method of the autonomous driving vehicle can be applied to the operation supervision platform, including the following steps:

[0197] Step S601, in response to monitoring an abnormality in vehicle data associated with the autonomous driving vehicle, sending a first video instruction to the autonomous driving vehicle according to the abnormal time of the vehicle data.

[0198] The first video instruction carries the abnormal moment, the first duration parameter and the second duration parameter.

[0199] For the explanation of step S601, please refer to the relevant description in any embodiment of the present application, and will not be repeated here.

[0200] In any embodiment of the present application, when the vehicle data includes CAN data sent by an autonomous driving vehicle or a multi-mode vehicle-mounted domain controller, the vehicle data anomaly may include at least one of the following:

[0201] First, CAN data indicates emergency braking of the autonomous vehicle.

[0202] The second item is that the CAN data indicates that the driving speed of the autonomous driving vehicle drops to zero within a set time period, wherein the duration of the set time period is less than the first set threshold value. That is, the speed of the autonomous driving vehicle drops to zero within a short period of time.

[0203] The third item is that CAN data indicates that the autonomous driving vehicle accelerates urgently.

[0204] Fourth, CAN data indicates that at least one system in the autonomous vehicle has failed.

[0205] Fifth, CAN data indicates that at least one component in the autonomous vehicle has failed.

[0206] Sixth, CAN data indicates dashboard warnings for autonomous vehicles.

[0207] Item 7: CAN data indicates that the autonomous driving vehicle has been taken over manually.

[0208] Item 8: The difference between the actual mileage of the autonomous driving vehicle indicated by the CAN data and the reference mileage counted by the odometer is greater than the set difference threshold. That is, the mileage difference is large.

[0209] Item 9: The transmission success rate of CAN data is lower than a second set threshold.

[0210] In any embodiment of the present application, when the vehicle data includes video data sent by an autonomous driving vehicle, the vehicle data anomaly may include: a transmission success rate of the video data is lower than a third set threshold.

[0211] There is no restriction on the size relationship between the second set threshold and the third set threshold. For example, the second set threshold may be equal to the third set threshold, or the second set threshold may be greater than the third set threshold, or the second set threshold may be less than the third set threshold.

[0212] Step S602, receiving first target data sent by the autonomous driving vehicle in response to the first video instruction.

[0213] Among them, the first target data includes: first video data collected inside and / or outside the cabin of the autonomous driving vehicle within a first time period before the abnormal moment, and second video data collected inside and / or outside the cabin within a second time period after the abnormal moment; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter.

[0214] Step S603: Supervise the autonomous driving vehicle based on the first target data and the vehicle data.

[0215] The explanation of steps S602 to S603 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0216] Step S604, in response to monitoring the autonomous driving vehicle driving to the intersection, a second video instruction is sent to the roadside unit and the autonomous driving vehicle at the intersection according to the arrival time of the autonomous driving vehicle at the intersection.

[0217] It should be noted that in the embodiment of the present application, there is no restriction on the size relationship between the first duration parameter, the second duration parameter, the third duration parameter and the fourth duration parameter.

[0218] It should also be noted that there is no restriction on the execution sequence of step S604. In the embodiment of the present application, step S604 is only executed after step S603 as an example. In actual application, step S604 can also be executed before step S601, or step S604 can also be executed in parallel with step S601, and so on.

[0219] In the embodiment of the present application, when the operation supervision platform detects that the autonomous driving vehicle is driving to a certain intersection, it can send a second video instruction to the roadside unit and the autonomous driving vehicle at the intersection at the same time according to the time when the autonomous driving vehicle arrives at or passes through the intersection (recorded as the arrival time in this application). The second video instruction carries the arrival time, the third duration parameter and the fourth duration parameter.

[0220] Step S605, receiving the second target data sent by the roadside unit in response to the second video instruction, and receiving the third target data sent by the autonomous driving vehicle in response to the second video instruction.

[0221] Among them, the second target data includes: the third video data collected by the roadside unit at the intersection within the third time period before the arrival time, and the fourth video data collected at the intersection within the fourth time period after the arrival time; wherein, the duration of the third time period matches or is consistent with the third duration parameter, and the duration of the fourth time period matches with the fourth duration parameter.

[0222] Among them, the third target data includes: the fifth video data collected by the autonomous driving vehicle inside and / or outside the cabin during the third time period, and the sixth video data collected by the autonomous driving vehicle inside and / or outside the cabin during the fourth time period.

[0223] In an embodiment of the present application, when the roadside unit receives the second video instruction, it can respond to the second video instruction to obtain the third video data collected by the roadside unit at the intersection during the third time period before the arrival time, and the fourth video data collected at the intersection during the fourth time period after the arrival time, so that the second target data can be generated based on the third video data and the fourth video data, and the second target data can be sent to the operation supervision platform.

[0224] Similarly, when the autonomous driving vehicle receives the second video instruction, it can respond to the second video instruction to obtain the fifth video data collected by the autonomous driving vehicle inside and / or outside the cabin during the third time period, and the sixth video data collected inside and / or outside the cabin during the fourth time period, so that it can generate third target data based on the fifth video data and the sixth video data, and send the third target data to the operation supervision platform.

[0225] Correspondingly, the operation supervision platform can receive the second target data sent by the roadside unit in response to the second video instruction, and receive the third target data sent by the autonomous driving vehicle in response to the second video instruction.

[0226] Step S606: Supervise the autonomous driving vehicle based on the second target data and the third target data.

[0227] In an embodiment of the present application, the operation supervision platform can supervise the autonomous driving vehicle based on the second target data and the third target data.

[0228] In the method for supervising the autonomous driving vehicle of the embodiment of the present application, when the operation supervision platform detects that the autonomous driving vehicle arrives at or passes through the intersection, it simultaneously sends a second video instruction to the autonomous driving vehicle and the roadside unit, and actively obtains the video data of the autonomous driving vehicle (referred to as vehicle video data) and the video data of the intersection (referred to as intersection video data), which can not only realize real-time monitoring of whether the autonomous driving vehicle violates traffic regulations such as speeding, driving on the line, driving in the wrong direction, running a red light, etc. at the intersection, but also greatly reduce the transmission time of video data and data screening efficiency, which is conducive to the rapid analysis of supervision data and the control of the operating status of the autonomous driving vehicle, so that when the autonomous driving vehicle encounters a traffic accident or violation, it can quickly find the corresponding data analysis, and provide effective data support for the optimization and improvement of the autonomous driving algorithm. In addition, the operation supervision platform has CAN data at abnormal times, and instruction video data from the vehicle and road ends, and collects more autonomous driving data on actual roads, which can enrich the autonomous driving technology database, facilitate the training and learning of subsequent autonomous driving models, and continuously improve the autonomous driving algorithm.

[0229] In summary, the current autonomous driving technology is not yet mature, and autonomous driving vehicles are prone to violations such as driving in the wrong direction, driving on the line, running red lights, occupying non-motorized vehicle lanes, etc., which affect traffic order. In order to achieve better multi-dimensional and comprehensive evaluation and supervision of the operating data of autonomous driving vehicles, this application mainly proposes a supervision method and system for autonomous driving vehicles, requiring autonomous driving vehicles to upload in-vehicle video data, out-of-vehicle video data and vehicle CAN data to the operation supervision platform. The operation supervision platform monitors the operation data of autonomous driving vehicles in real time. When the autonomous driving vehicle reports abnormal vehicle data (such as sudden braking, fault warning, manual takeover, large mileage difference, low data transmission success rate and other abnormal phenomena) or the autonomous driving vehicle arrives at the intersection, the operation supervision platform actively issues a video command, requiring the autonomous driving vehicle or the roadside unit at the intersection to upload the video data before the operation supervision platform issues the video command. The video data of the T1 period (15 seconds by default) and the video data of the T2 period (15 seconds by default) after the operation and supervision platform issues the video command ensure that the short video data collected by the autonomous driving vehicle or the roadside unit at the intersection within 30 seconds is quickly uploaded to the operation and supervision platform. On the one hand, it can greatly reduce the transmission time of video data and improve the efficiency of data screening, which is conducive to the rapid analysis of supervision data and the control of the operating status of autonomous driving vehicles; on the other hand, the operation and supervision platform has both CAN data and command video data at abnormal times, collects more autonomous driving data on actual roads, enriches the autonomous driving database, and is conducive to the subsequent training and learning of autonomous driving models, and continuously improves the autonomous driving algorithm.

[0230] In addition, for the supervision data being uploaded by the autonomous driving vehicle, if the network is abnormal or the autonomous driving vehicle is powered off, resulting in data upload failure, the multi-mode vehicle-mounted domain controller in the autonomous driving vehicle automatically stores the failed upload data locally, and after the link is restored to normal or the device is powered on, based on the data retransmission or retransmission mechanism, actively triggers data retransmission to the operation supervision platform, and after the operation supervision platform confirms the feedback and the data storage time reaches the set time, deletes the locally stored retransmission data, which can effectively ensure the integrity of the supervision data of the autonomous driving vehicle. The use of this system and method can effectively enhance the supervision of autonomous driving vehicles, help reduce the probability of traffic accidents, and is also the development trend of the future autonomous driving vehicle operation management technology system, and continuously improves and builds the regulatory system for autonomous driving supervision.

[0231] As an example, the supervision system of an autonomous driving vehicle may include key parts such as an autonomous driving vehicle, a multi-mode vehicle-mounted domain controller, a smart intersection, and an operation supervision platform. Among them, the autonomous driving vehicle deploys a multi-mode vehicle-mounted domain controller, and the autonomous driving vehicle has no less than 2 channels of video acquisition data, including at least 1 channel of in-vehicle video data and 1 channel of out-of-vehicle video data. The autonomous driving vehicle uploads CAN data (i.e., CAN bus data) and video data to the multi-mode vehicle-mounted domain controller in real time.

[0232] Among them, the multi-mode vehicle-mounted domain controller has hardware capabilities such as multiple heterogeneous network communication standards (including but not limited to EUHT, 4G, 5G, LTE-V, etc.), high processing power, and hardware encryption; at the software level, it has application algorithms such as regulatory data processing, video data processing, instruction video processing, data retransmission and supplementary transmission, and security protocols. These algorithms can ensure the timeliness and integrity of data processing, thereby meeting the data requirements of high-standard operation and supervision platforms.

[0233] Among them, the RSU (Road Side Unit) of the smart intersection is also required to have functions such as command video processing, data retransmission and supplementary transmission, and respond to the instructions issued by the operation and supervision platform in real time.

[0234] Among them, in order to improve the security of data transmission and interaction, the multi-mode vehicle-mounted domain controller can be based on TCP (Transmission Control Protocol) SSL (Secure Socket Layer) protocol two-way authentication, use the operation supervision platform to issue the client's security certificate, and establish two TCP connection communications with the operation supervision platform, one as a reporting channel for high-frequency real-time data, and the other as a channel for issuing various vehicle-cloud function instructions.

[0235] Among them, the operation supervision platform can detect the vehicle data (including CAN data and / or video data) of the autonomous driving vehicle reported by the multi-mode vehicle domain controller in real time as the supervision data. When the autonomous driving vehicle reports abnormal vehicle data (including emergency braking, continuous rapid deceleration, fault warning, manual takeover, large mileage difference, low success rate of data transmission and other abnormal behavior phenomena), or when the autonomous driving vehicle arrives at the intersection, the operation supervision platform actively issues the corresponding video command, requiring the multi-mode vehicle domain controller to upload the video data of the T1 period (default 15 seconds) before the operation supervision platform issues the video command and the video data of the T2 period (default 15 seconds) after the operation supervision platform issues the video command, to ensure that the short video data collected by the autonomous driving vehicle within 30 seconds is quickly uploaded to the operation supervision platform. Among them, T1 and T2 can be configured based on the scene.

[0236] Among them, in order to achieve real-time monitoring of whether the autonomous driving vehicle has speeding, driving on the line, driving in the wrong direction, running red lights and other violations of traffic regulations, the operation and supervision platform can simultaneously send video instructions to the multi-mode vehicle-mounted domain controller and the roadside unit when monitoring the autonomous driving vehicle arriving at the intersection, and actively obtain vehicle video data and intersection video data. The multi-mode vehicle-mounted domain controller and the roadside unit also decompose the planned uploaded video data into two time periods of T3+T4, that is, the video data of the T3 period before the video instruction is issued and the video data of the T4 period after the video instruction is issued. The default is 15 seconds before and after, a total of 30 seconds of video data, among which T3 and T4 can be configured based on the scene. The benefits of this implementation include: on the one hand, it greatly reduces the transmission time and data screening efficiency of video data of abnormal behavior, which is conducive to the rapid analysis of supervision data and the control of the operating status of autonomous driving vehicles; on the other hand, the operation and supervision platform has CAN data at abnormal times, vehicle-side and road-side instruction video data, collects more autonomous driving data on actual roads, enriches the autonomous driving technology database, and is conducive to the subsequent training and learning of autonomous driving models, and continuously improves autonomous driving algorithms.

[0237] In addition, in order to ensure the integrity of the supervision data of the operation supervision platform, the multi-mode vehicle domain controller needs to have a data loss prevention function. For the supervision data being uploaded by the multi-mode vehicle domain controller, if the network is abnormal or the vehicle is powered off, resulting in data upload failure, the multi-mode vehicle domain controller automatically stores the failed uploaded data locally, and after the link is restored to normal or the device is powered on, based on the data retransmission or retransmission mechanism, it actively triggers data retransmission to the operation supervision platform, and after the operation supervision platform confirms the feedback, the multi-mode vehicle domain controller deletes the locally stored retransmission data, which can effectively ensure the integrity of the supervision data of the autonomous driving vehicle. The use of this system and method can effectively enhance the supervision of autonomous driving vehicles and help reduce the probability of traffic accidents.

[0238] As an example, the various functional modules and their functions in the supervision system of an autonomous driving vehicle may be:

[0239] 1. Autonomous driving vehicles.

[0240] Among them, the autonomous driving vehicle has the function of collecting multi-channel video data, and reports at least one channel of in-vehicle video data, at least one channel of outside-vehicle video data, and the CAN data of the autonomous driving vehicle to the multi-mode vehicle domain controller in real time.

[0241] 2. Multi-mode vehicle domain controller.

[0242] Among them, the hardware of the multi-mode vehicle-mounted domain controller has multi-mode communication standards (including but not limited to: EUHT, 4G, 5G, LTE-V, etc.), hardware encryption, high-precision positioning, computing power of not less than 4TOPS (Tera Operations Per Second), etc.; the software has application functions such as regulatory data processing, video data processing, instruction video processing, data retransmission and supplementary transmission, and security protocols.

[0243] 3. Smart intersection.

[0244] We can use RSU, MEC (Mobile Edge Computing), lidar, millimeter-wave radar, cameras, signs and other traffic infrastructure to build a holographic perception smart intersection traffic situation system to realize V2I (Vehicle to Infrastructure) or V2N (Vehicle to Network) functions. RSU also needs to have functions such as command video processing, data retransmission and supplementary transmission to respond to instructions issued by the operation supervision platform.

[0245] 4. Operation and supervision platform.

[0246] The operation supervision platform has functions such as vehicle operation management (including CAN data supervision, video data supervision, command video supervision), roadside operation management (including roadside data supervision, intersection video data supervision, command video supervision) and command control. It can conduct multi-dimensional analysis, display and control of the operation status data of autonomous driving vehicles in real time.

[0247] As an example, the implementation process of the supervision method of autonomous driving vehicles may include the following steps:

[0248] 1. Data collection process.

[0249] 1.1. The internal camera of the autonomous vehicle needs to collect video data including the driver or safety officer’s head, hands, steering wheel, co-pilot, and back seat;

[0250] 1.2. The external cameras of the autonomous vehicle need to ensure that the shooting picture is centered and can monitor the front of the engine hood, the left and right edges, and the road in front;

[0251] 1.3. CAN data of autonomous vehicles, including but not limited to: vehicle speed data, driving mode, brake system data, fault status data, tire pressure data, steering system data, fuel consumption data, etc., are mainly used to obtain the operating status, driving data, fault status, etc. of autonomous vehicles, so as to supervise autonomous vehicles and ensure the safe driving of autonomous vehicles.

[0252] 2. Data interaction process.

[0253] 2.1. The autonomous driving vehicle periodically sends data to the multi-mode vehicle domain controller. The data communication frequency can be 10 Hz (Hertz). Considering that the autonomous driving vehicle and the multi-mode vehicle domain controller are directly connected at the near end and the interactive communication of high-frequency real-time data is carried out, the UDP (User Datagram Protocol) protocol is recommended to reduce the waiting delay of the confirmation response (ie, ACK (Acknowledge)); the TCP protocol can also be supported.

[0254] 2.2. The data interaction between the multi-mode vehicle domain controller and the operation supervision platform takes into account the data communication security. Based on the TCP SSL protocol two-way authentication, the client security certificate issued by the operation supervision platform can be used. The multi-mode vehicle domain controller and the operation supervision platform establish two TCP connections, one as the reporting channel for high-frequency real-time data, and the other as the issuing channel for implementing various vehicle-cloud function instructions.

[0255] 3. Data supervision process.

[0256] Among them, the operation supervision platform has the ability to supervise the real-time operation status, location information, driving data, mileage difference (the deviation between the kilometers counted by the autonomous driving vehicle and the kilometers counted by the operation supervision platform), driving mode, fault status, etc. of the autonomous driving vehicle. Once abnormal vehicle data (including CAN data, supervision platform evaluation indicators, etc.) is detected or the autonomous driving vehicle arrives at the intersection, it will actively send a video command, requiring the multi-mode vehicle domain controller to upload the vehicle or the RSU at the intersection to upload the video data of a period of time before and after the operation supervision platform sends the video command, accurately record the short video data of the abnormal period (such as 30 seconds), greatly reduce the transmission time of the video data of abnormal behavior and the efficiency of data screening, which is conducive to the rapid analysis of supervision data and control of the operation status of autonomous driving vehicles. The specific implementation process is as follows:

[0257] 3.1. Vehicle data anomalies: When the operation supervision platform detects that the CAN data reported by the autonomous driving vehicle contains abnormal phenomena such as sudden braking, fault warning, and manual takeover, it will immediately send a video command to the multi-mode vehicle domain controller to actively obtain the video data collected by the autonomous driving vehicle. The multi-mode vehicle domain controller will decompose the video data planned to be uploaded into two time periods, T1+T2, that is, the video data of the T1 period before the video command is issued and the video data of the T2 period after the video command is issued. The default is 15 seconds before and after, for a total of 30 seconds of video data. T1 and T2 can be configured based on the scenario.

[0258] 3.2. Autonomous driving vehicles arrive at intersections: In order to monitor in real time whether autonomous driving vehicles violate traffic regulations such as speeding, driving on the line, driving in the wrong direction, running red lights, etc., the operation and supervision platform can simultaneously send video instructions to the multi-mode vehicle domain controller and the roadside unit when monitoring the autonomous driving vehicle arriving at the intersection, and actively obtain vehicle video data and intersection video data. The multi-mode vehicle domain controller and the roadside unit also decompose the planned uploaded video data into two time periods of T3+T4, that is, the video data of the T3 period before the video instruction is issued and the video data of the T4 period after the video instruction is issued. The default is 15 seconds before and after, for a total of 30 seconds of video data. T3 and T4 can be configured based on the scenario.

[0259] In one example, the process of the operation supervision platform issuing video instructions and receiving video data can be as follows: Figure 7 shown.

[0260] 3.3. The evaluation indicators of the operation supervision platform shall include at least the following indicators:

[0261] Mileage difference: The mileage difference between the kilometers counted by the operation supervision platform based on CAN data and the kilometers counted by the odometer of the autonomous driving vehicle is less than 1%, meeting the supervision requirements of autonomous driving companies and operation supervision platforms;

[0262] Data transmission success rate: For regulatory data items, the data transmission success rate (number of data uploaded successfully / number of data to be sent) shall not be less than 99%;

[0263] Success rate of video transmission: A regulatory data item, the success rate of video transmission (the number of successfully uploaded videos / the number of instructions issued by the operation and supervision platform) shall not be less than 95%.

[0264] It should be noted that the above evaluation indicators can be appropriately adjusted and improved based on the maturity of autonomous driving technology and the big data analysis collected by the operation supervision platform as a regulatory test evaluation system.

[0265] 4. Supervise the data integrity process.

[0266] 4.1. Rules for retransmission of regulatory data: During the process of the multi-mode vehicle domain controller reporting regulatory data to the operation and supervision platform, if a network anomaly occurs, resulting in upload failure or operation and supervision platform writing failure, the data record can be written to a local storage file. The total number of locally stored data records to be retransmitted does not exceed the set number threshold (such as 400,000). If the total number of data records to be retransmitted exceeds the set number threshold, rolling deletion is performed, that is, the first data record written is deleted and a new data record is added.

[0267] 4.2. The process of retransmission of regulatory data:

[0268] Autonomous driving vehicle side (i.e. vehicle side): After each successful connection between the multi-mode vehicle-mounted domain controller and the operation and supervision platform, it will detect whether there are data records to be re-transmitted locally. If so, it will apply for re-transmission. If not, it will be skipped. After receiving the confirmation response sent by the operation and supervision platform, the vehicle side will remove the data record corresponding to the local file. If the vehicle side does not receive the confirmation response sent by the operation and supervision platform within 3 seconds after sending the re-transmitted data record, it will enter a timeout.

[0269] For example, when the vehicle side does not receive a confirmation response, the vehicle side adjusts the supplementary transmission reporting interval, and the interval between the Nth supplementary transmission attempt and the N-1th supplementary transmission attempt is N*T. After receiving the confirmation response, the supplementary transmission reporting interval is restored to T, where the recommended value of T can be 1s.

[0270] Operation supervision platform (i.e. cloud): after receiving the supplementary data uploaded by the vehicle, the status of the supplementary data is fed back to the vehicle, and the data is unpacked, parsed and stored into the database according to the agreed number of supplementary data in the data packet. The storage path is different from the real-time supervision data, which is convenient for searching.

[0271] 4.3. Video data retransmission: The multi-mode vehicle domain controller divides the uploaded video data into a real-time video data channel and a video data retransmission channel. Multiple communication channels work in parallel to ensure data integrity. When the following abnormal behaviors occur, the multi-mode vehicle domain controller actively starts the video data retransmission mechanism to ensure the success rate of video data upload. It is recommended that the number of videos retransmitted at the same time should not exceed 8.

[0272] As an example, when the multi-mode vehicle-mounted domain controller detects at least one of the following conditions, the following Figure 8 The process shown in the figure re-sends the data records to be re-transmitted to the operation supervision platform:

[0273] (1) When the multi-mode vehicle domain controller is uploading video data, the vehicle is turned off, the video transmission is interrupted, or the device is restarted;

[0274] (2) When the multi-mode vehicle-mounted domain controller is uploading video data, the network upload rate is low, the upload fails within the specified time (such as within 3 minutes, which is configurable), and the operation supervision platform actively disconnects the link.

[0275] As an example, the interaction process between the autonomous driving vehicle, the multi-mode vehicle domain controller, the operation supervision platform and the smart intersection can be as follows: Fig. 9 shown.

[0276] In summary, in the supervision system of the autonomous driving vehicle provided by this application, each functional module has at least the following characteristics:

[0277] 1) The multi-mode vehicle-mounted domain controller has innovative algorithms such as supervision data processing, video data processing, command video processing, data retransmission and supplementary transmission, and security protocols; the RSU also has command video processing, data retransmission and supplementary transmission functions based on the arrival of the autonomous driving vehicle at the intersection, and responds to the instructions issued by the operation supervision platform and other innovative algorithms; the operation supervision platform interacts with the autonomous driving vehicle and the roadside unit, and the supervision dimensions are more extensive and comprehensive, which is conducive to big data collection and analysis, and can be used for the test and evaluation of autonomous driving technology;

[0278] 2) The multi-mode vehicle domain controller has multi-mode communication standards (EUHT, 4G, 5G, LTE-V, etc., especially EUHT), highly integrated 5-in-1 antenna, supports no less than 6-channel video processing, and has a performance processor computing power of no less than 4TOPS;

[0279] 3) When an autonomous vehicle reports abnormal vehicle data (including CAN data, regulatory platform evaluation indicators, etc.), the operation and supervision platform can quickly identify and immediately require the autonomous vehicle to report the video data collected within a period of time before and after the abnormal moment, which greatly reduces the transmission time of video data and data screening efficiency, and is conducive to the rapid analysis of regulatory data and the control of the operating status of autonomous vehicles;

[0280] 4) When the autonomous vehicle arrives at the intersection, the operation supervision platform immediately requires the autonomous vehicle and the roadside unit at the intersection to simultaneously report the video data collected within a period of time before and after the arrival at the intersection, which is conducive to the full supervision of the operation status of the autonomous vehicle at the intersection. When the autonomous vehicle encounters a traffic accident or violation, it can quickly find the corresponding data analysis, providing effective data support for the optimization and improvement of the autonomous driving algorithm;

[0281] 5) When the supervision data fails to be uploaded due to communication anomalies or fails to be written to the operation supervision platform, the multi-mode vehicle-mounted domain controller automatically stores the failed uploaded data locally, and automatically detects when the communication link is normal, and actively triggers data retransmission or video retransmission strategies, which can effectively ensure the integrity of the supervision data of the autonomous driving vehicle;

[0282] 6) The operation supervision platform can supervise the operation data of autonomous driving vehicles in multiple dimensions based on real-time supervision data, real-time video data, and command video data. This is conducive to more comprehensive and reasonable evaluation and supervision of the operation data of autonomous driving vehicles, and continuously improves and builds the legal system for autonomous driving supervision, which will be promoted and applied on a large scale in the future.

[0283] The autonomous driving vehicle supervision system and method provided in this application have at least the following advantages over the prior art:

[0284] 1. When an autonomous vehicle reports abnormal vehicle data (including CAN data, regulatory platform evaluation indicators, etc.), the operation and supervision platform stores video data records for a period of time before and after the corresponding abnormal moment, which is conducive to quickly and accurately finding the corresponding data of the problem and reducing a large amount of data screening and analysis;

[0285] 2. When an autonomous vehicle passes through an intersection, the operation and supervision platform stores video data records of the intersection and the autonomous vehicle for a period of time before and after the corresponding moment, which provides more comprehensive supervision methods for autonomous vehicles;

[0286] 3. RSU has functions such as command video processing, data retransmission and supplementary transmission. The difference from existing technologies also includes: RSU can respond to instructions issued by the operation supervision platform;

[0287] 4. More comprehensive and complete regulatory data for autonomous vehicles will be the development trend of autonomous vehicle management technology in the future.

[0288] The technical solution provided in the embodiment of the present application can be applicable to a variety of systems, especially 5G systems. For example, the applicable system can be a global system of mobile communication (Global System of Mobile communication, referred to as GSM) system, code division multiple access (Code Division Multiple Access, referred to as CDMA) system, wideband code division multiple access (Wideband Code Division Multiple Access, referred to as WCDMA) general packet radio service (General Packet Radio Service, referred to as GPRS) system, long term evolution (long term evolution, referred to as LTE) system, LTE frequency division duplex (Frequency Division Duplex, referred to as FDD) system, LTE time division duplex (time division duplex, referred to as TDD) system, advanced long term evolution (Long Term Evolution Advanced, referred to as LTE-A) system, Universal Mobile Telecommunication System (Universal Mobile Telecommunication System, referred to as UMTS), Worldwide interoperability for Microwave Access (Worldwide interoperability for Microwave Access, referred to as WiMAX) system, 5G new air interface (NewRadio, referred to as NR) system, etc. These various systems include terminals and network equipment. The system may also include a core network part, such as an evolved packet system (EPS), a 5G system (5GS), etc.

[0289] In order to implement the above embodiments, the present application also provides a supervision system for an autonomous driving vehicle.

[0290] Fig.10 It is a structural diagram of a supervision system for an autonomous driving vehicle provided according to an embodiment of the present application.

[0291] like Fig.10 As shown, the supervision system of the autonomous driving vehicle may include: an autonomous driving vehicle 1010 and an operation supervision platform 1020.

[0292] Among them, the autonomous driving vehicle 1010 is used to send vehicle data to the operation and supervision platform.

[0293] The operation supervision platform 1020 is used to receive vehicle data and, when the vehicle data is abnormal, send a first video instruction to the autonomous driving vehicle 1010 according to the abnormal time of the vehicle data; wherein the first video instruction carries the abnormal time, the first duration parameter and the second duration parameter.

[0294] The autonomous driving vehicle 1010 is also used to send the first target data to the operation supervision platform 1020 in response to the first video instruction; wherein the first target data includes: the first video data collected inside and / or outside the cabin of the autonomous driving vehicle 1010 in a first time period before the abnormal moment, and the second video data collected inside and / or outside the cabin in a second time period after the abnormal moment; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter.

[0295] The operation supervision platform 1020 is also used to supervise the autonomous driving vehicle 1010 based on the first target data and the vehicle data.

[0296] As a possible implementation method, the operation supervision platform 1020 is specifically used to: send a first video instruction to the multi-mode vehicle-mounted domain controller in the autonomous driving vehicle 1010 based on the first communication channel; and receive first target data sent by the multi-mode vehicle-mounted domain controller through the second communication channel.

[0297] As a possible implementation, the operation supervision platform 1020 is further used to: send a first authentication request to the multi-mode vehicle-mounted domain controller; wherein the first authentication request carries a first security certificate of the operation supervision platform; in response to receiving a first authentication pass response sent by the multi-mode vehicle-mounted domain controller, establish a first communication channel with the multi-mode vehicle-mounted domain controller; wherein the first authentication pass response is generated when the multi-mode vehicle-mounted domain controller authenticates the first security certificate and passes the authentication;

[0298] The multi-mode vehicle-mounted domain controller is also used to: send a second authentication request to the operation supervision platform 1020; wherein the second authentication request carries the second security certificate of the multi-mode vehicle-mounted domain controller; in response to receiving a second authentication pass response sent by the operation supervision platform 1020, establish a second communication channel with the operation supervision platform 1020; wherein the second authentication pass response is generated when the operation supervision platform authenticates the second security certificate and passes the authentication.

[0299] As a possible implementation method, the number of second communication channels is multiple, and the multi-mode vehicle-mounted domain controller is specifically used to: determine whether there is a first data record to be transmitted in the local storage file of the multi-mode vehicle-mounted domain controller; if the first data record exists in the storage file, the first data record and the first target data are transmitted in parallel through multiple second communication channels; if the first data record does not exist in the storage file, the first target data is transmitted in parallel through multiple second communication channels.

[0300] As a possible implementation method, the timing of writing the first data record to the storage file includes at least one of the following: the multi-mode vehicle-mounted domain controller or the autonomous driving vehicle 1010 loses power or has network communication abnormalities during the process of sending the first data record; the multi-mode vehicle-mounted domain controller does not receive a first confirmation response sent by the operation supervision platform 1020 within a set time period after sending the first data record, wherein the first confirmation response is used to indicate that the operation supervision platform 1020 has received the first data record; after receiving the first confirmation response, the multi-mode vehicle-mounted domain controller receives a write failure message sent by the operation supervision platform 1020; wherein the write failure message is used to indicate that the operation supervision platform 1020 has failed to store the first data record.

[0301] As a possible implementation method, the operation supervision platform 1020 is also used to: in response to receiving a first data record, store the first data record, and send a second confirmation response to the multi-mode vehicle domain controller; wherein the second confirmation response is used to indicate that the operation supervision platform 1020 has received the first data record; in response to receiving the first target data, store the first target data, and send a third confirmation response to the multi-mode vehicle domain controller; wherein the third confirmation response is used to indicate that the operation supervision platform 1020 has received the first target data; wherein the storage path of the first data record is different from that of the first target data.

[0302] As a possible implementation method, the first data record is provided with first label information, and the first label information is used to mark the first data record as supplementary data. The vehicle data is provided with second label information, and the second label information is used to indicate that the vehicle data is abnormal data. The first data record, the abnormal vehicle data and the first target data have different storage paths in the operation supervision platform 1020; the operation supervision platform 1020 is specifically used to: query the supplementary data with the first label information sent by the multi-mode vehicle domain controller from the stored data; query the abnormal data with the second label information sent by the multi-mode vehicle domain controller from the stored data; supervise the autonomous driving vehicle 1010 based on the supplementary data with the first label information, the abnormal data with the second label information and the first target data.

[0303] As a possible implementation method, the multi-mode vehicle-mounted domain controller is also used to: determine whether a second confirmation response is received within a set time period after the first data record is transmitted through multiple second communication channels; if the second confirmation response is not received, determine the first supplementary transmission reporting interval based on the number of supplementary transmissions of the first data record; after waiting for the first supplementary transmission reporting interval, re-transmit the first data record through the second communication channel; if the second confirmation response is still not received within the set time period after the re-transmission of the first data record, amplify the first supplementary transmission reporting interval to obtain a second supplementary transmission reporting interval; after waiting for the second supplementary transmission reporting interval, transmit the first data record again through the second communication channel until the second confirmation response is received and the set conditions are met, the first data record in the storage file is deleted; wherein the set conditions include: the storage time of the first data record reaches the set time.

[0304] As a possible implementation, a multi-mode vehicle-mounted domain controller is used to receive and store video data and CAN data sent in real time by the autonomous driving vehicle 1010, where the CAN data is the transmission data on the CAN bus of the autonomous driving vehicle 1010;

[0305] The vehicle data associated with the autonomous driving vehicle 1010 includes: video data and / or CAN data sent in real time by the multi-mode vehicle domain controller;

[0306] Among them, CAN data includes at least one of the following: driving data; driving mode; brake system data; fault status data; tire pressure data; steering system data; fuel consumption data.

[0307] As a possible implementation method, the multi-mode vehicle-mounted domain controller is specifically used to: determine whether there is a second data record to be transmitted in the local storage file of the multi-mode vehicle-mounted domain controller; if the second data record exists in the storage file, send the second data record to the operation supervision platform 1020, and send video data and / or CAN data; if the second data record does not exist in the storage file, send video data and / or CAN data to the operation supervision platform 1020.

[0308] As a possible implementation method, the multi-mode vehicle domain controller has a multi-mode communication standard, and the communication standards include: EUHT; 4G; 5G; LTE-V.

[0309] As a possible implementation method, the vehicle data includes CAN data sent by the autonomous driving vehicle 1010, and the vehicle data abnormality includes at least one of the following: CAN data indicates that the autonomous driving vehicle 1010 performs emergency braking; CAN data indicates that the driving speed of the autonomous driving vehicle 1010 drops to zero within a set time period, wherein the duration of the set time period is less than a first set threshold; CAN data indicates that the autonomous driving vehicle 1010 accelerates urgently; CAN data indicates that at least one system in the autonomous driving vehicle 1010 fails; CAN data indicates that at least one component in the autonomous driving vehicle 1010 fails; CAN data indicates a dashboard warning of the autonomous driving vehicle 1010; CAN data indicates that the autonomous driving vehicle 1010 is taken over manually; the difference between the actual mileage of the autonomous driving vehicle 1010 indicated by the CAN data and the reference mileage counted by the odometer is greater than the set difference threshold; the transmission success rate of the CAN data is lower than the second set threshold.

[0310] As a possible implementation method, the vehicle data includes video data sent by the autonomous driving vehicle 1010, and the vehicle data abnormalities include: a transmission success rate of the video data is lower than a third set threshold.

[0311] As a possible implementation method, the supervision system of the autonomous driving vehicle may further include: a roadside unit;

[0312] The operation supervision platform 1020 is further used to: in response to monitoring that the autonomous driving vehicle 1010 drives to the intersection, send a second video instruction to the roadside unit and the autonomous driving vehicle 1010 at the intersection according to the arrival time of the autonomous driving vehicle 1010 at the intersection, receive second target data sent by the roadside unit in response to the second video instruction and third target data sent by the autonomous driving vehicle 1010 in response to the second video instruction, and supervise the autonomous driving vehicle 1010 according to the second target data and the third target data;

[0313] The roadside unit is used to receive the second video instruction sent by the operation supervision platform 1020, and send the second target data to the operation supervision platform 1020 in response to the second video instruction;

[0314] The autonomous driving vehicle 1010 is further configured to: receive a second video instruction sent by the operation supervision platform 1020, and send third target data to the operation supervision platform 1020 in response to the second video instruction;

[0315] The second video instruction carries the arrival time, the third duration parameter and the fourth duration parameter; the second target data includes: the third video data collected at the intersection within the third time period before the arrival time, and the fourth video data collected at the intersection within the fourth time period after the arrival time; the duration of the third time period matches the third duration parameter, and the duration of the fourth time period matches the fourth duration parameter; the third target data includes: the fifth video data collected inside and / or outside the cabin within the third time period, and the sixth video data collected inside and / or outside the cabin within the fourth time period.

[0316] It should be noted that the autonomous driving vehicle supervision system provided in the embodiment of the present application can achieve the above Figures 2 to 6 All the method steps implemented in the method embodiment can achieve the same technical effect, and the parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0317] In order to implement the above embodiments, the present application also provides an operation supervision platform.

[0318] Fig.11 It is a structural diagram of an operation supervision platform provided according to an embodiment of the present application.

[0319] like Fig.11 As shown, the operation supervision platform may include a transceiver 1100, a processor 1110, and a memory 1120, wherein:

[0320] The transceiver 1100 is used to receive and send data under the control of the processor 1110 .

[0321] Among them, Fig.11 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 1110 and various circuits of memory represented by memory 1120 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1100 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which transmission medium may include a wireless channel, a wired channel, an optical cable, and other transmission media. The processor 1110 is responsible for managing the bus architecture and general processing, and the memory 1120 may store data used by the processor 1110 when performing operations.

[0322] The processor 1110 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0323] Processor 1110 calls a computer program stored in a memory and executes any of the aforementioned methods for supervising an autonomous driving vehicle.

[0324] It should be noted that the operation supervision platform provided in the embodiment of the present application can achieve the above Figures 2 to 6 All the method steps implemented in the method embodiment can achieve the same technical effect, and the parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0325] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0326] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network side device, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0327] On the other hand, an embodiment of the present application further provides a processor-readable storage medium, the processor-readable storage medium storing a computer program, the computer program is used to enable the processor to execute the present application Figures 2 to 6 The method shown in any embodiment.

[0328] Among them, the above-mentioned processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0329] In order to implement the above embodiments, the present application also proposes a computer program product.

[0330] The computer program product includes a computer program which, when executed by a processor, implements the present application. Figures 2 to 6 The method shown in any embodiment.

[0331] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0332] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0333] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0334] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0335] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for supervising an autonomous vehicle, It is characterized in that Applied to the operation and supervision platform, including: In response to detecting an abnormality in vehicle data associated with the autonomous driving vehicle, a first video instruction is sent to the autonomous driving vehicle according to the abnormal time of the vehicle data; wherein the first video instruction carries the abnormal time, a first duration parameter, and a second duration parameter; Receiving first target data sent by the autonomous driving vehicle in response to the first video instruction; wherein the first target data includes: first video data collected from inside and / or outside the cockpit of the autonomous driving vehicle in a first time period before the abnormal moment, and second video data collected from inside and / or outside the cockpit in a second time period after the abnormal moment; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter; The autonomous driving vehicle is supervised based on the first target data and the vehicle data.

2. The method according to claim 1, It is characterized in that In response to detecting that vehicle data associated with the autonomous driving vehicle is abnormal, sending a first video instruction to the autonomous driving vehicle according to the abnormal time of the vehicle data, including: In response to detecting an abnormality in vehicle data associated with the autonomous driving vehicle, sending a first video instruction to a multi-mode vehicle-mounted domain controller in the autonomous driving vehicle based on a first communication channel; The receiving first target data sent by the autonomous driving vehicle in response to the first video instruction includes: Receive the first target data sent by the multi-mode vehicle-mounted domain controller through the second communication channel.

3. The method according to claim 2, It is characterized in that The first communication channel is established by the operation supervision platform using the following steps: Sending a first authentication request to the multi-mode vehicle-mounted domain controller; wherein the first authentication request carries a first security certificate of the operation supervision platform; In response to receiving a first authentication success response sent by the multi-mode vehicle-mounted domain controller, establishing a first communication channel with the multi-mode vehicle-mounted domain controller; The first authentication success response is generated when the multi-mode vehicle-mounted domain controller authenticates the first security certificate and the first security certificate passes the authentication; Accordingly, the second communication channel is established by the multi-mode vehicle-mounted domain controller using the following steps: Sending a second authentication request to the operation supervision platform; wherein the second authentication request carries a second security certificate of the multi-mode vehicle-mounted domain controller; In response to receiving a second authentication pass response sent by the operation supervision platform, establishing a second communication channel with the operation supervision platform; The second authentication pass response is generated when the operation supervision platform authenticates the second security certificate and the second security certificate passes the authentication.

4. The method according to claim 2, It is characterized in that The number of the second communication channels is multiple, The first target data is sent to the operation supervision platform by the multi-mode vehicle-mounted domain controller using the following steps: Determine whether there is a first data record to be supplemented in the local storage file of the multi-mode vehicle-mounted domain controller; If the first data record exists in the storage file, transmitting the first data record and the first target data in parallel through a plurality of the second communication channels; If the first data record does not exist in the storage file, the first target data is transmitted in parallel through the multiple second communication channels.

5. The method according to claim 4, It is characterized in that The timing of writing the first data record into the storage file includes at least one of the following: The multi-mode vehicle-mounted domain controller or the autonomous driving vehicle loses power or has a network communication abnormality during the process of sending the first data record; The multi-mode vehicle-mounted domain controller does not receive a first confirmation response sent by the operation supervision platform within a set period of time after sending the first data record; wherein the first confirmation response is used to indicate that the operation supervision platform has received the first data record; After receiving the first confirmation response, the multi-mode vehicle-mounted domain controller receives a write failure message sent by the operation supervision platform; wherein the write failure message is used to indicate that the operation supervision platform has failed to store the first data record.

6. The method according to claim 4, It is characterized in that The method further comprises: In response to receiving the first data record, storing the first data record, and sending a second confirmation response to the multi-mode vehicle-mounted domain controller; wherein the second confirmation response is used to indicate that the operation supervision platform has received the first data record; In response to receiving the first target data, storing the first target data, and sending a third confirmation response to the multi-mode vehicle-mounted domain controller; wherein the third confirmation response is used to indicate that the operation supervision platform has received the first target data; The first data record and the first target data have different storage paths.

7. The method according to claim 6, It is characterized in that The first data record is provided with first tag information, the first tag information is used to mark the first data record as supplementary data, the vehicle data is provided with second tag information, the second tag information is used to indicate that the vehicle data is abnormal data, and the first data record, the abnormal vehicle data and the first target data have different storage paths; The supervising the autonomous driving vehicle based on the first target data and the vehicle data includes: Querying the stored data for the supplementary data having the first tag information sent by the multi-mode vehicle-mounted domain controller; querying the stored data for abnormal data having second tag information sent by the multi-mode vehicle-mounted domain controller; The autonomous driving vehicle is supervised based on the supplementary data with the first label information, the abnormal data with the second label information and the first target data.

8. The method according to claim 6, It is characterized in that After transmitting the first data record and the first target data in parallel through the plurality of the second communication channels, the multi-mode vehicle-mounted domain controller is further configured to perform the following operations: determining whether the second confirmation response is received within a set period of time after the first data record is transmitted through the plurality of second communication channels; If the second confirmation response is not received, determining a first supplementary transmission reporting interval according to the number of supplementary transmissions of the first data record; After waiting for the first supplementary transmission reporting interval, retransmitting the first data record through the second communication channel; If the second confirmation response is still not received within a set period of time after the first data record is retransmitted, the first supplementary transmission reporting interval is enlarged to obtain a second supplementary transmission reporting interval; After waiting for the second supplementary transmission reporting interval, the first data record is transmitted again through the second communication channel until the second confirmation response is received and the first data record in the storage file is deleted if the set conditions are met; wherein the set conditions include: the storage time of the first data record reaches the set time.

9. The method according to claim 2, It is characterized in that The multi-mode vehicle-mounted domain controller is used to receive and store video data and CAN data sent in real time by the autonomous driving vehicle, wherein the CAN data is transmission data on the CAN bus of the autonomous driving vehicle; The vehicle data associated with the autonomous driving vehicle includes: the video data and / or the CAN data sent in real time by the multi-mode vehicle-mounted domain controller; Wherein, the CAN data includes at least one of the following: Driving data; Driving modes; Braking system data; Fault status data; Tire pressure data; Steering system data; Fuel consumption data.

10. The method according to claim 9, It is characterized in that The multi-mode vehicle-mounted domain controller adopts the following steps to send the video data and / or the CAN data to the operation supervision platform in real time: Determine whether there is a second data record to be supplemented in the local storage file of the multi-mode vehicle-mounted domain controller; If the second data record exists in the storage file, sending the second data record to the operation supervision platform, and sending the video data and / or the CAN data; If the second data record does not exist in the storage file, the video data and / or the CAN data are sent to the operation supervision platform.

11. The method according to claim 9, It is characterized in that The multi-mode vehicle-mounted domain controller has a multi-mode communication format, and the communication format includes: EUHT; 4G； 5G； LTE-V.

12. The method according to claim 1, It is characterized in that The vehicle data includes CAN data sent by the autonomous driving vehicle, and the vehicle data anomaly includes at least one of the following: The CAN data indicates emergency braking of the autonomous driving vehicle; The CAN data indicates that the driving speed of the autonomous driving vehicle drops to zero within a set time period, wherein the duration of the set time period is less than a first set threshold; The CAN data indicates that the autonomous driving vehicle accelerates urgently; The CAN data indicates that at least one system in the autonomous vehicle has failed; The CAN data indicates that at least one component in the autonomous vehicle has failed; The CAN data indicates a dashboard warning of the autonomous driving vehicle; The CAN data indicates that the autonomous driving vehicle is manually taken over; The difference between the actual mileage of the autonomous driving vehicle indicated by the CAN data and the reference mileage counted by the odometer is greater than a set difference threshold; The transmission success rate of the CAN data is lower than a second set threshold.

13. The method according to claim 1, It is characterized in that The vehicle data includes video data sent by the autonomous driving vehicle, and the vehicle data anomaly includes: a transmission success rate of the video data is lower than a third set threshold.

14. The method according to any one of claims 1 to 13, It is characterized in that The method further comprises: In response to monitoring that the autonomous driving vehicle drives to an intersection, according to the arrival time of the autonomous driving vehicle at the intersection, a second video instruction is sent to the roadside unit at the intersection and the autonomous driving vehicle; wherein the second video instruction carries the arrival time, the third duration parameter and the fourth duration parameter; receiving second target data sent by the roadside unit in response to the second video instruction, and receiving third target data sent by the autonomous driving vehicle in response to the second video instruction; Supervising the autonomous driving vehicle according to the second target data and the third target data; The second target data includes: third video data collected at the intersection in a third time period before the arrival time, and fourth video data collected at the intersection in a fourth time period after the arrival time; the duration of the third time period matches the third duration parameter, and the duration of the fourth time period matches the fourth duration parameter; The third target data includes: fifth video data collected from inside and / or outside the cabin during the third time period, and sixth video data collected from inside and / or outside the cabin during the fourth time period.

15. A monitoring system for an autonomous vehicle, It is characterized in that include: Autonomous driving vehicles, used to send vehicle data to the operation supervision platform; An operation supervision platform, configured to receive the vehicle data, and, when the vehicle data is abnormal, send a first video instruction to the autonomous driving vehicle according to the abnormal time of the vehicle data; wherein the first video instruction carries the abnormal time, a first duration parameter, and a second duration parameter; The autonomous driving vehicle is further configured to send first target data to the operation supervision platform in response to the first video instruction; wherein the first target data includes: first video data collected from inside and / or outside the cockpit of the autonomous driving vehicle in a first time period before the abnormal moment, and second video data collected from inside and / or outside the cockpit in a second time period after the abnormal moment; the duration of the first time period matches the first duration parameter, and the duration of the second time period matches the second duration parameter; The operation supervision platform is also used to supervise the autonomous driving vehicle based on the first target data and the vehicle data.

16. The monitoring system according to claim 15, It is characterized in that The operation supervision platform is specifically used for: Based on the first communication channel, sending a first video instruction to the multi-mode vehicle-mounted domain controller in the autonomous driving vehicle; Receive the first target data sent by the multi-mode vehicle-mounted domain controller through the second communication channel.

17. The monitoring system according to claim 16, It is characterized in that The operation supervision platform is further used to: send a first authentication request to the multi-mode vehicle-mounted domain controller; wherein the first authentication request carries a first security certificate of the operation supervision platform; in response to receiving a first authentication pass response sent by the multi-mode vehicle-mounted domain controller, establish a first communication channel with the multi-mode vehicle-mounted domain controller; The first authentication success response is generated when the multi-mode vehicle-mounted domain controller authenticates the first security certificate and the first security certificate passes the authentication; The multi-mode vehicle-mounted domain controller is further used to: send a second authentication request to the operation supervision platform; wherein the second authentication request carries a second security certificate of the multi-mode vehicle-mounted domain controller; in response to receiving a second authentication pass response sent by the operation supervision platform, establish a second communication channel with the operation supervision platform; The second authentication pass response is generated when the operation supervision platform authenticates the second security certificate and the second security certificate passes the authentication.

18. The monitoring system according to claim 16, It is characterized in that The number of the second communication channels is multiple, and the multi-mode vehicle-mounted domain controller is specifically used for: Determine whether there is a first data record to be supplemented in the local storage file of the multi-mode vehicle-mounted domain controller; If the first data record exists in the storage file, transmitting the first data record and the first target data in parallel through a plurality of the second communication channels; If the first data record does not exist in the storage file, the first target data is transmitted in parallel through the multiple second communication channels.

19. The monitoring system according to claim 18, It is characterized in that The timing of writing the first data record into the storage file includes at least one of the following: The multi-mode vehicle-mounted domain controller or the autonomous driving vehicle loses power or has a network communication abnormality during the process of sending the first data record; The multi-mode vehicle-mounted domain controller does not receive a first confirmation response sent by the operation supervision platform within a set period of time after sending the first data record, wherein the first confirmation response is used to indicate that the operation supervision platform has received the first data record; After receiving the first confirmation response, the multi-mode vehicle-mounted domain controller receives a write failure message sent by the operation supervision platform; wherein the write failure message is used to indicate that the operation supervision platform has failed to store the first data record.

20. The monitoring system according to claim 18, It is characterized in that The operation supervision platform is also used for: In response to receiving the first data record, storing the first data record, and sending a second confirmation response to the multi-mode vehicle-mounted domain controller; wherein the second confirmation response is used to indicate that the operation supervision platform has received the first data record; In response to receiving the first target data, storing the first target data, and sending a third confirmation response to the multi-mode vehicle-mounted domain controller; wherein the third confirmation response is used to indicate that the operation supervision platform has received the first target data; The first data record and the first target data have different storage paths.

21. The monitoring system according to claim 20, It is characterized in that The first data record is provided with first tag information, the first tag information is used to mark the first data record as supplementary data, the vehicle data is provided with second tag information, the second tag information is used to indicate that the vehicle data is abnormal data, and the first data record, the abnormal vehicle data and the first target data have different storage paths in the operation supervision platform; The operation supervision platform is specifically used to: query the supplementary data with first label information sent by the multi-mode vehicle domain controller from the stored data; query the abnormal data with second label information sent by the multi-mode vehicle domain controller from the stored data; and supervise the autonomous driving vehicle based on the supplementary data with the first label information, the abnormal data with the second label information and the first target data.

22. The monitoring system according to claim 20, It is characterized in that The multi-mode vehicle-mounted domain controller is also used for: determining whether the second confirmation response is received within a set period of time after the first data record is transmitted through the plurality of second communication channels; If the second confirmation response is not received, determining a first supplementary transmission reporting interval according to the number of supplementary transmissions of the first data record; After waiting for the first supplementary transmission reporting interval, retransmitting the first data record through the second communication channel; If the second confirmation response is still not received within a set period of time after the first data record is retransmitted, the first supplementary transmission reporting interval is enlarged to obtain a second supplementary transmission reporting interval; After waiting for the second supplementary transmission reporting interval, the first data record is transmitted again through the second communication channel until the second confirmation response is received and the first data record in the storage file is deleted if the set conditions are met; wherein the set conditions include: the storage time of the first data record reaches the set time.

23. The monitoring system according to claim 16, It is characterized in that The multi-mode vehicle-mounted domain controller is used to receive and store video data and CAN data sent in real time by the autonomous driving vehicle, wherein the CAN data is transmission data on the CAN bus of the autonomous driving vehicle; The vehicle data associated with the autonomous driving vehicle includes: the video data and / or the CAN data sent in real time by the multi-mode vehicle-mounted domain controller; Wherein, the CAN data includes at least one of the following: Driving data; Driving modes; Braking system data; Fault status data; Tire pressure data; Steering system data; Fuel consumption data.

24. The monitoring system according to claim 23, It is characterized in that The multi-mode vehicle-mounted domain controller is specifically used for: Determine whether there is a second data record to be supplemented in the local storage file of the multi-mode vehicle-mounted domain controller; If the second data record exists in the storage file, sending the second data record to the operation supervision platform, and sending the video data and / or the CAN data; If the second data record does not exist in the storage file, the video data and / or the CAN data are sent to the operation supervision platform.

25. The monitoring system according to claim 23, It is characterized in that The multi-mode vehicle-mounted domain controller has a multi-mode communication format, and the communication format includes: EUHT; 4G； 5G； LTE-V.

26. The monitoring system according to claim 15, It is characterized in that The vehicle data includes CAN data sent by the autonomous driving vehicle, and the vehicle data anomaly includes at least one of the following: The CAN data indicates emergency braking of the autonomous driving vehicle; The CAN data indicates that the driving speed of the autonomous driving vehicle drops to zero within a set time period, wherein the duration of the set time period is less than a first set threshold; The CAN data indicates that the autonomous driving vehicle accelerates urgently; The CAN data indicates that at least one system in the autonomous vehicle has failed; The CAN data indicates that at least one component in the autonomous vehicle has failed; The CAN data indicates a dashboard warning of the autonomous driving vehicle; The CAN data indicates that the autonomous driving vehicle is manually taken over; The difference between the actual mileage of the autonomous driving vehicle indicated by the CAN data and the reference mileage counted by the odometer is greater than a set difference threshold; The transmission success rate of the CAN data is lower than a second set threshold.

27. The monitoring system according to claim 15, It is characterized in that The vehicle data includes video data sent by the autonomous driving vehicle, and the vehicle data anomaly includes: a transmission success rate of the video data is lower than a third set threshold.

28. A monitoring system according to any one of claims 15 to 27, It is characterized in that The supervision system further includes: a roadside unit; The operation supervision platform is further used for: in response to monitoring that the autonomous driving vehicle drives to an intersection, sending a second video instruction to the roadside unit at the intersection and the autonomous driving vehicle according to the arrival time of the autonomous driving vehicle at the intersection, receiving second target data sent by the roadside unit in response to the second video instruction and third target data sent by the autonomous driving vehicle in response to the second video instruction, and supervising the autonomous driving vehicle according to the second target data and the third target data; The roadside unit is used to receive a second video instruction sent by the operation supervision platform, and send second target data to the operation supervision platform in response to the second video instruction; The autonomous driving vehicle is further configured to: receive a second video instruction sent by the operation supervision platform, and send third target data to the operation supervision platform in response to the second video instruction; Wherein, the second video instruction carries the arrival time, the third duration parameter and the fourth duration parameter; The second target data includes: third video data collected at the intersection in a third time period before the arrival time, and fourth video data collected at the intersection in a fourth time period after the arrival time; the duration of the third time period matches the third duration parameter, and the duration of the fourth time period matches the fourth duration parameter; The third target data includes: fifth video data collected from inside and / or outside the cabin during the third time period, and sixth video data collected from inside and / or outside the cabin during the fourth time period.

29. An operation supervision platform, It is characterized in that Including memory, transceiver, processor; Memory for storing computer programs; a transceiver, for transmitting and receiving data under the control of the processor; A processor, configured to read the computer program in the memory and execute the method according to any one of claims 1 to 14.

30. A processor-readable storage medium, It is characterized in that The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method according to any one of claims 1 to 14.

Images