Automatic driving commercial vehicle remote assistance system and method

Abstract

The invention discloses an automatic driving commercial vehicle remote assistance system and method, and belongs to the technical field of automatic driving remote assistance, the system comprises a platform end, a network and a vehicle end, and the platform end performs data interaction with the vehicle end through the network; the vehicle end is arranged on an automatic driving commercial vehicle; the platform end receives monitoring data uploaded by the vehicle end through the network, processes and stores the monitoring data, generates a corresponding control instruction according to the monitoring data, issues the control instruction to the vehicle end through the network, and performs remote assistance on the vehicle end according to the control instruction. Through the remote assistance system, remote assistance of abnormal events of the automatic driving commercial vehicle is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving remote assistance, and in particular relates to a remote assistance system and method for autonomous driving commercial vehicles. Background Art

[0002] In principle, autonomous vehicles operating in road freight transport must be equipped with a safety officer, in accordance with the standards and requirements for safety officers for autonomous vehicles operating in road transport. To encourage technological innovation and industrial development, fully autonomous vehicles operating as taxi passenger transport may utilize a remote safety officer. However, the composition and methods of the remote assistance systems and assistance methods relied upon by remote safety officers in the commercial application of autonomous vehicles remain unclear, posing risks to the commercial application of autonomous vehicles without human personnel inside. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention proposes a remote assistance system and method for autonomous driving operating vehicles to solve the problems existing in the above-mentioned prior art.

[0004] To achieve the above objectives, the present invention provides a remote assistance system for autonomous driving commercial vehicles, comprising:

[0005] The platform side, the network and the vehicle side, the platform side exchanges data with the vehicle side through the network; the vehicle side is set on the autonomous driving operating vehicle; the platform side receives the monitoring data uploaded by the vehicle side through the network, processes and stores the monitoring data, generates corresponding control instructions according to the monitoring data, sends the control instructions to the vehicle side through the network, and remotely assists the vehicle side according to the control instructions.

[0006] Optionally, the monitoring data on the vehicle side includes external driving environment information, external driving environment abnormality information, internal vehicle environment information, internal vehicle environment abnormality information, vehicle motion status information, automatic driving system status information, vehicle operation status abnormality information, voice trigger signal, audio information and assistance request signal.

[0007] Optionally, the network is connected to the platform via a wired connection, and the network is connected to the vehicle via a wireless connection.

[0008] Optionally, the platform includes a high-performance server, which receives, processes and stores monitoring data, obtains control instructions based on the monitoring data, and sends the control instructions to the vehicle; the control instructions include: continued monitoring, voice calls, assisted control and emergency parking.

[0009] Optionally, the platform also includes a status monitoring device, wherein the working status of the remote safety officer is monitored by the status detection device, and the remote safety officer is prompted and reported based on the working status monitoring results.

[0010] On the other hand, the present application provides a remote assistance method for an autonomous driving vehicle, comprising:

[0011] The platform receives and monitors the monitoring data uploaded by the vehicle through the network. When an abnormal situation occurs in the monitoring data, it handles the abnormal event and issues instructions to the vehicle for voice calls, remote assistance and emergency parking.

[0012] Optionally, when the platform monitors the monitoring data and no abnormal signal is received in the monitoring data, the monitoring data is categorized and monitored by a remote safety officer, wherein the platform receives and displays the uploaded outdoor environment video through the network, and the remote safety officer monitors whether the outdoor environment is abnormal; in the in-vehicle environment monitoring, the platform receives and displays the uploaded in-vehicle environment video through the network, and the remote safety officer monitors whether the in-vehicle environment is abnormal; in the vehicle operation status monitoring, the platform receives and displays the uploaded vehicle operation status data through the network, and the remote safety officer monitors whether the operation status is abnormal; during the monitoring process, the target vehicle, i.e. the monitored vehicle, is switched manually or automatically, and the monitoring field of view is switched manually or automatically.

[0013] Optionally, the process of handling abnormal events includes: processing voice call events, wherein when the vehicle side receives an assistance request signal triggered by a passenger, a voice trigger signal is provided to the platform side, a voice call connection is established between the platform side and the vehicle side, and audio data is transmitted in real time in both directions for voice calls between the remote safety officer and the passenger.

[0014] Optionally, the abnormal event handling process also includes: processing remote assistance events, wherein, when a remote assistance instruction is triggered during vehicle driving to continue to complete an operation event, the remote safety officer sends a corresponding remote assistance instruction to the vehicle side with the help of the platform side according to the event type. After the vehicle side controls the vehicle's automatic driving system, it determines whether to execute the remote assistance instruction. If it is executed, the remote safety officer and passengers are prompted and the instruction is broadcast to the passengers. If it is not executed, the assistance failure information is returned to the platform side for the remote safety officer and passengers to prompt and report.

[0015] Optionally, the abnormal event handling process also includes: handling emergency stop events, wherein, when the remote safety officer detects an emergency stop event on the platform side, the remote safety officer interacts with the platform side to send an emergency stop command to the vehicle side, and the vehicle side controls the vehicle braking execution system to perform an emergency stop. When the emergency stop occurs, if the vehicle's automatic driving system executes the vehicle's longitudinal acceleration control, the vehicle only performs an emergency stop and the vehicle's automatic driving system switches to the exit state. After the vehicle stops, a voice call connection is established between the vehicle and the remote safety officer.

[0016] Compared with the prior art, the present invention has the following advantages and technical effects:

[0017] The remote assistance system proposed in the present invention has added, developed, and created vehicle-side data filtering technology, remote safety officer status monitoring technology, remote emergency parking technology, and multi-functional screen display technology based on specific operational needs; in addition, it effectively combines the hardware of each system to work together to ensure that the remote safety officer can complete the safety protection and operation services of unmanned commercial applications in the vehicle through the remote assistance system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0019] Figure 1 This is a schematic diagram of the basic structure of the remote assistance system for an autonomous vehicle according to an embodiment of the present invention;

[0020] Figure 2 This is a conventional monitoring flow chart of an embodiment of the present invention;

[0021] Figure 3 This is a flowchart of handling three types of abnormal events according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] The present invention provides a remote assistance system for autonomous driving commercial vehicles, comprising: a platform side, a network and a vehicle side, wherein the platform side exchanges relevant data and issues instructions with the vehicle side via the network, the vehicle side collects monitoring data inside and outside the vehicle, uploads the monitoring data via the network, and transmits the data to the platform side via the network, the platform side receives, processes and stores the monitoring data, remotely monitors the monitoring data, and issues relevant control instructions based on the monitoring data, which are transmitted to the vehicle side via the network, the vehicle side performs control according to the control instructions, and at the same time, the vehicle side also establishes a voice connection with the platform side via the network for remote communication.

[0025] The platform includes a high-performance server that receives, processes, and stores relevant monitoring data and issues control commands based on it. This monitoring data includes in-vehicle and out-of-vehicle environmental data and vehicle operating status data. This data includes basic operating status information, assistance request information, assistance result information, voice call function status information, and emergency stop function status information. Relevant control commands include continued monitoring, voice call, assistance control, and emergency stop.

[0026] The platform also includes a display system for displaying the above-mentioned monitoring data and alarm information, and for displaying the functional status of voice calls, auxiliary control, emergency parking, etc.

[0027] The platform also includes an operating system, which is used as an interactive means for remote security personnel to perform related monitoring or control functions.

[0028] The platform also includes a status monitoring device for remote safety officers, which is used to detect the working status of remote safety officers.

[0029] The above technical solution is described in detail:

[0030] Basic system composition:

[0031] The remote assistance system for autonomous vehicles consists of three parts: platform, network, and vehicle. Figure 1 shown.

[0032] 1.1 Platform terminal system

[0033] 1.1.1 High-performance server

[0034] Deployed on the platform side, it is capable of receiving, processing, and storing data from the vehicle side, as well as transmitting monitoring, voice calls, auxiliary control, emergency stop and other instructions issued by the operating system to the vehicle side.

[0035] in:

[0036] (1) The in-vehicle and out-vehicle environment monitoring data is transmitted from the vehicle-side camera to a high-performance server via the 5G network. The high-performance server performs video reception and video decoding to achieve remote monitoring.

[0037] Furthermore, the video image outside the vehicle is a combination of image images that is no less than the field of view of the driver's seat inside the vehicle.

[0038] Furthermore, the in-car video picture is a combination of image pictures covering a visual range of no less than the passenger seat area and upper and lower door areas in the car.

[0039] (2) Vehicle operation status monitoring data is wirelessly transmitted from the vehicle-side CAN bus to a high-performance server.

[0040] Furthermore, the vehicle uploads no less than the following five types of information to the remote platform:

[0041] (a) Basic information on the vehicle's operating status, including vehicle identification (frame number or license plate information, etc.), vehicle control mode, vehicle position, vehicle speed, acceleration, driving direction, and other motion status;

[0042] (b) Vehicle assistance request information (if any), including the type of abnormal incident;

[0043] (c) Assistance results (if any)

[0044] (d) Voice call function status

[0045] Furthermore, when the voice call function is triggered, a clear light signal on the vehicle side prompts passengers that the function has been activated and displays whether the connection is successful.

[0046] Furthermore, when the voice call function is triggered, a clear light signal is emitted from the remote interactive operation device to prompt the remote security officer that the function has been activated and to indicate whether the connection is successful.

[0047] Furthermore, if the voice call fails, the system attempts to re-establish the voice call at intervals not exceeding the interval threshold until the connection is successful or the attempt time exceeds the maximum time limit. The interval threshold can be set to 2 minutes, and the maximum time limit can be set to 60 minutes. Both the interval threshold and the maximum time limit can be adjusted based on actual needs.

[0048] Furthermore, once a voice call connection is established, it can only be interrupted by the remote safety officer and not by passengers in the car. If the voice call connection is interrupted due to an abnormal situation, the voice call connection will be automatically re-established.

[0049] (e) Emergency stop function status

[0050] (3) Voice call data: audio data is sent by the vehicle-side or platform-side operating system and transferred through a high-performance server; at the same time, during audio transmission, it is used to manage the establishment and disconnection of audio connections between the vehicle-side and platform operating system segments.

[0051] (4) Assist in controlling the command data, which is sent from the platform-side operating system to the vehicle-side automatic driving system via the high-performance server. After receiving the command, the automatic driving system sends the decision result of whether to execute the command to the platform-side display system via the high-performance server.

[0052] (5) The emergency stop command data is sent from the platform-side operating system to the vehicle-side brake execution system via the high-performance server. The vehicle-side brake execution system replies with a confirmation message and sends the "emergency stop command execution successful" message to the platform-side display system.

[0053] 1.1.2 Display System

[0054] Deployed on the platform side, the system is capable of displaying monitoring videos of the vehicle interior and exterior environments, alarm information of various abnormal events, and the functional status of voice calls, assisted control, emergency parking, etc.

[0055] 1.1.3 Operating System

[0056] Deployed on the platform side, the system supports manual selection of monitoring functions, opening and closing of voice call functions, issuing and canceling assisted control commands, and issuing emergency stop commands through interactive touch screens, input key keyboards, etc.

[0057] 1.1.4 Remote safety officer status monitoring device

[0058] Deployed on the platform side, it has equipment and instruments that support monitoring the working status of remote security officers through cameras and other means based on image recognition and other technologies. When abnormal behavior of remote security officers is detected, the remote security officers are promptly notified or reported to the enterprise operation management platform.

[0059] Furthermore, the abnormal status and prompt information conditions of the remote security officer are shown in Table 1.

[0060] Table 1

[0061]

[0062] 1.2 Vehicle side

[0063] 1.2.1 Data Upload Requirements

[0064] (1) Support data upload required for remote vehicle external environment monitoring function

[0065] Including the external driving environment information collected by the vehicle environment perception sensor and abnormal information of the external driving environment (if any).

[0066] Among them, abnormal information about the external environment of the vehicle is judged by the automatic driving system. The remote safety officer uses remote equipment to send abnormal signals, passenger help signals, etc., and passively handles the time represented by the signal reasonably. The remote safety officer does not need to monitor each vehicle in real time, but only needs to be able to respond and handle abnormal signals reasonably.

[0067] (2) Support data uploaded for remote in-vehicle environment monitoring

[0068] Including in-vehicle environment information collected by in-vehicle environment perception sensors and in-vehicle environment abnormality information (if any).

[0069] (3) Support data uploaded for remote vehicle operation status monitoring function

[0070] Including vehicle motion status information, automatic driving system status and other information, as well as abnormal vehicle operation status information (if any).

[0071] (4) Data that needs to be uploaded to support the voice call function

[0072] Including in-car audio signals and SOS signals triggered by passengers (if any).

[0073] 1.2.2 Data Receiving Requirements

[0074] Supports receiving monitoring, voice call, auxiliary control, emergency stop and other command information manually issued by the platform operating system.

[0075] 1.3 Network

[0076] The vehicle-side network must provide 24 / 7 5G network coverage within the vehicle operating area and operating route, and meet the requirements of low latency and high reliability. The platform-side network must be directly connected via network cable.

[0077] On the other hand, the present invention also provides a remote assistance method for autonomous driving commercial vehicles, including real-time uploading of monitoring data by the vehicle side, and receiving and routine monitoring of the monitoring data by the platform side. When an abnormal signal occurs, the abnormal event handling process is triggered, and assistance control instructions are issued to the vehicle in a timely manner to conduct voice calls, remote assistance control and emergency parking.

[0078] The above technical solution is described in detail:

[0079] How the remote assistance system works

[0080] 2.1 General monitoring process

[0081] When the platform does not receive any abnormal signals, the remote safety officer monitors the vehicle's interior and exterior environments during the working hours using the video uploaded from the vehicle to the platform; monitors the vehicle's operating status using the vehicle's operating status data uploaded from the vehicle to the platform; and changes the target vehicle in the main monitoring field of view according to the rules set by the remote assistance system or manual switching of the monitored vehicle. Figure 2 shown.

[0082] Because a single remote safety officer is responsible for three or more autonomous vehicles simultaneously, it's impossible for them to monitor every vehicle in real time. Instead, when a problem occurs with one of the vehicles, the remote officer uses the main screen to display the problem vehicle's field of view and information, allowing them to take appropriate action. Currently, the industry has the following technical approach: once the remote assistance system identifies a problem vehicle, it automatically switches the remote safety officer's main screen view to the area around and inside the problem vehicle. However, in some cases, when the vehicle doesn't actively sound an alarm, such as when a passenger inside is unusual, the remote officer needs to actively switch and zoom in on the image.

[0083] Among them, the vehicle end uploads the corresponding monitoring data through the network, and monitors each monitoring data separately by category. In the external vehicle environment monitoring, the platform end receives and displays the uploaded external vehicle environment video through the network, and the remote safety officer monitors whether the external vehicle environment has any abnormalities; in the internal vehicle environment monitoring, the platform end receives and displays the uploaded internal vehicle environment video through the network, and the remote safety officer monitors whether the internal vehicle environment has any abnormalities; in the vehicle operation status monitoring, the platform end receives and displays the uploaded vehicle operation status data through the network, and the remote safety officer monitors whether the operation status has any abnormalities; during the monitoring process, the target vehicle, i.e. the monitoring vehicle, is switched manually or automatically, and the monitoring field of view is switched manually or automatically.

[0084] 2.2 Abnormal Event Handling Process

[0085] When the platform receives an abnormal signal and the remote safety officer determines it as an abnormal event, the event category is divided into voice call category, remote assistance category and emergency stop category according to the remote safety officer's handling method. The handling process of each type of event is shown in Figure 3 .

[0086] 2.2.1 Voice Call Event Handling Process

[0087] When a passenger in the car manually triggers SOS or the vehicle performs an emergency stop and rear lights up event, the two-way voice call function between the vehicle and platform will be activated. In addition, when the remote safety officer monitors abnormalities of the passengers in the car, the two-way voice call function will be actively activated. After the call is connected, the remote safety officer will handle the incident through voice.

[0088] 2.2.2 Remote Assistance Incident Handling Process

[0089] When the vehicle encounters an event during driving that requires the remote safety officer to issue a remote assistance command in order to continue the operation, the remote safety officer will issue the corresponding assistance command based on the specific event type. After receiving the assistance command, the vehicle-side automatic driving system will determine whether to execute it. If it is executed, the corresponding information will be displayed during and after the assistance process to prompt the remote safety officer, passengers in the car, and traffic participants around the vehicle. If it is not executed, the remote safety officer and passengers in the car will be promptly prompted by signals. After the remote safety officer learns of "assistance failure" through the platform-side display system, he will immediately report the information.

[0090] The above-mentioned event types include: the vehicle encounters a solid line and is unable to cross the solid line and change lanes on its own. At this time, the autonomous driving system automatically uploads the information to the remote safety officer for processing; the traffic lights at the intersection are temporarily off, the current road is temporarily closed, and the vehicle needs to use the non-motorized vehicle lane, etc.

[0091] It should be noted that remote assistance involves sending recommendations to the autonomous driving system. The autonomous driving system will execute the assistance command after determining there is no danger based on its own perception capabilities. However, it may not execute the assistance command if it determines there is danger. Therefore, the assistance control command is only a control recommendation, not direct control.

[0092] Furthermore, the classification of assistance control instructions includes three categories: driving direction suggestions, driving speed suggestions, and driving path (trajectory) guidance. The specific instructions included in each category are shown in Table 2, which is the classification of remote assistance control instructions.

[0093] Table 2

[0094]

[0095] Furthermore, when the autonomous driving system executes the assistance control command, the remote assistance system interactive operation device has a clear light signal to prompt the remote safety officer, and the graphic symbol displays keywords such as "assistance instruction type (see Table 1)" + "assisting"; when the assistance is completed, the graphic symbol displays keywords such as "assistance completed".

[0096] Furthermore, when the autonomous driving system executes assist control instructions, there are clear light signals on the vehicle side to alert passengers.

[0097] Furthermore, when the remote safety officer issues an assistance control command but the autonomous driving system does not execute it, the remote assistance system interactive operation device will give a clear light signal to prompt the remote safety officer, and the graphic symbol will display keywords such as "assistance command type (see Table 1)" + "assistance failure".

[0098] Furthermore, after assisting in the issuance of control instructions and the autonomous driving system determines not to execute the instructions, it prompts the remote safety officer within the allowable range of (5G network transmission delay + system display delay).

[0099] Furthermore, the autonomous driving system has the highest priority on whether to execute the assist control instruction.

[0100] Furthermore, before the remote safety officer issues the assistance control command, the video surveillance object is switched to the target vehicle, and the headset uploads the ambient sound of the target vehicle until the assistance is completed or fails.

[0101] 2.2.3 Emergency Stop Incident Handling Process

[0102] If a vehicle encounters an emergency stop requiring a remote safety officer to ensure operational safety, the remote officer will issue the command to the vehicle's brake actuation system. Once the emergency stop is triggered, if the autonomous driving system also performs longitudinal acceleration, only the emergency stop brake control signal will be activated. Simultaneously, the autonomous driving system transitions from active to deactivated. After a safe stop, the voice communication function between the vehicle and the platform is automatically activated.

[0103] Emergency stops may occur in extreme situations, such as social security incidents occurring in the car, passenger hijacking, etc.

[0104] Furthermore, within 0.8 seconds after the emergency stop command is issued, the vehicle continues to emit rear-end collision warning signals in accordance with GB4785 regulations and keeps honking the horn to alert the following vehicles.

[0105] Furthermore, 0.8 seconds after the emergency stop command is issued, the vehicle initiates the emergency stop command and gives clear sound and light signals to remind passengers in the vehicle that the system has been activated. The sound signal is a continuous rapid beeping sound; at the same time, the vehicle's brake signal lights up and remains on until the vehicle stops.

[0106] Furthermore, when the vehicle initiates the emergency stop command, the remote assistance system interactive operation device has clear sound and light signals to remind the remote safety officer that the system has been activated. The sound signal is a continuous rapid beep, and the light signal device on the interactive operation device complies with the emergency stop symbol specified in GB / T 16754-2021. The signal device display color is red.

[0107] Furthermore, when the emergency stop function is triggered, if the automatic driving system simultaneously executes the vehicle's longitudinal acceleration control, the vehicle will only sound the braking control signal of the emergency stop function; at the same time, the automatic driving system will change from the activated state to the exited state.

[0108] Furthermore, after the vehicle stops, the voice call function between the vehicle and the remote assistance system is automatically turned on.

[0109] Technological advancements include the following:

[0110] 1. In accordance with the operational supervision requirements of the autonomous vehicle industry and the work convenience of remote safety officers, by adding "data filtering technology (nested in the data box)" to the high-performance server, specific data uploaded and received by the vehicle side is processed and processed, rather than uploading all vehicle-side data, and the vehicle side does not receive all remote assistance system data.

[0111] 2. The remote safety officer status monitoring device improves the existing driver status monitoring device. First, the abnormal status judgment parameter basis is improved, and new judgment factors such as "taking off the headset, leaving the workstation" are added (the specific judgment algorithm can be expanded by combining image recognition, deep learning, etc.); second, the status monitoring device is connected to the enterprise operation and management platform, including timely reporting of abnormal status information to the enterprise operation and management platform.

[0112] 3. Innovation in the mechanism or system of remote control parking, including the technology of directly controlling the vehicle's braking execution system EBS and ESC remotely, and a series of handling processes (logic) after the remote control command is issued.

[0113] 4. Display logic and algorithm innovation of the display system, such as routine monitoring described in 2.1, display content when abnormal events occur described in 2.2, screen switching algorithm, screen segmentation, etc.

[0114] 5. The proposed remote assistance system has been developed and created, based on specific operational needs, vehicle-side data filtering technology, remote safety officer status monitoring technology, remote emergency parking technology, and multi-functional screen display technology; in addition, the hardware of each system is effectively combined to work together to ensure that the remote safety officer can complete the safety protection and operation services of unmanned commercial applications in the vehicle through the remote assistance system.

[0115] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A remote assistance system for autonomous driving commercial vehicles, characterized in that: include: The platform side, the network and the vehicle side, wherein the platform side exchanges data with the vehicle side via the network; The vehicle end is arranged on an autonomous driving operating vehicle; The platform receives the monitoring data uploaded by the vehicle through the network, processes and stores the monitoring data, generates corresponding control instructions based on the monitoring data, sends the control instructions to the vehicle through the network, and provides remote assistance to the vehicle based on the control instructions.

2. The system according to claim 1, wherein: The vehicle-side monitoring data includes external driving environment information, external driving environment abnormality information, internal vehicle environment information, internal vehicle environment abnormality information, vehicle motion status information, automatic driving system status information, vehicle operation status abnormality information, voice trigger signals, audio information and assistance request signals.

3. The system according to claim 1, wherein: The network is connected to the platform via a wired connection, and the network is connected to the vehicle via a wireless connection.

4. The system according to claim 1, wherein: The platform includes a high-performance server, which receives, processes and stores monitoring data, obtains control instructions based on the monitoring data, and sends the control instructions to the vehicle. The control instructions include: continued monitoring, voice call, auxiliary control and emergency stop.

5. The system according to claim 1, wherein: The platform also includes a status monitoring device, wherein the working status of the remote safety officer is monitored by the status detection device, and prompts and reports are given to the remote safety officer based on the working status monitoring results.

6. The assistance method of the remote assistance system for an autonomous driving vehicle according to any one of claims 1 to 5, characterized in that: include: The platform receives and monitors the monitoring data uploaded by the vehicle through the network. When an abnormal situation occurs in the monitoring data, it handles the abnormal event and issues instructions to the vehicle for voice calls, remote assistance and emergency parking.

7. The method according to claim 6, characterized in that When the platform monitors the monitoring data and no abnormal signal is received in the monitoring data, the monitoring data is categorized and monitored by the remote safety officer, wherein the platform receives and displays the uploaded outdoor environment video through the network, and the remote safety officer monitors whether the outdoor environment is abnormal; in the in-vehicle environment monitoring, the platform receives and displays the uploaded in-vehicle environment video through the network, and the remote safety officer monitors whether the in-vehicle environment is abnormal; in the vehicle operation status monitoring, the platform receives and displays the uploaded vehicle operation status data through the network, and the remote safety officer monitors whether the operation status is abnormal; during the monitoring process, the target vehicle, i.e. the monitored vehicle, is switched manually or automatically, and the monitoring field of view is switched manually or automatically.

8. The method according to claim 6, characterized in that The process of handling abnormal events includes: processing voice call events, among which, when the vehicle side receives an assistance request signal triggered by a passenger, it provides a voice trigger signal to the platform side, establishes a voice call connection between the platform side and the vehicle side, and transmits audio data in real time in both directions for voice calls between the remote security officer and the passenger.

9. The method according to claim 6, characterized in that The process of handling abnormal events also includes: handling remote assistance events, where when a remote assistance instruction is triggered during vehicle driving to continue to complete an operation event, the remote safety officer, based on the event type, uses the platform to send the corresponding remote assistance instruction to the vehicle side. After the vehicle side controls the vehicle's automatic driving system, it determines whether to execute the remote assistance instruction. If it is executed, the remote safety officer and passengers are prompted and the instruction is broadcast to the passengers. If it is not executed, the assistance failure information is returned to the platform side for the remote safety officer and passengers to prompt and report.

10. The method according to claim 6, characterized in that The process of handling abnormal events also includes: handling emergency stop events, where when the remote safety officer detects an emergency stop event on the platform side, the remote safety officer interacts with the platform side to send an emergency stop command to the vehicle side, and the vehicle side controls the vehicle braking execution system to perform an emergency stop. When an emergency stop occurs, if the vehicle's automatic driving system executes the vehicle's longitudinal acceleration control, the vehicle only performs an emergency stop and the vehicle's automatic driving system switches to the exit state. After the vehicle stops, a voice call connection is established between the vehicle and the remote safety officer.

Images