Remote assistance method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a remote assistance method and device, computer equipment, a storage medium and a computer program product. The method is applied to an automatic driving vehicle, and comprises the following steps: after an operation of requesting remote assistance is received, sending a remote assistance request and a first call request to a vehicle remote control center; wherein the remote assistance request is used for requesting the vehicle remote control center to establish a remote assistance link with the autonomous vehicle, and the first call request is used for requesting the vehicle remote control center to establish a first call link with the autonomous vehicle. By adopting the method, the vehicle can get out of trouble in time when the remote assistance link is abnormal.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a remote assistance method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the continuous development of autonomous driving technology, many autonomous driving vehicles have gradually begun to be put into operation. When an autonomous driving vehicle is running on the road, if it gets stuck or other situations occur, it will send a remote assistance request to the vehicle remote control center, and operate according to the remote operation instructions sent by the vehicle remote control center to get out of trouble.

[0003] However, when the remote assistance link between the vehicle remote control center and the autonomous driving vehicle is abnormal, the vehicle remote control center may have issued a remote operation command, but the autonomous driving vehicle has not received it, and the people on the vehicle have been waiting, resulting in the vehicle being unable to escape in time. Summary of the invention

[0004] Based on this, it is necessary to provide a remote assistance method, apparatus, computer equipment, storage medium and computer program product to address the above technical problems, which can enable the vehicle to escape from trouble in time when the remote assistance link is abnormal.

[0005] In a first aspect, the present application provides a remote assistance method applied to an autonomous driving vehicle, the method comprising: after receiving an operation requesting remote assistance, sending a remote assistance request and a first call request to a vehicle remote control center; wherein the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

[0006] In one of the embodiments, the method further includes: if the first call link fails to be established, detecting the number of failures in establishing the first call link; if the number of failures is less than or equal to a threshold, resending the first call request to the vehicle remote control center.

[0007] In one of the embodiments, the method further includes: if the number of failures is greater than a threshold number, instructing the vehicle remote control center to send a second call request to a target terminal corresponding to a person on the autonomous driving vehicle, the second call request being used to request the target terminal to establish a second call link with the vehicle remote control center.

[0008] In one of the embodiments, the method further includes: detecting whether the autonomous driving vehicle meets the conditions for terminating remote assistance; if the conditions for terminating remote assistance are met, disconnecting the remote assistance link, and / or the first call link.

[0009] In one of the embodiments, detecting whether the autonomous driving vehicle meets the conditions for terminating remote assistance includes: detecting whether the autonomous driving vehicle is taken over manually; if it is detected that the autonomous driving vehicle is taken over manually, then the autonomous driving vehicle meets the conditions for terminating remote assistance.

[0010] In one of the embodiments, detecting whether the autonomous driving vehicle meets the conditions for terminating remote assistance includes: detecting whether an operation requesting to hang up is received; if an operation requesting to hang up is detected, then the autonomous driving vehicle meets the conditions for terminating remote assistance.

[0011] In one of the embodiments, the method further includes: controlling a camera in a cockpit of the autonomous driving vehicle to turn on so as to capture an operation image of a person in the cockpit through the camera; and sending the operation image to a vehicle remote control center through a remote assistance link.

[0012] In one embodiment, the operation of requesting remote assistance includes: triggering the remote assistance control on the human-computer interaction interface in the autonomous driving vehicle, or, voice command, or gesture command, or, triggering the physical button in the autonomous driving vehicle.

[0013] In a second aspect, the present application also provides a remote assistance device, which is arranged in an autonomous driving vehicle, and the device includes: a sending module, which is used to send a remote assistance request and a first call request to a vehicle remote control center after receiving an operation requesting remote assistance; wherein, the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

[0014] In one of the embodiments, the sending module is further used to detect the number of failures in establishing the first call link if the first call link fails to be established; if the number of failures is less than or equal to the number threshold, re-send the first call request to the vehicle remote control center.

[0015] In one of the embodiments, the sending module is also used to instruct the vehicle remote control center to send a second call request to the target terminal corresponding to the person on the autonomous driving vehicle if the number of failures is greater than a threshold number, and the second call request is used to request the target terminal to establish a second call link with the vehicle remote control center.

[0016] In one of the embodiments, the device also includes a detection module for detecting whether the autonomous driving vehicle meets the conditions for terminating remote assistance; if the conditions for terminating remote assistance are met, the remote assistance link and / or the first call link are disconnected.

[0017] In one of the embodiments, the detection module is specifically used to detect whether the autonomous driving vehicle is manually taken over; if it is detected that the autonomous driving vehicle is manually taken over, the autonomous driving vehicle meets the conditions for terminating remote assistance.

[0018] In one of the embodiments, the detection module is specifically used to detect whether an operation requesting to hang up is received; if an operation requesting to hang up is detected, the autonomous driving vehicle meets the conditions for terminating the remote assistance.

[0019] In one of the embodiments, the sending module is also used to control the camera in the cockpit of the autonomous driving vehicle to turn on, so as to capture the operation image of the people in the cockpit through the camera; and send the operation image to the vehicle remote control center through the remote assistance link.

[0020] In one embodiment, the operation of requesting remote assistance includes: triggering the remote assistance control on the human-computer interaction interface in the autonomous driving vehicle, or, voice command, or gesture command, or, triggering the physical button in the autonomous driving vehicle.

[0021] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in any one of the first aspects are implemented.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects above are implemented.

[0023] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0024] According to the above-mentioned remote assistance method, apparatus, computer equipment, storage medium and computer program product, after receiving the operation requesting remote assistance, the autonomous driving vehicle sends a remote assistance request and a first call request to the vehicle remote control center; wherein the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle. In this way, personnel on the vehicle (such as vehicle safety officers or passengers) can communicate with the vehicle remote control center in real time through the first call link. After the remote operation instruction is issued, the personnel on the vehicle can know that the instruction has been issued. If the vehicle does not respond in time, the personnel on the vehicle do not need to wait any longer and can take over the vehicle in time, thereby helping the vehicle to get out of trouble in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 is an application environment diagram of a remote assistance method in an embodiment;

[0027] Figure 2 is a flowchart of a remote assistance method in an embodiment;

[0028] Figure 3 A diagram of a human-computer interaction interface in remote assistance in one embodiment;

[0029] Figure 4 is a structural block diagram of a remote assistance device in one embodiment;

[0030] Figure 5 is a structural block diagram of another remote assistance device in one embodiment;

[0031] Figure 6 The figure is a diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] With the continuous development of autonomous driving technology, many autonomous driving vehicles have gradually begun to be put into operation. When an autonomous driving vehicle is running on the road, if it gets stuck or other situations occur, it will send a remote assistance request to the vehicle remote control center to operate according to the remote operation instructions sent by the vehicle control center to achieve escape.

[0034] However, when the remote assistance link between the vehicle remote control center and the autonomous driving vehicle is abnormal, or the remote operation instruction is issued abnormally, the vehicle remote control center may have issued the remote operation instruction, but the autonomous driving vehicle has not received it, and the people on the vehicle have been waiting, resulting in the vehicle being unable to escape in time.

[0035] In addition, if the vehicle remote control center responds slowly, the vehicle safety officer may not be able to wait to take over in advance, but in fact the autonomous vehicle can solve the problem autonomously. For example, the vehicle is stuck. From the moment it gets stuck, at the 4th second, the vehicle has not responded accordingly and the vehicle safety officer takes over the vehicle. But in fact, at the 5th second, the vehicle will receive the instruction from the vehicle remote control center and take corresponding actions. This will lead to unnecessary takeover problems and will also have a negative impact on MPI (Multi Point Interface).

[0036] Based on this, there are currently problems that the vehicle cannot be rescued in time, unnecessary takeovers, and negative impacts on MPI. Therefore, the embodiment of the present application provides a remote assistance method, where the autonomous driving vehicle sends a remote assistance request and a first call request method to the vehicle remote control center after receiving an operation requesting remote assistance, so that the personnel on the vehicle can talk to the vehicle remote control center in real time, so that when the remote assistance link is abnormal, the personnel on the vehicle do not need to continue waiting, and can directly take over the vehicle to rescue the vehicle, and can avoid unnecessary takeover problems and avoid negative impacts on MPI.

[0037] It should be noted that the beneficial effects brought about by the embodiments of the present application or the technical problems solved are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0038] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0039] The remote assistance method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the autonomous driving vehicle 102 communicates with the vehicle remote control center 104 through the network. The remote control center 104 can be an independent server or a server cluster composed of multiple servers, or various personal computers, etc. Figure 1 A personal computer is used as an example.

[0040] In an exemplary embodiment, Figure 2 As shown, a remote assistance method is provided, which is applied to Figure 1 The automatic driving vehicle in the embodiment is described, including the following steps 201 and 202.

[0041] Step 201, after receiving the operation of requesting remote assistance, sending a remote assistance request to the vehicle remote control center; wherein the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle.

[0042] Among them, the operation of requesting remote assistance includes: triggering the remote assistance control on the human-computer interaction interface in the autonomous driving vehicle, or, voice command, or gesture command, or, triggering the physical button in the autonomous driving vehicle.

[0043] Optionally, the autonomous driving vehicle includes a human-machine interaction interface and a remote assistance module. The human-machine interaction interface displays a remote assistance control. When the human-machine interaction interface detects a trigger operation for the remote assistance control, the human-machine interaction interface sends a notification message to the remote assistance module. After receiving the notification message, the remote assistance module sends a remote assistance request to the vehicle remote control center. Figure 3 As shown, a diagram of a human-computer interaction interface in remote assistance is provided.

[0044] Alternatively, after the autonomous driving vehicle receives a voice command instructing the autonomous driving vehicle to send a remote assistance request, such as "request remote assistance", the autonomous driving vehicle recognizes the voice command and sends a remote assistance request to the vehicle remote control center based on the recognition result.

[0045] Alternatively, after the human-computer interaction interface detects a gesture instruction on the human-computer interaction interface for instructing the autonomous driving vehicle to send a remote assistance request, for example, after the human-computer interaction interface detects that the human-computer interaction interface has been double-clicked, the human-computer interaction interface sends a notification message to the remote assistance module, and after receiving the notification message, the remote assistance module sends a remote assistance request to the vehicle remote control center.

[0046] Alternatively, the autonomous driving vehicle includes a physical button and a pressure sensor. When the pressure sensor detects that the physical button is triggered, it sends a pressure signal to the remote assistance module. After receiving the pressure signal, the remote assistance module determines whether the pressure signal is greater than a signal threshold. If so, it sends a remote assistance request to the vehicle remote control center.

[0047] When the vehicle remote control center receives the remote assistance request, it will establish a remote assistance link with the autonomous vehicle. After the remote assistance link is successfully established, the autonomous vehicle can send vehicle status information, road information, etc. to the vehicle remote control center through the remote assistance link, and the vehicle remote control center can send remote operation instructions, etc. to the autonomous vehicle through the remote assistance link.

[0048] Step 202, sending a first call request to the vehicle remote control center; wherein the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

[0049] In one scenario, there is a vehicle safety officer on the autonomous driving vehicle. The first call link is used for the vehicle safety officer to conduct voice calls and / or video calls with the vehicle remote control center to facilitate real-time communication between the vehicle safety officer and the vehicle remote control center.

[0050] However, with the development of autonomous vehicles, vehicle safety officers may no longer be needed in autonomous vehicles. In this scenario, passengers may be required to perform some simple operations on the vehicle. In this case, the first call link is used for passengers to make voice calls and / or video calls with the vehicle remote control center, which can facilitate real-time communication between passengers and the vehicle remote control center.

[0051] In another scenario, after the autonomous driving vehicle completes a task according to a remote operation instruction issued by the vehicle remote control center, the voice broadcast is completed. Then, the first call link is used for the autonomous driving vehicle to conduct a voice call and / or video call with the vehicle remote control center. The vehicle remote control center can quickly know that the previous remote operation instruction has been executed, and then issue the next remote operation instruction.

[0052] Optionally, when the human-computer interaction interface detects a trigger operation for the remote assistance control, in addition to sending a notification message to the remote assistance module, it also sends a first call request to the vehicle remote control center. After the vehicle remote control center receives the first call request, it establishes a first call link with the autonomous driving vehicle.

[0053] In order to facilitate understanding of how to establish the first call link, a vivid description is given here, that is, after the human-computer interaction interface sends the first call request, it enters the call room and waits until the vehicle remote control center receives the first call request and enters the call room. The first call link is successfully established. At this time, the vehicle remote control center and the autonomous driving vehicle can talk. If the vehicle remote control center does not enter the call room, the first call link fails to be established.

[0054] In the above-mentioned remote assistance method, after receiving the operation requesting remote assistance, the autonomous driving vehicle sends a remote assistance request and a first call request to the vehicle remote control center; wherein, the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle. In this way, the personnel on the vehicle can communicate with the vehicle remote control center in real time through the first call link. After the remote operation instruction is issued, the personnel on the vehicle can know that the instruction has been issued. If the vehicle does not respond in time, the personnel does not need to wait any longer and can take over the vehicle in time, thereby helping the vehicle to get out of trouble in time.

[0055] In addition, since the personnel on the vehicle can have real-time conversations with the vehicle remote control center through the first communication link, the personnel on the vehicle can know the command issuance status of the vehicle remote control center in real time, thereby avoiding unnecessary takeover and further avoiding negative impact on MPI.

[0056] In addition, for unmanned autonomous driving vehicles, the above-mentioned first communication link can also be used for passengers to communicate with the vehicle remote control center.

[0057] In one of the embodiments, the method further includes: if the first call link fails to be established, detecting the number of failures in establishing the first call link; if the number of failures is less than or equal to a threshold, resending the first call request to the vehicle remote control center.

[0058] The number threshold can be set as needed, for example, 3 times, and is not limited here.

[0059] Optionally, after the human-computer interaction interface sends the first call request, if no response message is received from the vehicle remote control center in response to the first call request within a preset time period, it means that the first call link has failed to be established. If the number of failures is less than or equal to the number threshold, the first call request is resent to the vehicle remote control center; if the number of failures is greater than the number threshold, the first call request is no longer sent to the vehicle remote control center.

[0060] In this embodiment, since the vehicle remote control center has to maintain a large number of autonomous driving vehicles, and the vehicle remote center can only establish a limited number of call links within the same time period, if more than this number of autonomous driving vehicles simultaneously request to establish call links within the same time period, there will be a busy line problem. If the autonomous driving vehicle fails to establish a call link and sends a large number of call requests to the vehicle remote control center multiple times, the busy line problem will be aggravated. If the first call link fails to be established, the first call request will be resent only when the number of failures is less than or equal to the number threshold, which can effectively avoid aggravating the busy line problem.

[0061] In one of the embodiments, if the number of failures is greater than a threshold number, the vehicle remote control center is instructed to send a second call request to a target terminal corresponding to the person on the autonomous driving vehicle, and the second call request is used to request the target terminal to establish a second call link with the vehicle remote control center.

[0062] The person in the vehicle can be a vehicle safety officer or a passenger. The second call link is used for the person in the vehicle to make a voice call and / or a video call with the vehicle remote control center, and can also be used for the autonomous driving vehicle to make a voice call and / or a video call with the vehicle remote control center.

[0063] When there is a vehicle safety officer on the autonomous driving vehicle, there are two optional ways to implement this embodiment:

[0064] The first method is that if it is detected that the number of failures to establish the first call link is greater than the number threshold, a first indication message carrying the target vehicle identifier is sent to the vehicle remote control center through the remote assistance link. The target vehicle identifier is the identifier of the autonomous driving vehicle, such as the license plate number. After receiving the first indication message, the vehicle remote control center queries the first database according to the target vehicle identifier to obtain the target terminal identifier of the target terminal corresponding to the vehicle safety officer. The first database stores multiple sets of correspondence between vehicle identifiers and terminal identifiers. The vehicle remote control center calls the target terminal according to the target terminal identifier to establish the second call link.

[0065] The second method is that if it is detected that the number of failures in establishing the first call link is greater than the number threshold, the second database is queried to obtain the target terminal identifier of the target terminal corresponding to the vehicle safety officer, and then a second instruction message carrying the target terminal identifier is sent to the vehicle remote control center through the remote assistance link. After receiving the second instruction message, the vehicle remote control center makes a call to the target terminal according to the target terminal identifier to establish the second call link.

[0066] In the case where there is no vehicle safety officer on the autonomous driving vehicle, that is, for an unmanned autonomous driving vehicle, there are two optional ways to implement this embodiment:

[0067] The first method is that if it is detected that the number of failures in establishing the first call link is greater than the number threshold, with the authorization of the passenger, the waybill information is queried to obtain the passenger information, and a third instruction message carrying the passenger information is sent to the vehicle remote control center through the remote assistance link. After receiving the third instruction message, the vehicle remote control center makes a call to the target terminal corresponding to the passenger according to the passenger information to establish the second call link.

[0068] The second method is that if it is detected that the number of failures in establishing the first call link is greater than the number threshold, a fourth instruction message carrying the target vehicle identifier is sent to the vehicle remote control center through the remote assistance link, and the target vehicle identifier is the identifier of the autonomous driving vehicle. After receiving the fourth instruction message, the vehicle remote control center queries the waybill information and obtains the passenger information with the authorization of the passenger, and calls the target terminal corresponding to the passenger according to the passenger information to establish the second call link.

[0069] In this embodiment, when the first call link fails to be established, a second call link is established, thereby avoiding the problem that the personnel in the vehicle cannot communicate with the vehicle remote control center due to the failure to establish the first call link.

[0070] In one of the embodiments, the method further includes: detecting whether the autonomous driving vehicle meets the conditions for terminating remote assistance; if the conditions for terminating remote assistance are met, disconnecting the remote assistance link, and / or the first call link.

[0071] Optionally, it is detected whether the autonomous driving vehicle is manually taken over; if it is detected that the autonomous driving vehicle is manually taken over, the autonomous driving vehicle meets the conditions for terminating the remote assistance. At this time, the remote assistance link and / or the first call link are disconnected.

[0072] For example, if it is detected that the brakes of the autonomous driving vehicle are stepped on, or the steering wheel is manually turned to the left, it means that the autonomous driving vehicle is taken over manually. In this case, the autonomous driving vehicle disconnects the remote assistance link but does not disconnect the first call link, or disconnects the remote assistance link and the first call link, or does not disconnect the remote assistance link but disconnects the first call link.

[0073] Among them, if the conditions for terminating remote assistance are met, the remote assistance link and / or the first call link are disconnected, including: if the conditions for terminating remote assistance are met, the remote assistance link is disconnected first, and when a request to hang up the first call link is detected, the first call link is disconnected.

[0074] In another optional embodiment, it is detected whether a hang-up request operation is received; if a hang-up request operation is detected, the autonomous driving vehicle meets the condition for terminating the remote assistance. At this time, the remote assistance link and / or the first call link are disconnected.

[0075] For example, after the first call link is successfully established, a hang-up control is displayed on the human-computer interaction interface. When a trigger operation for the hang-up control is detected, the remote assistance link is disconnected, the first call link is disconnected, or the remote assistance link and the first call link are disconnected, or the remote assistance link is disconnected without disconnecting the first call link.

[0076] In yet another optional embodiment, after the vehicle remote control center hangs up the first call link and the remote assistance link, it also disconnects the remote assistance link and the first call link.

[0077] In this embodiment, after the autonomous driving vehicle or the vehicle remote control center hangs up the remote assistance link, the remote assistance link is disconnected after 10 seconds. However, in this application, after detecting that the autonomous driving vehicle is manually taken over, the remote assistance link and / or the first call link are disconnected.

[0078] In one of the embodiments, the method further includes: controlling a camera in a cockpit of the autonomous driving vehicle to turn on so as to capture an operation image of a person in the cockpit through the camera; and sending the operation image to a vehicle remote control center through a remote assistance link.

[0079] Among them, the person in the cockpit can be a vehicle safety officer or a passenger.

[0080] Optionally, the autonomous driving vehicle sends a power-on signal to the camera. After receiving the power-on signal, the camera starts taking pictures of the cockpit, thereby obtaining operation images of the people in the cockpit. The camera then sends the operation images to the remote assistance module in the autonomous driving vehicle, and the remote assistance module sends the operation images to the vehicle remote control center through the remote assistance link.

[0081] In this embodiment, the camera in the cockpit of the autonomous driving vehicle is controlled to turn on, so as to capture the operation image of the person in the cockpit through the camera, and then send the operation image to the vehicle remote control center through the remote assistance link. In this way, the vehicle remote control center can monitor the person's operation of the vehicle in real time, and then guide the person's operation through the first call link or the second call link to achieve timely escape of the vehicle.

[0082] In addition, it should be noted that in the present application, the first call link and the remote assistance link are independent of each other. Failure to establish the first call link does not affect the establishment of the remote assistance link, nor does it affect the communication between the autonomous driving vehicle and the vehicle remote control center through the remote assistance link.

[0083] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0084] Based on the same inventive concept, the embodiment of the present application also provides a remote assistance device for implementing the remote assistance method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more remote assistance device embodiments provided below can refer to the limitations on the remote assistance method above, and will not be repeated here.

[0085] In an exemplary embodiment, Figure 4 As shown, a remote assistance device is provided, the remote assistance device 400 includes: a sending module 401, the sending module is arranged in the autonomous driving vehicle, wherein:

[0086] The sending module 401 is used to send a remote assistance request and a first call request to the vehicle remote control center after receiving an operation requesting remote assistance; wherein the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

[0087] In one of the embodiments, the sending module 401 is further used to detect the number of failures in establishing the first call link if the first call link fails to be established; if the number of failures is less than or equal to the number threshold, resend the first call request to the vehicle remote control center.

[0088] In one of the embodiments, the sending module 401 is also used to instruct the vehicle remote control center to send a second call request to the target terminal corresponding to the person on the autonomous driving vehicle if the number of failures is greater than a threshold number, and the second call request is used to request the target terminal to establish a second call link with the vehicle remote control center.

[0089] In one embodiment, if Figure 5As shown, another remote assistance device is provided, and the remote assistance device 400 also includes a detection module 402, which is used to detect whether the autonomous driving vehicle meets the conditions for ending the remote assistance; if the conditions for ending the remote assistance are met, the remote assistance link and / or the first call link are disconnected. The detection module is set in the autonomous driving vehicle.

[0090] In one of the embodiments, the detection module 402 is specifically used to detect whether the autonomous driving vehicle is manually taken over; if it is detected that the autonomous driving vehicle is manually taken over, the autonomous driving vehicle meets the conditions for terminating remote assistance.

[0091] In one embodiment, the detection module 402 is specifically used to detect whether a hang-up request operation is received; if a hang-up request operation is detected, the autonomous driving vehicle meets the conditions for terminating the remote assistance.

[0092] In one of the embodiments, the sending module 401 is also used to control the camera in the cockpit of the autonomous driving vehicle to turn on, so as to capture the operation image of the people in the cockpit through the camera; and send the operation image to the vehicle remote control center through the remote assistance link.

[0093] In one embodiment, the operation of requesting remote assistance includes: triggering the remote assistance control on the human-computer interaction interface in the autonomous driving vehicle, or, voice command, or gesture command, or, triggering the physical button in the autonomous driving vehicle.

[0094] Each module in the above remote assistance device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0095] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a remote assistance method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0096] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0097] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0098] After receiving the operation of requesting remote assistance, a remote assistance request and a first call request are sent to the vehicle remote control center; wherein the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

[0099] In one of the embodiments, when the processor executes the computer program, the following steps are also implemented: if the first call link fails to be established, the number of failures in establishing the first call link is detected; if the number of failures is less than or equal to the number threshold, the first call request is resent to the vehicle remote control center.

[0100] In one of the embodiments, the processor further implements the following steps when executing the computer program: if the number of failures is greater than a threshold number, the vehicle remote control center is instructed to send a second call request to the target terminal corresponding to the person on the autonomous driving vehicle, and the second call request is used to request the target terminal to establish a second call link with the vehicle remote control center.

[0101] In one of the embodiments, when the processor executes the computer program, it also implements the following steps: detecting whether the autonomous driving vehicle meets the conditions for ending remote assistance; if the conditions for ending remote assistance are met, disconnecting the remote assistance link, and / or the first call link.

[0102] In one of the embodiments, when the processor executes the computer program, it also implements the following steps: detecting whether the autonomous driving vehicle is manually taken over; if it is detected that the autonomous driving vehicle is manually taken over, the autonomous driving vehicle meets the conditions for ending remote assistance.

[0103] In one of the embodiments, when the processor executes the computer program, it also implements the following steps: detecting whether an operation requesting to hang up is received; if an operation requesting to hang up is detected, the autonomous driving vehicle meets the conditions for ending the remote assistance.

[0104] In one of the embodiments, when the processor executes the computer program, it also implements the following steps: controlling the camera in the cockpit of the autonomous driving vehicle to turn on so as to capture the operation image of the people in the cockpit through the camera; and sending the operation image to the vehicle remote control center through the remote assistance link.

[0105] In one embodiment, the operation of requesting remote assistance includes: triggering the remote assistance control on the human-computer interaction interface in the autonomous driving vehicle, or, voice command, or gesture command, or, triggering the physical button in the autonomous driving vehicle.

[0106] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0107] After receiving the operation of requesting remote assistance, a remote assistance request and a first call request are sent to the vehicle remote control center; wherein the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

[0108] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: if the first call link fails to be established, the number of failures in establishing the first call link is detected; if the number of failures is less than or equal to a threshold, the first call request is resent to the vehicle remote control center.

[0109] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: if the number of failures is greater than a threshold number, the vehicle remote control center is instructed to send a second call request to the target terminal corresponding to the person on the autonomous driving vehicle, and the second call request is used to request the target terminal to establish a second call link with the vehicle remote control center.

[0110] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: detecting whether the autonomous driving vehicle meets the conditions for ending remote assistance; if the conditions for ending remote assistance are met, disconnecting the remote assistance link, and / or the first call link.

[0111] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: detecting whether the autonomous driving vehicle is manually taken over; if it is detected that the autonomous driving vehicle is manually taken over, the autonomous driving vehicle meets the conditions for ending remote assistance.

[0112] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: detecting whether an operation requesting to hang up is received; if an operation requesting to hang up is detected, the autonomous driving vehicle meets the conditions for ending the remote assistance.

[0113] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: controlling the camera of the cockpit of the autonomous driving vehicle to turn on so as to capture the operation image of the people in the cockpit through the camera; and sending the operation image to the vehicle remote control center through the remote assistance link.

[0114] In one embodiment, the operation of requesting remote assistance includes: triggering the remote assistance control on the human-computer interaction interface in the autonomous driving vehicle, or, voice command, or gesture command, or, triggering the physical button in the autonomous driving vehicle.

[0115] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0116] After receiving the operation of requesting remote assistance, a remote assistance request and a first call request are sent to the vehicle remote control center; wherein the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

[0117] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: if the first call link fails to be established, the number of failures in establishing the first call link is detected; if the number of failures is less than or equal to a threshold, the first call request is resent to the vehicle remote control center.

[0118] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: if the number of failures is greater than a threshold number, the vehicle remote control center is instructed to send a second call request to the target terminal corresponding to the person on the autonomous driving vehicle, and the second call request is used to request the target terminal to establish a second call link with the vehicle remote control center.

[0119] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: detecting whether the autonomous driving vehicle meets the conditions for ending remote assistance; if the conditions for ending remote assistance are met, disconnecting the remote assistance link, and / or the first call link.

[0120] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: detecting whether the autonomous driving vehicle is manually taken over; if it is detected that the autonomous driving vehicle is manually taken over, the autonomous driving vehicle meets the conditions for ending remote assistance.

[0121] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: detecting whether an operation requesting to hang up is received; if an operation requesting to hang up is detected, the autonomous driving vehicle meets the conditions for ending the remote assistance.

[0122] In one of the embodiments, when the computer program is executed by the processor, the following steps are also implemented: controlling the camera of the cockpit of the autonomous driving vehicle to turn on so as to capture the operation image of the people in the cockpit through the camera; and sending the operation image to the vehicle remote control center through the remote assistance link.

[0123] In one embodiment, the operation of requesting remote assistance includes: triggering the remote assistance control on the human-computer interaction interface in the autonomous driving vehicle, or, voice command, or gesture command, or, triggering the physical button in the autonomous driving vehicle.

[0124] It should be noted that the solutions described in this specification and the embodiments, if personal information is involved, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0125] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0126] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A remote assistance method, characterized in that: Applied to an autonomous driving vehicle, the method comprises: After receiving the operation of requesting remote assistance, sending the remote assistance request and the first call request to the vehicle remote control center; Among them, the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

2. The method according to claim 1, characterized in that: The method further comprises: If the first call link fails to be established, detecting the number of failures in establishing the first call link; If the number of failures is less than or equal to the number threshold, the first call request is resent to the vehicle remote control center.

3. The method according to claim 2, characterized in that The method further comprises: If the number of failures is greater than the threshold, the vehicle remote control center is instructed to send a second call request to the target terminal corresponding to the person on the autonomous driving vehicle, and the second call request is used to request the target terminal to establish a second call link with the vehicle remote control center.

4. The method according to claim 1, characterized in that: The method further comprises: Detecting whether the autonomous driving vehicle meets the conditions for terminating remote assistance; If the condition for ending the remote assistance is met, the remote assistance link and / or the first call link are disconnected.

5. The method according to claim 4, characterized in that The detecting whether the autonomous driving vehicle meets the condition for terminating the remote assistance includes: Detecting whether the autonomous driving vehicle has been manually taken over; If it is detected that the autonomous driving vehicle is taken over manually, the autonomous driving vehicle meets the conditions for ending remote assistance.

6. The method according to claim 4, characterized in that The detecting whether the autonomous driving vehicle meets the condition for terminating the remote assistance includes: Check whether a request to hang up is received; If the hang-up request operation is detected, the autonomous driving vehicle meets the conditions for terminating the remote assistance.

7. The method according to claim 1, characterized in that The method further comprises: Controlling a camera in the cockpit of the autonomous driving vehicle to turn on, so as to capture operation images of the personnel in the cockpit through the camera; The operation image is sent to the vehicle remote control center through the remote assistance link.

8. The method according to any one of claims 1 to 7, characterized in that: The operation of requesting remote assistance includes: triggering an operation of a remote assistance control on a human-computer interaction interface in the autonomous driving vehicle, or a voice command, or a gesture command, or a triggering operation of a physical button in the autonomous driving vehicle.

9. A remote assistance device, characterized in that: Set in an autonomous driving vehicle, the device includes a sending module, which is used to send a remote assistance request and a first call request to a vehicle remote control center after receiving an operation requesting remote assistance; Among them, the remote assistance request is used to request the vehicle remote control center to establish a remote assistance link with the autonomous driving vehicle, and the first call request is used to request the vehicle remote control center to establish a first call link with the autonomous driving vehicle.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images