Remote driving control method and device, cockpit end, and storage medium

Abstract

The embodiment of the application discloses a remote driving control method and device, a cockpit terminal and a storage medium. The method is applied to the cockpit terminal and includes the following steps: determining a target mode to be switched by the cockpit terminal from at least two different remote driving modes; different remote driving modes correspond to different control authorities, and the control authorities are authorities opened to the cockpit terminal when the cockpit terminal remotely controls vehicle driving; and the cockpit terminal is controlled to switch to the target mode, so as to remotely drive the vehicle through the control authority corresponding to the target mode. The embodiment of the application can improve the safety of remote driving control.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, specifically to remote driving control methods, devices, cockpit terminals, and storage media. Background Technology

[0002] As low-latency wireless communication technology matures, various remote control technologies have become feasible. In the automotive field, the application of low-latency wireless communication technology enables remote control of vehicle operation. However, in practice, existing remote driving control solutions still suffer from insufficient safety. Summary of the Invention

[0003] This application discloses a remote driving control method, device, cockpit terminal, and storage medium, which can improve the safety of remote driving control.

[0004] This application discloses a remote driving control method applied to a cockpit. The method includes: determining a target mode to be switched to in the cockpit from at least two different remote driving modes; different remote driving modes correspond to different control permissions, the control permissions being the permissions granted to the cockpit when remotely controlling the vehicle's movement; controlling the cockpit to switch to the target mode, so as to remotely control the vehicle through the control permissions corresponding to the target mode.

[0005] As an optional implementation, the remote driving mode includes at least two of the following modes: observation mode, cloud control mode, and monitoring mode; the control authority corresponding to the observation mode is lower than the control authority corresponding to the monitoring mode; the control authority corresponding to the monitoring mode is lower than the control authority corresponding to the cloud control mode.

[0006] As an optional implementation, after the cockpit is switched to the target mode, the method further includes: when the cockpit is in the observation mode, outputting the vehicle's surrounding information collected when the vehicle is not in autonomous driving mode, and prohibiting the sending of driving control commands to the vehicle; or, when the cockpit is in the cloud control mode, sending driving control commands to the vehicle so that the vehicle performs corresponding driving operations in response to the driving control commands; or, when the cockpit is in the monitoring mode, outputting the vehicle's surrounding information collected when the vehicle is in autonomous driving mode.

[0007] As an optional implementation, determining the target mode to be switched to in the cockpit from at least two different remote driving modes includes: when the cockpit is in the current mode and a mode switching condition corresponding to the current mode is detected, determining the target mode to be switched to in the cockpit from the at least two different remote driving modes; the current mode is any one of the at least two different remote driving modes, and the target mode is different from the current mode; and controlling the cockpit to switch to the target mode includes: controlling the cockpit to switch from the current mode to the target mode.

[0008] As an optional implementation, the current mode includes an observation mode, and the target mode includes a cloud control mode; and, when the cockpit is in the current mode and a mode condition corresponding to the current mode is detected, determining the target mode to be switched to from the at least two different remote driving modes includes: when the cockpit is in the observation mode and a control activation condition is detected, selecting the cloud control mode as the target mode to be switched to from the at least two different remote driving modes; the detection of the control activation condition includes: receiving a first confirmation command sent by the vehicle, the first confirmation command being input in the vehicle in response to a first control request sent by the cockpit; or, detecting the cockpit... The safety operator inputs a second confirmation command, which is input in the cockpit in response to a second control request sent by the vehicle; or, the current mode includes: cloud control mode, and the target mode includes: observation mode; and, when the cockpit is in the current mode and a mode condition corresponding to the current mode is detected, determining the target mode to be switched to from the at least two different remote driving modes includes: when the cockpit is in cloud control mode and an interrupt command sent by the vehicle is received, selecting observation mode as the target mode to be switched to from the at least two different remote driving modes; the interrupt command is sent when the vehicle switches from autonomous driving mode to manual driving mode.

[0009] As an optional implementation, the current mode includes a cloud control mode, and the target mode includes a monitoring mode; and, when the cockpit is in the current mode and a mode condition corresponding to the current mode is detected, determining the target mode to be switched to from the at least two different remote driving modes includes: when the cockpit is in the cloud control mode, detecting a safety confirmation command input by the safety operator in the cockpit; the safety confirmation command is input by the safety operator after confirming that the vehicle's autonomous driving control information is correct; when the safety confirmation command is detected, selecting the monitoring mode from the at least two different remote driving modes as the target mode to be switched to from the cockpit.

[0010] As an optional implementation, the autonomous driving control information includes: driving path information and driving operation information; the step of detecting the safety confirmation command input by the safety operator when the cockpit is in cloud control mode includes: receiving driving path information generated by the vehicle based on the starting point location information and the ending point location information when the cockpit is in cloud control mode; generating driving operation information corresponding to different driving stages when the vehicle is driving autonomously according to the instructions of the driving path information; outputting the driving path information and the driving operation information; and detecting the safety confirmation command input by the safety operator for the driving path information and the driving operation information.

[0011] As an optional implementation, the current mode includes a monitoring mode, and the target mode includes a cloud control mode; and, when the cockpit is in the current mode and a mode condition corresponding to the current mode is detected, determining the target mode to be switched to from the at least two different remote driving modes includes: when the cockpit is in the monitoring mode and a takeover condition is detected, selecting the cloud control mode as the target mode to be switched to from the at least two different remote driving modes; the detection of the takeover condition includes: receiving a first takeover command sent by the vehicle when the autonomous driving state is paused; or, detecting a second takeover command input by the safety driver in the cockpit.

[0012] As an optional implementation, the current mode includes a monitoring mode, and the target mode includes an observation mode; and, when the cockpit is in the current mode and a mode condition corresponding to the current mode is detected, determining the target mode to be switched to from the at least two different remote driving modes includes: when the cockpit is in the monitoring mode and an interrupt command sent by the vehicle is received, selecting the observation mode as the target mode to be switched to from the at least two different remote driving modes; the interrupt command is sent when the vehicle switches from autonomous driving mode to manual driving mode.

[0013] As an optional implementation, the target mode includes: observation mode; determining the target mode to be switched on the cockpit from at least two different remote driving modes includes: when it is detected that the wireless communication connection between the cockpit and the vehicle is successfully established, selecting observation mode as the target mode to be switched on the cockpit from at least two different remote driving modes.

[0014] As an optional implementation, after switching the cockpit to the observation mode, the method further includes: exiting the observation mode when the wireless communication connection between the cockpit and the vehicle is detected to be disconnected.

[0015] This application discloses a remote driving control device applied to a cockpit. The device includes: a determining module, configured to determine a target mode to be switched to in the cockpit from at least two different remote driving modes; different remote driving modes correspond to different control permissions, the control permissions being the permissions granted to the cockpit when remotely controlling the vehicle's movement; and a switching module, configured to control the cockpit to switch to the target mode, so as to remotely control the vehicle through the control permissions corresponding to the target mode.

[0016] This application discloses a cockpit terminal, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor enables the processor to implement any of the remote driving control methods disclosed in this application.

[0017] This application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the remote driving control methods disclosed in this application.

[0018] Compared with related technologies, the embodiments of this application have the following beneficial effects:

[0019] First, multiple remote driving modes corresponding to different control permissions can be pre-set. During actual remote driving control, the cockpit can determine the target mode from these different modes and switch to it, thereby remotely controlling the vehicle using the control permissions corresponding to the target mode. This reduces driving safety issues caused by unclear control permissions on the cockpit. Furthermore, it saves on the manpower and computing resources required for permission management, improving the efficiency of remote driving. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the structure of a remote driving control system disclosed in one embodiment;

[0022] Figure 2 This is a schematic flowchart of a remote driving control method disclosed in one embodiment;

[0023] Figure 3 This is a schematic flowchart of another remote driving control method disclosed in one embodiment;

[0024] Figure 4 This is a schematic flowchart of another remote driving control method disclosed in one embodiment;

[0025] Figure 5 This is an example diagram of switching between three different remote driving modes disclosed in one embodiment;

[0026] Figure 6 This is a schematic diagram of the structure of a remote driving control device disclosed in one embodiment;

[0027] Figure 7 This is a schematic diagram of the cockpit end structure disclosed in one embodiment. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0030] This application discloses a remote driving control method, device, cockpit terminal, and storage medium, which can improve the safety of remote driving control. These will be described in detail below.

[0031] First, please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a remote driving control system disclosed in one embodiment. For example... Figure 1 As shown, the remote driving control system may include: a cockpit 10, a vehicle 20, and service equipment 30.

[0032] The service device 30 can establish wireless communication connections with both the cockpit 10 and the vehicle 20, enabling data transmission between them. Alternatively, the cockpit 10 and the vehicle 20 can also establish a direct wireless communication connection. This wireless communication connection can be based on low-latency communication technologies, such as 5G.

[0033] It should be noted that the cockpit 10 and the vehicle 20 can be two independent and different devices. The geographical locations of the cockpit 10 and the vehicle 20 can be completely different.

[0034] The cockpit 10 can be a simulator for the vehicle 20, including various components related to vehicle driving control, such as, but not limited to, a steering wheel, accelerator, brake, and turn signal switch. The safety operator in the cockpit 10 can trigger input driving control commands corresponding to these components by operating one or more of them. The cockpit 10 can send driving control commands to the vehicle 20, and the vehicle 20 can respond to the received commands and execute corresponding driving operations. The driving operations performed by the vehicle 20 can include: single driving operations, such as acceleration, deceleration, and steering; or automated driving operations composed of combinations of different driving operations, such as automatic parking and vehicle summoning, etc., without specific limitations.

[0035] The cockpit 10 may also include a display device, such as a screen. The display device can be used to display information related to remote driving control, such as weather, vehicle condition and other information about the vehicle's surroundings, as well as remote driving orders, safety reminders and other information.

[0036] Vehicle 20 may be equipped with one or more different sensors for collecting data on the vehicle's surrounding environment. For example, vehicle 20 may be equipped with sensors such as millimeter-wave radar, lidar, global navigation satellite system, and camera devices. Vehicle 20 may transmit data collected by one or more of the aforementioned sensors to the cockpit 10 for output or for generating remote driving control decisions based on the received data, without any specific limitation.

[0037] Based on the aforementioned remote driving control system, please refer to Figure 2 , Figure 2 This is a schematic flowchart of a remote driving control method disclosed in one embodiment, which can be applied to any of the aforementioned cockpit terminals. For example... Figure 2 As shown, the method may include:

[0038] 210. Determine the target mode to be switched at the cockpit end from at least two different remote driving modes.

[0039] In this embodiment, at least two different remote driving modes can be preset. Different remote driving modes correspond to different control permissions, which refer to the permissions granted to the cockpit when remotely controlling vehicle operation via the cockpit terminal. That is, the permissions granted to the cockpit terminal differ depending on the remote driving mode. The remote driving mode settings can be configured according to actual business needs and are not specifically limited.

[0040] In one embodiment, the remote driving mode may include at least two of the following modes: observation mode, cloud control mode, and monitoring mode. The control permissions corresponding to the observation mode are lower than those corresponding to the monitoring mode; the control permissions corresponding to the monitoring mode are lower than those corresponding to the cloud control mode.

[0041] That is, the various remote driving modes are sorted in descending order of control authority, which can be: cloud control mode, monitoring mode, and observation mode. Among them, the higher the control authority, the more types of driving operations can be performed by controlling the vehicle through the cockpit.

[0042] 220. Switch the control cockpit to the target mode to remotely control the vehicle using the control permissions corresponding to the target mode.

[0043] When the cockpit switches to the target mode, the cockpit has the control permissions corresponding to the target mode and can execute remote driving control operations corresponding to the control permissions.

[0044] As can be seen, in this embodiment, multiple remote driving modes corresponding to different control permissions can be pre-set. During actual remote driving control, the cockpit can determine the target mode from among these different remote driving modes based on actual needs and switch to the corresponding target mode. Therefore, driving safety issues caused by unclear control permissions on the cockpit can be reduced. Furthermore, it can save the manpower and computing resources required for permission management, thereby improving the efficiency of remote driving.

[0045] In one embodiment, when the cockpit switches to observation mode, the cockpit can output information about the vehicle's surroundings collected when the vehicle is not in autonomous driving mode, and the cockpit is prohibited from sending driving control commands to the vehicle.

[0046] For example, the cockpit can display images of the vehicle's surroundings captured by the vehicle's cameras. The safety driver can then observe the weather conditions outside the vehicle and the road conditions at the vehicle's current location using these images. Furthermore, the cockpit is prohibited from sending driving control commands to the vehicle, meaning the vehicle is in an uncontrolled state, and the safety driver cannot remotely control the vehicle's movement by operating the various driving-related components included in the cockpit. Optionally, the observation mode can be used in the initial stage of remote driving, where a wireless communication connection has been established between the cockpit and the vehicle, but the cockpit has not obtained any control permissions over the vehicle; alternatively, the observation mode can also be used when the vehicle's autonomous driving is forcibly interrupted.

[0047] When the cockpit is in observation mode, the safety operator in the cockpit can remotely observe the vehicle's surroundings. However, the safety operator cannot remotely control the vehicle at this time. The vehicle can be driven manually by the safety operator inside the vehicle; or, the vehicle can be driven automatically by the vehicle's autonomous driving function.

[0048] In another embodiment, when the cockpit switches to cloud control mode, it sends driving control commands to the vehicle, causing the vehicle to perform corresponding driving operations in response. These driving control commands may be generated by the cockpit when it detects a safety operator manipulating driving-related components.

[0049] For example, when the cockpit is in cloud control mode, when the safety operator in the cockpit presses the accelerator, the cockpit can generate an acceleration command and send it to the vehicle. The vehicle responds to the received acceleration command by performing an acceleration operation, thereby increasing the vehicle's speed.

[0050] When the cockpit is in cloud control mode, it is equivalent to remote manual driving, where a safety operator in the cockpit can be responsible for controlling the vehicle's movement. Optionally, when the cockpit is in cloud control mode, control of the vehicle can be completely transferred to the cockpit, and neither the safety operator nor the vehicle's autonomous driving function will control the vehicle's movement.

[0051] In another embodiment, when the cockpit switches to monitoring mode, it can output information about the vehicle's surroundings collected by the vehicle while it is in autonomous driving mode. The monitoring mode generally corresponds to the vehicle's autonomous driving state, which may include, but is not limited to, automatic parking, memory parking, and Navigation-Guided Pilot (Navigation Guided Pilot) modes.

[0052] For example, the cockpit can display information about the vehicle's surroundings collected by the vehicle through a display device, and the safety operator can monitor the vehicle's autonomous driving process through the information about the vehicle's surroundings displayed on the display device, providing safety assurance for the vehicle to perform any of the aforementioned autonomous driving operations.

[0053] It's important to note that while the cockpit can output information about the vehicle's surroundings in both observation and monitoring modes, the difference lies in the mode. In monitoring mode, the vehicle can be in autonomous driving mode, and the cockpit can be used to monitor the safety of this autonomous driving state. Furthermore, in monitoring mode, the cockpit has the authority to send control commands to the vehicle, allowing safety operators to remotely control its operation. However, in observation mode, remote control of the vehicle is not permitted.

[0054] In other words, when the cockpit is in monitoring mode, it can be considered "human-vehicle co-driving." Both the cockpit and the vehicle itself can control the vehicle's movement. The cockpit can be used to monitor the vehicle's movement and take over the vehicle if any abnormalities occur.

[0055] The above description illustrates various remote driving modes disclosed in the embodiments of this application. It should be noted that the observation mode, cloud control mode, and monitoring mode are just some examples of the remote driving modes disclosed in this application. In other possible embodiments, other remote driving modes with corresponding control permissions different from the observation mode, cloud control mode, and monitoring mode may also be included, without specific limitations.

[0056] In this embodiment, the cockpit can switch between different remote driving modes, which will be described separately below.

[0057] Please see Figure 3 , Figure 3This is a schematic flowchart of another remote driving control method disclosed in one embodiment, which can be applied to any of the aforementioned cockpit terminals. For example... Figure 3 As shown, the method may include:

[0058] 310. When the cockpit is in the current mode and a mode switching condition corresponding to the current mode is detected, determine the target mode to be switched to in the cockpit from at least two different remote driving modes.

[0059] 320. Switch the control cockpit from the current mode to the target mode to remotely control the vehicle using the control permissions corresponding to the target mode.

[0060] In step 310, the current mode can be any one of at least two different remote driving modes, while the target mode can be different from the current mode.

[0061] The mode switching conditions can be used to trigger mode switching on the cockpit side. The mode switching conditions corresponding to different remote driving modes can be different or the same, and there is no specific limitation.

[0062] In one embodiment, if at least two different remote driving modes include an observation mode and a cloud control mode, then the implementation of step 310 may include any one or more of the following:

[0063] Method 1: When the cockpit is in observation mode and a control activation condition is detected, select the cloud control mode from at least two different remote driving modes as the target mode to be switched to on the cockpit. That is, the current mode is observation mode, the target mode is cloud control mode, and the mode switching condition for the cockpit to switch from observation mode to cloud control mode includes: detecting a control activation condition.

[0064] Among them, the control activation condition can be used to indicate that the request for remote driving control of the vehicle has been successfully confirmed.

[0065] Optionally, the control initiation condition may include: the cockpit receiving a first confirmation command sent by the vehicle. This first confirmation command can be input within the vehicle in response to a first control request sent by the cockpit. In other words, the cockpit can actively initiate a first control request to the vehicle. In this embodiment, in addition to a safety operator being configured in the cockpit, a vehicle-side safety operator can also be configured within the vehicle. When the vehicle receives the first control request sent by the cockpit, it can output the first control request for confirmation by the vehicle-side safety operator. If the vehicle detects that the vehicle-side safety operator has input a first confirmation command in response to the first control request, it indicates that the takeover between the cockpit and the vehicle has been successful, and the vehicle can send a first confirmation command to the cockpit. When the cockpit receives the first confirmation command, it can be determined that the control initiation condition has been detected.

[0066] Optionally, the control initiation condition may also include: detecting a second confirmation command input by the safety operator, which is input in the cockpit in response to the second control request sent by the vehicle. That is, the vehicle can also actively initiate the second control request, which is then confirmed by the safety operator in the cockpit. If the cockpit detects the second confirmation command input by the safety operator in response to the second control request, it can be determined that the control initiation condition has been detected.

[0067] Method 2: When the cockpit is in cloud control mode and receives an interrupt command from the vehicle, select the observation mode from at least two different remote driving modes as the target mode to be switched to on the cockpit. That is, the current mode is cloud control mode, the target mode is observation mode, and the mode switching conditions for the cockpit to switch from cloud control mode to observation mode include receiving an interrupt command from the vehicle.

[0068] The interrupt command can be sent when the vehicle switches from autonomous driving to manual driving. When the vehicle is in autonomous driving mode, it may encounter some unexpected situations, which the safety operator on the vehicle can handle in a timely manner. When the vehicle detects that the safety operator on the vehicle has operated any driving-related components, such as braking, accelerating, or turning the steering wheel, it can determine that the vehicle is switching from autonomous driving to manual driving mode, generate an interrupt command, and send the interrupt command to the cockpit.

[0069] In one embodiment, if at least two different remote driving modes include a cloud control mode and a monitoring mode, then the implementation of step 310 may include any one or more of the following:

[0070] Method 3: When the cockpit is in cloud control mode and a safety confirmation command input by the safety operator is detected, the monitoring mode is selected as the target mode to be switched from at least two different remote driving modes. The aforementioned safety confirmation command can be input by the safety operator after confirming that the vehicle's autonomous driving control information is correct. That is, the current mode is cloud control mode, the target mode is monitoring mode, and the mode switching condition for the cockpit to switch from cloud control mode to monitoring mode includes: detecting a safety confirmation command input by the safety operator in the cockpit.

[0071] The autonomous driving control information may include information related to the vehicle's autonomous driving, including but not limited to: vehicle travel path information or driving operation information. Driving operation information may include a single driving operation; or an autonomous driving operation composed of a combination of various driving operations.

[0072] Optionally, in order to detect the safety confirmation command input by the safety operator, the cockpit can receive the driving path information generated by the vehicle based on the starting point location information and the ending point location information when it is in cloud control mode; and generate driving operation information corresponding to different driving stages when the vehicle is driving autonomously according to the instructions of the driving path information.

[0073] For example, after receiving the driving path information, the cockpit can break down the vehicle's autonomous driving process into driving stages such as "the vehicle parking out of the parking space, summoning the vehicle to the parking lot exit, and remembering the parking space." Each driving stage can correspond to a set of autonomous driving operations composed of different driving maneuvers. When the vehicle performs an autonomous driving operation, it is in an autonomous driving state. The cockpit can then identify the corresponding autonomous driving operation as the corresponding driving operation information.

[0074] Furthermore, the cockpit can output driving route information and driving operation information, and detect the safety confirmation commands input by the safety officer based on the driving route information and driving operation information.

[0075] As can be seen, when the cockpit is in cloud control mode, if the vehicle can drive from the starting point to the destination via autonomous driving, the cockpit can first output driving information and the corresponding autonomous driving operations for different driving stages, so that the safety operator can confirm whether it is safe. After the safety operator confirms, the cockpit can switch to monitoring mode. The safety operator no longer needs to manually control the vehicle's driving, but can instead monitor the vehicle during the autonomous driving process.

[0076] Method 4: When the cockpit is in monitoring mode and a takeover condition is detected, select the cloud control mode from at least two different remote driving modes as the target mode to be switched to on the cockpit. That is, the current mode is monitoring mode, the target mode is cloud control mode, and the mode switching condition for the cockpit to switch from monitoring mode to cloud control mode includes: detecting a takeover condition.

[0077] The takeover conditions can be detected when the vehicle's autonomous driving needs to be taken over. They can include the cockpit actively requesting takeover of the vehicle's autonomous driving, or the vehicle passively requesting takeover from the cockpit.

[0078] Optionally, the detection of takeover conditions may include receiving a first takeover command sent by the vehicle when it is paused in autonomous driving mode. During autonomous driving, if the vehicle detects situations affecting driving safety, such as sensors failing to accurately identify road conditions or obstacles, it can pause the autonomous driving mode. For example, it can stop sending a first takeover command to the cockpit, and the vehicle actively requests assistance from the cockpit to handle such scenarios. From the cockpit's perspective, this can be a passive takeover situation.

[0079] Optionally, the detection of takeover conditions may include: detecting a second takeover command input by the safety operator. When the cockpit is in monitoring mode, it can output information about the vehicle's surroundings collected by the vehicle. Therefore, the safety operator in the cockpit can monitor the safety of the vehicle's autonomous driving at any time based on this information. When the safety operator detects that the vehicle's current driving actions may pose a safety hazard, the safety operator can input a second takeover command in any of the following ways, such as pressing the brake pedal in the cockpit, clicking a virtual takeover button on the cockpit's touchscreen display, etc., but not limited to these. For the cockpit, this can be a case of active takeover.

[0080] As can be seen, when the cockpit is in monitoring mode, it can take over the vehicle's autonomous driving through active or passive takeover, thus allowing the cockpit to switch to cloud control mode.

[0081] In one embodiment, if at least two different remote driving modes include an observation mode and a monitoring mode, then the implementation of step 310 may include:

[0082] Method 5: When the cockpit is in monitoring mode and receives an interrupt command from the vehicle, select observation mode from at least two different remote driving modes as the target mode to be switched to on the cockpit. The aforementioned interrupt command is sent when the vehicle switches from autonomous driving mode to manual driving mode. That is, the current mode is monitoring mode, the target mode is observation mode, and the mode switching conditions for the cockpit to switch from monitoring mode to observation mode include receiving an interrupt command from the vehicle.

[0083] Optionally, to improve the safety of remote driving control, the remote driving mode on the cockpit can be switched from monitoring mode to observation mode, but not from observation mode to surveillance mode.

[0084] In summary, in the foregoing embodiments, the various remote driving modes included in the cockpit can be switched between each other, and the specific mode switching conditions can be bound to the current mode of the cockpit. Different remote driving modes currently in the cockpit correspond to different mode switching conditions. Furthermore, the same remote driving mode can correspond to multiple different mode switching conditions, and the target modes corresponding to different mode switching conditions are also different. Therefore, in the remote driving control method disclosed in this application, the cockpit can switch between different remote driving modes according to the actual problems encountered at different stages of remote driving, effectively improving the safety of remote driving.

[0085] When the remote driving mode includes an observation mode, since the observation mode corresponds to lower control permissions, it can be used as the initial remote driving mode on the cockpit side. Please refer to [link / reference]. Figure 4 , Figure 4This is a schematic flowchart of another remote driving control method disclosed in one embodiment, which can be applied to any of the aforementioned cockpit terminals. For example... Figure 4 As shown, the method may include:

[0086] 410. Check whether the wireless communication connection between the cockpit and the vehicle has been successfully established; if yes, proceed to step 420; if no, continue to step 410.

[0087] In step 410, the wireless communication connection between the cockpit and the vehicle is successfully established, which may include the successful establishment of a wireless communication connection between the cockpit and the service equipment, and the successful establishment of a wireless communication connection between the service equipment and the vehicle.

[0088] 420. Select the observation mode from at least two different remote driving modes as the target mode to be switched on the cockpit, and control the cockpit to switch to the observation mode so as to remotely control the vehicle through the control permissions corresponding to the observation mode.

[0089] 430. Check if the wireless communication connection between the cockpit and the vehicle is disconnected; if yes, proceed to step 450; if no, proceed to step 440.

[0090] 440. When the cockpit is in observation mode, perform the operation corresponding to the observation mode, and continue to perform step 430.

[0091] In step 440, the operation corresponding to the observation mode may include any one or more of the following operations, but is not limited to: outputting the vehicle's surrounding information collected when the vehicle is not in autonomous driving mode, and prohibiting the cockpit from sending driving control commands to the vehicle. Alternatively, detecting any of the following control initiation conditions: a first confirmation command sent by the vehicle or a second confirmation command input by the safety operator in the cockpit.

[0092] 450. Exit observation mode.

[0093] It should be noted that before switching to observation mode in step 420 on the cockpit side, and after exiting observation mode in step 450 on the cockpit side, the cockpit side can be in the default mode. The default mode can be any operating mode of the cockpit side other than the aforementioned observation mode, monitoring mode, and cloud control mode, including but not limited to remote driving mode, without any specific limitation.

[0094] To more clearly describe the remote driving control method disclosed in this application, the following description uses remote driving modes including observation mode, cloud control mode, and monitoring mode as examples to illustrate the entry, exit, and switching of each remote driving mode. Please refer to... Figure 5 , Figure 5This is an example diagram illustrating switching between three different remote driving modes, as disclosed in one embodiment. Figure 5 As shown:

[0095] The conditions for entering observation mode 510 include: the vehicle and the cockpit successfully establishing a wireless communication connection; the conditions for exiting observation mode 510 include: the wireless communication connection between the vehicle and the cockpit is disconnected.

[0096] The conditions for switching from observation mode 510 to cloud control mode 520 include: detecting a control start condition; the conditions for switching from cloud control mode 520 to observation mode 510 include: the vehicle sending an interrupt command.

[0097] The conditions for switching from cloud control mode 520 to monitoring mode 530 include: detecting a safety confirmation command input by the safety officer in the cockpit; the conditions for switching from monitoring mode 530 to cloud control mode 520 include: detecting a takeover condition.

[0098] The conditions for switching from monitoring mode 530 to observation mode 510 include: the vehicle sending an interrupt command.

[0099] Take, for example, a scenario where the cockpit remotely controls a vehicle to drive itself to a charging station based on order information. After receiving the order, the cockpit can establish a wireless communication connection with the vehicle to be charged as indicated in the order information, and the cockpit enters observation mode 510.

[0100] When the vehicle enters observation mode 510 at the cockpit, it can upload information about its surroundings collected by sensors such as cameras and radar to the cockpit, which will then output the information. The safety operator at the cockpit can perform a series of pre-departure safety checks based on the uploaded information, including but not limited to: safety checks of the vehicle's surrounding environment and checks of the vehicle's own operational status. Once the safety checks are complete, the operator can send a first control request to the vehicle through the cockpit to initiate remote control. Passengers or the safety operator inside the vehicle can determine whether to accept the first control request; if accepted, they can input a first confirmation command within the vehicle. The vehicle can then send the first confirmation command to the cockpit. Upon receiving the first confirmation command, the cockpit can switch from observation mode 510 to cloud control mode 520.

[0101] When the cockpit enters cloud control mode 520, it can obtain the vehicle's autonomous driving control information. This information may be generated based on order information, including, but is not limited to, vehicle travel route information and driving operation information. The cockpit can output the aforementioned autonomous driving control information when in cloud control mode 510 for confirmation by the safety driver.

[0102] When the safety operator confirms that the autonomous driving control information is correct and enters a safety confirmation command, the cockpit can switch from cloud control mode 520 to monitoring mode 530.

[0103] It should be noted that when the cockpit enters cloud control mode 520, the cockpit can also detect the safety operator's operation of driving-related components in the cockpit, so as to remotely control the vehicle in real time through the operation performed by the safety operator.

[0104] When the cockpit enters monitoring mode 530, the vehicle can correspondingly enter autonomous driving mode according to the aforementioned autonomous driving control information, and upload the vehicle's surrounding information collected during autonomous driving mode to the cockpit in real time. The cockpit can output the collected vehicle surrounding information in monitoring mode 530, and the safety operator can monitor the safety of the vehicle's autonomous driving through the vehicle surrounding information output by the cockpit.

[0105] When a safety operator discovers a safety issue with the vehicle's autonomous driving system and needs to take over, they can press the brake pedal in the cockpit. If the cockpit detects a second takeover command, it will switch from monitoring mode 530 back to cloud control mode 520. This allows the safety operator to remotely control the vehicle in real time in the cloud control mode 520 in the cockpit to resolve the current safety issues facing the vehicle.

[0106] Furthermore, when the cockpit enters cloud control mode 520 or monitoring mode 530, if the passenger or safety officer in the vehicle operates any driving-related components, the cockpit can exit cloud control mode 520 or monitoring mode 530 and enter observation mode 510. This allows control of the vehicle to be transferred from the cockpit to the vehicle itself, enabling the personnel inside the vehicle to control the driving, thus avoiding conflicts in control authority between the cockpit and the vehicle.

[0107] Please see Figure 6 , Figure 6 This is a schematic diagram of a remote driving control device disclosed in one embodiment, which is applied to any of the aforementioned cockpit terminals. For example... Figure 6 As shown, the remote driving control device 600 may include: a determination module 610 and a switching module 620.

[0108] The determining module 610 is used to determine the target mode to be switched on the cockpit from at least two different remote driving modes; different remote driving modes correspond to different control permissions, and the control permissions are the permissions granted to the cockpit when the vehicle is remotely controlled by the cockpit.

[0109] The switching module 620 is used to control the cockpit to switch to the target mode, so as to remotely control the vehicle through the control permissions corresponding to the target mode.

[0110] Optionally, the remote driving mode includes at least two of the following modes: observation mode, cloud control mode, and monitoring mode; the control authority corresponding to the observation mode is lower than the control authority corresponding to the monitoring mode; the control authority corresponding to the monitoring mode is lower than the control authority corresponding to the cloud control mode.

[0111] In one embodiment, the remote driving control device 600 may further include an output module or a communication module.

[0112] The output module can be used to output vehicle surrounding information collected by the vehicle when it is not in autonomous driving mode, when the cockpit is in the observation mode; and the communication module can be used to prevent the sending of driving control commands to the vehicle when the cockpit is in the observation mode; or...

[0113] The communication module can be used to send driving control commands to the vehicle when the cockpit is in the cloud control mode, so that the vehicle responds to the driving control commands and performs corresponding driving operations; or...

[0114] The output module can also be used to output the vehicle's surrounding information collected when the vehicle is in autonomous driving mode, when the cockpit is in the monitoring mode.

[0115] In one embodiment, the determining module 610 may include a detection unit and a selection unit.

[0116] The detection unit can be used to detect the mode switching conditions corresponding to the current mode when the cockpit is in the current mode;

[0117] The selection unit can be used to determine the target mode to be switched on the cockpit from the at least two different remote driving modes when the detection unit detects a mode switching condition corresponding to the current mode; the current mode is any one of the at least two different remote driving modes, and the target mode is different from the current mode;

[0118] The switching module 620 can also be used to control the cockpit to switch from the current mode to the target mode.

[0119] In one embodiment, the current mode includes: observation mode, and the target mode includes: cloud control mode;

[0120] The detection unit can also be used to detect control activation conditions when the cockpit is in observation mode; the detection of control activation conditions includes: receiving a first confirmation command sent by the vehicle, the first confirmation command being input in the vehicle in response to a first control request sent by the cockpit; or, detecting a second confirmation command input by the safety officer in the cockpit, the second confirmation command being input in the cockpit in response to a second control request sent by the vehicle.

[0121] The selection unit can also be used to select the cloud control mode as the target mode to be switched on the cockpit when the detection unit detects the control start condition;

[0122] In one embodiment, the current mode includes: cloud control mode, and the target mode includes: observation mode;

[0123] The detection unit can also be used to receive an interrupt command sent by the vehicle when the cockpit is in cloud control mode; the interrupt command is sent when the vehicle switches from autonomous driving mode to manual driving mode.

[0124] The selection unit can also be used to select the observation mode as the target mode to be switched on the cockpit end from at least two different remote driving modes when the detection unit receives the interrupt command sent by the vehicle.

[0125] In one embodiment, the current mode includes: cloud control mode, and the target mode includes: monitoring mode;

[0126] The detection unit can also be used to detect the safety confirmation command input by the safety operator in the cockpit when the cockpit is in cloud control mode; the safety confirmation command is input by the safety operator after confirming that the vehicle's autonomous driving control information is correct.

[0127] The selection unit can also be used to select a monitoring mode from at least two different remote driving modes as the target mode to be switched on the cockpit side when the detection unit detects the safety confirmation command.

[0128] Optionally, the autonomous driving control information may include: driving path information and driving operation information;

[0129] The detection unit can also be used to receive driving path information generated by the vehicle based on the starting point location information and the ending point location information when the cockpit is in cloud control mode; generate driving operation information corresponding to different driving stages when the vehicle is automatically driving according to the instructions of the driving path information; output the driving path information and the driving operation information; and detect the safety confirmation command input by the safety operator for the driving path information and the driving operation information.

[0130] In one embodiment, the current mode includes: monitoring mode, and the target mode includes: cloud control mode;

[0131] The detection unit can also be used to detect takeover conditions when the cockpit is in monitoring mode; detecting takeover conditions includes: receiving a first takeover command sent by the vehicle when it is paused in autonomous driving mode; or, detecting a second takeover command input by the safety operator.

[0132] The selection unit can also be used to select the cloud control mode from at least two different remote driving modes as the target mode to be switched on the cockpit when the detection unit detects the takeover condition.

[0133] In one embodiment, the current mode includes a monitoring mode, and the target mode includes an observation mode;

[0134] The detection unit can also be used to receive an interrupt command sent by the vehicle when the cockpit is in monitoring mode;

[0135] The selection unit can also be used to select the observation mode as the target mode to be switched on the cockpit end from at least two different remote driving modes when the detection unit receives the interrupt command sent by the vehicle.

[0136] In one embodiment, the detection unit can also be used to detect the wireless communication connection between the cockpit and the vehicle;

[0137] The selection unit can also be used to select the observation mode as the target mode to be switched from at least two different remote driving modes when the detection unit detects that the wireless communication connection between the cockpit and the vehicle has been successfully established.

[0138] The aforementioned switching module 620 can also be used to exit the observation mode when the detection unit detects that the wireless communication connection between the cockpit and the vehicle is disconnected.

[0139] Implementing the remote driving control device disclosed in this application allows for the determination of a target mode from various remote driving modes during actual remote driving control, and switching to the corresponding target mode. Different remote driving modes correspond to different control permissions, thus reducing driving safety issues caused by unclear control permissions at the cockpit end. Furthermore, it saves on the manpower and computing resources required for permission management, thereby improving the efficiency of remote driving.

[0140] Please see Figure 7 , Figure 7 This is a schematic diagram of the cockpit end structure disclosed in one embodiment. For example... Figure 7As shown, the cockpit end 700 may include:

[0141] Memory 710 storing executable program code;

[0142] Processor 720 coupled to memory 710;

[0143] The processor 720 calls the executable program code stored in the memory 710 to execute any of the remote driving control methods disclosed in the embodiments of this application.

[0144] This application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor implements any of the remote driving control methods disclosed in this application.

[0145] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program, when executed by a processor, implements any of the remote driving control methods disclosed in this application.

[0146] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0147] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0148] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0151] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0152] The remote driving control method, device, cockpit terminal, and storage medium disclosed in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A remote driving control method, characterized in that, The method is applied to the cockpit end; the method includes: The target mode to be switched on the cockpit is determined from at least two different remote driving modes; different remote driving modes correspond to different control permissions, the control permissions being the permissions granted to the cockpit when the vehicle is remotely controlled to drive, and the remote driving modes include at least two of the following modes: observation mode, cloud control mode, and monitoring mode; The cockpit is controlled to switch to the target mode so that the vehicle can be remotely driven by the control permissions corresponding to the target mode. When the cockpit is in the observation mode, it outputs the vehicle's surrounding information collected when the vehicle is not in autonomous driving mode, and prohibits sending driving control commands to the vehicle; or, When the cockpit is in the cloud control mode, a driving control command is sent to the vehicle, causing the vehicle to perform corresponding driving operations in response to the driving control command; or... When the cockpit is in the monitoring mode, it outputs the vehicle's surrounding information collected when the vehicle is in autonomous driving mode.

2. The method according to claim 1, characterized in that, The control permissions corresponding to the observation mode are lower than those corresponding to the monitoring mode; the control permissions corresponding to the monitoring mode are lower than those corresponding to the cloud control mode.

3. The method according to claim 2, characterized in that, Determining the target mode to be switched on the cockpit from at least two different remote driving modes includes: When the current mode of the cockpit is observation mode and a control activation condition is detected, the cloud control mode is selected from at least two different remote driving modes as the target mode to be switched to by the cockpit. The detection of the control activation condition includes: receiving a first confirmation command sent by the vehicle, the first confirmation command being input in the vehicle in response to a first control request sent by the cockpit; or, detecting a second confirmation command input by the safety operator in the cockpit, the second confirmation command being input in the cockpit in response to a second control request sent by the vehicle.

4. The method according to claim 2, characterized in that, Determining the target mode to be switched on the cockpit from at least two different remote driving modes includes: When the current mode of the cockpit is cloud control mode and an interrupt command is received from the vehicle, the observation mode is selected from at least two different remote driving modes as the target mode to be switched to on the cockpit; the interrupt command is sent when the vehicle switches from autonomous driving mode to manual driving mode.

5. The method according to claim 2, characterized in that, Determining the target mode to be switched on the cockpit from at least two different remote driving modes includes: When the current mode of the cockpit is cloud control mode, the safety confirmation command input by the safety operator in the cockpit is detected; the safety confirmation command is input by the safety operator after confirming that the vehicle's autonomous driving control information is correct. Upon detecting the safety confirmation command, the monitoring mode is selected from at least two different remote driving modes as the target mode to be switched on the cockpit.

6. The method according to claim 5, characterized in that, The autonomous driving control information includes: driving path information and driving operation information; the system in cloud control mode at the cockpit detects safety confirmation commands input by the safety operator, including: When the cockpit is in cloud control mode, it receives driving path information generated by the vehicle based on the starting point location information and the ending point location information; When the vehicle is driving autonomously according to the instructions of the driving path information, driving operation information corresponding to different driving stages is generated. Output the driving path information and the driving operation information; The safety officer checks the safety confirmation commands input by the driver based on the driving route information and the driving operation information.

7. The method according to claim 2, characterized in that, Determining the target mode to be switched on the cockpit from at least two different remote driving modes includes: When the current mode of the cockpit is monitoring mode and a takeover condition is detected, the cloud control mode is selected from at least two different remote driving modes as the target mode to be switched to on the cockpit. The detected takeover condition includes: receiving a first takeover command sent by the vehicle when it is paused in autonomous driving mode; or, detecting a second takeover command input by the safety driver on the cockpit.

8. The method according to claim 2, characterized in that, Determining the target mode to be switched on the cockpit from at least two different remote driving modes includes: When the current mode of the cockpit is monitoring mode and an interrupt command is received from the vehicle, the observation mode is selected from at least two different remote driving modes as the target mode to be switched to on the cockpit; the interrupt command is sent when the vehicle switches from autonomous driving mode to manual driving mode.

9. The method according to claim 2, characterized in that, Determining the target mode to be switched on the cockpit from at least two different remote driving modes includes: When a successful wireless communication connection is detected between the cockpit and the vehicle, the observation mode is selected from at least two different remote driving modes as the target mode to be switched to by the cockpit.

10. The method according to claim 9, characterized in that, After switching the cockpit to the observation mode, the method further includes: When the wireless communication connection between the cockpit and the vehicle is detected to be disconnected, exit the observation mode.

11. A remote driving control device, characterized in that, The device is applied to the cockpit end; the device includes: The determination module is used to determine the target mode to be switched on the cockpit from at least two different remote driving modes; different remote driving modes correspond to different control permissions, the control permissions being the permissions granted to the cockpit when the vehicle is remotely controlled to drive, and the remote driving modes include at least two of the following modes: observation mode, cloud control mode, and monitoring mode. A switching module is used to control the cockpit to switch to the target mode, so as to remotely control the vehicle through the control permissions corresponding to the target mode; An output module is configured to output vehicle surrounding information collected by the vehicle when it is not in autonomous driving mode, when the cockpit is in the observation mode; and a communication module is configured to prevent the transmission of driving control commands to the vehicle when the cockpit is in the observation mode; or... The communication module is used to send driving control commands to the vehicle when the cockpit is in the cloud control mode, so that the vehicle responds to the driving control commands and performs corresponding driving operations; or... The output module is also used to output the vehicle's surrounding information collected when the vehicle is in autonomous driving mode, when the cockpit is in the monitoring mode.

12. A cockpit end, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 10.

Images