Remote control method and device for unmanned vehicle, electronic equipment and storage medium

Abstract

The application discloses an unmanned vehicle remote control method and device, electronic equipment and a storage medium. The method comprises the following steps: determining the running state of a target unmanned vehicle; determining a target remote control mode according to the running state of the target unmanned vehicle; and controlling the target unmanned vehicle according to the target remote control mode. The method determines the target remote control mode according to the running state of the target unmanned vehicle, controls the target unmanned vehicle according to the target remote control mode, can timely match the corresponding control mode to control the target unmanned vehicle according to the running state of the target unmanned vehicle, and improves the safety of the target unmanned vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method, device, electronic device, and storage medium for remote control of autonomous vehicles. Background Technology

[0002] Autonomous driving technology integrates technologies such as intelligent networking, artificial intelligence perception and decision-making, radar positioning and perception data fusion, high-precision map positioning, and vehicle sensor monitoring, enabling vehicles to drive without human intervention. In recent years, with the development of autonomous driving technology, intelligent transportation tools such as driverless buses and driverless taxis have gradually begun to operate.

[0003] However, due to the high complexity of autonomous driving systems and actual road conditions, safety drivers are usually assigned to driverless vehicles to handle unexpected anomalies. This results in high labor costs and makes it difficult to record the vehicle's operating status and road conditions when anomalies occur, hindering subsequent iterations and upgrades of the autonomous driving system. Furthermore, since human monitoring of vehicles or simultaneous monitoring of multiple vehicles via displays is necessary, it is impossible to monitor multiple driverless vehicles simultaneously. This leads to an inability to accurately assess the status of driverless vehicles and the driving environment, resulting in untimely responses from driverless vehicles. Summary of the Invention

[0004] This invention provides a remote control method, device, electronic device, and storage medium for unmanned vehicles, in order to solve the problem that the monitoring of multiple unmanned vehicles cannot be carried out simultaneously, resulting in the inability to accurately determine the status of unmanned vehicles and the driving road conditions, thus causing untimely processing of unmanned vehicles.

[0005] According to one aspect of the present invention, a method for remotely controlling an unmanned vehicle is provided, comprising:

[0006] The operating status of the target autonomous vehicle is determined. The operating status of the target autonomous vehicle is used to describe the abnormal operating status of the target autonomous vehicle. The abnormal operating status of the target autonomous vehicle includes: equipment failure status and operation failure status. The equipment failure status is the state when the software or hardware of the target autonomous vehicle fails during operation. The operation failure status is the state when the target autonomous vehicle encounters an obstacle that cannot be passed during operation.

[0007] The target remote control mode is determined based on the operating status of the target unmanned vehicle. The target remote control mode is a mode for remotely controlling the target unmanned vehicle. The target remote control mode includes: a first control mode and a second control mode. The first control mode is a mode for directly remotely controlling the target unmanned vehicle through a remote control cockpit. The second control mode is a mode for remote control triggered by a takeover determination module and for remotely controlling the target unmanned vehicle through a remote control cockpit.

[0008] The target unmanned vehicle is controlled according to the target remote control mode.

[0009] According to another aspect of the present invention, a remote control device for a target unmanned vehicle is provided, comprising:

[0010] The operating status determination module is used to determine the operating status of the target autonomous vehicle. The operating status of the target autonomous vehicle is used to describe the abnormal operating status of the target autonomous vehicle. The abnormal operating status of the target autonomous vehicle includes: equipment failure status and operation failure status. The equipment failure status is the state when the software or hardware of the target autonomous vehicle fails during operation. The operation failure status is the state when the target autonomous vehicle encounters a road obstacle that cannot be passed during operation.

[0011] The control mode determination module is used to determine the target remote control mode based on the operating status of the target unmanned vehicle. The target remote control mode is a mode for remotely controlling the target unmanned vehicle. The target remote control mode includes: a first control mode and a second control mode. The first control mode is a mode for directly remotely controlling the target unmanned vehicle through a remote control cockpit. The second control mode is a mode for remote control triggered by a takeover determination module and for remotely controlling the target unmanned vehicle through a remote control cockpit.

[0012] The control module is used to control the target unmanned vehicle according to the target remote control mode.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the remote control method for unmanned vehicles according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the remote control method for unmanned vehicles according to any embodiment of the present invention.

[0018] The technical solution of this invention determines the target remote control mode by the operating status of the target unmanned vehicle, and controls the target unmanned vehicle according to the target remote control mode. This enables timely matching of the corresponding control mode to control the target unmanned vehicle based on its operating status, thereby improving the safety of the target unmanned vehicle's operation.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of a remote control method for an unmanned vehicle provided in an embodiment of the present invention;

[0022] Figure 2 An overall architecture diagram of an unmanned vehicle provided in an embodiment of the present invention;

[0023] Figure 3 A modular structure diagram of an overall autonomous vehicle provided in an embodiment of the present invention;

[0024] Figure 4 A flowchart of a remote control method for an unmanned vehicle provided in an embodiment of the present invention;

[0025] Figure 5 A flowchart for terminating remote control of an unmanned vehicle is provided as an embodiment of the present invention;

[0026] Figure 6This is a schematic diagram of the structure of a remote control device for an unmanned vehicle provided in an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of an electronic device for implementing the remote control method for unmanned vehicles according to embodiments of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Figure 1 This is a flowchart illustrating a remote control method for an unmanned vehicle provided in an embodiment of the present invention. This embodiment is applicable to situations involving the control of unmanned vehicles. The method can be executed by a target unmanned vehicle remote control device, which can be implemented in hardware and / or software. This target unmanned vehicle remote control device can be configured in any electronic device with network communication capabilities. Figure 1 As shown, the method includes:

[0031] S110. Determine the operating status of the target unmanned vehicle.

[0032] The operating status of the target autonomous vehicle is used to describe the abnormal operating status of the target autonomous vehicle. The abnormal operating status of the target autonomous vehicle includes: equipment failure status and operational failure status. The equipment failure status is the state when the software or hardware of the target autonomous vehicle fails during operation, and the operational failure status is the state when the target autonomous vehicle encounters an obstacle that it cannot pass during operation.

[0033] The target unmanned vehicle is the one that exhibits abnormal operating status among multiple unmanned vehicles that need to be monitored.

[0034] Determine whether the target autonomous vehicle is in an abnormal operating state during operation based on its current driving status.

[0035] Optionally, determine the operating status of the target autonomous vehicle, including steps A1-A3:

[0036] Step A1: Obtain the driving data of the target autonomous vehicle and the environmental data within a preset range around the target autonomous vehicle.

[0037] The camera video data acquisition module configured on the target autonomous vehicle collects environmental data in real time within a preset range around the vehicle body; the autonomous vehicle status monitoring module configured inside the target autonomous vehicle collects driving data during the driving process.

[0038] The driving data of the target unmanned vehicle includes information such as steering wheel angle, vehicle speed, acceleration, braking status, gear status, throttle status, and drive-by-wire status.

[0039] Furthermore, the camera video data acquisition module is used to collect 360° environmental video data information around the autonomous vehicle, and to encode the video data in hardware, encapsulate data packets, store video files, and upload the stored video files to the target autonomous vehicle monitoring cloud platform server module and remote display interface module in a timely manner via an industrial control computer.

[0040] The industrial control computer is configured in the takeover determination module. The industrial control computer contains a remote control determination program, which continuously monitors vehicle status information and triggers a vehicle stop and reports a remote control request when the vehicle status is abnormal. Furthermore, upon receiving a remote control command, it determines whether to enter remote control mode based on the target autonomous vehicle's condition information and network quality monitoring data. The target autonomous vehicle's condition information includes its driving data and environmental data within a preset range around the vehicle.

[0041] Furthermore, the camera video data acquisition module includes six cameras and a time synchronization device. The time synchronization device is used to perform time synchronization processing on the video data acquired by the cameras.

[0042] For example, such as Figure 2 As shown, the camera video data acquisition module includes six cameras, which are respectively positioned at the front, left front, right front, left rear, right rear, and rear of the vehicle.

[0043] The remote display interface module includes a display interface that shows the target autonomous vehicle's driving data, environmental data within a preset range around the vehicle, and network status data, based on the target autonomous vehicle's device number. The remote display interface module connects to the target autonomous vehicle monitoring cloud platform server module, and the corresponding server forwards the target autonomous vehicle's driving data and environmental data within the preset range around the vehicle to the display interface.

[0044] The target unmanned vehicle monitoring cloud platform server module is used to forward and store data such as the target unmanned vehicle driving data sent by the industrial control computer, environmental data within a preset range around the target unmanned vehicle, and remote control cockpit commands.

[0045] Furthermore, the target unmanned vehicle monitoring cloud platform server module includes: a cloud server.

[0046] Among them, the autonomous vehicle status monitoring module is used to collect driving data of the target autonomous vehicle, such as steering wheel angle, vehicle speed, acceleration, braking status, gear status, throttle status, and drive-by-wire status.

[0047] The remote control cockpit includes a scene display screen and a simulated cockpit. It displays environmental data within a preset range around the target autonomous vehicle and simulates the operation of a real cockpit, translating these actions into remote control commands and sending them to the corresponding autonomous vehicle. The simulated cockpit includes a steering wheel, pedals, gear shift lever, and handbrake.

[0048] Furthermore, the autonomous vehicle status monitoring module forwards the collected driving data of various target autonomous vehicles to the industrial control computer via the CAN protocol; the industrial control computer formats and encodes the received driving data of the target autonomous vehicles, stores the formatted data file, and uploads it to the target autonomous vehicle monitoring cloud platform server module and remote display interface module in a timely manner.

[0049] Furthermore, the various modules communicate with each other via network transmission and the quality monitoring module, which includes a 5GCPE for accessing the 5G mobile network and transmitting driving data of the target autonomous vehicle, environmental data within a preset range around the target autonomous vehicle, and remote control cockpit commands.

[0050] Furthermore, the network status determination program inside the industrial control computer determines the communication status and network connection status between the network transmission and quality monitoring module and other modules in real time, and synchronizes the obtained information to the network.

[0051] For example, such as Figure 3The diagram shows the overall modular structure of the target autonomous vehicle. As can be seen from the diagram, the remote control cockpit and remote display interface module are configured on the monitoring end; the target autonomous vehicle monitoring cloud platform server module is configured on the cloud; and the autonomous vehicle status monitoring module, camera video data acquisition module, network transmission and quality monitoring module, and takeover judgment module are configured on the autonomous vehicle end.

[0052] Step A2: If the driving data of the target unmanned vehicle is abnormal, the operating status of the target unmanned vehicle is determined to be a device malfunction.

[0053] If the takeover determination module detects an anomaly in the driving data of the target unmanned vehicle, it will take the equipment fault status as the operating status of the target unmanned vehicle.

[0054] Step A3: Determine whether the target unmanned vehicle has encountered a road obstacle based on the environmental data within a preset range around the vehicle. If so, determine the operating status of the target unmanned vehicle as an obstacle-prone state.

[0055] The takeover determination module determines whether the target autonomous vehicle has encountered a road obstacle in the current scene based on environmental data within a preset range around the vehicle. If a road obstacle is encountered, the operating status of the target autonomous vehicle is changed to an obstacle-running status.

[0056] Optionally, determine whether the target autonomous vehicle has encountered an obstacle based on environmental data within a preset range around the vehicle, including steps B1-B3:

[0057] Step B1: The takeover determination module detects environmental data within a preset range around the target unmanned vehicle in real time.

[0058] The takeover determination module detects environmental data within a preset range around the target autonomous vehicle in real time to determine whether the target autonomous vehicle has a malfunction in the current scenario.

[0059] Step B2: If there is a roadblock in the current scene and the target unmanned vehicle stays in the current scene for more than a preset time, it is determined that the target unmanned vehicle has encountered a roadblock that cannot be passed.

[0060] The current scenario refers to the environment within a preset range around the target autonomous vehicle in the current operating section of the road.

[0061] The preset time can be set in advance according to the actual situation or based on experience.

[0062] If the takeover determination module detects that there is a road obstacle in the environmental data collected by the target unmanned vehicle in the current scene within a preset range around the vehicle, and the target unmanned vehicle stays in front of the road obstacle for more than a preset time, then it is determined that the target unmanned vehicle has encountered an insurmountable road obstacle.

[0063] Step B3: If there are no roadblocks in the current scene, or if there are roadblocks in the current scene and the target autonomous vehicle stays in the current scene for no more than the preset time, then the target autonomous vehicle will operate normally.

[0064] If the takeover determination module detects that there are no road obstacles in the environmental data collected by the target unmanned vehicle within a preset range around the vehicle in the current scene, it determines that the target unmanned vehicle is operating normally.

[0065] If the takeover determination module detects that there is a roadblock in the environmental data collected by the target unmanned vehicle in the current scene within a preset range around the vehicle, but the target unmanned vehicle stays in the current scene for no more than a preset time, it is considered that the target unmanned vehicle has passed the roadblock or is passing the roadblock, and at this time it is determined that the target unmanned vehicle is operating normally.

[0066] S120. Determine the target remote control mode based on the operating status of the target unmanned vehicle.

[0067] The target remote control mode is a mode for remotely controlling the target unmanned vehicle. The target remote control mode includes: a first control mode and a second control mode. The first control mode is a mode for directly controlling the target unmanned vehicle through the remote control cockpit. The second control mode is a mode for triggering remote control through the takeover determination module and remotely controlling the target unmanned vehicle through the remote control cockpit.

[0068] Optionally, the target remote control mode is determined based on the operating status of the target autonomous vehicle, including steps C1-C2:

[0069] Step C1: If the target unmanned vehicle is in a fault state, then the target remote control mode is the second control mode.

[0070] If the target unmanned vehicle is in a faulty operating state, the takeover determination module will change the target remote control mode to the second control mode.

[0071] C2. If the target unmanned vehicle is in an obstacle state, then the target remote control mode is the first control mode.

[0072] If the target unmanned vehicle is in an obstructed state, the takeover determination module will change the target remote control mode to the first control mode.

[0073] S130. Control the target unmanned vehicle according to the target remote control mode.

[0074] If the target remote control mode is the first control mode, the remote control cockpit sends a remote control request to the takeover determination module. After receiving the remote control request, the takeover determination module determines whether it can directly enter the control state based on the network status and feeds back the result of whether it can be directly controlled to the remote control cockpit. The remote control cockpit then controls the target unmanned vehicle based on the feedback result.

[0075] For example, if the current network connection is good, the remote control cockpit can directly control the target autonomous vehicle. If the current network connection is poor, the remote control cockpit display will show that the current network connection is bad.

[0076] If the target remote control mode is the second control mode, the takeover determination module sends a remote control request to the target unmanned vehicle monitoring cloud platform server module. After receiving the remote control request, the target unmanned vehicle monitoring cloud platform server module will display the remote control information of the target unmanned vehicle in real time and control the target unmanned vehicle through the remote control cockpit.

[0077] Optionally, the target unmanned vehicle is controlled according to the target remote control mode, including steps D1-D3:

[0078] Step D1: If the current target remote control mode is the second control mode, the takeover determination module determines whether it is necessary to control the target unmanned vehicle.

[0079] If the current remote control mode of the target is the second control mode, the takeover determination module determines whether it is necessary to control the target unmanned vehicle based on the equipment fault status of the target unmanned vehicle.

[0080] Step D2: If necessary, the target driverless vehicle enters the second control mode.

[0081] If not required, the target autonomous vehicle will be monitored separately to ensure that it can be controlled in a timely manner when necessary.

[0082] If necessary, the target driverless vehicle enters the second control mode.

[0083] Optionally, if necessary, the target autonomous vehicle enters a second control mode, including steps E1-E3:

[0084] Step E1: If the target unmanned vehicle fails to enter the second control mode, the target unmanned vehicle enters the protection mode and sends a remote control request to the remote control cockpit at a preset frequency.

[0085] The protection mode is a mode that ensures the safe operation of the target unmanned vehicle.

[0086] The protection mode includes measures such as reducing vehicle speed or emergency braking.

[0087] For example, if the equipment failure status of the target autonomous vehicle is a problem such as camera failure or brake failure, the protection mode is to reduce the vehicle speed.

[0088] If the equipment malfunction of the target unmanned vehicle is due to problems such as exceeding the speed limit or not following the designated path, the protection mode is emergency stop.

[0089] If the target unmanned vehicle fails to enter the second control mode, the target unmanned vehicle determines the protection mode to enter based on the equipment failure status. After the target unmanned vehicle enters the protection mode, it sends a remote control request to the remote control cockpit at a preset frequency.

[0090] Step E2: If the number of remote control requests sent does not exceed the preset number and the target unmanned vehicle successfully enters the second control mode, then stop sending remote control requests.

[0091] If the remote control cockpit successfully receives the remote control request, the target unmanned vehicle successfully enters the second control mode. If the number of remote control requests sent does not exceed the preset number, the remote control requests will stop being sent and the vehicle will exit the protection mode.

[0092] Step E3: If the number of remote control requests sent exceeds the preset number, the target unmanned vehicle enters the first control mode and issues an abnormal status warning.

[0093] If the number of remote control requests sent exceeds the preset number, it indicates that the target unmanned vehicle is unable to enter the second control mode. The takeover determination module will then change the target remote control mode back to the first control mode and issue an abnormal status warning.

[0094] Step D3: Control the target unmanned vehicle according to the second control mode.

[0095] The takeover determination module sends a remote control request to the target unmanned vehicle monitoring cloud platform server module. After receiving the remote control request, the remote display interface module will display the remote control information of the target unmanned vehicle in real time and control the target unmanned vehicle through the remote control cockpit.

[0096] For example, such as Figure 4 As shown, the system monitors the target autonomous vehicle's status in real time, including its driving data and environmental data within a preset range around the vehicle. Based on the acquired data, it determines whether the target autonomous vehicle is in an abnormal operating state. If so, the takeover decision module sends a remote control request to the remote control cockpit. Upon receiving the request, the remote control cockpit sends a remote control request to the takeover decision module, which simultaneously puts the target autonomous vehicle into protection mode and determines whether it can enter remote control mode. If it can, it enters remote control mode and sends the entry status information back to the remote control cockpit. If it cannot directly enter remote control mode, it checks the entry failure status information and the target autonomous vehicle's network status. If the network status is poor, it checks the maximum number of checks; if the limit is exceeded, it sends a remote control status error message. If the network status is good, it enters remote control mode and sends the entry status information back to the remote control cockpit. Once the remote control cockpit receives a status message indicating successful entry into remote control mode, it determines whether it has successfully entered remote control mode. If successful, it directly controls the target unmanned vehicle. If entry fails, it attempts to send the remote control request again. If that fails, it checks if the number of requests has reached the limit. If it has, it sends a request to end remote control.

[0097] Optionally, after controlling the target unmanned vehicle according to the target remote control mode, steps F1-F2 are included:

[0098] Step F1: If the current target remote control mode is the first control mode, the remote control cockpit determines whether the target unmanned vehicle has passed the obstacle. If it has, the first control mode is released and the autonomous driving state is restored.

[0099] If the current remote control mode of the target is the first control mode, the remote control cockpit will determine whether the target unmanned vehicle has passed the obstacle by displaying the current scene of the target unmanned vehicle on the display interface. If it has passed the obstacle, the first control mode will be released and the autonomous driving state will be restored.

[0100] Furthermore, if releasing the first control mode fails, the target unmanned vehicle enters a protection mode, and the takeover determination module continues to release the first control mode until it is successfully released.

[0101] Step F2: If the current remote control mode of the target is the second control mode, the takeover judgment module determines whether the current environment of the target unmanned vehicle is an open and safe area. If so, the second control mode is released and the steering wheel of the target unmanned vehicle is straightened and the handbrake is put into the parking space.

[0102] If the current remote control mode of the target is the second control mode, the takeover determination module will determine in real time whether the target unmanned vehicle is currently in an open and safe area. If it is, the target unmanned vehicle will stop, and the remote control cockpit will send a second control mode release request to the takeover determination module. After the release request is approved, the second control mode will be released, and the steering wheel of the target unmanned vehicle will be straightened and the handbrake will be engaged in the parking position. If it is not an open and safe area, the system will continue to control the target unmanned vehicle to search for an open and safe area.

[0103] Furthermore, if releasing the second control mode fails, the target unmanned vehicle enters a protection mode, and the takeover determination module continues to release the second control mode until it is successfully released.

[0104] For example, such as Figure 4 As shown, the simulated cockpit within the remote control cabin sends a request to the takeover determination module to terminate the remote control mode. Upon receiving this request, the takeover determination module puts the target autonomous vehicle into protection mode. The module then determines whether the target autonomous vehicle can terminate the remote control mode in the current scenario. If it can, it provides feedback indicating that the remote control request can be terminated; otherwise, it provides feedback indicating that the remote control request cannot be terminated. The simulated driving end, i.e., the monitoring end, receives the feedback and determines whether remote control can be terminated. If it can, it exits the control mode and enters a reset state, i.e., the target autonomous vehicle straightens its steering wheel and places the handbrake in the parking position; otherwise, it maintains the remote control state. Here, the remote control mode refers to the target's remote control mode.

[0105] The technical solution of this embodiment determines the target remote control mode by the operating status of the target autonomous vehicle, and controls the target autonomous vehicle according to the target remote control mode. It can timely match the corresponding control mode to control the target autonomous vehicle according to the operating status of the target autonomous vehicle, thereby improving the driving safety of the target autonomous vehicle.

[0106] Figure 6 This is a schematic diagram of a remote control device for an unmanned vehicle provided in an embodiment of the present invention. This embodiment is applicable to situations involving the control of unmanned vehicles. The target unmanned vehicle remote control device can be implemented in hardware and / or software, and can be configured in any electronic device with network communication capabilities. Figure 6As shown, the device includes: an operating status determination module 210, a control mode determination module 220, and a control module, wherein:

[0107] Operational Status Determination Module 210: Determines the operational status of the target unmanned vehicle. The operational status of the target unmanned vehicle is used to describe the abnormal operational status of the target unmanned vehicle. The abnormal operational status of the target unmanned vehicle includes: equipment failure status and operational failure status. The equipment failure status is the status when the software or hardware of the target unmanned vehicle fails during operation. The operational failure status is the status when the target unmanned vehicle encounters a road obstacle that cannot be passed during operation.

[0108] Control mode determination module 220: used to determine the target remote control mode based on the operating status of the target unmanned vehicle. The target remote control mode is a mode for remotely controlling the target unmanned vehicle. The target remote control mode includes: a first control mode and a second control mode. The first control mode is a mode for directly remotely controlling the target unmanned vehicle through the remote control cockpit. The second control mode is a mode for remote control triggered by the takeover determination module and for remotely controlling the target unmanned vehicle through the remote control cockpit.

[0109] Control module 230: Used to control the target unmanned vehicle according to the target remote control mode.

[0110] Optionally, the running status determination module 210 includes:

[0111] Environmental data acquisition unit: used to acquire the driving data of the target unmanned vehicle and the environmental data within a preset range around the target unmanned vehicle.

[0112] Equipment Fault Status Determination Unit: Used to determine the operating status of the target unmanned vehicle as an equipment fault state if the driving data of the target unmanned vehicle is abnormal.

[0113] Operational Obstacle Status Determination Unit: Used to determine whether the target unmanned vehicle has encountered an obstacle based on environmental data within a preset range around the vehicle body. If so, the operating status of the target unmanned vehicle is determined to be an operational obstacle status.

[0114] Optional, the running status determination unit is specifically used for:

[0115] The takeover determination module monitors environmental data within a preset range around the target unmanned vehicle in real time.

[0116] If there is a roadblock in the current scene and the target unmanned vehicle stays in the current scene for more than a preset time, it is determined that the target unmanned vehicle has encountered a roadblock that cannot be passed. The current scene is the environment within a preset range around the target unmanned vehicle in the current running section of the road.

[0117] If there are no roadblocks in the current scene, or if there are roadblocks in the current scene and the target autonomous vehicle stays in the current scene for no more than the preset time, then the target autonomous vehicle will operate normally.

[0118] Optionally, the control mode determination module 220 includes:

[0119] Second control mode determination unit: if the target unmanned vehicle is in a fault state, then the target remote control mode is the second control mode.

[0120] First control mode determination unit: if the target unmanned vehicle is in an obstacle operation state, then the target remote control mode is the first control mode.

[0121] Optional, the control module 230 includes:

[0122] Remote control request sending unit: used to determine whether the takeover judgment module needs to control the target unmanned vehicle if the current target remote control mode is the second control mode;

[0123] Second control mode entry unit: used to allow the target unmanned vehicle to enter the second control mode if needed;

[0124] Control unit: Used to control the autonomous vehicle according to the second control mode target.

[0125] Optionally, the second control mode determination unit is specifically used for:

[0126] If the target unmanned vehicle fails to enter the second control mode, it will enter the protection mode and send remote control requests to the remote control cockpit at a preset frequency. The protection mode is a mode that can ensure the safe operation of the target unmanned vehicle.

[0127] If the number of remote control requests sent does not exceed the preset number and the target autonomous vehicle successfully enters the second control mode, then the sending of remote control requests will stop.

[0128] If the number of remote control requests sent exceeds the preset number, the target unmanned vehicle will enter the first control mode and issue an abnormal status warning.

[0129] Optional, the control module 230 includes:

[0130] First control mode release unit: If the current target remote control mode is the first control mode, the remote control cockpit determines whether the target unmanned vehicle has passed the obstacle. If it has passed, the first control mode is released and the autonomous driving state is restored.

[0131] Second control mode release unit: If the current target remote control mode is the second control mode, the takeover judgment module determines whether the current environment of the target unmanned vehicle is an open and safe area. If so, the second control mode is released and the steering wheel of the target unmanned vehicle is straightened and the handbrake is put into the parking position.

[0132] The remote control device for the target unmanned vehicle provided in the embodiments of the present invention can execute the remote control method for the unmanned vehicle provided in any of the embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the remote control method for the unmanned vehicle. For details, please refer to the relevant operations of the remote control method for the unmanned vehicle in the foregoing embodiments.

[0133] Figure 7 This is a schematic diagram of the structure of an electronic device for implementing the remote control method for unmanned vehicles according to embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0134] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0135] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0136] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as remote control methods for autonomous vehicles.

[0137] In some embodiments, the autonomous vehicle remote control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the autonomous vehicle remote control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the autonomous vehicle remote control method by any other suitable means (e.g., by means of firmware).

[0138] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0140] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0142] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0143] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0144] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0145] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for remote control of an unmanned vehicle, characterized in that, include: The operating status of the target autonomous vehicle is determined. The operating status of the target autonomous vehicle is used to describe the abnormal operating status of the target autonomous vehicle. The abnormal operating status of the target autonomous vehicle includes: equipment failure status and operation failure status. The equipment failure status is the state when the software or hardware of the target autonomous vehicle fails during operation. The operation failure status is the state when the target autonomous vehicle encounters an obstacle that cannot be passed during operation. The target remote control mode is determined based on the operating status of the target unmanned vehicle. The target remote control mode is a mode for remotely controlling the target unmanned vehicle. The target remote control mode includes: a first control mode and a second control mode. The first control mode is a mode for directly remotely controlling the target unmanned vehicle through a remote control cockpit. The second control mode is a mode for determining whether the target unmanned vehicle has entered the remote control mode through a takeover determination module and then remotely controlling the target unmanned vehicle through a remote control cockpit. The target unmanned vehicle is controlled according to the target remote control mode; Whether the target unmanned vehicle has entered the remote control mode is determined based on the target unmanned vehicle's driving data, environmental data within a preset range around the target unmanned vehicle, and network quality monitoring data. Determining the target remote control mode based on the operating status of the target autonomous vehicle includes: If the target unmanned vehicle is in a faulty operating state, then the target remote control mode is the second control mode. If the target unmanned vehicle is in an obstacle state, then the target remote control mode is the first control mode.

2. The method according to claim 1, characterized in that, Determining the operating status of the target unmanned vehicle includes: Acquire the driving data of the target autonomous vehicle and the environmental data within a preset range around the target autonomous vehicle. If the driving data of the target unmanned vehicle is abnormal, the operating status of the target unmanned vehicle is determined to be a device malfunction. Based on environmental data within a preset range around the target autonomous vehicle, it is determined whether the target autonomous vehicle has encountered an obstacle. If so, the operating state of the target autonomous vehicle is determined to be an obstacle-prone state.

3. The method according to claim 2, characterized in that, The step of determining whether the target autonomous vehicle has encountered a road obstacle based on environmental data within a preset range around the vehicle includes: The takeover determination module monitors environmental data within a preset range around the target unmanned vehicle in real time. If there is a roadblock in the current scene and the target unmanned vehicle stays in the current scene for more than a preset time, it is determined that the target unmanned vehicle has encountered a roadblock that cannot be passed. The current scene is the environment within a preset range around the target unmanned vehicle in the current operating section of the target unmanned vehicle. If there are no roadblocks in the current scene, or if there are roadblocks in the current scene and the target unmanned vehicle stays in the current scene for no more than a preset time, then the target unmanned vehicle will operate normally.

4. The method according to claim 1, characterized in that, The step of controlling the target unmanned vehicle according to the target remote control mode includes: If the current target remote control mode is the second control mode, the takeover determination module determines whether it is necessary to control the target unmanned vehicle. If necessary, the target driverless vehicle enters a second control mode; The target unmanned vehicle is controlled according to the second control mode.

5. The method according to claim 4, characterized in that, If necessary, the target unmanned vehicle enters a second control mode, including: If the target unmanned vehicle fails to enter the second control mode, the target unmanned vehicle enters the protection mode and sends remote control requests to the remote control cockpit at a preset frequency. The protection mode is a mode that can ensure the safe operation of the target unmanned vehicle. If the number of remote control requests sent does not exceed the preset number and the target unmanned vehicle successfully enters the second control mode, then the sending of remote control requests will stop. If the number of remote control requests sent exceeds a preset number, the target unmanned vehicle enters the first control mode and issues an abnormal status warning.

6. The method according to claim 1, characterized in that, After controlling the target unmanned vehicle according to the target remote control mode, the following steps are included: If the current remote control mode of the target is the first control mode, the remote control cockpit determines whether the target unmanned vehicle has passed the obstacle. If it has, the first control mode is released and the autonomous driving state is restored. If the current remote control mode of the target is the second control mode, the takeover determination module determines whether the current environment of the target unmanned vehicle is an open and safe area. If so, the second control mode is released and the steering wheel of the target unmanned vehicle is straightened and the handbrake is put into the parking position.

7. A remote control device for a target unmanned vehicle, characterized in that, include: The operating status determination module is used to determine the operating status of the target autonomous vehicle. The operating status of the target autonomous vehicle is used to describe the abnormal operating status of the target autonomous vehicle. The abnormal operating status of the target autonomous vehicle includes: equipment failure status and operation failure status. The equipment failure status is the state when the software or hardware of the target autonomous vehicle fails during operation. The operation failure status is the state when the target autonomous vehicle encounters a road obstacle that cannot be passed during operation. The control mode determination module is used to determine the target remote control mode based on the operating status of the target unmanned vehicle. The target remote control mode is a mode for remotely controlling the target unmanned vehicle. The target remote control mode includes: a first control mode and a second control mode. The first control mode is a mode for directly remotely controlling the target unmanned vehicle through a remote control cockpit. The second control mode is a mode for determining whether the target unmanned vehicle has entered the remote control mode through a takeover determination module and then remotely controlling the target unmanned vehicle through a remote control cockpit. The control module is used to control the target unmanned vehicle according to the target remote control mode; Whether the target unmanned vehicle has entered the remote control mode is determined based on the target unmanned vehicle's driving data, environmental data within a preset range around the target unmanned vehicle, and network quality monitoring data. The control mode determination module includes: The second control mode determination unit is used to determine the target remote control mode as the second control mode if the target unmanned vehicle is in a device failure state. The first control mode determination unit is configured to determine the target remote control mode as the first control mode if the target unmanned vehicle is in an obstacle operation state.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the remote control method for an unmanned vehicle as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the remote control method for an unmanned vehicle as described in any one of claims 1-6.

Images