A vehicle remote control method, system and vehicle

Abstract

The application provides a vehicle remote control method, system and vehicle, applied to the technical field of vehicles, and aims to solve the problem that it is difficult to meet the remote control vehicle demand of users. The method comprises the following steps: in response to receiving a vehicle search instruction sent by a remote controller, obtaining a historical position of the vehicle, the historical position being the last vehicle position obtained by a UAV; determining an initial position of the vehicle based on the historical position and aerial images collected by the UAV; moving to a first position and sending an information collection instruction to the vehicle; receiving vehicle perception data sent by the vehicle and sending the vehicle perception data and aerial images collected by the UAV to the remote controller; in response to receiving a vehicle control instruction sent by the remote controller, sending the vehicle control instruction to the vehicle to enable the vehicle to move based on the vehicle control instruction; and the first position is located within a range with the initial position as the center and a preset distance as the radius.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a vehicle remote control method, system and vehicle. Background Technology

[0002] In real-world application scenarios such as outdoor recreation, complex environments, emergency rescue, and remote operations, vehicles are often located in areas with weak mobile network coverage or blocked signals, resulting in the interruption or inability to establish a direct communication link between the user terminal and the vehicle.

[0003] In related technologies, remote vehicle control primarily relies on mobile communication networks or short-range wireless communication to enable direct interaction between the user terminal and the vehicle. Therefore, when a valid connection cannot be established between the user terminal and the vehicle, remote vehicle control is impossible, failing to meet the user's need for remote vehicle control. Summary of the Invention

[0004] The purpose of this application is to provide a method, system, and vehicle for remote vehicle control, aiming to solve the problem that related technologies cannot meet users' needs for remote vehicle control.

[0005] Firstly, a remote vehicle control method is provided, applied to a drone. The method includes: upon receiving a vehicle search command from a remote controller, acquiring the vehicle's historical position; determining the vehicle's initial position based on the historical position and aerial images collected by the drone; moving to the first position and sending an information collection command to the vehicle; receiving vehicle perception data sent by the vehicle; sending the vehicle perception data and the aerial images collected by the drone to the remote controller; and, upon receiving a vehicle control command from the remote controller, sending the vehicle control command to the vehicle to cause the vehicle to move based on the vehicle control command. The historical position is the last vehicle position acquired by the drone, the first position is located within a radius centered on the initial position and a preset distance, and the information collection command instructs the vehicle to collect perception data of the surrounding environment.

[0006] The vehicle remote control method provided in this application utilizes the aerial photography and relay transmission capabilities of a drone, combining the vehicle's historical location with drone aerial images to determine the vehicle's initial position. This improves the accuracy of vehicle location identification, avoids the bias problems inherent in single positioning methods, and ensures the reliability and efficiency of long-distance vehicle search. Simultaneously, the drone moves to a preset range around the vehicle, acting as a relay, establishing a communication link between the remote controller and the vehicle, and conducting data interaction. Specifically, it receives vehicle perception data sent by the vehicle and sends this data, along with its own collected aerial images, to the remote controller. This allows the user on the remote controller side to understand the vehicle's environment and issue more accurate vehicle control commands. Furthermore, the remote controller forwards the vehicle control commands to the vehicle via the drone, enabling the vehicle to move or perform corresponding control actions based on these commands, thereby achieving remote vehicle control. This solves the problem of users being unable to directly control vehicles in long-distance scenarios, improves the stability and feasibility of long-distance vehicle control, and meets users' needs for remote vehicle control.

[0007] Optionally, the initial position of the vehicle can be determined based on historical location and aerial images collected by the drone, including: moving to the historical location and acquiring the wireless positioning signal emitted by the vehicle during the movement; adjusting its own flight direction to approach the vehicle based on the signal strength change of the wireless positioning signal; and determining the initial position of the vehicle based on the aerial images and the vehicle's body feature information.

[0008] Optionally, the vehicle remote control method further includes: in response to receiving motion control parameters sent by the vehicle, sending the motion control parameters to a remote controller for user confirmation via the remote controller; and in response to receiving a motion confirmation command sent by the remote controller, sending a motion confirmation command to the vehicle to cause the vehicle to begin moving based on the motion control parameters. The motion control parameters are parameters generated by the vehicle based on vehicle control commands and used to control the actions of the vehicle's actuators.

[0009] Optionally, in response to receiving a vehicle control command from a remote controller, sending the vehicle control command to the vehicle includes: performing a safety verification on the vehicle control command, determining a risk value for the vehicle control command, and sending the vehicle control command to the vehicle when the risk value is less than a risk threshold. The vehicle control command includes at least one of a steering command, an acceleration command, a braking command, and a gear shifting command. The method further includes: generating an alarm signal when the risk value is greater than the risk threshold, and sending the alarm signal to the remote controller.

[0010] In a second aspect, a vehicle remote control system is provided, comprising: a drone, a remote controller, and a vehicle, wherein the remote controller is communicatively connected to the drone, and the drone performs the vehicle remote control method as described in the first aspect.

[0011] Optionally, the drone is specifically configured to, in response to receiving a first vehicle control command from a remote controller, send the first vehicle control command and the aerial data it has collected to the vehicle. The first vehicle control command includes target location information and driving preference settings. The vehicle is configured to: in response to receiving a first vehicle control command and aerial photography data, perform path planning based on target location information, driving preference settings, and aerial photography data, and determine the vehicle's motion control parameters, including: driving speed, steering angle, and braking timing.

[0012] Optionally, the remote controller is configured to receive and display vehicle perception data from the vehicle and aerial data collected by the drone. Target location information is generated by the remote controller based on user operations, which include selecting a target point, drawing a driving path, or inputting coordinates on the remote controller's display interface.

[0013] Optionally, the drone is specifically configured to, in response to receiving a second vehicle control command from a remote controller, send the second vehicle control command to the vehicle. The second vehicle control command includes a target control action, which may include one or more of steering, acceleration, braking, gear shifting, and emergency stop. The vehicle is configured to, in response to receiving the second vehicle control command, execute the corresponding target control action, and during the execution of the target control action, send real-time acquired vehicle perception data to the drone. The drone is configured to, send the vehicle perception data and aerial images to the remote controller, enabling the user to determine the vehicle's execution status based on the remote controller.

[0014] Optionally, the drone is configured to: after moving to the first location, if it is determined that the vehicle is in a dormant state, send a vehicle wake-up command to the vehicle to wake it up, and the vehicle is configured to: switch to working state in response to receiving the vehicle wake-up command sent by the drone.

[0015] Thirdly, a vehicle is provided, comprising: a control system and a drive system, wherein the control system receives vehicle control commands sent by a drone and controls the drive system based on the vehicle control commands, and the drive system is used to drive the vehicle to move, wherein the drone is used to execute the vehicle remote control method described in the first aspect. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the structure of a vehicle remote control system provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a vehicle provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of another vehicle provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of a drone provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a remote controller provided in an embodiment of this application; Figure 6 A flowchart illustrating a vehicle remote control method provided in an embodiment of this application; Figure 7 A flowchart illustrating yet another vehicle remote control method provided in this application embodiment; Figure 8 A flowchart illustrating yet another vehicle remote control method provided in this application embodiment; Figure 9 This is a flowchart illustrating another vehicle remote control method provided in an embodiment of this application.

[0018] Figure label: 1000 - Vehicle Remote Control System; 100 - Vehicles; 101 - Control System; 1011-Central Controller; 1012-Data Acquisition Module; 1013-First Communication Wake-up Module; 1014-First Positioning Module; 102 - Drive system; 200 - Unmanned Aerial Vehicles (UAVs); 201-First wireless communication module; 202-Image acquisition module; 203-Second communication wake-up module; 204-Second positioning module; 205-Flight control module; 300 - Remote Controller; 301 - Second wireless communication module; 302 - Display module; 303 - Signal input module; 304 - Third positioning module. Detailed Implementation

[0019] In the embodiments of this application, the terms "first," "second," "third," "fourth," "fifth," and "sixth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," "third," "fourth," "fifth," and "sixth" may explicitly or implicitly include one or more of that feature.

[0020] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0021] "A and / or B" includes the following three combinations: A only, B only, and a combination of A and B.

[0022] With the deep integration of intelligent connected vehicles and the low-altitude economy, some vehicles have begun to be equipped with drones for auxiliary functions such as road inspection and surrounding environment perception. The integration of drones has become a core development trend in vehicles, with systems designed to work in tandem with drones being deployed. Simultaneously, efforts are being made to improve the synchronization accuracy and automated collaboration capabilities between drones and vehicles to leverage the unique advantages of drones' aerial perspective in complex traffic scenarios.

[0023] In practical application scenarios such as outdoor recreation, complex environments, emergency rescue, and remote operations, vehicles are often located in areas with weak mobile network coverage or blocked signals, resulting in the interruption or inability to establish a direct communication link between the user terminal and the vehicle. Communication between drones and vehicles relies on GPS positioning and a one-way communication link. When the user is unable to approach the vehicle due to accidents, injuries, or dangerous environments, the drone also leaves the direct communication range with the vehicle, making it impossible for the user terminal to establish an effective connection with the vehicle.

[0024] For example, in outdoor leisure scenarios, when users carry drones for activities such as camping, travel photography, off-road adventures, and geological exploration, they are often several kilometers or even tens of kilometers away from the vehicle. At this time, the drone is out of the vehicle's direct communication range, and the signal in the wild environment is weak, so the user terminal (i.e., the mobile terminal carried by the user, such as a mobile phone) cannot establish an effective connection with the vehicle.

[0025] In complex environments, when vehicles are parked in enclosed or signal-blocked areas such as underground parking lots, tunnels, or deep mountain valleys, mobile network and Bluetooth signals are severely blocked, and the communication link between the vehicle and the user terminal is completely interrupted.

[0026] In emergency rescue scenarios, users may encounter accidents such as injury or being trapped while off-roading or working outdoors, and may be unable to reach the vehicle's location independently or establish an effective connection with it.

[0027] In remote vehicle troubleshooting scenarios, a vehicle experiences a non-fatal malfunction in a remote area, and the user is not near the vehicle and cannot communicate with it.

[0028] However, in related technologies, remote vehicle control mainly relies on mobile communication networks or short-range wireless communication to achieve direct interaction between the user terminal and the vehicle. Therefore, in the above scenario, when the user terminal and the vehicle cannot establish an effective connection, remote control of the vehicle cannot be achieved, making it difficult to meet the user's need for remote vehicle control and affecting user experience and travel safety.

[0029] Therefore, there is an urgent need for a remote vehicle control technology that can overcome communication limitations and adapt to multiple scenarios.

[0030] Based on this, this application proposes a vehicle remote control method. Utilizing the aerial photography and relay transmission capabilities of a drone, and combining the vehicle's historical location with drone aerial images, the initial position of the vehicle is determined, improving the accuracy of vehicle location identification, avoiding the bias problems inherent in single positioning methods, and ensuring the reliability and efficiency of long-distance vehicle search. Simultaneously, the drone moves to a preset range around the vehicle, acting as a relay, establishing a communication link between the remote controller and the vehicle, and conducting data interaction. Specifically, it receives vehicle perception data sent by the vehicle and sends this data, along with its own collected aerial images, to the remote controller. This allows the user on the remote controller side to understand the vehicle's environment and issue more accurate vehicle control commands. Furthermore, the remote controller forwards the vehicle control commands to the vehicle via the drone, enabling the vehicle to move or perform corresponding control actions based on the commands, thereby achieving remote vehicle control. This solves the problem of users being unable to directly control vehicles in long-distance scenarios, improves the stability and feasibility of long-distance vehicle control, and meets users' needs for remote vehicle control.

[0031] The vehicle remote control method proposed in this application can be applied to vehicle remote control systems. For example... Figure 1 As shown, the vehicle remote control system 1000 includes: a vehicle 100, a drone 200, and a remote controller 300.

[0032] Vehicle 100 can be, but is not limited to, pure electric vehicles, hybrid vehicles, range-extended electric vehicles, plug-in hybrid vehicles, new energy vehicles, etc.

[0033] The UAV 200 refers to an unmanned aircraft operated by radio remote control equipment and its own program control device. In the embodiments of this application, the UAV 200 serves as a communication relay and aerial sensing carrier. The UAV 200 is equipped with camera equipment to collect real-time images or video data of the environment around the vehicle 100 and the target area.

[0034] The remote controller 300 is a mobile terminal carried by the user, serving as the user's operation and sensing terminal. It can be a mobile phone, tablet computer, laptop computer, netbook, personal digital assistant (PDA), or a dedicated remote control device matched with the drone. This application embodiment does not limit this.

[0035] In some embodiments, the remote controller may be a touch screen or a combination of a touch screen and a remote controller, so that users can issue control commands through the remote controller.

[0036] For example, a simulated interface of a vehicle control panel is displayed on the touch screen. The user touches and operates the virtual controls in the simulated interface. The remote controller responds to the user's operation, generates corresponding vehicle control commands, and sends them to the drone. In another example, the remote controller is equipped with a physical joystick and / or physical buttons corresponding to the vehicle control panel. The user operates the physical joystick or physical buttons, and the remote controller responds to the user's operation, generates corresponding vehicle control commands, and sends them to the drone.

[0037] In other examples, the touch screen displays aerial and / or road images. The user selects a target location on the image displayed on the touch screen, or draws a driving path and inputs coordinates. In response to the user's operation, the remote controller generates a vehicle control command containing the target location and / or target path, and sends the vehicle control command to the drone to instruct the vehicle to switch to autonomous driving mode and drive according to the target location and / or target path.

[0038] In this embodiment, a communication link is formed between the vehicle, the drone, and the remote controller, with the drone acting as a relay node. The remote controller is a user-carried operating terminal responsible for issuing control commands and receiving feedback information. The drone, as an aerial relay node, responds to the remote controller's vehicle search command by first establishing a communication connection with the vehicle, then acquiring aerial images of the area where the vehicle is located, and simultaneously forwarding vehicle perception data uploaded by the vehicle. The remote controller generates vehicle control commands based on the received aerial images and vehicle perception data, and then sends them to the vehicle via the drone, allowing the vehicle to execute the corresponding control actions. Thus, the drone establishes a two-way data transmission channel between the remote controller and the vehicle, enabling the remote controller to still achieve information interaction and remote control with the vehicle through the drone even when the vehicle's communication signal is poor or it is outside the remote control range.

[0039] In some embodiments, the drone 200 is equipped with a wake-up module, which functions as a wireless key. When the distance between the drone 200 and the vehicle 100 is less than or equal to a preset distance, the vehicle can be woken up, enabling the vehicle to be in working condition.

[0040] In some examples, when the user is far from vehicle 100 and vehicle 100 is in a dormant state, the drone 200 disconnects from vehicle 100. In this scenario, the remote controller 300 sends a vehicle search command to the drone 200. Upon receiving the command, the drone 200 locates vehicle 100 and moves to its vicinity. Then, if vehicle 100 is in a dormant state, the drone 200 sends a vehicle wake-up command to wake it up. Vehicle 100 responds to this command, waking up and becoming operational. Next, the remote controller 300 sends a vehicle control command to the drone 200. The drone 200 receives this command and forwards it to vehicle 100, causing vehicle 100 to move to the target location.

[0041] In some embodiments, such as Figure 2 As shown, the vehicle 100 includes a control system 101 and a drive system 102. The control system 101, as the decision-making core of the vehicle 100, establishes a communication link with the drone and receives information (such as aerial images) and commands (such as vehicle control commands and vehicle wake-up commands) sent by the drone. The drive system 102, as the underlying actuator of the vehicle 100, is typically located at the front-wheel drive and / or rear-wheel drive positions of the vehicle body and is fixedly connected to the vehicle body. Upon receiving power, the drive system 102 converts electrical energy into mechanical energy to drive the vehicle. In this embodiment, the control system 101 controls the drive system 102 based on motion control parameters or vehicle control commands, and the drive system 102 drives the vehicle to move.

[0042] like Figure 3 As shown, in the vehicle 100, the control system 101 includes: a central controller 1011, a data acquisition module 1012, a first communication wake-up module 1013, and a first positioning module 1014.

[0043] The central controller 1011 serves as the core computing and control unit of the vehicle 100, responsible for intelligent decision-making and communication management of the entire vehicle. In some embodiments, the central controller 1011 integrates a multi-mode wireless communication unit, a video signal and path information fusion processing unit, an advanced autonomous driving algorithm unit, and a vehicle control command parsing and execution unit.

[0044] The central controller 1011's built-in multi-mode wireless communication unit can establish a stable two-way communication link with the drone, enabling bidirectional data transmission through wireless communication protocols. These protocols include, but are not limited to: Wireless Fidelity (WiFi) communication protocol, 5G cellular network communication protocol, Bluetooth protocol, and short-range wireless communication protocol.

[0045] In some examples, the multi-mode wireless communication unit achieves high-speed, low-latency video and command transmission based on the WiFi communication protocol, ultra-long-range public network communication based on 5G cellular networks, stable medium- and short-range communication based on the Bluetooth protocol, and serves as a backup communication link based on a distance-based wireless communication protocol. It should be noted that the multi-mode wireless communication unit can also achieve bidirectional data transmission with the drone based on other wireless communication methods; this is only illustrated here.

[0046] The central controller 1011's built-in video signal and path information fusion processing unit fuses aerial images (including pictures and videos) transmitted by the UAV, path planning information, remote road condition information, and vehicle perception data collected by the data acquisition module 1012, and sends the fusion result to the advanced autonomous driving algorithm unit. Based on the fusion result, the advanced autonomous driving algorithm built into the central controller 1011 performs global path planning, local obstacle avoidance, and trajectory tracking, specifically in determining motion control parameters. The central controller 1011's built-in vehicle control command parsing and execution unit parses the commands sent by the UAV and executes control commands or motion control parameters to control the drive system 102, such as vehicle driving, steering, braking, gear shifting, lighting control, door control, and other vehicle control functions.

[0047] The data acquisition module 1012 is used to collect vehicle perception data around the vehicle. The data acquisition module 1012 can be a camera or radar installed on the vehicle. As the core of the vehicle 100's visual perception, it is typically located at the front, side, or rear of the vehicle. The data acquisition module 1012 integrates a vehicle surrounding environment image acquisition unit and a video signal compression and transmission unit. The vehicle surrounding environment image acquisition unit is used to acquire real-time images of the environment around the vehicle, and the video signal compression and transmission unit is used to transmit the environmental images to the central controller 1011, providing visual input for the autonomous driving mode to achieve local obstacle avoidance and road condition recognition.

[0048] The first communication wake-up module 1013, as an independent low-power resident module in the vehicle, is used to wake up the vehicle when it is in sleep mode in response to a vehicle wake-up command sent by the drone, thus putting the vehicle into working mode. The first communication wake-up module 1013 includes an independent power supply device and a wake-up command receiving and triggering unit. The power supply device provides power to the wake-up command receiving and triggering unit, ensuring that the wake-up command receiving and triggering unit receives and responds to the vehicle wake-up command sent by the drone when the vehicle 100 is in sleep mode, causing the vehicle 100 to switch to working mode. Specifically, this requires waking up non-electrical modules such as the central controller 1011, data acquisition module 1012, and drive system 102. The power supply device can be a backup lithium battery or the vehicle's mains battery.

[0049] In this embodiment, the wake-up command receiving and triggering unit supports the Bluetooth Low Energy (BLE) communication protocol. It operates in low-power mode and continuously emits wireless positioning signals (i.e., BLE characteristic signals) to trigger a handshake with the drone, thereby receiving and responding to the vehicle wake-up command issued by the drone to wake up the vehicle 100.

[0050] The first positioning module 1014 can acquire the vehicle's own longitude, latitude, altitude, and other positioning information in real time, and transmit this information to the drone via a wireless communication link. It's important to note that when the vehicle and drone are connected, the vehicle can send its positioning information to the drone in real time or periodically. The first positioning module 1014 integrates a first satellite positioning unit and a first positioning information sharing and transmission unit. The satellite positioning unit can acquire the vehicle's own longitude, latitude, altitude, and other positioning information in real time. The positioning information sharing and transmission unit shares the positioning information to the drone via a wireless communication link, providing location support for vehicle-drone collaborative positioning and path planning.

[0051] The drive system 102 integrates a drive module, a steering module, and a braking module. The drive system 102 receives control commands from the central controller 1011 and performs various actions related to vehicle driving.

[0052] In some embodiments, such as Figure 4 As shown, the UAV 200 includes: a first wireless communication module 201, an image acquisition module 202, a second communication wake-up module 203, a second positioning module 204, and a flight control module 205.

[0053] The first wireless communication module 201 serves as the bidirectional communication unit of the UAV 200, possessing both vehicle communication adaptation capabilities and remote controller communication adaptation capabilities. The first wireless communication module 201 includes a vehicle-side communication unit, a user-side communication unit, and a data relay and forwarding unit. The vehicle-side communication unit can establish a remote wireless connection with the central controller in the vehicle, enabling wireless communication with the vehicle. This includes bidirectional transmission of vehicle perception data collected by the vehicle (specifically, transmitted in the form of video signals), aerial images collected by the UAV, control commands, and positioning information. The user-side communication unit can establish a wireless connection with the remote controller, sending aerial images collected by the UAV, vehicle perception data, vehicle location information, and vehicle status information to the remote controller. The data relay and forwarding unit acts as a relay, enabling data forwarding between the vehicle and the remote controller, forming a complete communication link.

[0054] The image acquisition module 202 is used to acquire images of the vehicle's surrounding environment from the air, generating aerial images. The image acquisition module 202 includes a high-altitude all-domain environmental video acquisition unit and a video signal integration and transmission unit. The high-altitude all-domain environmental video acquisition unit acquires a wide-range, long-distance, high-altitude view of road conditions and the surrounding environment, i.e., aerial images. The video signal integration and transmission unit simultaneously sends the aerial images to the vehicle's central controller and remote controller. The image acquisition module 202 and the vehicle's data acquisition module 1012 form an aerial and ground-based dual-vision perception system, providing all-domain environmental intelligence for the vehicle's advanced autonomous driving and remote user decision-making.

[0055] The second communication wake-up module 203 communicates with the first communication wake-up module in the vehicle using a compatible communication protocol. When the UAV 200 moves within a preset distance range of the vehicle, it sends a vehicle wake-up command to the first communication wake-up module to wake up the vehicle, laying the foundation for subsequent communication and control. The second communication wake-up module 203 includes a short-range communication unit and a wake-up command sending unit. The short-range communication unit uses a compatible communication protocol with the first communication wake-up module in the vehicle, and the wake-up command sending unit is used to send the vehicle wake-up command to the first communication wake-up module in a targeted manner.

[0056] The second positioning module 204 can acquire its own location information in real time and forward it to the vehicle and the remote controller, enabling location information exchange among the three. This provides location data for the remote controller to remotely control the vehicle, the vehicle to plan its autonomous driving path, and the UAV 200 to follow the vehicle. The second positioning module 204 integrates a second satellite positioning unit and a second positioning information sharing and transmission unit. The second satellite positioning unit can acquire the UAV 200's own location information in real time. The second positioning information sharing and transmission unit can synchronously share positioning data with the vehicle and the remote controller.

[0057] The flight control module 205 is used to maintain the stable flight of the UAV 200. It can automatically adjust the flight attitude, altitude, and distance according to the vehicle's position, and maintain the stability of relay communication and video acquisition. The flight control module 205 integrates a flight attitude and altitude control unit, a vehicle-following flight and position adjustment unit, and an obstacle avoidance unit.

[0058] In some embodiments, such as Figure 5 As shown, the remote controller 300 includes: a second wireless communication module 301, a display module 302, a signal input module 303, and a third positioning module 304.

[0059] The second wireless communication module 301 is used to establish a stable wireless connection with the UAV, enabling indirect data interaction with the vehicle through the UAV acting as a relay node. The second wireless communication module 301 includes a UAV communication unit and a video positioning command transceiver unit. It can receive aerial images collected by the UAV, vehicle perception data, vehicle location information, and vehicle status information sent by the UAV, and can also upload user-input commands (such as vehicle control commands and vehicle wake-up commands) and remote controller location information to the UAV.

[0060] Display module 302 is used to display data received by the remote controller. Display module 302 includes: a vehicle camera video display unit, a drone camera video display unit, and a positioning information device status display unit. The vehicle camera video display unit displays video of the vehicle's surrounding environment (i.e., image data from the vehicle's perception data) collected by the vehicle's data acquisition module, while the drone camera video display unit displays high-altitude, full-domain environmental video (i.e., aerial images) collected by the drone's image acquisition module. In some examples, display module 302 can simultaneously display two video signals in split-screen or switch between displaying the image data from the vehicle's perception data and the aerial images, allowing users to have a complete understanding of the vehicle and drone's surrounding environment and providing visual basis for remote control.

[0061] The signal input module 303 serves as a user command input unit, receiving user input information such as vehicle control commands, route commands, and vehicle wake-up commands. The signal input module 303 is divided into a touch input unit (such as a touchscreen), a physical input unit (such as a remote control device compatible with the drone), a voice command input unit, and a control command encoding and verification unit. The touch input unit can use virtual buttons or a virtual joystick on a touchscreen, allowing users to select target points and draw paths through an image (aerial image or map) interface. The physical input unit can use physical buttons, a physical joystick, or a knob, allowing users to control the drone or vehicle by manipulating specific buttons or joysticks. The voice command input unit has voice command recognition capabilities, recognizing user-inputted voice. The control command encoding and verification unit encodes user-input information to form specific commands and also verifies these commands.

[0062] The third positioning module 304 integrates a positioning information acquisition unit and a positioning information transmission unit. The positioning information acquisition unit can acquire its own location information in real time. The positioning information transmission unit is used to send the location data to the UAV, enabling the UAV to forward the location information to the vehicle. This location information can be used as the target location.

[0063] As can be seen from the above, the vehicle remote control system provided in this application embodiment can construct a closed-loop control architecture based on a drone when there is a physical distance between the user and the vehicle and no direct communication link. The drone acts as a relay, receiving vehicle control commands (including steering, acceleration, braking, sleep / wake-up, etc.) output by the remote controller and relaying them to the vehicle (specifically, the vehicle's control system). At the same time, it collects aerial images of the vehicle's surrounding environment and relays them back to the remote controller, providing a basis for the user to remotely control the vehicle through the remote controller. It not only supports autonomous driving of the vehicle but also supports remote control of the remote controller, solving the scenario limitations and filling the gaps in extreme conditions such as long-distance physical isolation between the user and the vehicle, lack of public network signal, and inability of the user to approach the vehicle, thereby achieving remote control of the vehicle and meeting the user's remote vehicle control needs.

[0064] The following describes a vehicle remote control method provided in this application, using a vehicle remote control system as an example. This method is applied to a drone within the vehicle remote control system. Figure 6 As shown, the vehicle remote control method includes: S601. Upon receiving a vehicle search command from the remote controller, obtain the vehicle's historical location.

[0065] The vehicle search command instructs the drone to search for vehicles. The historical location is the last vehicle location acquired by the drone.

[0066] This application does not limit the scenarios in which the remote controller sends vehicle search requests. For example, a vehicle search request can be initiated by a user via the remote controller in the following scenarios: the user needs to find the current location or historical driving trajectory of a target vehicle, but the target vehicle is not within the user's direct control range, or the ground communication network signal in the area where the target vehicle is located is poor, preventing the user from directly obtaining the vehicle's information through conventional methods (such as cloud platforms or cellular networks). In this case, the user operates the remote controller to issue a vehicle search command to a standby drone, utilizing the drone's aerial maneuverability and aerial communication coverage capabilities as a communication relay to obtain the target vehicle's data.

[0067] Specifically, users can input vehicle search commands through the signal input module of the remote controller and send the vehicle search commands to the drone through the second wireless communication module.

[0068] After receiving the vehicle search command from the remote controller, the drone parses the command and extracts the unique identification information of the target vehicle (such as the vehicle identification number VIN or the communication address of the vehicle terminal) and the time window to be queried.

[0069] Since there is currently no communication connection between the drone and the target vehicle, the drone cannot directly send query requests to the target vehicle. Instead, the drone retrieves the target vehicle's historical location data from its own local storage based on the target vehicle's unique identifier. Specifically, during past missions or while in standby mode, the drone continuously receives and stores location information proactively reported by the target vehicle; the stored historical location data includes timestamps and corresponding latitude and longitude coordinates. The drone then filters out all historical location records falling within a specified time frame from its local storage.

[0070] In some embodiments, if historical location data for the target vehicle is not present in local storage, or if the stored data is incomplete (e.g., missing some records within the query time window), the drone sends a data supplementation request to the remote controller. In response to this request, the remote controller retrieves the corresponding missing data from its local storage or a cloud database and returns the retrieval results to the drone. The drone then merges the data returned by the remote controller with its existing local data to obtain a complete historical location dataset.

[0071] In some embodiments, while the remote controller sends vehicle search commands to the drone, it can also send its own location information, i.e., user-end location information, to the drone. This location information can then be used as a reference location for the vehicle's target position when it subsequently performs autonomous driving based on vehicle control commands.

[0072] In some embodiments, before the UAV responds to the vehicle search command, an initialization process is performed first, that is, the first wireless communication module, the image acquisition module, the second communication wake-up module, the second positioning module and the flight control module are powered on and started respectively. After each module is powered on, a self-test operation is performed to ensure that each module is in normal working condition.

[0073] In some embodiments, before responding to a vehicle search command, the UAV sends a reference clock pulse signal to the remote controller to synchronize the remote controller with its own clock. This ensures that subsequent command transmissions and information interactions are not out of sync, guaranteeing the consistency of information transmission and the real-time nature of command responses.

[0074] The first wireless communication module of the UAV sends a reference clock pulse signal to the second wireless communication module of the remote controller. In response to the received reference clock pulse signal, the remote controller sends a feedback clock acknowledge frame to the UAV, and the UAV checks the timing deviation to complete the precise calibration.

[0075] In some examples, clock synchronization can be achieved based on the Controller Area Network Time Synchronization (CANTsyn) method in the Automotive Open System Architecture (AUTOSAR).

[0076] In other examples, clock synchronization can be achieved based on the IEEE Standard for Local and Metropolitan Area Networks – Timing and Synchronization for Time-Sensitive Applications (IEEE 802.1AS).

[0077] In some other embodiments, clock synchronization can be achieved based on network time protocol-based time synchronization mechanisms.

[0078] S602. Determine the initial position of the vehicle based on historical location data and aerial images collected by drones.

[0079] After the drone acquires the historical location of the target vehicle, it can control the onboard image acquisition module (such as a visible light camera or an infrared camera) to acquire aerial images of the area corresponding to the first location.

[0080] The drone performs image recognition processing on the acquired aerial images to identify whether target vehicles are present in the images. Specifically, the drone performs target detection and matching in the aerial images based on a pre-stored target vehicle appearance feature model (including feature information such as vehicle color, model, and license plate area).

[0081] If a target vehicle is identified in the aerial image, the actual position of the target vehicle in the geographic coordinate system is calculated by coordinate conversion based on the pixel position of the target vehicle in the image and the current positioning information of the UAV (including latitude and longitude, flight altitude and gimbal attitude angle), and this actual position is determined as the initial position of the target vehicle.

[0082] If the target vehicle is not identified in the aerial image, the image acquisition range is expanded, and the drone is controlled to fly in a spiral motion outward from the first position information as the center, and aerial images are continuously acquired and identified during the flight until the target vehicle is identified and its initial position is determined.

[0083] As a specific implementation method, S602 can be implemented as follows: S6021. Move to the historical location and acquire the wireless positioning signal emitted by the vehicle during the movement.

[0084] Since the historical location is the vehicle's last reported location information, the vehicle may be at or near that historical location. Using the historical location as a baseline can narrow down the search area to quickly determine the vehicle's initial location.

[0085] A wireless positioning signal is a uniquely identified signal capable of medium to long-distance transmission; it can be a Bluetooth Low Energy (BLE) characteristic signal. The vehicle's first communication wake-up module has an independent power supply. Even when the vehicle is in sleep mode, the first communication wake-up module continues to emit wireless positioning signals to wake the vehicle when a compatible drone, Bluetooth key, or other mobile terminal approaches.

[0086] Because wireless positioning signals attenuate during transmission, and the degree of attenuation varies depending on the location, the drone can use the signal strength of the wireless positioning signal as an auxiliary reference to determine the vehicle's initial position. It should be noted that the drone can only acquire the vehicle's wireless positioning signal when the distance between the drone and the vehicle is less than a first distance threshold.

[0087] In one example, the drone's flight control unit combines its own location information with the vehicle's historical location to plan a preliminary, zone-based scanning search path. It then flies smoothly towards the historical location, automatically avoiding aerial obstacles throughout the journey, ensuring both search efficiency and flight safety. Furthermore, it acquires the wireless positioning signals emitted by the vehicle as the drone moves towards the historical location.

[0088] In some embodiments, if the drone fails to capture a wireless positioning signal during its movement, it automatically expands the search range and re-executes the positioning process within the expanded search range until the target vehicle is found and its initial position is determined.

[0089] S6022: Based on changes in the signal strength of wireless positioning signals, adjust its flight direction to approach the vehicle.

[0090] The strength of the wireless positioning signal acquired by the drone varies depending on its distance from the vehicle. Based on the changes in the strength of the wireless positioning signal, the drone can determine whether it is approaching or moving away from the vehicle, and then adjust its flight direction to get closer to the vehicle.

[0091] In some examples, the drone detects a signal strength increase in the wireless positioning signal that is greater than or equal to a first change threshold (i.e., a significant increase in strength), indicating that the drone is approaching the vehicle. The drone can maintain its current flight direction and continue to approach the vehicle.

[0092] In some other examples, the drone detects a signal strength enhancement of the wireless positioning signal that is greater than or equal to a second change threshold and less than a first change threshold (i.e., the strength is enhanced, but the enhancement is small). This indicates that the drone is approaching the vehicle and there is a directional deviation. The drone can adjust its flight direction to get closer to the vehicle.

[0093] In some other examples, the drone detected a signal strength reduction of the wireless positioning signal greater than or equal to the third change threshold (i.e., a significant decrease in strength), indicating that the drone was moving away from the vehicle. The drone could then turn around or significantly adjust its flight direction to approach the vehicle.

[0094] In some other examples, the drone detected a signal attenuation of the wireless positioning signal that was greater than or equal to the fourth change threshold and less than the third change threshold (i.e., the intensity was weakened, but the weakening was small). This indicates that the drone was moving away from the vehicle, but not completely away. The drone could adjust its direction to move closer to the vehicle.

[0095] It should be noted that the above-mentioned change threshold is only an example. In practical applications, more change thresholds can be defined based on the flight and rotation attitude of the drone to determine the gradient change of signal strength, narrow the search range, and adjust the flight direction to continuously approach the vehicle.

[0096] S6023. Determine the initial position of the vehicle based on aerial images and vehicle body feature information.

[0097] When the drone enters the coverage area of ​​the wireless positioning signal, it locks onto the location of the signal source and then adjusts its flight attitude to approach the target area. Simultaneously, it activates the image acquisition module to capture aerial images of the vehicle's surroundings. These aerial images contain a wealth of information; when a vehicle is present in the image, the drone can determine its initial position based on the image. However, aerial images may contain multiple vehicles, making it difficult for the drone to directly identify the target vehicle. Therefore, the drone acquires the target vehicle's body feature information, including vehicle model, outline, license plate number, and color.

[0098] Furthermore, the drone can use its built-in image recognition algorithm to match the collected aerial images with vehicle body feature information, identify the target vehicle that matches the vehicle body feature information from multiple vehicles, and then determine the position of the vehicle in the image. Combined with the drone's current positioning information, the actual position of the target vehicle in the geographic coordinate system can be calculated, thus completing the accurate identification and positioning of the vehicle, i.e., determining the initial position of the vehicle.

[0099] In some embodiments, if the drone fails to determine the initial location of the vehicle within a preset time period, the drone can send a message of vehicle search failure to the remote controller, prompting the user to supplement vehicle feature information before restarting the vehicle search.

[0100] As can be seen from S6021-S6023 above, this embodiment uses the historical location as the initial reference area and adjusts its flight direction to approach the vehicle based on the signal strength of the wireless positioning signal. By utilizing the physical characteristic of signal strength changing with distance as a basis for real-time navigation, the UAV can dynamically approach the signal source even without precise vehicle coordinates, thereby improving the efficiency of direction adjustment during the search process and reducing invalid flights caused by trajectory deviations. Furthermore, aerial images provide visual information about the vehicle's location. By using vehicle exterior features and other vehicle characteristic information as identification criteria, target matching of the aerial images can eliminate misjudgments caused by interference sources or environmental reflections, thus improving the spatial accuracy and reliability of the initial position determination. In summary, this application can achieve a relatively stable and accurate positioning process when the vehicle's position is uncertain through the coordinated cooperation of signal guidance from the wireless positioning signal and visual confirmation from the aerial images.

[0101] S603, Move to the first position and send an information collection command to the vehicle.

[0102] After determining the vehicle's initial position, the drone adjusts its flight altitude and attitude, hovering at a first position near the vehicle. This first position is located within a radius of a preset distance centered on the initial position. While in this first position, the drone can establish a communication connection with the vehicle and maintain a communication connection with the remote controller, thus ensuring the stability of two-way data transmission.

[0103] After the drone moves to its initial position, it establishes a communication connection with the vehicle and sends an information collection command to instruct the vehicle to collect perception data of the surrounding environment. Upon receiving the command, the vehicle collects vehicle perception data of the surrounding environment, which may include: visual information such as road images, traffic signs, pedestrians, and vehicles captured by cameras; 3D point cloud data captured by LiDAR; distance and relative speed of objects ahead captured by millimeter-wave radar; and nearby obstacles detected by ultrasonic sensors. This environmental perception data, combined with aerial images, helps the user comprehensively assess the environment around the vehicle, providing a visual basis for vehicle control.

[0104] In some embodiments, after the drone moves to the first location, or after the first wireless positioning signal is detected, the drone sends a reference clock pulse signal to the vehicle to synchronize the vehicle's clock with its own. This ensures that subsequent command transmissions and information interactions are not out of sync, guaranteeing the consistency of information transmission and the real-time nature of command responses.

[0105] The drone's first wireless communication module sends a reference clock pulse signal to the vehicle's first communication wake-up module. In response to the received reference clock pulse signal, the vehicle sends a feedback clock acknowledge frame to the drone. The drone then checks the timing deviation and completes precise calibration. The specific clock synchronization method can be referenced from the clock synchronization method between the drone and the remote controller. At this point, the drone, vehicle, and remote controller achieve three-way clock synchronization.

[0106] In some embodiments, the drone establishes other communication connections with the vehicle, specifically establishing a communication connection between the first wireless communication module in the drone and the central controller of the vehicle.

[0107] When the vehicle is in operation, the UAV establishes an initial communication handshake connection with the vehicle's central controller through its configured first wireless communication module. This initial communication handshake connection is a bidirectional data connection. The UAV sends a protocol negotiation request to the vehicle's central controller through this bidirectional data connection. This protocol negotiation request contains a list of communication protocol types supported by the first wireless communication module.

[0108] Upon receiving the protocol negotiation request, the vehicle's central controller selects a target communication protocol from the list of communication protocol types based on the current environmental interference conditions, and returns a protocol negotiation response to the UAV via the bidirectional data connection. This protocol negotiation response contains the selected target communication protocol type.

[0109] The UAV switches the current operating protocol of the first wireless communication module to the target communication protocol according to the target communication protocol in the negotiation response, and re-establishes a communication connection with the vehicle's central controller according to the target communication protocol.

[0110] During the communication connection between the drone and the vehicle, the drone continuously monitors the channel quality index of the current communication link. When the channel quality index falls below a preset threshold, the drone resends a protocol negotiation request to the vehicle's central controller through the communication connection, triggering a protocol renegotiation and switching process. This allows the drone and the vehicle's central controller to dynamically select a communication protocol that meets the current environmental conditions from among WiFi, 5G cellular network, Bluetooth, or short-range wireless communication protocols. After the communication connection is established, the drone receives vehicle status data from the vehicle's central controller through this connection and sends control commands from the remote controller to the vehicle's central controller through the same connection, ensuring low latency and high stability in information and command transmission between the drone and the vehicle.

[0111] The communication connection is a two-way data connection. The wireless communication protocol between the drone and the vehicle can be automatically selected according to the environment, that is, switching between WiFi communication protocol, 5G cellular network communication protocol, Bluetooth protocol and short-range wireless communication protocol to ensure low latency and high stability of information and command transmission.

[0112] S604 receives vehicle perception data sent by the vehicle and sends the vehicle perception data, along with the aerial images it has collected, to the remote controller.

[0113] The drone receives vehicle perception data transmitted by the vehicle through an established communication connection. This vehicle perception data includes information about the surrounding environment collected by onboard sensors while the vehicle is in motion or in standby mode. Optionally, the onboard sensors include, but are not limited to, one or more of onboard cameras, millimeter-wave radar, lidar, and ultrasonic radar. Correspondingly, the vehicle perception data includes one or more of the following: road surface images and / or video streams collected by the onboard camera, obstacle distance and relative speed information detected by millimeter-wave radar, point cloud data generated by lidar, and near-range obstacle information detected by ultrasonic radar.

[0114] Meanwhile, the drone continuously acquires aerial images of the area where the vehicle is located using its onboard image acquisition module (such as one or more of a visible light camera, infrared camera, or multispectral camera). Optionally, the aerial images acquired by the drone include images of the vehicle itself and a larger area of ​​its surrounding environment, and the field of view of the aerial images is greater than the field of view of the road surface images in the vehicle's perception data.

[0115] The drone associates the received vehicle perception data with the aerial images it has captured before sending it to a remote controller. This association processing includes, but is not limited to: aligning timestamps to establish a correspondence between vehicle perception data and aerial images at the same moment; or fusing spatial locations to map obstacle and road information from the vehicle perception data to corresponding geographic coordinates in the aerial images, creating a multi-angle environmental view that includes both overhead and ground-level perspectives.

[0116] Aerial imagery provides a high-altitude, bird's-eye view of the vehicle's surroundings, showcasing the road topology, surrounding building distribution, distant traffic conditions, and overall environmental layout. Vehicle perception data, on the other hand, provides a ground-level, horizontal view of the vehicle's surroundings, displaying precise road details, nearby obstacles, and the accurate location and distance information of pedestrians or other road users. The complementary perspectives of aerial imagery and vehicle perception data allow the remote controller to simultaneously acquire both large-scale, long-range environmental situational information and high-precision, close-range local environmental information.

[0117] In some embodiments, the drone also receives vehicle status data (such as battery level, operating status of each module, and status of each device) sent by the vehicle. The drone sends the vehicle status data and drone status data (such as flight attitude, communication quality, and positioning information) to a remote controller, enabling the user to have a comprehensive understanding of the vehicle and drone status through the remote controller, providing the user with a basis for vehicle control.

[0118] After receiving vehicle perception data, vehicle status data, aerial images, and drone status data from the drone, the remote controller can send vehicle control commands to the drone and grant control permissions, i.e., authorize the drone to forward vehicle control commands to the vehicle.

[0119] S605. In response to receiving a vehicle control command from a remote controller, the vehicle control command is sent to the vehicle.

[0120] In some embodiments, the remote controller receives and displays aerial images and vehicle perception data transmitted by the drone. After the user simultaneously views the aerial images and vehicle perception data in split-screen or picture-in-picture mode, vehicle control commands are generated based on the aforementioned image information.

[0121] Specifically, users use aerial images to understand the overall road topology, traffic conditions, and macroscopic environment of the area where the vehicle is located, and use vehicle perception data to understand the distribution of nearby obstacles, road surface details, and precise distance information around the vehicle. By combining the above two types of image information, users determine whether the current environment is suitable for passage, whether there are obstacles, and whether it is necessary to avoid or turn, and generate corresponding vehicle control commands accordingly.

[0122] For example, when a user observes a congested or closed section of road ahead of the vehicle through aerial images, and at the same time confirms through vehicle perception data that there are no other vehicles or obstacles in the immediate vicinity in front of the vehicle, a steering control command is generated to steer the vehicle to an alternative route shown in the aerial images.

[0123] For example, when a user detects a nearby obstacle in front of the vehicle through vehicle perception data, and confirms the location of the obstacle and the surrounding passable area through aerial images, a deceleration or obstacle avoidance control command is generated to control the vehicle to decelerate or detour.

[0124] For example, when a user confirms through aerial images that the vehicle is approaching the target location (such as a designated parking area) and at the same time confirms through vehicle perception data that the surrounding environment is safe, a parking or parking control command is generated to control the vehicle to drive into the target location and stop.

[0125] After generating vehicle control commands, the remote controller sends the commands to the drone. Upon receiving the vehicle control commands from the remote controller, the drone forwards them to the vehicle via the established communication link. The vehicle then executes the control commands.

[0126] In other embodiments, the remote controller also automatically generates vehicle control commands based on received aerial images and vehicle perception data, without requiring manual operation by the user, based on a preset autonomous driving algorithm. For example, the remote controller has built-in path planning and obstacle avoidance algorithms. Based on the road network in the aerial images and the obstacle distribution in the vehicle perception data, it automatically plans the vehicle's driving path and generates corresponding speed control and steering control commands, which are then sent to the drone for forwarding to the vehicle.

[0127] As shown above, this application utilizes the aerial photography and relay transmission capabilities of drones, combining the vehicle's historical location with drone aerial images to determine the vehicle's initial position. This improves the accuracy of vehicle location identification, avoids the bias problems inherent in single positioning methods, and ensures the reliability and efficiency of long-distance vehicle search. Simultaneously, the drone moves to a preset range around the vehicle, acting as a relay transmission unit to establish a communication link between the remote controller and the vehicle, facilitating data interaction. Specifically, it receives vehicle perception data sent by the vehicle and transmits this data, along with its own collected aerial images, to the remote controller, allowing the user on the remote controller side to understand the vehicle's environment. This addresses the problem of users being unable to directly control vehicles in long-distance scenarios, improving the stability and feasibility of long-distance vehicle control and meeting users' needs for remote vehicle control.

[0128] In some embodiments, to facilitate user verification of the autonomously planned driving path, the vehicle feeds back the calculated control parameters and / or planned path to the remote controller via a drone for user confirmation.

[0129] As a feasible implementation method, after sending vehicle control commands to the vehicle, combined with... Figure 6 ,like Figure 7 As shown, the vehicle remote control method also includes: S701, in response to receiving motion control parameters sent by the vehicle, sends the motion control parameters to the remote controller.

[0130] Motion control parameters are parameters generated by the vehicle based on vehicle control commands and used to control the actions of the vehicle's actuators. These parameters include: driving speed, steering angle, and braking timing.

[0131] After receiving vehicle control commands from a remote controller via a drone, the vehicle can activate an advanced autonomous driving algorithm. Then, based on its current location, the target location information indicated by the vehicle control commands, aerial images captured by the drone, and its own collected vehicle perception data, the advanced autonomous driving algorithm performs global path planning and local trajectory tracking to determine the control parameters required for the vehicle to travel from its current location to the target location. This application does not limit the specific method used to determine these control parameters.

[0132] In one implementation, the drone, through its onboard first wireless communication module, receives motion control parameters sent by the vehicle according to a predetermined communication cycle. These motion control parameters include the vehicle's speed, steering angle, and braking timing. The drone converts the received motion control parameters into data packets according to a preset format and sends these data packets to a remote controller. The remote controller then parses the data packets to obtain the motion control parameters, allowing the user to access them. It is important to note that the preset format conforms to the communication protocol of the remote controller's second wireless communication module, and the data packets comply with the remote controller's parsing standards.

[0133] After the motion control parameters are sent to the remote controller, the user can confirm them through the remote controller. Through the remote controller, the user not only sees the aerial images from the drone's perspective but also the environmental perception data from the vehicle's perspective, thus gaining a relatively comprehensive understanding of the vehicle's surroundings. To ensure the safety and reliability of remote vehicle control, the motion control parameters generated by the vehicle in autonomous driving mode are fed back to the remote controller for the user to confirm. This allows the user to verify whether the motion strategy planned by the vehicle in autonomous driving mode meets their needs, or to assess the potential risks of the motion control parameters from an external perspective, preventing the output of motion control parameters even if the vehicle's intelligent driving algorithm malfunctions.

[0134] S702, upon receiving a motion confirmation command from the remote controller, sends a motion confirmation command to the vehicle.

[0135] After the user confirms the planning results are correct on the remote controller, the remote controller generates a motion confirmation command and sends it to the drone. The drone receives the motion confirmation command from the remote controller through its onboard first wireless communication module and forwards it to the vehicle. The vehicle then responds to the motion confirmation command and executes the corresponding motion operation based on the motion control parameters.

[0136] As can be seen from the above, if the intelligent driving algorithm on the vehicle malfunctions but still outputs motion control parameters, the vehicle directly executes corresponding motion operations based on these parameters, which can easily lead to risks. In this embodiment, the drone acts as a relay to receive and forward motion control parameters, allowing the user to assess the risks of the motion control parameters through a remote controller. Additionally, the user can also use the remote controller to determine whether the motion strategy corresponding to the motion control parameters meets their needs. Correspondingly, the drone acts as a relay to receive and forward motion confirmation commands, enabling the vehicle to receive confirmation responses from the remote controller, thus forming a complete closed-loop control link from vehicle-side parameter reporting to remote controller command feedback. This application establishes a two-way data relay mechanism between the vehicle and the remote controller using a drone, enabling a complete interactive process of motion parameter uploading and control command issuance even when the vehicle's long-distance communication capabilities are limited, ensuring the safety and reliability of remote vehicle control.

[0137] In some embodiments, when the vehicle control command is a specific motion command, combined with Figure 6 ,like Figure 8 As shown, in S605, in response to receiving a vehicle control command from the remote controller, the vehicle control command is sent to the vehicle, specifically including: S801. In response to receiving a vehicle control command sent by a remote controller, perform a security check on the vehicle control command and determine the risk value of the vehicle control command.

[0138] The vehicle control commands include at least one of the following: steering commands, acceleration commands, braking commands, and gear shifting commands.

[0139] In this embodiment of the application, the user can directly remotely control the vehicle through the remote controller. That is, the user inputs vehicle control commands through the signal input module of the remote controller, and then relays them to the vehicle through the drone to achieve remote control of the vehicle.

[0140] It's important to understand that users might mistakenly input incorrect vehicle control commands. If the drone directly relays these commands to the vehicle, it could pose a danger. In one example, the vehicle is in a complex environment with high density of vegetation or buildings. If the vehicle control command instructs the vehicle to travel at 140 km / h, it might collide with vegetation or buildings. In another example, the vehicle is in a scene with many surrounding vehicles. If the vehicle control command instructs the vehicle to travel in the first direction, and there are vehicles or obstacles in that direction, a collision could also occur. Therefore, to prevent vehicle control commands generated by user error from being directly applied to the vehicle, the drone needs to perform safety verification on the vehicle control commands and determine their risk values.

[0141] In some embodiments, safety verification includes retrieving the corresponding current vehicle state parameters and control parameters based on the type of vehicle control command. For example, for a steering command, the current vehicle speed and current steering angle are retrieved; for an acceleration command, the current vehicle speed is retrieved; for a braking command, the current vehicle speed and braking deceleration are retrieved; and for a gear shifting command, the current vehicle speed and engine speed are retrieved. The UAV compares the target control parameters (such as target steering angle, target acceleration, target deceleration, and target gear) in the vehicle control command with the current vehicle state parameters to determine whether the target control parameters exceed a safety threshold, thereby determining the risk value.

[0142] In other embodiments, the risk value is determined by calculating the deviation between the target control parameters and the current vehicle state parameters, and determining the risk level based on the magnitude of the deviation; the larger the deviation, the higher the risk level and the greater the risk value; the smaller the deviation, the lower the risk level and the smaller the risk value.

[0143] In some embodiments, when the UAV performs safety verification on vehicle control commands, it also combines aerial images collected by the UAV itself or vehicle perception data uploaded by the vehicle to jointly determine the risk value. For example, if the UAV identifies road conditions ahead (such as curves, pedestrians, and obstacles) in the aerial images, and the vehicle control command requires acceleration through curves or areas with dense pedestrian traffic, the risk value will be increased accordingly during safety verification, thereby raising the risk assessment level.

[0144] S802. When the risk value is less than the risk threshold, a vehicle control command is sent to the vehicle.

[0145] When the risk value is less than the risk threshold, it means that there is no risk in moving the vehicle based on the vehicle control command, or the risk is small and controllable. The drone sends the vehicle control command to the vehicle, so that the vehicle moves based on the vehicle control command.

[0146] As can be seen from the above, this application uses a risk threshold as a dynamic decision-making limit, which not only allows routine commands to pass quickly but also blocks and isolates high-risk operations, preventing vehicles from going out of control due to miscontrol or attacks. This ensures that remote vehicle control maintains verifiable security even in complex network environments, thereby guaranteeing accurate and reliable remote operation of vehicles.

[0147] In some embodiments, combined with Figure 8 ,like Figure 9 As shown, the vehicle remote control method also includes: S901. When the risk value is greater than the risk threshold, an alarm signal is generated and sent to the remote controller.

[0148] Based on the above embodiments, when the risk value exceeds the risk threshold, it indicates that the risk of the vehicle moving based on the vehicle control command is significant and uncontrollable. In this case, the drone refuses to forward the vehicle control command to the vehicle and generates an alarm signal, sending it to the remote controller. This alerts the user to the risk associated with the vehicle control command and prompts them to reissue the vehicle control command via the remote controller.

[0149] In some embodiments, after the control of the vehicle ends, the UAV responds to the control end command sent by the received remote controller, stops collecting aerial images and information relays, and returns to a preset recovery point or remains in place according to user instructions.

[0150] In some application scenarios, the vehicle and the user are far apart, but satellite positioning signals are available. After the drone establishes a connection with the vehicle and the remote controller, the vehicle can collect its positioning information and send it to the drone, which then relays it to the remote controller. Similarly, the remote controller can collect the user's positioning information (i.e., its own positioning information) and send it to the drone, which then relays it to the vehicle. The drone can also collect its own positioning information and send it to both the vehicle and the remote controller. In addition to positioning information, the drone can also act as a relay to receive and forward other information, such as vehicle perception data, aerial images, vehicle status data, and drone status data, establishing a communication link to achieve location and information exchange among the three parties.

[0151] The above embodiments primarily focus on the remote vehicle control method from the UAV side, briefly introducing the functions and implementation methods of the vehicle-side remote controller in the process of implementing the remote vehicle control method. For the implementation methods of the vehicle-side and remote controller-side remote vehicle control method, please refer to the above embodiments or the embodiments of the vehicle remote control system described below; these will not be repeated here.

[0152] This application also provides a vehicle remote control system, which may include a drone for executing the vehicle remote control method provided in the above embodiments; a remote controller, which is communicatively connected to the drone; and a vehicle.

[0153] In some embodiments, the remote controller receives and displays aerial images and vehicle perception data transmitted by the drone. After observing the aerial images and vehicle perception data on the remote controller's display interface, the user performs a first operation on the display interface. The first operation includes one or more of the following: selecting a target point on the aerial image, drawing a driving path on the display interface, or inputting coordinates. Optionally, the target point includes the remote controller's own location.

[0154] In response to the first operation, the remote controller generates a first vehicle control command, which includes target location information and driving preference settings, including one or more of driving speed preferences and driving style preferences. The remote controller then sends the first vehicle control command to the drone.

[0155] Upon receiving the first vehicle control command, the drone sends the first vehicle control command and the aerial photography data it has collected to the vehicle.

[0156] Upon receiving the first vehicle control command and the aerial photography data, the vehicle initiates an advanced autonomous driving algorithm. Based on the target location information, driving preference settings, aerial photography data, and vehicle perception data collected by the vehicle itself, it performs global path planning and local trajectory tracking to determine the motion control parameters required for the vehicle to travel from its current location to the target location. The motion control parameters include driving speed, steering angle, and braking timing.

[0157] After completing path planning, the vehicle transmits the planning results to a remote controller via a drone for user confirmation. Upon user confirmation, the vehicle enters the autonomous driving execution phase, controlling the vehicle to travel to the target location according to the stated motion control parameters.

[0158] In this embodiment, after observing aerial images and vehicle perception data through a remote controller, the user selects a target point or draws a path. The remote controller then generates a first vehicle control command. The drone sends the first vehicle control command and aerial data to the vehicle. The vehicle autonomously completes path planning based on the target location, driving preferences, and dual-source visual data, and provides feedback confirmation. This fully utilizes the complete environmental perception brought by both aerial and ground-level perspectives, while also combining the user's subjective intent with the accuracy of the autonomous driving algorithm, thus balancing the flexibility and safety of remote control.

[0159] As a feasible implementation method, the remote controller is configured to receive and display vehicle perception data of the vehicle and aerial photography data collected by the UAV; the target location information is generated by the remote controller according to user operation, the user operation including selecting a target point, drawing a driving path or inputting coordinates on the display interface of the remote controller.

[0160] The vehicle perception data includes information about the surrounding environment collected by onboard sensors when the vehicle is in motion or in standby mode, such as road images captured by onboard cameras; the aerial data includes aerial views of the area where the vehicle is located, captured by drones through their onboard image acquisition modules. The remote controller displays both on the same screen, allowing users to simultaneously view the aerial and road images via split-screen or picture-in-picture mode, thus gaining an intuitive understanding of the overall layout and local details of the vehicle's surroundings.

[0161] After observing aerial images and vehicle perception data on the remote controller's display interface, the user comprehensively judges the vehicle's current environment and drivable direction based on the road topology and traffic conditions in the aerial images, and the obstacle distribution and road surface details in the vehicle perception data. The user then performs corresponding operations on the display interface, including selecting a target point on the aerial image, drawing a driving path on the display interface, or inputting coordinates. Selecting a target point involves the user directly clicking on a location in the aerial image; drawing a driving path involves the user planning the desired path for the vehicle by drawing lines or curves on the aerial image; and inputting coordinates involves the user manually inputting latitude and longitude values ​​to specify the target location. Optionally, the target point may also include the remote controller's own current location. When the user wants the vehicle to travel to their location, they can directly select the remote controller's own location as the target point.

[0162] In response to the user operation, the remote controller generates a first vehicle control command. This command includes target location information and driving preference settings, which include one or more of a driving speed preference (e.g., economy mode, standard mode, or sport mode) and a driving style preference (e.g., aggressive or smooth). The remote controller sends the first vehicle control command to the drone, which then forwards it to the vehicle, enabling the vehicle to perform corresponding autonomous driving operations based on the command.

[0163] In other embodiments, the remote controller receives and displays aerial images and vehicle perception data transmitted by the drone. After observing the aerial images and vehicle perception data on the remote controller's display interface, the user performs a second operation. The second operation includes one or more of the following: touching virtual buttons, operating a physical joystick, inputting voice commands, or drawing a driving trajectory on a map interface.

[0164] In response to the second operation, the remote controller generates a second vehicle control command. This second vehicle control command includes a target maneuver, which may include one or more of the following: steering, acceleration, braking, gear shifting, and emergency stop. After encoding and verifying the second vehicle control command, the remote controller sends it to the drone.

[0165] After the drone forwards the command to the vehicle, the vehicle is configured to: execute the corresponding target control action in response to receiving the second vehicle control command; and send the real-time collected vehicle perception data to the drone during the execution of the target control action. For example, if the second vehicle control command includes a steering action and a corresponding target steering angle, the vehicle controls the steering system to turn to the target steering angle; if it includes an acceleration action and a corresponding target speed or target acceleration, the vehicle controls the powertrain system to output the corresponding drive torque to achieve acceleration; if it includes a braking action and a corresponding target deceleration, the vehicle controls the braking system to apply the corresponding braking force to achieve deceleration or stopping; if it includes a gear shifting action and a corresponding target gear, the vehicle controls the transmission to shift to the target gear; and if it includes an emergency stop action, the vehicle controls the powertrain system and braking system to perform an emergency stop operation.

[0166] The drone is configured to send vehicle perception data and aerial images to the remote controller, enabling the user to determine the vehicle's operational status based on the remote controller. The drone also sends the received vehicle perception data and its own real-time aerial images to the remote controller, allowing the user to simultaneously observe both the aerial images and vehicle perception data on the remote controller's display interface. This allows the user to confirm in real-time whether the vehicle has performed the expected target control actions, monitor the vehicle's operational status and changes in the surrounding environment during the control actions, and promptly issue subsequent control commands based on the actual situation.

[0167] As a feasible implementation method, the UAV, upon receiving the second vehicle control command, performs a secondary safety verification on the second vehicle control command. The secondary safety verification includes: determining whether the control parameters corresponding to the target manipulation action exceed a preset safety threshold. The control parameters include one or more of the following: steering angle, acceleration value, deceleration value, and gear shifting sequence. The safety threshold includes one or more of the following: vehicle mechanical steering limit value, vehicle maximum acceleration value, vehicle maximum braking deceleration value, and allowable speed range for gear shifting.

[0168] If the control parameters corresponding to the target manipulation action do not exceed the safety threshold, the second vehicle control command is deemed safe, and the UAV forwards the second vehicle control command to the vehicle. Upon receiving the second vehicle control command, the vehicle executes the corresponding target manipulation action.

[0169] If the control parameters corresponding to the target control action exceed the safety threshold, it is determined that the second vehicle control command has a safety risk. The drone refuses to forward the second vehicle control command and sends a warning to the remote controller to alert the user that the current control action has a safety risk.

[0170] In this embodiment, the user can issue manual control commands such as steering, acceleration, braking, gear shifting, and emergency stop via a remote controller. During the execution of the control actions, the vehicle's real-time perception data is transmitted back to the remote controller via a drone and aerial images. This allows the user to simultaneously observe the vehicle's dynamics and changes in the surrounding environment from both aerial and ground-level perspectives, and to confirm the execution effect of the control actions in real time. This improves the real-time performance, intuitiveness, and monitorability of remote manual control, and reduces the risk of misoperation due to information delays or limited viewing angles.

[0171] In some embodiments, the remote vehicle may be in a dormant state to reduce power consumption and extend standby time. In this case, after the drone moves to the first location, it needs to wake up the vehicle before it can establish a communication connection with the vehicle and conduct subsequent data interaction.

[0172] As a feasible implementation method, the drone is also configured to detect the vehicle's current communication status after moving to the first location. Specifically, the drone sends a connection request or handshake signal to the vehicle. If no response signal is received from the vehicle within a preset time, the drone determines that the vehicle is in a dormant state. Alternatively, if the drone cannot complete a handshake confirmation with the vehicle when initially establishing a V2X communication link, the drone determines that the vehicle is in a dormant state.

[0173] After determining that the vehicle is in a dormant state, the drone sends a vehicle wake-up command to the vehicle. This command is transmitted via V2X communication module to the vehicle's low-power wake-up receiver module in a broadcast or point-to-point manner. The vehicle's low-power wake-up receiver module is independent of the vehicle's main control unit, maintains a constant monitoring state, and consumes extremely low power.

[0174] After receiving the wake-up command from the drone, the vehicle's low-power wake-up receiver module parses and verifies the command to ensure its validity. Once verified, the low-power wake-up receiver module sends a power-on signal to the vehicle's main control unit and onboard communication unit, switching the vehicle's main control unit, onboard T-Box, and related sensors from sleep mode to operating mode.

[0175] In one example, combining Figure 3 , Figure 4The schematic diagram shows that the second communication wake-up module of the drone communicates with the first communication wake-up module of the vehicle to send a vehicle wake-up command encrypted with a wake-up code (this wake-up code is unique and only known by the vehicle and its paired drone and mobile terminal). Upon receiving the vehicle wake-up command and completing verification, the first communication wake-up module of the vehicle triggers the vehicle wake-up mechanism, waking up the central controller, data acquisition module, and first positioning module, and supplying power to each module through an independent power supply device. At this point, the vehicle switches to operational status.

[0176] After the vehicle switches to operational status, it establishes a communication connection with the drone and sends a wake-up confirmation response to the drone, notifying it that the vehicle is now operational and ready for data exchange. Upon receiving the wake-up confirmation response, the drone establishes a communication connection with the vehicle and continues with subsequent information collection or control command interaction processes.

[0177] Optionally, if the vehicle does not receive a wake-up command from the drone within a preset time, or if the low-power wake-up receiving module fails to verify the wake-up command, the vehicle will remain in sleep mode and will not respond.

[0178] Optionally, after sending the wake-up command, the drone starts a timeout timer. If no wake-up confirmation response is received from the vehicle within the preset time, the wake-up is determined to have failed. The drone then sends the wake-up failure information to the remote controller to inform the user that the vehicle cannot be woken up at this time.

[0179] In this embodiment, the vehicle is equipped with an independent low-power wake-up receiver module, which can always maintain a listening state when the main control unit is in sleep mode. After the drone detects that the vehicle is in sleep mode, it actively sends a wake-up command to remotely wake up the vehicle and switch it to working mode. This eliminates the need for the vehicle to maintain high power consumption when in standby mode, greatly reducing the vehicle's power consumption and extending standby time. At the same time, it ensures that the vehicle can be quickly woken up and a communication connection can be established by the drone when remote control is needed, thus taking into account both the energy-saving requirements of the vehicle and the real-time requirements of remote control.

[0180] In summary, this application utilizes a drone as the core of global relay and aerial perception to construct a remote vehicle control system. Based on the drone's aerial photography capabilities and wireless signal (wireless positioning signal) scanning capabilities, it achieves precise vehicle search and positioning. Using a communication wake-up module onboard the drone and vehicle, it enables remote power-on wake-up when the vehicle is in sleep mode, reducing reliance on vehicle online status, public network signals, satellite positioning, and short-range communication. Simultaneously, the drone acts as a relay between the vehicle and the remote controller, enabling bidirectional transmission of various data. The remote controller can achieve real-time vehicle control via the drone and also control the vehicle for autonomous driving. This enables vehicle search and remote control in scenarios where the user and vehicle are physically far apart, lack direct communication, or the vehicle is in sleep mode or without public network signal. Specific examples include remote vehicle control in the wild, vehicle control in areas without signal, emergency rescue vehicle control (such as vehicles stuck in dangerous areas like heavy rain, blizzards, or mudslides), emergency disaster relief vehicle control, and remote area operations (such as power line inspection and geological exploration). Users can control vehicles via drone relay even when far from the vehicle and with weak signal, enabling the transfer of work materials and adjustment of parking positions. It can not only improve the user experience, but also reduce operation and rescue costs.

[0181] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for remote vehicle control, characterized in that, Applied to drones, the method includes: Upon receiving a vehicle search command from a remote controller, the drone acquires the vehicle's historical location; the historical location is the last vehicle location acquired by the drone. Based on the historical location and the aerial images collected by the drone, the initial position of the vehicle is determined; The vehicle moves to a first position and sends an information collection command to the vehicle; wherein the first position is located within a range centered on the initial position and with a preset distance as the radius; the information collection command is used to instruct the vehicle to collect perception data of the surrounding environment; Receive vehicle perception data sent by the vehicle, and send the vehicle perception data, along with the aerial images collected by itself, to the remote controller; In response to receiving a vehicle control command from the remote controller, the vehicle control command is sent to the vehicle to cause the vehicle to move based on the vehicle control command.

2. The method according to claim 1, characterized in that, Determining the initial position of the vehicle based on the historical location and the aerial images collected by the drone includes: Move to the historical location and, during the movement, acquire the wireless positioning signal emitted by the vehicle; Based on the changes in the signal strength of the wireless positioning signal, the aircraft adjusts its flight direction to approach the vehicle. Based on the aerial image and the vehicle's body feature information, the initial position of the vehicle is determined.

3. The method according to claim 1, characterized in that, The method further includes: In response to receiving motion control parameters sent by the vehicle, the motion control parameters are sent to the remote controller for user confirmation via the remote controller; the motion control parameters are parameters generated by the vehicle based on the vehicle control commands and are used to control the actions of the vehicle's actuators. Upon receiving a motion confirmation command from the remote controller, the motion confirmation command is sent to the vehicle to cause the vehicle to begin moving based on the motion control parameters.

4. The method according to claim 1, characterized in that, The step of responding to receiving a vehicle control command from the remote controller and sending the vehicle control command to the vehicle includes: In response to receiving a vehicle control command sent by the remote controller, the vehicle control command is subjected to a safety verification to determine the risk value of the vehicle control command; the vehicle control command includes at least one of the following: steering command, acceleration command, braking command, and gear shifting command; When the risk value is less than the risk threshold, the vehicle control command is sent to the vehicle; The method further includes: When the risk value exceeds the risk threshold, an alarm signal is generated and sent to the remote controller.

5. A vehicle remote control system, characterized in that, include: The drone performs the vehicle remote control method as described in any one of claims 1-4; A remote controller, which is communicatively connected to the UAV; vehicle.

6. The vehicle remote control system according to claim 5, characterized in that, The drone is specifically configured to, in response to receiving a first vehicle control command sent by the remote controller, send the first vehicle control command and the aerial photography data it has collected to the vehicle; The first vehicle control command includes target location information and driving preference settings; The vehicle is configured as follows: In response to receiving the first vehicle control command and the aerial photography data, the vehicle performs path planning based on the target location information, the driving preference settings, and the aerial photography data, and determines the vehicle's motion control parameters; the motion control parameters include: driving speed, steering angle, and braking timing.

7. The vehicle remote control system according to claim 6, characterized in that, The remote controller is configured to receive and display vehicle perception data of the vehicle and aerial photography data collected by the drone. The target location information is generated by the remote controller based on user operations, which include selecting a target point, drawing a driving path, or inputting coordinates on the display interface of the remote controller.

8. The vehicle remote control system according to claim 5, characterized in that, The drone is specifically configured to send the second vehicle control command to the vehicle in response to receiving the second vehicle control command sent by the remote controller; the second vehicle control command includes a target control action, which includes one or more of steering, acceleration, braking, gear shifting, and emergency stop. The vehicle is configured as follows: In response to receiving the second vehicle control command, execute the corresponding target control action; During the execution of the target control action, the vehicle perception data collected in real time is sent to the drone; The drone is configured to send the vehicle perception data and aerial images to the remote controller, so that the user can determine the vehicle's execution status based on the remote controller.

9. The vehicle remote control system according to any one of claims 5-8, characterized in that, The drone is also configured to: after moving to the first position, if it is determined that the vehicle is in a dormant state, send a vehicle wake-up command to the vehicle to wake up the vehicle; The vehicle is configured to switch to a working state in response to receiving a vehicle wake-up command sent by the drone.

10. A vehicle, characterized in that, include: A control system and a drive system, wherein the control system receives vehicle control commands sent by the UAV and controls the drive system based on the vehicle control commands; The drone is used to perform the vehicle remote control method as described in any one of claims 1-4; The drive system is used to drive the vehicle to move.

Images