Unmanned aerial vehicle based vehicle field of vision supplement and passage guidance method and system

Abstract

The application discloses a kind of vehicle visual field supplement and traffic guide method and system based on unmanned aerial vehicle, belong to intelligent driving auxiliary field.Method includes that user initiates guide staff mode opening instruction;When vehicle speed is less than or equal to preset safety threshold, open guide staff mode;At this time, unmanned aerial vehicle observation parameter is set and control instruction is generated to on-board controller accordingly;On-board controller controls unmanned aerial vehicle to reach target position after driving camera to carry out image acquisition, and the image collected is shown to user through man-machine interface to assist user decision-making;During the movement of vehicle, unmanned aerial vehicle and vehicle keep the relative position relationship synchronous motion;User initiates guide staff mode closing instruction or triggers guide staff mode closing condition, and triggers unmanned aerial vehicle to return.This application provides intuitive, stable, real-time third perspective visual aid for driver in narrow, complex or visual field severely limited traffic scene.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving assistance. Specifically, this invention relates to a method and system for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Existing vehicle vision assistance and traffic decision-making solutions are inherently limited by the vehicle's physical platform and static perception architecture, making it difficult to meet the driver's urgent need for global and dynamic visual information in narrow, complex, or off-road conditions.

[0003] In existing technologies, systems based on multi-camera stitched panoramic images and ultrasonic radar can provide a fused surround view of the vehicle body, but their perspective is a fixed virtual overhead view. They cannot provide a real, freely deployable aerial lateral or forward penetrating field of view, especially lacking an intuitive global perception of the upper contours of obstacles, road conditions in continuous curves, and the precise relationship between wheels and boundaries.

[0004] Remote vision assistance solutions based on fixed monitoring facilities at the field and V2X communication rely heavily on pre-deployed infrastructure and stable network coverage in specific areas. They are completely ineffective in scenarios such as wilderness areas, unstructured roads, construction areas, or temporary work sites where public facilities are lacking, and their applicability is extremely limited.

[0005] Therefore, this invention proposes a method and system for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles (UAVs). Summary of the Invention

[0006] This invention aims to overcome the shortcomings of the prior art and proposes a vehicle vision supplementation and traffic guidance method and system based on UAVs to achieve the following objectives: to provide drivers with intuitive, stable and real-time third-person visual assistance in narrow, complex or severely limited traffic scenarios.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: This invention provides a method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles (UAVs), the method comprising: Step S1: The user initiates the command to start the guide mode through the human-computer interaction interface; Step S2: The vehicle's main control unit determines whether the current vehicle speed is less than or equal to a preset safety threshold; if it is, the guide mode is activated; otherwise, the guide mode is refused to be activated. Step S3: In guide mode, the user sets the drone observation parameters through the human-computer interaction interface, including the relative position of the drone and the vehicle, as well as the camera parameters. Step S4: The vehicle-mounted main control unit generates control commands based on the observed parameters and sends them to the UAV's onboard controller; Step S5: After the airborne controller controls the UAV to reach the relative position according to the control command, it drives the camera to perform the image acquisition task. The acquired images are sent back to the vehicle main control unit and displayed to the user through the human-machine interface to assist the user's decision-making. During the vehicle's movement, the UAV and the vehicle maintain the relative position relationship and move synchronously. Step S6: The user initiates a guide mode shutdown command through the vehicle's human-machine interface; or after triggering the preset guide mode shutdown conditions, the vehicle's main control unit sends a return command to the drone.

[0008] Furthermore, in step S5, the image acquisition task includes: The camera captures real-time bird's-eye view and sends it to the onboard controller; The airborne controller processes the bird's-eye view in real time, including geometric correction, image enhancement, and overlaying of virtual guide signs; The airborne controller sends the processed video stream to the vehicle's main control unit.

[0009] Furthermore, the airborne controller sends the processed video stream to the vehicle's main control unit via a dual-mode communication architecture, which includes: High-speed direct link, i.e., direct communication between the airborne controller and the vehicle's main control unit; The relay control link is the relay communication between the airborne controller and the vehicle's main control unit via a cloud platform.

[0010] Furthermore, when the signal quality between the airborne controller and the vehicle's main control unit is greater than a preset threshold, a high-speed direct link is selected; when the signal quality between the airborne controller and the vehicle's main control unit is less than or equal to a preset threshold, a relay control link is selected.

[0011] Furthermore, during the vehicle's movement, the synchronized movement of the drone and the vehicle while maintaining the aforementioned relative positional relationship includes: The drone's onboard controller acquires its own positioning data and the vehicle's positioning data, and calculates the actual relative position between the drone and the vehicle based on this data. The airborne controller calculates the deviation between the actual relative position and the preset relative position; If the deviation is greater than a preset threshold, the airborne controller adjusts the UAV's position to move in the direction that reduces the deviation; If the deviation is less than or equal to a preset threshold, the airborne controller controls the drone to maintain its current position.

[0012] Furthermore, during the vehicle's movement, the synchronized movement of the drone and the vehicle while maintaining the aforementioned relative positional relationship includes: During the follow-up process, the drone continuously senses the surrounding environment. When it detects static or dynamic obstacles on the flight path, the onboard controller automatically triggers the preset obstacle avoidance algorithm, controls the drone to perform obstacle avoidance actions, and automatically returns to the preset relative position based on the preset path planning algorithm after the obstacle is removed, thus resuming the synchronous follow-up state.

[0013] Furthermore, during the synchronous movement of the drone and vehicle while maintaining the aforementioned relative positional relationship, the method also includes an environment adaptive strategy, including: When the ambient light level is detected to be below a preset threshold for a preset period of time, the onboard controller automatically controls the drone camera to switch to night mode and adjusts the sensitivity or enables supplemental lighting. When the panoramic image interface pops up on the human-machine interface, the vehicle's main control unit automatically displays the drone footage and the panoramic image interface simultaneously on the human-machine interface in a split-screen format, without interfering with each other. When the vehicle speed is detected to be greater than the preset safety threshold in guide mode, the vehicle's main control unit automatically controls the airborne controller to adjust the relative position of the drone to the preset safe position.

[0014] Furthermore, in step S6, the preset conditions for disabling the guide mode include: The drone's battery level is below the preset battery threshold; Communication between the drone and the vehicle's main control unit is interrupted and remains interrupted for a preset period of time; If any of the above conditions are met, the conditions for disabling the guide mode are considered met.

[0015] Furthermore, the method also includes: automatically charging the drone through the vehicle's cabin power supply interface after the drone returns to base.

[0016] This invention provides a vehicle vision supplementation and traffic guidance system based on unmanned aerial vehicles (UAVs), using the aforementioned UAV-based vehicle vision supplementation and traffic guidance method, characterized in that: the system includes: Vehicle side: vehicle main control unit, human-machine interface, vehicle speed sensor, vehicle positioning module, engine compartment power supply interface, and vehicle communication unit; the vehicle main control unit is connected to the human-machine interface, vehicle speed sensor, vehicle positioning module, engine compartment power supply interface, and vehicle communication unit respectively. The drone terminal includes an airborne controller, a camera, a drone positioning module, a drone communication unit, and a battery management system (BMS). The airborne controller is connected to the camera, the drone positioning module, the drone communication unit, and the battery management system (BMS). Among them, wireless communication is between the vehicle-mounted communication unit and the UAV communication unit.

[0017] The technical effects of this invention are as follows: This invention significantly improves vehicle safety and operational efficiency in narrow, complex, and limited-view scenarios. Its technical effects are specifically reflected in the following aspects: The dynamic aerial perspective based on the drone platform completely breaks through the physical limitations of traditional vehicle sensors fixed to the vehicle body. Through a mechanism of "one-click deployment - real-time observation - intelligent following," it provides the driver with an intuitive, stable, and real-time global bird's-eye view, particularly suitable for blind spots and weak areas of traditional perception systems, such as narrow passages, sharp bends, and off-road terrain. By having the drone pre-fly to the optimal observation point, the driver can gain advance insight into the road conditions ahead, the full view of obstacles, and the precise relationship between the wheels and the boundaries, thereby making accurate predictions and avoiding repeated advances and retreats, getting out of the vehicle to check, or even scratches and collisions caused by poor visibility, greatly saving travel time and reducing operational risks.

[0018] This invention employs a "preset guidance-closed-loop control" task logic and a "direct connection-relay" dual-mode communication data link to ensure control reliability and image transmission stability throughout the entire process, from field of view to real-time follow-up. Compared to traditional methods that rely on experience-based judgment or instructions from others, this invention significantly reduces the high demands on the pilot's spatial awareness and the psychological burden of operation. By enhancing the real-time perception and adaptive response capabilities to changes in lighting, communication interference, and dynamic obstacles, it ensures the continuous execution of auxiliary tasks in various complex environments and the safety of the UAV itself. The highly automated control throughout the entire process makes acquiring a third-party field of view as convenient as using in-vehicle functions, realizing a fundamental shift from the limited field of view of "human adapting to vehicle" to the unlimited field of view of "vehicle extending human." Attached Figure Description

[0019] Figure 1 A schematic diagram of a vehicle vision supplementation and traffic guidance system based on an unmanned aerial vehicle (UAV) is provided for an embodiment of the present invention. Figure 2 A flowchart of a vehicle vision supplementation and traffic guidance method based on unmanned aerial vehicles (UAVs) provided in an embodiment of the present invention. Detailed Implementation

[0020] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. This is to help those skilled in the art to have a more complete, accurate, and in-depth understanding of the inventive concept and technical solutions of the present invention, and to facilitate its implementation. It should be noted that the terms "first," "second," etc., used in this application are only for the convenience of describing the technical solutions and to distinguish components; the corresponding component configurations may be the same or different, and are not intended to limit the scope of this application. To make the technical solutions of the present invention clearer, the present invention will be explained and illustrated through the following embodiments.

[0021] This embodiment provides a method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles (UAVs), aiming to solve the following technical problems: 1. Address the core pain point that drivers face when navigating narrow roads, complex curves, or obstructed views, making it difficult to accurately judge the spatial relationships (such as height and gaps) of obstacles and the real-time position of the wheels due to the inherent limited field of vision of the vehicle, resulting in low traffic efficiency, hesitant operation, and the risk of scratches and collisions. 2. Enable drones to function as rapidly deployable aerial mobile sensing units, capable of autonomously flying to the optimal observation point according to traffic needs, and providing stable, high-definition, and low-latency global bird's-eye view, effectively supplementing the insufficient perception of vehicle-mounted sensors in blind spots and at medium and long distances, and transforming the traditional passive perspective from inside the vehicle into an active God's-eye view from outside the vehicle. 3. During the low-speed movement of the vehicle, ensure that the UAV can intelligently maintain a preset relative position (azimuth, altitude, distance) with the vehicle through high-precision positioning and real-time closed-loop control, and has the ability to autonomously avoid environmental obstacles and adapt to changes in lighting while following, thereby providing a continuous, reliable, and intuitive third-party dynamic view. 4. Construct an integrated and automated operation loop of "intent setting - automatic exit - real-time observation - intelligent following - safe return", which deeply integrates the UAV system with the vehicle-machine interface, vehicle dynamic information (vehicle speed, steering) and user commands, significantly reducing the driver's operational burden and psychological pressure in complex scenarios, and improving the accuracy, safety and overall experience of traffic decisions.

[0022] This embodiment provides a vehicle vision supplementation and traffic guidance method based on unmanned aerial vehicles (UAVs), which is applied to a UAV-based vehicle vision supplementation and traffic guidance system, such as... Figure 1 As shown, the system consists of two core modules: a new energy pickup truck mobile platform (vehicle-side) and an auxiliary observation drone (drone-side), specifically including: Vehicle side: vehicle main control unit, human-machine interface, vehicle speed sensor, vehicle positioning module, engine compartment power supply interface, and vehicle communication unit; the vehicle main control unit is connected to the human-machine interface, vehicle speed sensor, vehicle positioning module, engine compartment power supply interface, and vehicle communication unit respectively. The drone terminal includes an airborne controller, a camera, a drone positioning module, a drone communication unit, and a battery management system (BMS). The airborne controller is connected to the camera, the drone positioning module, the drone communication unit, and the battery management system (BMS). Among them, wireless communication is between the vehicle-mounted communication unit and the UAV communication unit.

[0023] The vehicle-mounted main control unit (VMU), as the main controller of the vehicle, is responsible for data processing and control. It is equipped with a high-performance multi-core processor, runs a real-time operating system, and possesses powerful data processing capabilities, multi-task scheduling capabilities, and communication and coordination capabilities with various vehicle subsystems. The human-machine interface (HMI) is used to enable intelligent interaction between the user and the VMU, including receiving and transmitting user-input commands and providing a visual interface. In this embodiment, a large VMU screen is used as the HMI. The vehicle speed sensor collects the vehicle's real-time speed and sends it to the VMU. The vehicle positioning module collects the vehicle's real-time location and sends it to the VMU; in this embodiment, a vehicle GPS is used. The engine compartment power supply interface is a pre-set charging interface in the dedicated engine compartment of the vehicle-mounted drone, used to charge the drone in the engine compartment in response to the control of the VMU. The vehicle-mounted communication unit is the vehicle's communication equipment, used to establish wireless communication between the vehicle and remote devices (mainly drones in this embodiment); in this embodiment, a vehicle-mounted TBOX can be used.

[0024] The airborne controller, as the main controller of the UAV, is responsible for data processing and control of the UAV. The camera is used to acquire images. In this embodiment, it is deployed on the UAV to acquire images of the vehicle and its surrounding environment and send them to the airborne controller. Specifically, a high-definition wide-angle camera can be used. This high-definition wide-angle camera adopts a large aperture and low distortion optical design, with a field of view of more than 120 degrees, which can cover a wide area around the vehicle. The UAV positioning module is used to acquire the real-time position of the UAV and send it to the airborne controller. In this embodiment, RTK-GNSS is used. The UAV communication unit is the communication device of the UAV and is used to establish wireless communication between the UAV and remote devices (mainly the vehicle-mounted system in this embodiment). The battery management system (BMS) is deployed on the UAV battery and is used to acquire battery status data such as battery power, temperature, and voltage and send them to the airborne controller.

[0025] Specifically, this embodiment provides a method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles (UAVs), such as... Figure 2 As shown, the specific steps include: Step S1: The user initiates the command to start the guide mode through the human-computer interaction interface; Step S2: The vehicle's main control unit determines whether the current vehicle speed is less than or equal to a preset safety threshold; if it is, the guide mode is activated; otherwise, the guide mode is refused to be activated. Step S3: In guide mode, the user sets the drone observation parameters through the human-computer interaction interface, including the relative position of the drone and the vehicle, as well as the camera parameters. Step S4: The vehicle-mounted main control unit generates control commands based on the observed parameters and sends them to the UAV's onboard controller; Step S5: After the airborne controller controls the UAV to reach the relative position according to the control command, it drives the camera to perform the image acquisition task. The acquired images are sent back to the vehicle main control unit and displayed to the user through the human-machine interface to assist the user's decision-making. During the vehicle's movement, the UAV and the vehicle maintain the relative position relationship and move synchronously. Step S6: The user initiates a guide mode shutdown command through the vehicle's human-machine interface; or after triggering the preset guide mode shutdown conditions, the vehicle's main control unit sends a return command to the drone.

[0026] Referring to step S1, the guide mode is an advanced feature of the vehicle's intelligent assistance system, designed specifically to handle complex traffic scenarios. When the vehicle is traveling at low speed or preparing to pass through complex road sections, the user can initiate the command to activate the guide mode through the App entry in the human-machine interface, voice commands, or desktop card shortcuts.

[0027] When a vehicle approaches narrow alleyways, rugged mountain roads, congested parking lots, construction zones, or intersections with severely obstructed visibility, the driver can initiate the guide mode activation command in one of the following three ways: First, through a dedicated App icon on the human-machine interface. After clicking the icon, the system will display a guide mode activation confirmation interface, showing the current function status, safety warnings, and the estimated drone deployment time. The command takes effect after the driver confirms. Second, through voice commands. The driver speaks a preset wake-up word and function activation command, such as "Activate guide mode" or "Activate aerial view." The voice recognition engine performs noise reduction, feature extraction, and semantic analysis on the collected voice signal. After recognizing the user's true intent, the system will also display a confirmation interface or directly execute the activation operation, depending on the system's preset security level. Third, through desktop card shortcuts. The main interface of the human-machine interface displays frequently used functions in card format. The guide mode card is presented with a prominent visual design. Users can quickly initiate commands by clicking the card, without navigating through multiple menus, greatly improving operational convenience. The design of the above three initiation methods fully considers the operational safety and convenience requirements in driving scenarios. Touch operation is suitable for fine settings when the vehicle is stationary or at extremely low speeds, voice commands are suitable for driving scenarios where both hands cannot take off the steering wheel, and desktop card shortcuts provide the most efficient operation path for high-frequency users.

[0028] Step S1, through the coexistence of multiple interaction methods, ensures that users can conveniently and safely trigger the function in different driving states, avoiding users abandoning the use of the assistance function at critical moments due to cumbersome operation or inconvenient interaction, thereby improving the usability of the system and the user experience.

[0029] Referring to step S2, after receiving the guide mode command, the vehicle's main control unit first obtains the current vehicle speed through the vehicle speed sensor and determines whether the current vehicle speed is less than or equal to a preset safety threshold. If it is lower, the guide mode is activated; otherwise, the guide mode is refused, the user is prompted "Vehicle speed too high, function unavailable," and the process is terminated. In this embodiment, the vehicle speed safety threshold is set to 10 km / h, but it can be flexibly selected according to actual needs in specific implementations.

[0030] Specifically, when the vehicle's main control unit receives a guide mode activation command from the human-machine interface, it immediately sends a data request to the vehicle speed sensor via the vehicle's CAN bus or Ethernet. The vehicle speed sensor calculates the vehicle's real-time speed and feeds it back to the main control unit. The main control unit compares the received real-time speed with a preset safety threshold, which is stored in the main control unit's non-volatile memory. The default setting is 10 km / h. This value is based on extensive real-world road testing and safety assessments, taking into account the time window required for the drone to take off, deploy, and reach the preset location, as well as the fact that even in the event of a sudden situation at this speed, the vehicle can safely brake within a short distance to avoid a collision with the drone. If the current vehicle speed is less than or equal to 10 km / h, the main control unit determines that the vehicle is in a safe and controllable low-speed state, meeting the basic safety conditions for drone deployment. It then sends a guide mode activation confirmation message to the human-machine interface and simultaneously initiates the drone communication link initialization process, preparing for the subsequent transmission of control commands. If the current vehicle speed is greater than 10km / h, the vehicle's main control unit determines that the current vehicle speed is too high. Releasing the drone in this state poses a safety risk, including the drone's potential for airflow disturbance during takeoff, the drone's inability to maintain stable tracking due to excessive relative speed, and the possibility of collisions during emergency braking. Therefore, the vehicle's main control unit refuses to activate the guide mode and clearly informs the user through visual prompts, voice broadcasts, or vibration feedback via the human-machine interface that the vehicle speed is too high and the function is unavailable. At the same time, it terminates the current guide mode activation process and waits for the user to reduce the vehicle speed before re-initiating the command.

[0031] Step S2 introduces a vehicle speed safety verification mechanism, which reflects the design principle of prioritizing functional safety. It strictly limits the deployment and operation of drones to low vehicle speeds or stationary states, fundamentally eliminating the potential safety hazards that may arise from releasing drones at high speeds, and ensuring that the guide mode operates within safe boundaries.

[0032] Referring to step S3, in guide mode, the user sets the drone observation parameters through the human-machine interface, including the relative position of the drone and the vehicle, as well as the camera parameters. Specifically, the user invokes the 3D vehicle model interactive interface on the human-machine interface. On this interface, the user sets the drone observation parameters and sends them to the vehicle's main control unit via touch swiping, including: The relative position of the drone and the vehicle includes the drone's azimuth angle, horizontal distance, and hovering height relative to the vehicle body; Camera parameters, including camera pitch angle and yaw angle.

[0033] Step S3 grants users autonomous control over the drone's observation perspective. Different complex traffic scenarios require significantly different observation perspectives. For example, when passing through narrow alleys, the drone needs to be positioned directly in front of the vehicle at a high vantage point to overlook the road conditions ahead. When passing through rugged mountain roads, the drone needs to be positioned to the side and front of the vehicle to observe the distance between the wheels and the curb. In a crowded parking lot, the drone may need to be positioned directly above the vehicle to observe the overall spatial layout. Through the 3D car model interactive interface, users can quickly set the most suitable observation parameters for the current scenario in an intuitive and visual way, without having to memorize complex parameter input rules.

[0034] Referring to step S4, the vehicle's main control unit dynamically generates a "deployment-observation" task sequence based on the received UAV observation parameters and packages it into control commands, which are then sent to the UAV's onboard controller. The task sequence is as follows: First, the UAV is controlled to automatically take off from the vehicle's dedicated cabin; then it is guided to fly to a preset relative position and hover; then the onboard high-definition wide-angle camera is adjusted to the designated direction to begin collecting real-time bird's-eye view images.

[0035] Step S4 enables the intelligent conversion from the user's high-level intent to the drone's low-level actions. Users only need to focus on high-level needs such as where to look and what angle to look at, without needing to understand the specific flight control details of the drone. The vehicle's main control unit undertakes the complex task decomposition and path planning work. This hierarchical control architecture not only lowers the user's operating threshold, but also ensures the safety and optimization of the drone's actions through centralized task planning. At the same time, the standardized encapsulation of task sequences provides a good technical foundation for subsequent functional expansion and cross-vehicle adaptation.

[0036] Referring to step S5, the airborne controller controls the UAV to reach the relative position according to the control command and then drives the camera to perform the image acquisition task; during the vehicle's movement, the UAV and the vehicle maintain the relative position relationship and move synchronously.

[0037] Specifically, after receiving the start command from the vehicle's main control unit, the drone automatically takes off from the vehicle's cabin. Following the instructions, the drone flies to a preset relative position (specified orientation, distance, and altitude) and adjusts its attitude to align the onboard high-definition wide-angle camera with the preset shooting direction. Once the drone reaches the specified relative position, the camera captures real-time bird's-eye view and sends it to the onboard controller.

[0038] Because drones are inevitably affected by airflow disturbances when hovering in the air, causing slight shaking of the fuselage, and because wide-angle lenses inherently have geometric distortion, the directly output raw images cannot meet the requirements for accurate traffic guidance. Therefore, the onboard controller processes the acquired raw images in real time. The onboard controller performs real-time processing of the bird's-eye view, including geometric correction, image enhancement, and overlaying virtual guidance markers (such as the intersection of red and green lines used to indicate wheel positions).

[0039] Specifically, the first step is geometric correction processing. The airborne controller uses the camera's intrinsic parameter matrix (including focal length, principal point coordinates, distortion coefficients, etc., which are precisely calibrated and stored in the airborne memory before leaving the factory) and the gimbal's current attitude angles (pitch, yaw, roll) to perform inverse perspective transformation and distortion correction on the original image. This maps the curved wide-angle view into a planar perspective image that conforms to human visual perception. Simultaneously, based on the UAV's actual position and attitude relative to the vehicle, it calculates the mapping relationship between each pixel in the image and its actual ground coordinates, establishing a precise correspondence between the image coordinate system and the ground world coordinate system. The second step is image enhancement processing. The airborne controller adaptively enhances the image based on current ambient lighting conditions, atmospheric visibility, and camera exposure parameters. This includes dynamic range adjustment, contrast enhancement, edge sharpening, and noise suppression, improving image clarity and recognizability to ensure the driver can clearly observe environmental details around the vehicle under different lighting conditions. Finally, the virtual guidance sign overlay process is performed. Based on the vehicle's geometric parameters (including body length, width, wheelbase, track width, and wheel radius, which are pre-stored in the vehicle's main control unit and synchronized to the airborne controller when the guide mode is activated) and the drone's real-time relative position to the vehicle, the onboard controller accurately calculates the projected position of the vehicle's wheels in the bird's-eye view and overlays virtual guidance signs onto the image. For example, a red circular sign (red-eye) is overlaid at the wheel's projected position to highlight the wheel's location, and a green line is overlaid on the vehicle's outline to clearly indicate the vehicle's boundaries. The intersection of the red-eye and green lines is used to precisely indicate the relative relationship between the wheels and curbs or obstacles. By observing these overlaid virtual guidance signs, the driver can intuitively judge the distance between the vehicle and its surroundings, thereby accurately controlling the vehicle's passage.

[0040] Subsequently, the airborne controller, using the UAV communication unit, transmits the processed video stream to the vehicle's main control unit via a dual-mode communication architecture to cope with complex and ever-changing real-world communication environments. The dual-mode communication architecture includes: High-speed direct link, i.e., direct communication between the airborne controller and the vehicle's main control unit; The relay control link is the relay communication between the airborne controller and the vehicle's main control unit via a cloud platform.

[0041] The communication mode is automatically selected based on real-time signal quality: During the operation of the guide mode, the airborne controller detects the signal quality of the high-speed direct link at a fixed period (e.g., once per second). In this example, wireless signal strength is selected as the signal quality indicator. When the wireless signal strength between the airborne controller and the vehicle's main control unit is greater than a preset threshold, it indicates that the direct link quality is good. The airborne controller selects the high-speed direct link as the main transmission channel and sends the high-definition video stream, which has been processed in real time, directly to the vehicle's main control unit at a high bit rate (e.g., 10-20Mbps) to ensure that the driver has the best real-time observation experience. At the same time, the relay control link serves as a backup channel and is only used to transmit low-bandwidth data such as heartbeat packets and key control commands to keep the link active for quick switching. When the wireless signal strength between the airborne controller and the vehicle's main control unit is less than or equal to a preset threshold, it indicates that the direct connection signal between the UAV and the vehicle has deteriorated due to obstruction, interference, or other reasons. At this time, the airborne controller immediately initiates a link switching process, switching the video stream transmission to a relay control link. Simultaneously, it automatically reduces the video encoding bitrate (e.g., to 1-3 Mbps) and resolution (e.g., from 4K to 720p) to prioritize the continuity of the video stream and the reliable transmission of critical control commands, preventing the driver from losing their aerial observation capability due to link interruption. When the wireless signal strength between the airborne controller and the vehicle's main control unit recovers to above the threshold, the airborne controller automatically switches the video stream transmission back to the high-speed direct connection link and restores high bitrate and high-resolution transmission. This mechanism, based on real-time signal quality dynamic selection of communication modes, ensures the continuous availability of the guide mode in complex electromagnetic environments such as urban canyons, mountain bends, and underground parking lot entrances and exits, significantly improving the environmental adaptability and reliability of the method in this embodiment. In specific implementations, in addition to wireless strength, signal quality indicators such as signal-to-noise ratio can also be selected as judgment criteria.

[0042] Ultimately, the vehicle's main control unit receives video data from the drone's communication unit via the vehicle's communication unit. After decoding the data, the main control unit presents a real-time bird's-eye view with virtual guide lines through the human-machine interface for the driver to observe.

[0043] Throughout the entire drone filming process, as the vehicle moves, the drone needs to maintain a synchronized relative position with it. That is, when the vehicle moves forward, turns, accelerates, or decelerates, the drone should make corresponding position adjustments in the air to maintain a constant azimuth angle, horizontal distance, and hovering height relative to the vehicle, thereby providing the driver with a stable and continuous bird's-eye view. To achieve this goal, the onboard controller executes dynamic position holding and intelligent obstacle avoidance control strategies, as follows: The UAV's onboard controller acquires its own positioning data through the UAV positioning module, while the vehicle's main control unit acquires the vehicle's positioning data through the vehicle positioning module and transmits it to the UAV's onboard controller via the vehicle communication unit. The UAV's onboard controller integrates these two sets of position data to calculate the actual relative position between the UAV and the vehicle. For example, using the vehicle's position as a reference origin, the UAV's position is transformed into a relative coordinate system with the vehicle as the origin, calculating the UAV's actual azimuth, horizontal distance, and relative altitude relative to the vehicle. Simultaneously, the onboard controller calculates the deviation between the actual relative position and a preset relative position and determines whether the deviation is within the allowable tolerance range. Correspondingly, if the deviation is greater than a preset threshold, it indicates that the UAV's current position deviates from the preset ideal observation position. The onboard controller then adjusts the UAV's position in the direction of decreasing deviation until the deviation between the actual relative position and the preset relative position is less than or equal to the preset threshold. If the deviation is less than or equal to the preset threshold, it indicates that the UAV's current position meets the observation requirements, and the onboard controller controls the UAV to maintain its current position.

[0044] While maintaining a dynamic position, the drone also needs intelligent obstacle avoidance capabilities to cope with static or dynamic obstacles that may appear on its flight path. Specifically, the drone continuously senses its surrounding environment during the follow-up process. When static or dynamic obstacles are detected on the flight path through vision or other sensors, the onboard controller automatically triggers a preset obstacle avoidance algorithm, controls the drone to perform obstacle avoidance actions, and automatically returns to the preset relative position based on a preset path planning algorithm after the obstacle is cleared, resuming synchronous following. The entire process forms a closed-loop control of "perception-computation-adjustment" to ensure a stable field of view. The entire obstacle avoidance process is autonomously decided and executed by the onboard controller without human intervention. At the same time, the onboard controller reports obstacle avoidance events (including obstacle type, location, and obstacle avoidance actions taken) to the vehicle's main control unit in real time via a communication link. The vehicle's main control unit prompts the driver through the human-machine interface, visually or verbally, that the drone is in obstacle avoidance mode, avoiding confusion caused by the drone temporarily deviating from the preset position. This combination of dynamic position keeping and intelligent obstacle avoidance control strategies ensures that the drone can stably follow the vehicle and provide a continuous observation perspective in complex real-world environments, while also autonomously avoiding collision risks, achieving a balance between safety and functionality.

[0045] Furthermore, during the synchronous movement of the drone and vehicle while maintaining the aforementioned relative positional relationship, the method of this embodiment also incorporates an environment adaptation strategy, including: Light Adaptive: When the ambient light is detected to be below the preset threshold and lasts for a preset period of time (such as when entering a tunnel or at night), the onboard controller automatically controls the drone camera to switch to night mode and adjusts the sensitivity or enables supplemental lighting to ensure clear images. This light adaptive strategy ensures the availability of the guide mode in all weather and all scenarios and does not fail due to changes in lighting conditions. Functional conflict management: When the human-machine interface automatically pops up the panoramic image interface (AVM screen) due to operations such as engaging reverse gear, the vehicle's main control unit automatically displays the drone screen and the AVM screen simultaneously on the human-machine interface in a split-screen format, without interfering with each other; this functional conflict management strategy reflects the deep integration of this application with the vehicle's original intelligent system, avoids the negative experience caused by function superposition, and realizes the collaborative work of multiple auxiliary systems. Vehicle speed-linked control: When the vehicle speed exceeds a preset safety threshold (e.g., 10 km / h) in guide mode, the vehicle's main control unit automatically controls the onboard controller to adjust the drone's relative position to a preset safe position. This safe position is typically set at a certain height directly above the vehicle (e.g., 10 meters). At this position, the horizontal distance between the drone and the vehicle is minimized, and the relative speed is lowest. Even if the vehicle continues to accelerate, the drone can most easily maintain its following position. This height also avoids most ground obstacles, reducing the risk of collision. Simultaneously with adjusting to the safe position, the vehicle's main control unit issues a warning to the driver through a prominent visual prompt and voice announcement via the human-machine interface, reminding the driver to reduce speed. Once the vehicle speed falls below the safety threshold and remains below it for a period of time, the vehicle's main control unit sends a command to the onboard controller to return the drone from the safe position to the user-preset observation relative position, resuming normal observation tasks. This vehicle speed-linked control strategy provides dynamic safety boundary protection for guide mode. Even if the driver increases the vehicle speed without actively disabling the function during mode operation, automatic protective measures can be taken to avoid safety hazards caused by the drone following at high speeds, further improving the safety of both the drone and the vehicle.

[0046] Referring to step S6, the vehicle's main control unit will only send a return-to-home command to the drone after the user initiates a guide mode shutdown command through the vehicle's human-machine interface or after the preset guide mode shutdown conditions are triggered.

[0047] The conditions for disabling the guide mode include: Condition 1: The drone's battery level is below a preset battery threshold (e.g., 20%). Condition 2: Communication between the drone and the vehicle's main control unit is interrupted and lasts for a preset period of time; If any of the above conditions are met, the conditions for closing the guide mode are considered met. The drone will then return to home and land in the cabin, while the user is notified to exit the guide mode.

[0048] In this embodiment, for condition 1, when the drone's battery level is detected to be below 20%, it is determined that the drone's remaining power is insufficient to support continued mission execution and safe return. A low battery alarm is immediately sent to the vehicle's main control unit via the communication link, and the return-to-home process is automatically triggered without waiting for confirmation from the vehicle's main control unit, ensuring the drone can safely return to the cabin before its battery is depleted. Upon receiving the low battery alarm, the vehicle's main control unit alerts the driver to the low drone battery level through the human-machine interface, guiding the driver's attention. For condition 2, the vehicle's main control unit and the airborne controller maintain connection monitoring by periodically sending heartbeat packets. If either party fails to receive a heartbeat packet or data packet from the other within a preset time window (e.g., 5 seconds), the communication link is determined to be interrupted, and the return-to-home process is automatically triggered. This two-way communication interruption detection and autonomous return-to-home mechanism avoids the risk of the drone losing connection due to a single node failure.

[0049] When the guide mode is off, the drone automatically flies back to its position in the vehicle's roof compartment after receiving the return command, and completes a precise landing and lock-on using sophisticated guidance devices (such as UWB or visual guidance). After returning to its position, the drone's charging port automatically connects to the power supply interface in the pickup truck's mobile platform, automatically charging the drone through the vehicle's power supply interface. The vehicle's main control unit intelligently selects fast charging, standard charging, or maintenance mode based on the drone's battery level, the vehicle's energy status, and subsequent mission plans, safely and efficiently replenishing the drone's battery through the V2L interface, preparing it for the next mission.

[0050] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution; or the direct application of the inventive concept and technical solution to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles (UAVs), characterized in that: The method includes: Step S1: The user initiates the command to start the guide mode through the human-computer interaction interface; Step S2: The vehicle's main control unit determines whether the current vehicle speed is less than or equal to a preset safety threshold; if it is, the guide mode is activated; otherwise, the guide mode is refused to be activated. Step S3: In guide mode, the user sets the drone observation parameters through the human-computer interaction interface, including the relative position of the drone and the vehicle, as well as the camera parameters. Step S4: The vehicle-mounted main control unit generates control commands based on the observed parameters and sends them to the UAV's onboard controller; Step S5: After the airborne controller controls the UAV to reach the relative position according to the control command, it drives the camera to perform the image acquisition task. The acquired images are sent back to the vehicle main control unit and displayed to the user through the human-machine interface to assist the user's decision-making. During the vehicle's movement, the UAV and the vehicle maintain the relative position relationship and move synchronously. Step S6: The user initiates a guide mode shutdown command through the vehicle's human-machine interface; or after triggering the preset guide mode shutdown conditions, the vehicle's main control unit sends a return command to the drone.

2. The method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 1, characterized in that: In step S5, the image acquisition task includes: The camera captures real-time bird's-eye view and sends it to the onboard controller; The airborne controller processes the bird's-eye view in real time, including geometric correction, image enhancement, and overlaying of virtual guide signs; The airborne controller sends the processed video stream to the vehicle's main control unit.

3. The method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 2, characterized in that: The airborne controller sends the processed video stream to the vehicle's main control unit via a dual-mode communication architecture, which includes: High-speed direct link, i.e., direct communication between the airborne controller and the vehicle's main control unit; The relay control link is the relay communication between the airborne controller and the vehicle's main control unit via a cloud platform.

4. The method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 3, characterized in that: When the signal quality between the airborne controller and the vehicle's main control unit is greater than a preset threshold, a high-speed direct link is selected; when the signal quality between the airborne controller and the vehicle's main control unit is less than or equal to a preset threshold, a relay control link is selected.

5. A method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 1, characterized in that: During the vehicle's movement, the drone and the vehicle maintain the aforementioned relative positional relationship and move synchronously, including: The drone's onboard controller acquires its own positioning data and the vehicle's positioning data, and calculates the actual relative position between the drone and the vehicle based on this data. The airborne controller calculates the deviation between the actual relative position and the preset relative position; If the deviation is greater than a preset threshold, the airborne controller adjusts the UAV's position to move in the direction that reduces the deviation; If the deviation is less than or equal to a preset threshold, the airborne controller controls the drone to maintain its current position.

6. A method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 1, characterized in that: During the vehicle's movement, the drone and the vehicle maintain the aforementioned relative positional relationship and move synchronously, including: During the follow-up process, the drone continuously senses the surrounding environment. When it detects static or dynamic obstacles on the flight path, the onboard controller automatically triggers the preset obstacle avoidance algorithm, controls the drone to perform obstacle avoidance actions, and automatically returns to the preset relative position based on the preset path planning algorithm after the obstacle is removed, thus resuming the synchronous follow-up state.

7. A method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 1, characterized in that: During the synchronous movement of the drone and vehicle while maintaining the aforementioned relative positional relationship, the method further includes an environment adaptive strategy, comprising: When the ambient light level is detected to be below a preset threshold for a preset period of time, the onboard controller automatically controls the drone camera to switch to night mode and adjusts the sensitivity or enables supplemental lighting. When the panoramic image interface pops up on the human-machine interface, the vehicle's main control unit automatically displays the drone footage and the panoramic image interface simultaneously on the human-machine interface in a split-screen format, without interfering with each other. When the vehicle speed is detected to be greater than the preset safety threshold in guide mode, the vehicle's main control unit automatically controls the airborne controller to adjust the relative position of the drone to the preset safe position.

8. A method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 1, characterized in that: In step S6, the preset conditions for disabling the guide mode include: The drone's battery level is below the preset battery threshold; Communication between the drone and the vehicle's main control unit is interrupted and remains interrupted for a preset period of time; If any of the above conditions are met, the conditions for disabling the guide mode are considered met.

9. A method for supplementing vehicle vision and guiding traffic based on unmanned aerial vehicles according to claim 1, characterized in that: The method also includes: after the drone returns to base, it is automatically charged through the vehicle's cabin power supply interface.

10. A vehicle vision supplementation and traffic guidance system based on unmanned aerial vehicles (UAVs), using a vehicle vision supplementation and traffic guidance method based on UAVs according to any one of claims 1-9, characterized in that: The system includes: Vehicle side: vehicle main control unit, human-machine interface, vehicle speed sensor, vehicle positioning module, engine compartment power supply interface, and vehicle communication unit; the vehicle main control unit is connected to the human-machine interface, vehicle speed sensor, vehicle positioning module, engine compartment power supply interface, and vehicle communication unit respectively. The drone terminal includes an airborne controller, a camera, a drone positioning module, a drone communication unit, and a battery management system (BMS). The airborne controller is connected to the camera, the drone positioning module, the drone communication unit, and the battery management system (BMS). Among them, wireless communication is between the vehicle-mounted communication unit and the UAV communication unit.

Images