An intelligent interaction system and method for a vehicle-mounted unmanned aerial vehicle, a storage medium and a computer program product

Abstract

This invention proposes a vehicle-mounted unmanned aerial vehicle (UAV) intelligent interaction system, method, storage medium, and computer program product. The method includes performing hangar self-checks, UAV self-checks, and environmental assessments to generate a comprehensive self-check report. Based on the comprehensive self-check report, abnormal information is displayed on the vehicle-mounted central control unit (CMU). The method also parses user input commands via the CMU; if the parsed command is a takeoff command and the comprehensive self-check report contains no critical abnormalities, a hangar opening command is generated. Upon receiving the hangar opening command, the system controls the hangar to open and displays a safety confirmation interface containing abnormal information from the comprehensive self-check report on the CMU. Upon receiving a user-issued safety confirmation command, a takeoff permission command is generated. This invention integrates multi-layered self-checks and abnormal handling, achieving mandatory pre-takeoff safety confirmation. Combined with automatic landing failure assistance, it improves the safety and efficiency of vehicle-mounted UAV operation.

Description

Technical Field

[0001] This invention belongs to the field of vehicle-mounted drone interaction technology, specifically relating to a vehicle-mounted drone intelligent interaction system, method, storage medium, and computer program product. Background Technology

[0002] With the popularization of drone technology, its application in vehicle-mounted scenarios is becoming increasingly widespread, such as aerial photography, inspection, and emergency response. However, existing vehicle-mounted drone systems typically suffer from problems such as complex interaction processes, insufficient security, and inconvenient user operation. In particular, the lack of systematic and intelligent interactive support in areas such as pre-flight self-checks, in-flight status monitoring, anomaly handling, and landing guidance leads to poor user experience and operational risks. In general, the following problems exist: First, the interactive interfaces are fragmented, meaning users typically need to switch between multiple devices, including the drone remote controller, a mobile app, and the vehicle's central control unit. For example, pre-flight self-checks require checking the drone's status via the remote controller, takeoff commands are sent via the mobile app, flight monitoring requires observing the remote controller screen, and landing requires switching back to the mobile app. This multi-device, multi-interface operation method leads to user distraction, low operational efficiency, and a risk of missing crucial information due to untimely device switching. Secondly, there is a lack of safety mechanisms. Existing systems rely solely on visual inspection of the environment by the user before takeoff, lacking a systematic environmental perception and risk assessment mechanism. When the drone takes off from the vehicle-mounted hangar, key safety factors such as whether there are obstacles around the hangar, whether the wind speed exceeds the safety threshold, and whether the drone's battery is sufficiently charged depend entirely on the user's subjective judgment. Once the user is negligent or makes a misjudgment, safety accidents such as takeoff collisions and crashes are very likely to occur. Thirdly, the anomaly handling is simplistic. When the system detects an anomaly, existing technologies usually only display simple error codes or prompts on the drone remote controller or mobile app, lacking hierarchical prompts and intelligent guidance. Users are often at a loss when faced with unfamiliar error codes, needing to consult the manual or contact technical support, resulting in low anomaly handling efficiency and delays in optimal handling. Fourthly, the level of automation is low. In the entire process from drone startup to recovery, most operations need to be performed manually by the user. For example, the hangar door needs to be opened manually, the drone needs to be placed at the takeoff point manually, flight parameters need to be adjusted manually after takeoff, and the drone needs to be manually returned to the hangar and connected to the charger after landing. This highly manual operation method contradicts the technological development trend of vehicle-mounted intelligence. Summary of the Invention

[0003] To address the current problem of low intelligence levels in vehicle-mounted drones, which prevents them from achieving safe, convenient, and automated operation in a vehicle environment, this invention proposes a vehicle-mounted drone intelligent interaction system, method, storage medium, and computer program product.

[0004] A vehicle-mounted unmanned aerial vehicle (UAV) intelligent interaction system, which achieves one of the objectives of this invention, includes: The self-test module performs hangar self-tests, drone self-tests, and environmental assessments, generating a comprehensive self-test report. Based on this report, it displays abnormal information on the vehicle-mounted central control unit. The hangar serves as a storage, charging, and takeoff / landing platform for drones. Specifically, the hangar includes a cabin for accommodating drones, an openable and closable door, a lifting platform for carrying the drones, and a charging interface for charging the drones. When a drone lands, the lifting platform rises to the top of the hangar to receive it, then descends to store the drone inside the cabin. When a drone takes off, the lifting platform rises to lift the drone to the top of the hangar, and the drone can take off after the door opens. The takeoff / landing platform support refers to the lifting platform providing a stable bearing surface for the drone during takeoff and landing. The command interaction module is used to parse the commands input by the user through the vehicle central control unit. If the command is parsed as a takeoff command and there are no key abnormalities in the comprehensive self-test report, a hangar opening command is generated. The parsing of user-inputted commands via the vehicle's central control unit specifically refers to the following: the command interaction module receives command signals generated by user actions such as clicking, swiping, or voice input on the vehicle's central control unit's touchscreen, performs semantic parsing and format conversion on the command signals, identifies the command type (such as takeoff, landing, taking a picture, returning to home, etc.), and extracts parameter information carried in the command (such as takeoff altitude, flight mode, etc.). The takeoff commands refer to various commands issued by the user that contain takeoff intentions, including but not limited to one-click takeoff commands using system default parameters, preset parameter takeoff commands that call user-saved parameter combinations, and custom parameter takeoff commands where the user temporarily sets parameters such as takeoff altitude.

[0005] The comprehensive self-inspection report specifically refers to a unified report generated by summarizing the test results from hangar self-inspection, UAV self-inspection, and environmental assessment. The report includes the name of each test item, the corresponding test data, the comparison results with the preset hangar status standard values, and anomaly indicators. Anomaly indicators are used to mark test items whose test data do not meet the standard requirements, so as to facilitate the subsequent anomaly handling unit to perform hierarchical processing.

[0006] The pre-flight interaction module is used to control the hangar to perform the opening action when a hangar opening command is received, and to pop up a safety confirmation interface containing the abnormal information in the comprehensive self-test report on the vehicle-mounted central control unit. When a user confirms the safety command issued through the safety confirmation interface is received, a takeoff permission command is generated.

[0007] Furthermore, the self-testing module includes one or more of the following: hangar self-testing subunit, UAV self-testing subunit, and environmental assessment subunit; The hangar self-inspection subunit is used to acquire hangar door opening / closing status data, lifting platform position status data, and charging interface connection status data through a sensor group installed on the hangar. It generates a hangar status data package, compares each detection data point in the data package with preset hangar status standard values, and adds an anomaly flag if they do not match. The data package with the anomaly flag is then merged into the comprehensive self-inspection report. The preset hangar status standard values ​​refer to reference values ​​when each component of the hangar is in normal working condition. For example: the standard value for a fully open door is the signal value output by a Hall sensor or microswitch when the door is fully open; the standard value for a fully closed door is the signal value output by a Hall sensor or microswitch when the door is fully closed; the standard value for a raised lifting platform is the position value output by a displacement sensor when the lifting platform reaches its highest point; the standard value for a lowered lifting platform is the position value output by a displacement sensor when the lifting platform reaches its lowest point; and the standard value for a charging interface connection is the conduction signal value of the contact switch when the charging interface is normally connected to the drone.

[0008] The above standard values ​​are preset at the factory and cannot be modified by the user; or they can be automatically calibrated during installation through a calibration procedure.

[0009] The UAV self-test subunit is used to obtain the UAV's power status data, motor status data, propeller status data and flight control system status data by establishing a communication link with the UAV, generate a UAV status data packet, compare each detection data in the UAV status data packet with a preset UAV status threshold, add an abnormality mark if a certain detection data exceeds the corresponding preset threshold, and merge the data packet with the abnormality mark into the comprehensive self-test report. The environmental assessment subunit is used to acquire wind speed data, rainfall data, and obstacle distance data around the hangar through environmental perception sensors deployed on the vehicle, generate an environmental data package, compare the detection data of each detection item in the environmental data package with a preset environmental safety threshold, add an anomaly mark if a certain detection data exceeds the corresponding preset threshold, and merge the data package with the anomaly mark into the comprehensive self-inspection report.

[0010] Furthermore, the self-test module also includes an anomaly handling unit, used to process test items with anomaly markers in the comprehensive self-test report. The processing method includes: Determine whether each detection item with an anomaly label belongs to a critical anomaly item. The critical anomaly items include anomalies that prevent the drone from taking off safely or affect the drone's flight. If a critical anomaly exists and the system is in the pre-flight self-check phase: an anomaly indicator will be displayed as a pop-up on the vehicle-mounted central control unit. The pre-flight self-check phase refers to the stage where the user has started the system and entered the self-check interface through the vehicle-mounted central control unit, but has not yet issued a takeoff command. At this time, the self-check module is performing or has completed the hangar self-check, UAV self-check, and environmental assessment. The user can view the comprehensive self-check report and anomaly information through the vehicle-mounted central control unit and prepare to issue a takeoff command. During this phase, the user is in an active operating state and needs to fully understand the system status in order to make a takeoff decision; therefore, a strong prompt strategy is adopted for anomaly handling.

[0011] Furthermore, if a critical anomaly exists and the system is in standby or driving mode, an anomaly indicator will be displayed in the status bar area of ​​the vehicle's central control unit by changing the icon color. The standby or driving mode refers to the period when the user has not entered the self-test interface, or when the vehicle is in motion and the drone is not in flight. At this time, the user may be driving or performing other tasks, and the pop-up window would interfere with the user's current operation and even affect driving safety. Therefore, anomaly handling adopts a weak prompt strategy, only displaying the anomaly indicator in the status bar area by changing the icon color. The user can click to view details at their convenience.

[0012] If there are abnormal items but no critical abnormal items: an abnormality indicator will be displayed in the status bar area of ​​the vehicle central control unit by changing the icon color.

[0013] Key anomalies are those that prevent the drone from taking off safely or affect its flight, including but not limited to: abnormal motor status (e.g., the motor speed feedback deviates from the target speed by more than the set range (e.g., ±10%)), abnormal propeller integrity (self-test feedback indicates damage or missing parts), abnormal hangar door status (the hangar door cannot be opened properly), abnormal charging interface (the charging interface is not properly connected to the drone and the battery level is below the takeoff threshold of 15%), abnormal flight control system (abnormal inertial measurement unit calibration status or heartbeat packet loss more than the set number of times), abnormal battery level (battery level is below the set percentage of 15%), and abnormal environment (wind speed exceeds the set wind speed value of 10 m / s or there are obstacles less than 1 meter away from the hangar).

[0014] When the system detects a critical anomaly during flight, the anomaly handling unit adopts different handling strategies based on the specific type and severity of the anomaly, including: When the system detects that the drone's battery level is lower than the first preset percentage, it will immediately display a return-to-home reminder. If the user does not respond within the set time, the system will automatically control the drone to return to home. If the battery level is lower than the second preset percentage, the system will force the drone to return to home immediately without waiting for user confirmation. The second preset percentage is lower than the first preset percentage.

[0015] When the system detects that the communication signal strength is continuously lower than the set value for more than the first set time, it displays a weak communication signal warning message on the vehicle central control unit. If the signal is completely interrupted for more than the second set time, the uncontrolled return-to-home mechanism is activated, and the drone automatically returns to the signal recovery area. If the signal is interrupted for more than the third set time, the drone automatically executes the preset emergency landing procedure, where the third set time > the first set time > the second set time.

[0016] When the system detects a sudden increase in wind speed exceeding the set wind speed, it will immediately display a return-to-home reminder and suggest that the user land the drone in a low-wind area. If the drone is performing an important mission such as emergency rescue, the user can ignore the reminder and continue flying, but the system will continue to monitor wind speed changes and update the reminder in real time.

[0017] When abnormal motor speed or propeller integrity is detected, the system immediately forces the drone to return to home and does not allow the user to cancel the return, because such abnormalities directly threaten flight safety and continuing to fly may lead to a crash.

[0018] Furthermore, the anomaly handling unit is also used to respond to the user's click operation on the anomaly icon. When the user clicks the anomaly icon, the complete anomaly information is displayed on the vehicle central control unit and the user is automatically redirected to the processing guidance page. After the anomaly is eliminated, the anomaly icon is made to disappear.

[0019] Furthermore, when the command interaction module parses the commands input by the user through the vehicle central control unit, it includes: When the user command is a takeoff command, the parameter identifier carried in the command is parsed, and the parameter source type is determined based on the parameter identifier. The parameter source type includes fixed parameters, preset parameters, and custom parameters. The corresponding takeoff parameters are obtained according to the parameter source type, and the takeoff parameters are passed to the pre-takeoff interaction module so that the pre-takeoff interaction module can control the UAV to take off according to the takeoff parameters after generating a takeoff permission command.

[0020] Furthermore, before controlling the hangar to perform the opening action, the pre-takeoff interaction module also includes: The system determines whether the drone is inside the hangar. If the drone is inside the hangar, the vehicle-mounted central control unit displays a reminder message that the hangar is opening. After the hangar opens, the vehicle-mounted central control unit displays a reminder message that the helipad has been raised. If the drone is not inside the hangar, the vehicle-mounted central control unit displays a reminder message requesting manual confirmation of the safety of the drone's surroundings, and a safety confirmation interface is displayed.

[0021] Furthermore, the safety confirmation interface includes a first button for confirming safety and a second button for canceling takeoff. Only after the user clicks the first button will the pre-takeoff interaction module generate the takeoff permission command.

[0022] Furthermore, in the hangar self-test subunit: The hatch opening and closing status data are obtained through Hall sensors or micro switches. If the hatch is detected to have not reached the preset open or closed position, it is determined to be abnormal. The position status data of the lifting platform is obtained through a displacement sensor. If the lifting platform is detected to have not reached the preset raised or lowered position, it is determined to be abnormal. The charging interface connection status data is obtained through a contact switch. If it is detected that the charging interface is not properly connected to the drone, it is determined to be abnormal.

[0023] Low-wind-speed areas refer to regions where wind speeds are below a preset safe landing threshold. This preset safe landing threshold is pre-set based on the drone model and flight performance, and is typically 60%-70% of the drone's maximum wind resistance. For example, if the drone's maximum wind resistance is 12 m / s, the safe landing threshold is 8 m / s. Low-wind-speed areas are defined using one or more of the following methods: (1) Based on the historical wind speed records on the vehicle's driving path, identify the visited location points where the wind speed is lower than the safety threshold, and form a circular area with the point as the center and the radius as the preset value. (2) Based on the micro-topography analysis around the vehicle, the wind speed at different locations is estimated by computational fluid dynamics model, and areas with wind speeds below the safety threshold are identified; (3) By using UAVs to sample and detect in real time at multiple points, the wind speed field around the vehicle is reconstructed, and areas with wind speeds below the safety threshold are identified.

[0024] Furthermore, when the UAV self-test subunit obtains the UAV status data packet by establishing a communication link with the UAV, it is specifically used to: establish a communication link with the UAV through the MAVLink micro UAV communication protocol or a private wireless communication protocol, parse and extract the UAV's battery percentage data, motor speed feedback data, propeller integrity self-test data, inertial measurement unit calibration status data, and flight control system heartbeat packet data from the communication link, and summarize the above data into the UAV status data packet.

[0025] Furthermore, in the environmental assessment subunit: The wind speed data is acquired through a wind speed sensor; the rainfall data is acquired through a rain gauge sensor or a rain detection sensor mounted on the drone itself; and the obstacle distance data is acquired through a surround-view camera or ultrasonic radar.

[0026] Furthermore, the vehicle-mounted drone intelligent interaction system also includes a flight interaction module, used to receive flight status parameters sent by the drone in real time via a wireless communication link during drone flight, display these parameters on the vehicle-mounted central control unit, and provide a flight control function interface. Specifically, the system monitors the communication link status through a heartbeat packet mechanism, receives status data frames broadcast by the drone at a preset frequency, and parses altitude data, horizontal distance data, remaining battery power data, and communication signal strength data from these status data frames. This data is then forwarded to the vehicle-mounted central control unit, where it is displayed numerically on the same interface. Function buttons for taking photos, recording videos, and gimbal control are generated on the vehicle-mounted central control unit, and corresponding control commands are sent to the drone in response to user clicks on these function buttons.

[0027] Furthermore, when the automatic landing unit calculates the relative positional deviation between the UAV and the hangar based on the collected guidance signals, it is specifically used for: The visual guidance target is a QR code pattern or an infrared LED dot matrix. The drone's downward-facing camera captures images of the visually guided target, and an image processing algorithm identifies the position coordinates and size of the visually guided target in the image. The location coordinates and size are converted into a three-dimensional relative positional deviation between the UAV and the hangar by combining the camera intrinsic parameters; The flight control system generates attitude adjustment commands based on the three-dimensional relative position deviation, guiding the UAV to land in the hangar.

[0028] Furthermore, after obtaining the reason for failure, the failure handling unit also includes: When the drone's downward-facing camera fails to identify the visual guidance target within a preset time, it generates positioning signal loss reason data. When the hangar door is blocked by an obstacle, preventing the drone from landing, data on the reason for the obstruction of the landing platform is generated.

[0029] Furthermore, after activating the virtual joystick interface on the vehicle central control unit and receiving manual control commands input by the user through the virtual joystick interface, the system further includes: A virtual joystick interaction interface containing a first touch area and a second touch area is displayed on the vehicle central control unit. The first touch area is used to simulate a flight joystick, and the second touch area is used to simulate a gimbal joystick. Detect the user's finger touch and slide operation in the first touch area or the second touch area, obtain the starting coordinates and current coordinates of the touch point, and calculate the direction and length of the slide vector; The type of control command is determined based on the direction of the sliding vector, and the strength of the control command is determined based on the length of the sliding vector, thereby generating UAV flight control commands or gimbal control commands. The control commands are sent to the drone to manually control its landing.

[0030] Furthermore, it also includes a post-landing interaction module for responding to return-to-home commands, controlling the drone to return to home, and executing post-landing procedures including automatic landing and / or manual guidance.

[0031] Furthermore, the post-landing interaction module includes an automatic landing unit, which controls a visual guidance target set on the hangar to emit a guidance signal, collects the guidance signal through the UAV's downward-facing camera, calculates the relative position deviation between the UAV and the hangar based on the collected guidance signal, and adjusts the UAV's flight attitude based on the relative position deviation to guide the UAV to land in the hangar.

[0032] Furthermore, the post-landing interaction module also includes a return-to-home trigger unit, which monitors the communication link status between the integrated control module and the UAV, and generates the return-to-home command when the communication link status is interrupted and continues for more than a preset time threshold.

[0033] Furthermore, the post-landing interaction module also includes a failure processing unit, which is used to obtain the reason for failure when the automatic landing unit fails to guide the system, and to display the reason for failure on the vehicle central control unit.

[0034] Furthermore, the failure handling unit is also used to acquire map data of the vehicle's surrounding environment and acquire real-time images of the hangar's surroundings through surround-view cameras deployed on the vehicle; identify flat, unobstructed areas based on the environmental map data and the real-time images, and determine the identified flat, unobstructed areas as alternative landing points; and display the location of the alternative landing points on the vehicle's central control unit.

[0035] Furthermore, the failure handling unit is also used to activate a virtual joystick interaction interface on the vehicle-mounted central control unit, receive manual control commands input by the user through the virtual joystick interaction interface, and control the drone to land according to the manual control commands.

[0036] Furthermore, when the automatic landing unit fails to guide the system, the failure handling unit executes the following multi-level processing strategy: If the failure is due to a brief loss of signal, such as when the drone flies over the hangar causing the target to briefly disappear from view, the system will automatically restart the automatic landing program and try again multiple times, such as 3 times. If multiple landing attempts fail, the system initiates an alternative landing point identification process, using panoramic camera images and environmental map data to identify flat, unobstructed areas around the hangar. The system then displays alternative landing points sorted by distance from the hangar, allowing the user to select one for the drone to land. If the user does not select a landing point, the system automatically chooses the nearest available alternative and performs the landing. If the user is not satisfied with the landing effect at the alternative landing points, or if the user wishes to control the drone landing personally, they can click the manual control button. The system will then activate the virtual joystick interface, handing control to the user. The user can manually control the drone to land at the designated location using the virtual joystick on the touchscreen. If the drone is unable to land and its battery level is below 5% of the set level, the system will initiate an emergency landing procedure: an emergency warning will be displayed on the vehicle's central control unit indicating that the battery is about to run out and requesting that the drone immediately find a safe landing area. The emergency landing mode will be activated, and the drone will automatically find the nearest flat ground to land.

[0037] A second objective of this invention is a vehicle-mounted unmanned aerial vehicle (UAV) intelligent interaction method, comprising: Perform hangar self-inspection, drone self-inspection and environmental assessment, generate comprehensive self-inspection report, and display abnormal information on the vehicle-mounted central control unit based on the comprehensive self-inspection report, wherein the hangar is used for drone storage, charging and take-off and landing platform support; The system parses the commands input by the user through the vehicle central control unit. If the command is a takeoff command and the comprehensive self-test report has no critical anomalies, a hangar opening command is generated. When a hangar opening command is received, the hangar is controlled to perform the opening action, and a safety confirmation interface containing the abnormal information in the comprehensive self-test report pops up on the vehicle central control unit. When a safety confirmation command is received from the user through the safety confirmation interface, a takeoff permission command is generated.

[0038] A non-transitory computer-readable storage medium for achieving the third objective of the present invention, wherein a computer program is stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the intelligent interaction method for the vehicle-mounted unmanned aerial vehicle.

[0039] A computer program product for achieving the fourth objective of the present invention includes a computer program / instruction, which, when executed by a processor, implements the steps of the vehicle-mounted drone intelligent interaction method.

[0040] The beneficial effects of this invention include: (1) This invention achieves comprehensive automatic detection of the pre-flight status by integrating hangar self-inspection, UAV self-inspection and environmental assessment through a multi-layer self-inspection mechanism, eliminating the subjectivity and uncertainty of human judgment; through the mandatory safety confirmation interface, it ensures that users must actively confirm the safety of the surrounding environment before each takeoff, thus eliminating safety accidents caused by negligence in the process; through the graded processing of key anomalies, it can monitor safety risks in real time during flight and automatically take protective measures such as returning to base when risks occur, thereby maximizing flight safety.

[0041] (2) By deeply integrating the drone control process into the vehicle central control system, users do not need to switch between multiple devices. All interactions can be completed on the vehicle central control screen. Through automated sequence control, the entire process is completed automatically, from opening the hangar, raising and lowering the lifting platform to the hangar recovery after returning to base. Users only need to confirm at key nodes, which greatly reduces manual operation steps. (3) The system integrates flight status monitoring and function control on the same screen through a one-stop interactive interface. Users can view key parameters such as the drone's altitude, distance, battery level, and signal in real time, and can quickly perform operations such as taking photos, recording videos, and gimbal control without switching between the monitoring interface and the control interface. Through non-intrusive labels and hierarchical prompts in pop-ups, the system can avoid information overload without missing key information, and users will not be distracted by frequent pop-ups during flight. (4) Through a three-level parameter system of fixed parameters, preset parameters and custom parameters, it can meet the quick start-up needs of novice users as well as the fine control needs of professional users; through the dual-mode design of visual guided landing and manual guided landing, it can achieve automated and precise landing, and also provide alternative solutions when automatic landing fails, ensuring that the drone can be safely recovered under any circumstances. (5) By reusing existing environmental perception sensors in the vehicle, such as surround-view cameras, rain sensors, and radar, there is no need to deploy a large number of dedicated sensors, which reduces the system hardware cost. Through modular software architecture design, it can be flexibly adapted to different models of UAVs and different configurations of vehicles, and has broad market application prospects. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the system architecture described in this invention; Figure 2 This is a schematic diagram of the self-test interaction process described in this invention. Detailed Implementation

[0043] The following detailed embodiments are provided to explain the technical solutions of the present invention, so that those skilled in the art can understand the present invention. The scope of protection of the present invention is not limited to the following specific embodiments. Any modifications or improvements made by those skilled in the art that incorporate the technical solutions of the present invention but differ from the following detailed embodiments are also within the scope of protection of the present invention.

[0044] Example 1 A vehicle-mounted unmanned aerial vehicle intelligent interaction system The vehicle-mounted drone intelligent interaction system of this invention includes: a vehicle-mounted central control unit, a hangar, a drone, and an integrated control module. The vehicle-mounted central control unit provides a user interface and displays information; the hangar is located on the vehicle (e.g., on the roof or in the trunk) and serves as a storage, charging, and take-off / landing platform for the drone; the drone performs flight missions; and the integrated control module is communicatively connected to the vehicle-mounted central control unit, the drone, and the hangar (e.g., via CAN bus, Ethernet, or a dedicated wireless communication module) to receive and process signals from all parties and issue control commands.

[0045] The integrated control module is functionally divided into a self-test module, a command interaction module, a pre-takeoff interaction module, an in-flight interaction module, and a post-landing interaction module, such as... Figure 1 As shown, the modules work together to achieve intelligent interaction throughout the entire process.

[0046] The self-test module is used to perform hangar self-test, drone self-test and environmental assessment, generate a comprehensive self-test report, and display abnormal information on the vehicle-mounted central control unit based on the comprehensive self-test report. The hangar is set on the vehicle and is used for drone storage, charging and take-off and landing platform support. The self-test module includes one or more of the following: hangar self-test sub-unit, UAV self-test sub-unit, and environmental assessment sub-unit; In one embodiment, the hangar self-inspection subunit is used to acquire hangar door opening and closing status data, lifting platform position status data and charging interface connection status data through a sensor group installed on the hangar, generate a hangar status data packet, compare each detection data in the data packet with a preset hangar status standard value, add an abnormality mark if they do not match, and merge the data packet with the abnormality mark into the comprehensive self-inspection report. Specifically, the Hall sensor determines whether the hatch has reached a predetermined position by detecting changes in magnetic flux between a magnet fixed to the hatch and the sensor: when the hatch moves to the open position, the magnet approaches the Hall sensor, the magnetic flux reaches a preset threshold, and the sensor outputs a high-level signal; when the hatch is closed, the magnet moves away, the magnetic flux falls below the preset threshold, and the sensor outputs a low-level signal. The microswitch determines whether the hatch is closed by the opening and closing of its mechanical contacts: when the hatch is closed, pressing the microswitch closes its contacts and outputs a closed signal; when the hatch is open, the microswitch contacts open and output an open signal.

[0047] Displacement sensors can be either resistive displacement sensors or optical encoders. Resistive displacement sensors determine the vertical position of the lifting platform by detecting changes in resistance caused by changes in the position of a sliding contact on a resistive element; optical encoders, on the other hand, read the scale on an optical encoder through photoelectric detection, output pulse signals, and determine the position of the lifting platform by counting the pulses.

[0048] The contact switch is located at the charging interface. When the drone lands in the hangar and docks with the charging interface, the charging terminal on the drone presses the contact switch to make it conductive. The system detects the conductive signal and determines that the charging interface is connected normally. Conversely, if the drone is not docked correctly, the contact switch is in the open state, and the system determines that the charging interface is not connected.

[0049] In one embodiment, the sensor group of the hangar self-test subunit can adopt different configuration schemes. For example, the door opening / closing status detection uses an infrared beam sensor instead of a Hall sensor or microswitch. That is, an infrared transmitter and receiver are respectively set at the door's open and closed positions. When the door moves to the corresponding position, it blocks the infrared beam, triggering a signal change. The lifting platform position detection uses a laser rangefinder instead of a displacement sensor. That is, the laser rangefinder is fixed to the top of the hangar and emits a laser beam downwards to measure the distance to the lifting platform, determining the platform's position based on the distance value. The charging interface connection status detection uses a current detection circuit instead of a contact switch. That is, when the drone connects to the charging interface, the charging circuit starts working, and the current detection circuit detects the charging current to determine that the connection is normal. The above different sensor configuration schemes can be flexibly selected according to requirements such as cost, accuracy, and reliability.

[0050] The hangar self-inspection subunit compares the acquired detection data with preset hangar status standard values ​​(such as the sensor signal threshold corresponding to the door opening position, the displacement value corresponding to the lifting platform raising position, etc.). If a detection data does not match the corresponding preset hangar status standard value, an abnormality mark is added to the detection item, and then the data packet with the abnormality mark is merged into the comprehensive self-inspection report.

[0051] In one embodiment, the UAV self-test unit obtains the UAV's battery status data, motor status data, propeller status data, and flight control system status data by establishing a communication link with the UAV (e.g., the MAVLink micro-UAV communication protocol or a proprietary wireless communication protocol), and generates a UAV status data packet. Specifically, it parses and extracts the UAV's battery percentage data, motor speed feedback data, propeller integrity self-test data, inertial measurement unit calibration status data, and flight control system heartbeat packet data from the communication link. Each detection data point is compared with preset UAV status thresholds (e.g., battery level below 15% is abnormal, motor speed deviation exceeding ±10% is abnormal, etc.). If any detection data point exceeds the corresponding preset threshold, an abnormality flag is added to that detection item, and then the data packet with the abnormality flag is incorporated into the comprehensive self-test report.

[0052] In one embodiment, the UAV self-test subunit can establish a communication link with the UAV using different communication protocols. For example, for UAVs supporting the MAVLink protocol, the UAV status data can be obtained directly using the MAVLink standard message set; for UAVs using proprietary protocols, the proprietary protocol can be converted into a unified data format through a protocol parsing module; for older UAVs that do not support wireless communication, the status data can be converted into wireless signals and transmitted by adding a data acquisition module. This multi-protocol compatible design allows the present invention to be adapted to different models and brands of UAVs.

[0053] In one embodiment, the environmental assessment subunit acquires wind speed data, rainfall data, and obstacle distance data around the hangar using environmental perception sensors deployed on the vehicle, and generates an environmental data package. Specifically, wind speed data is acquired through a wind speed sensor; rainfall data is acquired through a rain gauge sensor or a rain detection sensor mounted on the drone; and obstacle distance data is acquired through a surround-view camera or ultrasonic radar. Each detection data point is compared with preset environmental safety thresholds (e.g., wind speed greater than 10 m / s is abnormal, obstacle distance less than 1 meter is abnormal, etc.). If a detection data point exceeds the corresponding preset threshold, an anomaly flag is added to that detection item, and then the data package with the anomaly flag is incorporated into the comprehensive self-inspection report.

[0054] In one embodiment, the environmental assessment subunit can employ different sensor configurations. For example, wind speed data can be acquired using the wind speed sensor of the vehicle's air conditioning system, eliminating the need for an additional dedicated wind speed sensor; rainfall data can be acquired using the vehicle's rain sensor, which is typically mounted on the windshield for automatic wiper control; obstacle distance data can be acquired using the vehicle's millimeter-wave radar, which is commonly used in adaptive cruise control and automatic emergency braking systems. By reusing existing vehicle sensors, system costs can be reduced and installation and deployment simplified.

[0055] In one embodiment, the self-test module further includes a report generation subunit: summarizing the anomaly markers added to the hangar status data packet, the UAV status data packet, and the environmental data packet, as well as various detection data and comparison results, to generate a comprehensive self-test report.

[0056] In one embodiment, the self-test module further includes an anomaly handling unit, used to process test items with anomaly markers in the comprehensive self-test report, specifically including: First, the anomaly handling unit determines whether each detection item with an anomaly marker is a critical anomaly. Specifically, the system classifies an anomaly as critical when any of the following occurs: The deviation between the drone motor speed feedback and the target speed exceeds ±10%, indicating a motor malfunction; Damaged or missing propeller integrity self-check feedback indicates physical damage to the propellers; The hangar door cannot be opened fully, indicating a mechanical structural failure; The charging port is not properly connected to the drone and the battery level is below 15%, indicating the drone may have insufficient battery power and cannot be replenished; The inertial measurement unit calibration status of the flight control system is abnormal or the heartbeat packet is lost more than 3 times consecutively, indicating a flight control system malfunction; The drone's battery level is below 15%, indicating insufficient endurance; Wind speed exceeds 10 m / s, indicating unsuitable weather conditions for flight; Obstacles less than 1 meter away from the hangar indicate a restricted takeoff area. The criteria for determining these critical anomalies can be adaptively adjusted according to different drone models and application scenarios.

[0057] Then, perform the corresponding processing based on the current drone status, such as... Figure 2As shown, if a critical anomaly exists and the system is in flight: a forced return-to-home reminder will pop up on the vehicle's central control unit, and the drone will be automatically controlled to return to home. The flight phase refers to the stage where the drone has already taken off and is flying. Detecting a critical anomaly at this stage would directly threaten flight safety; therefore, the anomaly handling adopts the highest priority strategy, immediately forcibly popping up a return-to-home reminder and controlling the drone to automatically return to home. If a critical anomaly exists and the system is in the pre-flight self-check phase: an anomaly indicator will be displayed in a pop-up window on the vehicle's central control unit. If a critical anomaly exists and the system is in standby or driving phase: an anomaly indicator will be displayed in the status bar area of ​​the vehicle's central control unit with a color-changing icon. If an anomaly indicator exists but no critical anomaly exists: an anomaly indicator will be displayed in the status bar area of ​​the vehicle's central control unit with a color-changing icon. The anomaly handling unit also responds to user clicks on the anomaly indicator. When a user clicks on the anomaly indicator, the system detects the click event through the vehicle's central control unit's operating system touch event listening interface. In response to this click, the interface jump function is called to display complete anomaly information and automatically redirects to the processing guidance page. After the user completes the exception handling as prompted on the guided page, the system determines whether the exception has been eliminated by polling to check the exception status or by triggering an event. If the exception has been eliminated, the interface update function is called to remove the corresponding exception identifier from the screen. The polling to check the exception status may involve periodically (e.g., every 2 seconds) retrieving the comprehensive self-check report to check whether the original exception item has been recovered; the event triggering method may involve immediately re-checking after the user clicks the "Processing Complete" button.

[0058] Figure 2 In the initial stage, before the takeoff command is issued, the user has already started the system but has not yet issued the takeoff command. At this point, the system is in the takeoff preparation phase, and a comprehensive understanding of the system status is needed to make a decision on whether to take off. Therefore, the anomaly handling unit outputs an anomaly indicator in the form of a pop-up window on the vehicle's central control unit and directly displays the complete anomaly information to remind the user to handle it. After the user clicks on the anomaly information, the system automatically jumps to the troubleshooting guidance page to guide the user in troubleshooting. This strong prompting strategy ensures that critical safety information is not overlooked and eliminates the safety hazard of taking off with defects due to negligence from the process perspective.

[0059] Before takeoff, when the system is in standby mode, the vehicle is in motion, or the user is not ready to take off, the pop-up window could interfere with the user's current operation and even affect driving safety. Therefore, the anomaly handling unit only displays an anomaly indicator in the status bar area of ​​the vehicle's central control unit by changing the icon color, notifying the user of the anomaly without displaying details. The user can click on the indicator at their convenience, and the system will then display the complete anomaly information and automatically redirect to the handling guidance page. This weak prompting strategy allows the user to be aware of the system status without interrupting the current task, achieving a balance between security and user experience.

[0060] When the drone is in flight, the highest priority strategy is adopted for anomaly handling: if a critical anomaly is detected, the system will immediately force a return-to-home reminder to pop up on the vehicle-mounted central control unit and control the drone to return to home automatically to ensure flight safety.

[0061] In another embodiment, when the system detects a critical anomaly during flight, the anomaly handling unit adopts different handling strategies based on the specific type and severity of the anomaly, including: When the system detects that the drone's battery level is below the first set percentage, it will immediately display a return-to-home reminder. If the user does not respond within the set time of 10 seconds, the system will automatically control the drone to return to home. If the battery level is below the second set percentage, the system will force the drone to return to home immediately without waiting for user confirmation. The first set percentage is 15%, and the second set percentage is 8%.

[0062] When the system detects that the communication signal strength is continuously lower than the set value of -80dBm for more than the first set time, it will display a weak communication signal warning message on the vehicle central control unit. If the signal is completely interrupted for more than the second set time, the uncontrolled return-to-home mechanism will be activated, and the drone will automatically return to the signal recovery area. If the signal is interrupted for more than the third set time, the drone will automatically execute the preset emergency landing procedure. The third set time is 10 seconds, the first set time is 5 seconds, and the second set time is 3 seconds.

[0063] When the system detects a sudden increase in wind speed exceeding 12 m / s, it will immediately display a return-to-home reminder and advise the user to land the drone in a low-wind area. If the drone is performing an important mission such as emergency rescue, the user can ignore the reminder and continue flying, but the system will continue to monitor wind speed changes and update the reminder in real time.

[0064] When abnormal motor speed or propeller integrity is detected, the system immediately forces the drone to return to home and does not allow the user to cancel the return.

[0065] In one embodiment, one method for determining a low-wind-speed area includes: The integrated control module records GPS coordinates and real-time wind speed data of the vehicle's location at a sampling frequency of one sampling point every 10 meters during vehicle operation, constructing a wind speed distribution map along the vehicle's path. When searching for low-wind-speed areas, the system queries historical wind speed records, filters out geographical locations with wind speeds below 8 m / s, and sorts them from closest to furthest from the current vehicle location. The system then designates a circular area with a radius of 50 meters centered on the top-ranked location as the low-wind-speed zone, displays the location and navigation route of this zone on the vehicle's central control unit, and recommends that the user drive the vehicle into this zone before landing the drone.

[0066] The above-mentioned scheme for determining low wind speeds assumes that vehicles can be moved, thereby improving landing conditions by driving vehicles into low wind speed areas.

[0067] In one embodiment, a second method for determining low-wind-speed areas includes: The integrated control module continuously records wind speed data at different geographical locations during vehicle operation, constructing a wind speed distribution map along the vehicle's travel path. The wind speed data is stored in association with GPS location information, forming a mapping relationship between wind speed and location. When it is necessary to find a low wind speed area, the system queries historical wind speed records and identifies geographical locations where the wind speed is below the safe threshold (e.g., 8 m / s). If there are multiple candidate points, the system selects the point that is closest to the vehicle's current location and in the same direction of travel as the target low wind speed area. A circular area with a radius of 50 meters, centered on the target location, is designated as a low-wind-speed area. If this area overlaps with the current hangar location, the search area is expanded. The location of the low-wind-speed area is displayed on the vehicle's central control unit, and a navigation route is provided, suggesting that the user drive the vehicle into the area before landing the drone.

[0068] In one embodiment, a third method for determining low-wind-speed areas includes: (1) Grid delineation High-precision map data within a 500-meter radius of the vehicle's surroundings are obtained from the vehicle navigation system, including building outlines and heights, vegetation distribution, and terrain elevation information. The current wind direction and wind speed data are obtained through a wind speed sensor; Obtain the preset wind speed threshold for safe drone landing (the specific value depends on the drone model, but is usually 60%-70% of its maximum wind resistance). With the vehicle's current position as the origin, establish a 500m × 500m planar coordinate system, and divide the area into 5m × 5m grid units, with each grid unit's coordinates labeled P(x,y). (2) Calculate the influence coefficient For each grid cell, a wind speed reduction coefficient k(x,y) is obtained based on its relationship with buildings, vegetation, and terrain. The wind speed reduction coefficient is determined using one of the following methods: Method 1: The system has an embedded lightweight computational fluid dynamics solver that calculates the estimated wind speed of each grid cell in real time based on the current wind direction and geographic information data; Method 2: The system pre-stores empirical formulas or lookup tables based on wind tunnel experimental data, and calculates or looks up the wind speed reduction coefficient according to parameters such as the distance between the grid cell and the building / vegetation, and terrain features; Method 3: The system pre-trains a wind speed prediction model, inputs geographic information feature vectors, and outputs the estimated wind speed for each grid cell.

[0069] (3) Wind speed estimation Estimated wind speed = safe landing wind speed threshold for drones × wind speed reduction coefficient. Filter out grid cells with estimated wind speed ≤ safe landing wind speed threshold for drones, merge adjacent cells into continuous areas, delete isolated areas with an area less than 100 square meters of the preset threshold, and sort the remaining areas from closest to furthest from the vehicle.

[0070] (4) Area display The lowest wind speed area is highlighted in green on the map of the vehicle's central control unit, and navigation routes are provided to guide users to drive their vehicles into the area or guide drones to land in the area.

[0071] In one embodiment, the system displays the locations of detection points 50 meters away in eight directions (north, northeast, east, southeast, south, southwest, west, and northwest) around the vehicle on the in-vehicle central control unit. The user is guided to fly the drone sequentially to each detection point and hover for 10 seconds to collect real-time wind speed data. Based on the wind speed data from the eight points, the system reconstructs a wind speed field distribution map within a 100-meter radius of the vehicle using inverse distance weighted interpolation, identifying areas with wind speeds below 8 m / s. After the user confirms a low-wind-speed area, the system guides the drone to that area to land.

[0072] If none of the above methods can identify an area with wind speeds below 8 m / s, the system determines that the current environment is not suitable for drone landing and activates an emergency procedure: an emergency prompt is displayed on the vehicle's central control unit indicating that the current wind speed is too high and to find a sheltered location. The system also suggests that the user drive the vehicle into a sheltered location such as an underground parking lot or the leeward side of a building before attempting to land.

[0073] In another embodiment, the method for determining low wind speed areas includes: The vehicle's central control unit displays suggested locations for wind speed detection points in a 360-degree direction around the vehicle, guiding users to set temporary wind speed detection points in different directions and at different distances around the vehicle. Users hover the drone over each detection point, and the drone collects real-time wind speed data at that point through its own wind speed sensor and transmits the data back to the integrated control module. Based on the wind speed data from each detection point, an interpolation algorithm is used to reconstruct the wind speed field distribution map around the vehicle and identify areas where the wind speed is below the safety threshold. The system displays identified low-wind-speed areas on the vehicle's central control unit and guides the user to land the drone in that area.

[0074] The command interaction module is used to parse the commands input by the user through the vehicle central control unit. If the command is parsed as a takeoff command and there are no key abnormalities in the comprehensive self-test report, a hangar opening command is generated. The command interaction module retrieves the complete set of takeoff parameters from the corresponding storage location (fixed parameter storage area, preset parameter storage area, or temporary parameter cache area) based on the parameter source type, and then passes this parameter set to the pre-takeoff interaction module. Specifically, it first identifies whether the user command is a takeoff command. If it is a takeoff command, it further parses the parameter identifiers carried in the command and determines the parameter source type based on the parameter identifiers. Parameter source types include: Fixed parameters are a set of default parameters preset by the system. Users can take off with one click without any settings. They are suitable for general scenarios where there are no special requirements for flight parameters, such as a default takeoff altitude of 10 meters, a default return altitude of 30 meters, and a default flight radius of 500 meters. Preset parameters are a set of parameters that users store in advance according to specific scenarios. For example, users can preset a combination of parameters for urban aerial photography scenarios, such as a takeoff altitude of 50 meters, a return altitude of 80 meters, and a flight radius of 200 meters, and preset a combination of parameters for field inspection scenarios, such as a takeoff altitude of 30 meters, a return altitude of 50 meters, and a flight radius of 500 meters, to achieve rapid switching between scenarios. Custom parameters are parameters temporarily set by the user when issuing the takeoff command. They are suitable for special scenarios requiring precise control, such as setting the takeoff altitude to 15 meters when flying in a forest to avoid crashing into trees. This three-level parameter system ensures both ease of operation and flexibility in different scenarios.

[0075] The pre-flight interaction module is used to control the hangar to perform the opening action when a hangar opening command is received, and to pop up a safety confirmation interface containing the abnormal information in the comprehensive self-test report on the vehicle-mounted central control unit. When a user confirms the safety command issued through the safety confirmation interface is received, a takeoff permission command is generated.

[0076] In one embodiment, before controlling the hangar to perform the opening action, the pre-flight interaction module determines whether the drone is inside the hangar. If the drone is inside the hangar, the vehicle-mounted central control unit displays a reminder message indicating that the hangar is opening, and displays a reminder message indicating that the helipad has been raised after the hangar has opened. If the drone is not inside the hangar, the vehicle-mounted central control unit pops up a reminder message indicating that the drone's surroundings should be manually checked for safety, and displays a safety confirmation interface.

[0077] In one embodiment, the method for determining whether a drone is in the hangar can employ one or more combinations of the following: Charging interface detection: Read the status of the charging interface contact switch. If it is connected, the drone is in the hangar. Infrared beam detection: An infrared transmitter and receiver are installed at the hangar entrance. If the infrared beam is blocked, the drone is inside the hangar. Communication detection: The drone's position coordinates are obtained by communicating with it. If the deviation from the hangar's preset coordinates is less than 10 centimeters, it is determined that the drone is inside the hangar.

[0078] In one embodiment, the safety confirmation interface includes "Confirm Safety" and "Cancel Takeoff" buttons. Only after the user clicks the "Confirm Safety" button does the pre-takeoff interaction module generate a takeoff permission command, and there is no automatic confirmation after a timeout; subsequent actions are executed based on the user's selection. If the user clicks the "Cancel Takeoff" button, the takeoff process is canceled, the hangar (if already open) is shut down, and the system returns to standby mode. This safety confirmation interface not only includes the "Confirm Safety" and "Cancel Takeoff" buttons but also displays all abnormal information (including critical and non-critical anomalies) from the comprehensive self-check report. For critical anomalies, the system has already intercepted them in the command interaction module stage, so the user will not see this interface when a critical anomaly exists. For non-critical anomalies, they are displayed in yellow or orange to remind the user of matters that do not affect safe takeoff but require attention; the user must confirm after being informed before takeoff.

[0079] The in-flight interaction module is activated after the UAV takes off. This module receives flight status parameters sent by the UAV in real time via a wireless communication link. These parameters include altitude, horizontal distance, remaining battery power, communication signal strength, flight speed, and number of satellite signals. Specifically, it monitors the communication link status through a heartbeat mechanism, periodically (e.g., 5 times per second) receives status data frames broadcast by the UAV, parses the parameters from the frames, and forwards the data to the vehicle-mounted central control unit. The vehicle-mounted central control unit displays these parameters in real time on the flight control interface in numerical or graphical form (e.g., instrument panel, progress bar).

[0080] Meanwhile, the in-flight interaction module provides flight control function interfaces on the vehicle-mounted central control unit, including buttons for taking photos, recording videos, gimbal control (up, down, left, right), lens zoom, one-click return-to-home, and flight mode switching. The system responds to user clicks on these function buttons by sending corresponding control commands to the drone via a wireless communication link.

[0081] The post-landing interaction module responds to the return-to-home command, controls the drone to return to home, and performs automatic or manually guided landing. The return-to-home command can be issued by the user or triggered automatically by the system, such as when communication is interrupted for more than 5 seconds or the battery level is below 10%.

[0082] The post-landing interaction module includes an automatic landing unit, which controls a visual guidance target set on the hangar to emit guidance signals. The guidance signals are collected by the UAV's downward-facing camera. The relative position deviation between the UAV and the hangar is calculated based on the collected guidance signals. The UAV's flight attitude is adjusted according to the relative position deviation to guide the UAV to land in the hangar.

[0083] Specifically, the visually guided target can be a QR code pattern or an infrared LED dot matrix. Taking a QR code target as an example, the UAV's downward-facing camera captures the target image, and image processing algorithms (such as adaptive threshold segmentation and contour detection) identify the position of the QR code in the image, extracting the pixel coordinates of the four corner points of the QR code. Based on the actual physical size of the QR code (e.g., 20cm × 20cm) and the corner coordinates, combined with camera intrinsic parameters (focal length, principal point coordinates) and distortion coefficients, the PnP algorithm is used to solve the pose of the QR code target relative to the UAV camera, obtaining the three-dimensional relative position deviations Δx, Δy, and Δz. The flight control system generates horizontal position adjustment commands based on Δx and Δy, and vertical descent speed commands based on Δz, gradually adjusting the UAV's attitude to align the UAV with the hangar center. When Δx, Δy, and Δz are all less than the preset descent allowable thresholds (e.g., Δx < 2cm, Δy < 2cm, Δz < 5cm), the final descent is executed.

[0084] The post-landing interaction module also includes a failure processing unit, which is used to obtain the reason for failure when the automatic landing unit fails to guide the landing, and to display the reason for failure on the vehicle central control unit.

[0085] Specifically, the reasons for failure include loss of positioning signal, obstruction of landing platform, communication interruption, and insufficient power, and the judgment methods include: If the drone's downward-facing camera fails to detect the visual guidance target within a set time (e.g., 30 seconds) from the start of the automatic landing, or if it detects the target but then loses more than a set number of frames (e.g., 10 frames) afterward, it is determined that the positioning signal has been lost. If the hangar door is obstructed by an obstacle during the guided landing process (detected by infrared sensors or ultrasonic radar on the hangar), it is determined that the landing platform is obstructed. In one embodiment, the failure handling unit is further configured to acquire map data of the vehicle's surrounding environment and acquire real-time images of the hangar's surroundings through surround-view cameras deployed on the vehicle; identify flat, unobstructed areas based on the environmental map data and the real-time images, and determine the identified flat, unobstructed areas as alternative landing points; and display the location of the alternative landing points on the vehicle's central control unit.

[0086] The flat, unobstructed area refers to a flat area without protruding obstacles that is suitable for the safe landing of drones, and must meet the following standards: (1) The terrain undulation in the area is less than the allowable height difference of the UAV landing gear; for quadcopter UAVs, the height difference between any two points in the area is usually required to be no more than 5 cm and the slope of the area is no more than 5°.

[0087] (2) There are no protruding objects in the area that are higher than the height of the UAV's landing gear. Specifically, the distance from any point in the area to the nearest obstacle (such as trees, streetlights, vehicles, pedestrians, etc.) is not less than the diameter of the UAV's fuselage, and is usually required to be no less than 50 centimeters. (3) The area of ​​the area shall not be less than twice the projected area of ​​the UAV fuselage to ensure that the UAV has sufficient landing space. For UAVs with a wheelbase of 30 cm, the area shall not be less than 0.18 square meters (approximately equivalent to a square with a side length of 42 cm).

[0088] In one embodiment, the failure handling unit acquires map data of the vehicle's surrounding environment and obtains real-time panoramic images of the hangar's surroundings using surround-view cameras deployed on the vehicle. A semantic segmentation neural network (such as DeepLabV3+) is used to perform pixel-level classification on the panoramic images to identify ground areas. Combined with accessible area markers in the environmental map, areas that simultaneously meet the criteria of "image identified as ground" and "map marked as accessible" are selected. The area, minimum bounding rectangle size, and distance from the hangar are calculated for each candidate area. Areas with an area greater than a first multiple (e.g., 2 times) of the UAV's projected area, a minimum bounding rectangle size greater than a second multiple (1.5 times) of the UAV's diameter, and a set distance from the hangar (e.g., within 50 meters) are selected as candidate landing points and displayed on the vehicle's central control unit, sorted by distance from nearest to farthest. The user can select a candidate landing point, and the system controls the UAV to fly to and land at that point.

[0089] The second alternative solution includes: the failure handling unit launches a virtual joystick interaction interface on the vehicle-mounted central control unit, using a dual joystick layout. The left joystick controls the horizontal movement of the drone, and the right joystick controls the ascent, descent, and yaw. The system detects the user's finger sliding in the joystick area through touch event listening, calculates the direction and length of the sliding vector, the direction determines the command type, and the length is mapped to the command intensity (speed or angular velocity), and generates flight control commands to send to the drone to achieve manual control landing.

[0090] Example 2 A vehicle-mounted drone intelligent interaction method includes the following steps: Step S1: Perform hangar self-check, drone self-check and environmental assessment, generate a comprehensive self-check report, and display abnormal information on the vehicle central control unit according to the comprehensive self-check report, wherein the hangar is set on the vehicle and is used for drone storage, charging and take-off and landing platform support; Step S2: Parse the command input by the user through the vehicle central control unit. If the command is a takeoff command and there are no critical anomalies in the comprehensive self-test report, then generate a hangar opening command. Step S3: When a hangar opening command is received, the hangar is controlled to perform the opening action, and a safety confirmation interface containing the abnormal information in the comprehensive self-test report pops up on the vehicle central control unit. When a safety confirmation command is received from the user through the safety confirmation interface, a takeoff permission command is generated.

[0091] In one embodiment, step S4 is included after step S3: During the drone's flight, flight status parameters sent by the drone are received in real time via a wireless communication link, and the flight status parameters are displayed on the vehicle-mounted central control unit, providing a flight control function interface. This includes buttons for taking a photo, recording video, gimbal control (up, down, left, right), lens zoom, one-click return-to-home, and flight mode switching. The system responds to the user's clicks on these function buttons by sending corresponding control commands to the drone via the wireless communication link.

[0092] In one embodiment, step S5 is included after step S3 or step S4: in response to the return-to-home command, the UAV is controlled to return to home, and post-landing procedures including automatic landing and manual guidance are executed. The specific implementation of each step is consistent with the description of each module of the aforementioned system, and will not be repeated here.

[0093] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0094] This invention has broad industrial application value and can be applied to the following scenarios: 1. Vehicle-mounted aerial photography scene When traveling by car, users can launch a drone with a single click using the in-car central control system, allowing it to follow the vehicle and capture scenery along the way. The drone automatically follows the vehicle, and users can view the drone's footage in real time inside the car and control the camera to take photos and videos via the in-car screen.

[0095] 2. Vehicle-mounted inspection scenario In industrial scenarios such as power line inspection, pipeline inspection, and bridge inspection, inspection vehicles are equipped with drones that travel along the inspection route. The drones take off automatically to inspect the target facilities, and the inspection data is transmitted back to the vehicle's central control system in real time for analysis and processing. When an anomaly is detected, the system automatically records the location and generates a report.

[0096] 3. Emergency Response Scenarios In emergency scenarios such as traffic accidents and fire rescues, emergency vehicles equipped with drones can quickly arrive at the scene. The drones take off automatically and transmit real-time images of the scene through the vehicle's central control system, assisting commanders in quickly grasping the situation and formulating response plans.

[0097] This invention significantly improves the operational efficiency and safety of vehicle-mounted drones by systematically integrating and intelligently processing the drone control process, lowering the operational threshold and showing broad application prospects in civilian, industrial, and military fields.

[0098] Example 3 A computer program product includes a computer program / instructions that, when executed by a processor, implement the various steps of the method described in this invention.

[0099] Example 4 A non-transitory computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, implement the various steps of the method described in this invention, which will not be elaborated further here.

[0100] The computer-readable storage medium can be the data transmission apparatus or the internal storage unit of a computer device provided in any of the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be the external storage device of the computer device, such as the plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device.

[0101] Furthermore, the computer-readable storage medium may include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that is to be output or has already been output.

[0102] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0104] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0105] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0106] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A vehicle-mounted unmanned aerial vehicle intelligent interaction system, characterized in that, include: The self-test module is used to perform hangar self-test, UAV self-test and environmental assessment, generate a comprehensive self-test report, and display abnormal information on the vehicle central control unit based on the comprehensive self-test report; The command interaction module is used to parse the commands input by the user through the vehicle central control unit. If the command is parsed as a takeoff command and there are no key abnormalities in the comprehensive self-test report, a hangar opening command is generated. The pre-flight interaction module is used to control the hangar to perform the opening action when it receives the hangar opening command, and to pop up a safety confirmation interface containing the abnormal information in the comprehensive self-test report on the vehicle central control unit. When it receives the user's confirmation safety command issued through the safety confirmation interface, it generates a takeoff permission command.

2. The UAV-intelligent interaction system of claim 1, wherein, The self-testing module includes one or more of the following: hangar self-testing subunit, UAV self-testing subunit, and environmental assessment subunit; The hangar self-inspection subunit is used to acquire hangar door opening and closing status data, lifting platform position status data and charging interface connection status data, generate hangar status data package, compare each detection data in the data package with the preset hangar status standard value, and if it does not match the preset hangar status standard value, add an abnormality mark to the detection item, and include the detection items with abnormality marks into the comprehensive self-inspection report. The UAV self-test subunit is used to acquire the UAV's power status data, motor status data, propeller status data and flight control system status data, generate UAV status data packets, compare each detection data in the UAV status data packets with preset UAV status thresholds, and add an abnormality mark to the detection item if a certain detection data exceeds the corresponding preset threshold, and merge the data packets with abnormality marks into the comprehensive self-test report. The environmental assessment subunit is used to acquire wind speed data, rainfall data, and obstacle distance data around the hangar, generate an environmental data package, compare the detection data of each detection item in the environmental data package with a preset environmental safety threshold, and add an anomaly mark to the detection item if a certain detection data exceeds the corresponding preset threshold. The data package with the anomaly mark is then merged into the comprehensive self-inspection report.

3. The UAV-intelligent interaction system of claim 2, wherein, The self-test module further includes an anomaly handling unit, used to process test items marked with anomalies in the comprehensive self-test report. The processing method includes: Determine whether each detection item with an anomaly label is a critical anomaly item. The critical anomaly items include detection items that prevent the drone from taking off safely or affect the drone's flight. If there are critical anomalies and the system is in the pre-flight self-check phase, an anomaly indicator will be displayed in a pop-up window on the vehicle-mounted central control unit.

4. The UAV-intelligent interaction system of claim 3, wherein, The processing method further includes: if there is a critical anomaly and the system is in standby or driving mode, an anomaly indicator is output in the status bar area of ​​the vehicle central control unit in the form of an icon color change. 5.The vehicle-mounted UAV intelligent interaction system of claim 3 or 4, wherein, The anomaly handling unit is also used to respond to the user's click operation on the anomaly icon. When the user clicks the anomaly icon, the complete anomaly information is displayed on the vehicle central control unit and the user is automatically redirected to the processing guidance page. After the anomaly is eliminated, the anomaly icon is made to disappear.

6. The drone intelligent interaction system of claim 1, wherein, The command interaction module, when parsing commands input by the user through the vehicle's central control unit, includes: When the user command is a takeoff command, the parameter identifier carried in the command is parsed, and the parameter source type is determined based on the parameter identifier. The parameter source type includes fixed parameters, preset parameters, and custom parameters. The corresponding takeoff parameters are obtained according to the parameter source type, and the takeoff parameters are passed to the pre-takeoff interaction module.

7. The vehicle-mounted unmanned aerial vehicle intelligent interaction system as described in claim 1, characterized in that, Before controlling the hangar to perform the opening action, the pre-flight interaction module also includes: The system determines whether the drone is inside the hangar. If the drone is inside the hangar, the vehicle-mounted central control unit displays a reminder message that the hangar is opening. After the hangar opens, the vehicle-mounted central control unit displays a reminder message that the helipad has been raised. If the drone is not inside the hangar, the vehicle-mounted central control unit displays a reminder message requesting manual confirmation of the safety of the drone's surroundings, and a safety confirmation interface is displayed.

8. The UAV-integrated vehicle interaction system of claim 7, wherein, The safety confirmation interface includes a first button for confirming safety and a second button for canceling takeoff. The pre-takeoff interaction module generates the takeoff permission command only after the user clicks the first button.

9. The UAV-integrated vehicle interaction system of claim 1, wherein, It also includes a post-landing interaction module, which controls the drone to return to home after receiving a return command and executes post-landing procedures including automatic landing and / or manual guidance.

10. The UAV-integrated vehicle interaction system of claim 9, wherein, The post-landing interaction module includes an automatic landing unit, which controls a visual guidance target set on the hangar to emit guidance signals. The guidance signals are collected by the UAV's downward-facing camera. The relative position deviation between the UAV and the hangar is calculated based on the collected guidance signals. The UAV's flight attitude is adjusted according to the relative position deviation to guide the UAV to land in the hangar.

11. The UAV-integrated vehicle interaction system of claim 10, wherein, The post-landing interaction module also includes a failure processing unit, which is used to obtain the reason for failure when the automatic landing unit fails to guide the landing, and to display the reason for failure on the vehicle central control unit.

12. The drone intelligent interaction system of claim 11, wherein, The failure handling unit is also used to acquire map data of the vehicle's surrounding environment and acquire real-time images of the hangar's surroundings through surround-view cameras deployed on the vehicle; identify flat, unobstructed areas based on the environmental map data and the real-time images, and determine the identified flat, unobstructed areas as alternative landing points; and display the location of the alternative landing points on the vehicle's central control unit. 13.The vehicle-mounted UAV intelligent interaction system of claim 11 or 12, wherein, The failure handling unit is also used to activate a virtual joystick interface on the vehicle-mounted central control unit, receive manual control commands input by the user through the virtual joystick interface, and control the drone to land according to the manual control commands.

14. A vehicle-mounted unmanned aerial vehicle (UAV) intelligent interaction method for the system as described in claim 1, characterized in that, include: Perform hangar self-inspection, drone self-inspection and environmental assessment, generate a comprehensive self-inspection report, and display abnormal information on the vehicle-mounted central control unit based on the comprehensive self-inspection report; The system parses the commands input by the user through the vehicle central control unit. If the command is a takeoff command and the comprehensive self-test report has no critical anomalies, a hangar opening command is generated. When a hangar opening command is received, the hangar is controlled to perform the opening action, and a safety confirmation interface containing the abnormal information in the comprehensive self-test report pops up on the vehicle central control unit. When a safety confirmation command is received from the user through the safety confirmation interface, a takeoff permission command is generated.

15. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the vehicle-mounted unmanned aerial vehicle intelligent interaction method as described in claim 14.

16. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the vehicle-mounted unmanned aerial vehicle intelligent interaction method of claim 14.

Images