Landing control method and aircraft control system

Abstract

The present application relates to the field of unmanned aerial vehicle technology and provides a landing control method and an aircraft control system. The method includes: during the return process of the unmanned aerial vehicle, if the altitude above the ground is less than or equal to a first altitude threshold, photographing the predetermined landing position to obtain a first image; if the characteristic information in the first image does not include a helipad mark, if the altitude above the ground is less than or equal to a second altitude threshold, photographing the predetermined landing position to obtain a second image; based on the second image, determining the environmental characteristics of the area where the predetermined landing position is located; if the environmental characteristics do not meet the preset landing requirements, generating a first prompt message, and sending the first prompt message to a control terminal, prompting the adjustment of the predetermined landing position; responding to the adjustment instruction for the predetermined landing position, using the adjusted landing position as the target landing position, and controlling the unmanned aerial vehicle to land at the target landing position. Utilizing this method, the landing safety of the unmanned aerial vehicle can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to a landing control method and an aircraft control system. Background Art

[0002] During the UAV's return process, the UAV's position information is usually located through the Global Positioning System (GPS), and the distance and direction between the UAV and the return point are calculated. The UAV generates a flight path based on the distance and direction between the UAV and the return point and lands along the flight path to the return point.

[0003] However, due to the inherent inaccuracy of GPS positioning, the actual landing location of the UAV may differ from the intended return point by tens of centimeters to several meters. Unknown landing locations can adversely affect the landing safety of the UAV and the safety of people and objects in the landing area. Summary of the Invention

[0004] In view of this, the present application provides a landing control method and an aircraft control system to solve the problem that the landing control method of the unmanned aerial vehicle has errors, resulting in the actual landing position being unknown, which may have an adverse impact on the landing safety of the unmanned aerial vehicle and the safety of people and objects in the landing area of the unmanned aerial vehicle.

[0005] A first aspect of an embodiment of the present application provides a landing control method, which is applied to an unmanned aerial vehicle, wherein the unmanned aerial vehicle is communicatively connected to a control terminal, and the landing control method includes: during the return process of the unmanned aerial vehicle, if the height of the unmanned aerial vehicle above the ground is less than or equal to a first height threshold, controlling a preset shooting device to shoot a predetermined landing position to obtain a first image; if the feature information in the first image does not include a helipad sign, when the height of the unmanned aerial vehicle above the ground is less than or equal to a second height threshold, controlling the preset shooting device to shoot the predetermined landing position to obtain a second image, wherein the second height threshold is less than the first height threshold; based on the second image, determining the environmental characteristics of the area where the predetermined landing position is located; if the environmental characteristics do not meet the preset landing requirements, generating a first prompt message, and sending the first prompt message to the control terminal to prompt the user to adjust the predetermined landing position; in response to an adjustment instruction for the predetermined landing position, using the adjusted landing position as the target landing position, and controlling the unmanned aerial vehicle to land at the target landing position.

[0006] In some embodiments, if the characteristic information includes the apron identification, a second prompt message is generated based on the apron identification, and the second prompt message is sent to the control terminal, prompting the user to choose whether to land at the apron corresponding to the apron identification; in response to the control instruction to land at the apron, the unmanned aerial vehicle is controlled to land at the apron.

[0007] In some embodiments, determining the environmental characteristics of the area where the predetermined landing position is located based on the second image includes: using a preset classification model to identify object features in the second image; predicting the flatness of the area where the predetermined landing position is located based on a pixel position mapping relationship between pixel points in multiple second images; and using the object features and the flatness as the environmental characteristics.

[0008] In some embodiments, predicting the flatness of the area where the predetermined landing position is located based on the pixel position mapping relationship of pixel points in multiple second images includes: constructing a fitting model based on the pixel position mapping relationship; acquiring two consecutive frames of second images, the two consecutive frames of second images including a front image and a rear image; predicting the relative position of the pixel points of the front image in the rear image based on the fitting model; and calculating the flatness based on the relative position of the pixel points of the front image in the rear image and the positions of the corresponding pixel points in the rear image.

[0009] In some embodiments, the method further includes: if the object characteristics include preset characteristics, determining that the environmental characteristics do not meet the preset landing requirements, the preset characteristics including characteristics of the object that affect the landing safety of the unmanned aerial vehicle.

[0010] In some embodiments, the method further includes: if the flatness is less than a preset flatness threshold, determining that the environmental characteristics do not meet the preset landing requirements.

[0011] In some embodiments, the unmanned aerial vehicle includes a gimbal, which is connected to the preset shooting device. Controlling the preset shooting device to shoot a first image of the predetermined landing position includes: adjusting the posture of the preset shooting device by adjusting the posture of the gimbal, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained; and / or adjusting the posture of the preset shooting device by adjusting the posture of the unmanned aerial vehicle, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained.

[0012] In some embodiments, if the environmental characteristics do not meet the preset landing requirements, a video image of the area where the predetermined landing position is located is obtained; and the first prompt information is generated based on the video image and the environmental characteristics.

[0013] A second aspect of an embodiment of the present application provides an aircraft control system, comprising: an unmanned aerial vehicle, the unmanned aerial vehicle being used to execute the above-mentioned landing control method; a control terminal, the control terminal being communicatively connected to the unmanned aerial vehicle, the control terminal comprising a display device, the display device being used to display a first prompt message sent by the unmanned aerial vehicle.

[0014] In some embodiments, the first prompt information includes the image transmission screen, which is marked with a landing point icon corresponding to the predetermined landing position. The control terminal also includes a joystick, and the control terminal is used to: respond to the user's operation of adjusting the predetermined landing position through the joystick, or respond to the user's operation of moving the landing point icon, update the position of the landing point icon in the image transmission screen, and generate an adjustment instruction carrying the position of the updated landing point icon, and send the adjusted instruction to the unmanned aerial vehicle, so that the unmanned aerial vehicle determines the target landing position according to the position of the updated landing point icon.

[0015] A third aspect of an embodiment of the present application provides an unmanned aerial vehicle, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned landing control method when executing the computer-readable instructions.

[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions implement the above-mentioned landing control method when executed by a processor.

[0017] In a landing control method provided by an embodiment of the present application, during the return process of the unmanned aerial vehicle, the predetermined landing position is photographed by controlling the preset shooting device multiple times to obtain an image containing the surrounding environment of the predetermined landing position. When it is determined that the image does not contain the apron sign, the environmental characteristics of the area where the predetermined landing position is located are analyzed based on the captured image. If the environmental characteristics do not meet the preset landing requirements, a first prompt message is generated and sent to the control terminal to prompt the user to adjust the predetermined landing position. In response to the adjustment instruction for the predetermined landing position, the adjusted landing position is used as the target landing position, and the unmanned aerial vehicle is controlled to land at the target landing position. By detecting the environmental characteristics around the predetermined landing position during the return process of the unmanned aerial vehicle, and prompting the user to adjust the predetermined landing position when the environmental characteristics do not meet the preset landing requirements, the probability of the unmanned aerial vehicle landing in a dangerous area can be effectively reduced, thereby improving the landing safety of the unmanned aerial vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 It is a structural diagram of the aircraft control system provided in an embodiment of the present application.

[0020] Figure 2 This is a flowchart of the implementation of the landing control method provided in an embodiment of the present application.

[0021] Figure 3 This is an example diagram of the interface of the image transmission screen provided in an embodiment of the present application.

[0022] Figure 4 It is a structural schematic diagram of the landing control device provided in an embodiment of the present application.

[0023] Figure 5 It is a structural schematic diagram of the unmanned aerial vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the suggested technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can mean A or B. "And / or" in this application is merely a way to describe the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more. "Multiple" means two or more than two. For example, at least one of a, b or c can mean: a, b, c, a and b, a and c, b and c, a, b and c.

[0027] See also Figure 1 As shown in FIG. 1 , a schematic diagram of the structure of the aircraft control system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, aircraft control system 1000 includes UAV 100 and control terminal 200, which are in communication with each other. The landing control method provided in the embodiments of the present application can be applied to UAV 100 or a control device within UAV 100. Control terminal 200 can be used to receive data sent by UAV 100, such as videos, images, sensor data, and status information of UAV 100.

[0028] During the landing control process, the unmanned aerial vehicle 100 performing the return mission detects the height above the ground. If the height above the ground of the unmanned aerial vehicle 100 is less than or equal to the first height threshold, the preset camera is controlled to capture the predetermined landing position to obtain a first image. The unmanned aerial vehicle 100 analyzes the characteristic information in the first image. If the characteristic information in the first image does not include the apron mark, if the unmanned aerial vehicle 100 detects that the height above the ground of the unmanned aerial vehicle 100 is less than or equal to the second height threshold, the preset camera is controlled to capture the predetermined landing position to obtain a second image. The second height threshold is less than the first height threshold. Based on the second image, the unmanned aerial vehicle 100 determines the environmental characteristics of the area where the predetermined landing position is located. If the environmental characteristics do not meet the preset landing requirements, the unmanned aerial vehicle 100 generates a first prompt message based on the second image and the environmental characteristics, and sends the first prompt message to the control terminal 200 to prompt the user to adjust the predetermined landing position.

[0029] The control terminal 200 receives the first prompt information and displays the first prompt information on the display device. The user adjusts the predetermined landing position according to the first prompt information, and the control terminal 200 generates an adjustment instruction in response to the user's adjustment operation and sends the adjustment instruction to the unmanned aerial vehicle 100. The unmanned aerial vehicle 100 responds to the adjustment instruction for the predetermined landing position, uses the adjusted landing position as the target landing position, and controls the unmanned aerial vehicle 100 to land at the target landing position. The landing control method provided in the embodiment of the present application, if no helipad is detected in the area where the predetermined landing position is located, detects whether the environmental characteristics of the area where the predetermined landing position is located meet the landing requirements. If the environmental characteristics do not meet the preset landing requirements, a prompt information is generated and sent to the on-board control terminal to prompt the user to adjust the predetermined landing position, thereby effectively reducing the probability of the unmanned aerial vehicle landing in a dangerous area and improving the landing safety of the unmanned aerial vehicle.

[0030] In some embodiments, the first prompt information includes a video transmission image of the area where the predetermined landing location is located, with a landing point icon corresponding to the predetermined landing location marked in the video transmission image. The user can adjust the predetermined landing location by manipulating the joystick of the control terminal, or by moving the landing point icon (for example, by dragging the landing point icon, or selecting another location as the location of the landing point icon), thereby adjusting the predetermined landing location. In response to the user's operation of adjusting the predetermined landing location using the joystick, or in response to the user's operation of moving the landing point icon, the control terminal updates the position of the landing point icon in the video transmission image, generates an adjustment instruction carrying the updated position of the landing point icon, and sends the adjustment instruction to the unmanned aerial vehicle, so that the unmanned aerial vehicle determines the target landing location based on the updated position of the landing point icon.

[0031] In some embodiments, the UAV 100 may be a multi-rotor drone, an unmanned aerial vehicle, or the like. The control terminal 200 may be a remote controller or a user terminal, such as a mobile phone, a tablet computer, a wearable device, or any combination thereof. The embodiments of the present application do not limit the specific device types of the UAV 100 or the control terminal 200.

[0032] In some embodiments, the communication connection method includes, but is not limited to, a wireless communication connection. For example, the wireless channel from the UAV 100 to the control terminal 200 can be used to transmit data collected by the UAV 100, such as videos, images, sensor data, and UAV 100 status information. The wireless channel from the control terminal 200 to the UAV 100 can also be used to transmit remote control data, such as flight control commands and control commands such as return commands, photo capture commands, and video recording commands.

[0033] Figure 1 The illustrated scenarios are merely illustrative examples, and the return control method provided in this application can also be applied in other scenarios. The embodiments of this application do not limit the specific application scenarios of the return control method.

[0034] See also Figure 2 The figure is a flowchart of the implementation of the landing control method provided by the embodiment of the present application. The method is applied to unmanned aerial vehicles. The embodiment of the present application uses the method to Figure 1 The method is described using the unmanned aerial vehicle 100 in FIG.

[0035] S11: During the return process of the UAV, if the altitude of the UAV above the ground is less than or equal to a first altitude threshold, control a preset shooting device to shoot a predetermined landing position to obtain a first image.

[0036] In some embodiments, the predetermined landing location represents a predetermined location or area where the UAV will land. The predetermined landing location can be determined based on the location of the UAV's takeoff point, can be user-specified, or can be a location determined during autonomous return to home, such as during a low-battery return to home situation. The embodiments of the present application do not limit the method for determining the predetermined landing location.

[0037] In some embodiments, the first height threshold can be customized, for example, 10 meters, 9 meters, etc. The present embodiment does not limit the first height threshold.

[0038] In some embodiments, the first image represents an image of the area where the predetermined landing location is located, and is used to indicate the environmental conditions of the area where the predetermined landing location is located.

[0039] In some embodiments, when it is determined that the altitude of the unmanned aerial vehicle above the ground is less than or equal to a first altitude threshold, the unmanned aerial vehicle controls a preset shooting device to shoot a predetermined landing position to obtain a first image to ensure that the first image shot is relatively clear, so as to improve the accuracy of subsequent recognition of apron signs based on the first image.

[0040] In some embodiments, the method for detecting the UAV's altitude above the ground can be selected based on the specific application scenario, accuracy requirements, and cost budget of the UAV. For example, the UAV's altitude above the ground can be detected using one or more of sensor technology (e.g., an altitude sensor), radar technology, and Global Positioning System (GPS) technology. The present embodiments do not limit the specific detection method.

[0041] In some embodiments of the present application, the unmanned aerial vehicle includes a gimbal, which is connected to a preset shooting device and controls the preset shooting device to shoot a first image of a predetermined landing position, including: adjusting the posture of the preset shooting device by adjusting the posture of the gimbal, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained; and / or adjusting the posture of the preset shooting device by adjusting the posture of the unmanned aerial vehicle, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained.

[0042] In some embodiments, the preset shooting device may be one or more devices such as a camera, a video camera, etc. The embodiments of the present application are not limited thereto.

[0043] In some embodiments, the gimbal is connected to a pre-set camera device to stabilize and support the pre-set camera device. The gimbal is adjustable. The UAV can adjust the gimbal's posture (e.g., angle and direction) using built-in control devices. This adjustment can be horizontal, vertical, or tilt.

[0044] In some embodiments, the UAV can adjust the shooting direction of the preset shooting device by adjusting the posture of the gimbal, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained.

[0045] For example, when returning home, a UAV typically follows a pre-set flight path to a designated location (e.g., a predetermined landing location) and then performs a vertical landing to ensure a safe landing. When the UAV is directly below the predetermined landing location, the UAV can adjust the gimbal's posture so that the preset camera is pointing vertically downward to capture the predetermined landing location.

[0046] In other embodiments, the UAV may also adjust the posture of the preset camera by adjusting the posture of the UAV, so that the shooting direction of the preset camera is toward the predetermined landing position and the first image captured by the preset camera is obtained.

[0047] In other embodiments, the UAV can also adjust the posture of the preset shooting device by adjusting the posture of the gimbal and the posture of the UAV, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained.

[0048] S12: If the feature information in the first image does not include the apron mark, and the altitude of the UAV above the ground is less than or equal to a second altitude threshold, controlling a preset camera to capture the predetermined landing location to obtain a second image, wherein the second altitude threshold is less than the first altitude threshold.

[0049] In some embodiments, feature information may include geometric features (e.g., outlines, edge contours, and pixel values) of objects in the area of the predetermined landing location, topological features (e.g., surface features and moment features), and landmark information (e.g., specific patterns, building features, and terrain features). Objects include, but are not limited to, natural elements (e.g., trees, vegetation, rocks, and water bodies), as well as various man-made or constructed landmarks (e.g., buildings and landmarks).

[0050] In some embodiments, the UAV may obtain feature information by performing preprocessing, feature extraction, feature matching, and other methods on the first image. Preprocessing may include performing image denoising, image enhancement, image correction, and other processing on the first image. The UAV may perform feature extraction using one or more methods such as local features, deep learning, and optical flow. The UAV may perform feature matching using one or more methods such as similarity measurement, machine learning, and geometric constraints.

[0051] In some embodiments, the apron sign can be customized to indicate the apron. For example, the apron sign can be set to an "H" sign. This embodiment of the present application is not limited thereto.

[0052] In some embodiments, if the feature information in the first image does not include a landing pad marker, the UAV continues to land. When the UAV's altitude above the ground is less than or equal to a second altitude threshold, the UAV is relatively close to the predetermined landing location. In this case, the UAV controls a preset camera to capture the predetermined landing location, thereby obtaining a second image.

[0053] In some embodiments, the UAV can more accurately obtain the features of objects in the area where the predetermined landing position is located by analyzing the second image, so as to more accurately determine whether the area where the predetermined landing position is located is suitable for landing.

[0054] In some embodiments of the present application, if the characteristic information includes an apron identification, a second prompt message is generated based on the apron identification, and the second prompt message is sent to the control terminal, prompting the user to choose whether to land on the apron corresponding to the apron identification; in response to the control instruction to land on the apron, the unmanned aerial vehicle is controlled to land on the apron.

[0055] In some embodiments, if the characteristic information includes an apron mark, it means that there is a helipad in the area where the predetermined landing position is located. At this time, the unmanned aerial vehicle can generate a second prompt information based on the apron mark and send the second prompt information to the control terminal. The control terminal receives the second prompt information and can display the second prompt information on the display device. For example, the second prompt information can be "There is a helipad with an apron mark of "H" in the area where the predetermined landing position is located. Please confirm whether to choose to land on the apron." If the user chooses to land on the apron, the control terminal generates a control instruction for landing on the apron and sends the control instruction to the unmanned aerial vehicle. In response to the control instruction for landing on the apron, the unmanned aerial vehicle adjusts the predetermined landing position, uses the location of the apron as the target landing position, and controls the unmanned aerial vehicle to land on the apron.

[0056] S13: Determine environmental features of the area where the predetermined landing position is located based on the second image.

[0057] In some embodiments, environmental features include but are not limited to geometric features of objects in the area where the predetermined landing location is located (such as the object's outline, edge outline, pixel value), topological features (such as the object's texture features, moment features), and landmark information (such as specific patterns, building features, terrain features).

[0058] In some embodiments, the UAV may obtain feature information by performing preprocessing, feature extraction, feature matching, and other methods on the second image. Preprocessing may include performing image denoising, image enhancement, image correction, and other processing on the second image. The UAV may perform feature extraction using one or more methods such as local features, deep learning, and optical flow. The UAV may perform feature matching using one or more methods such as similarity measurement, machine learning, and geometric constraints.

[0059] In some embodiments of the present application, environmental features of the area where the predetermined landing position is located are determined based on the second image, including: using a preset classification model to identify object features in the second image; predicting the flatness of the area where the predetermined landing position is located based on a pixel position mapping relationship between pixel points in multiple second images; and using the object features and the flatness as environmental features.

[0060] In some embodiments, the object features include but are not limited to the object's category, location, shape, size, etc. Flatness indicates the flatness of the ground in the area where the predetermined landing position is located.

[0061] In some embodiments, the preset classification model is a deep learning model, such as a convolutional neural network model, a support vector machine, a random forest, etc.

[0062] In some embodiments, the UAV can train a preset classification model in the following manner: the UAV acquires multiple second images as training samples and preprocesses the acquired sample images, such as image denoising, enhancement, geometric correction, and distortion correction, to improve image quality and recognition accuracy. The UAV then extracts features from the preprocessed sample images, such as color feature extraction, texture feature extraction, and shape feature extraction. After extracting multiple features, the UAV performs feature selection and fusion to improve recognition accuracy and efficiency. The UAV uses the selected and fused features to train the initial classification model, resulting in a preset classification model.

[0063] In the process of using the preset classification model to identify the object features in the second image, the unmanned aerial vehicle inputs the second image into the preset classification model, and identifies the object features in the second image through comparison and classification.

[0064] In some embodiments of the present application, based on the pixel position mapping relationship of pixel points in multiple second images, the flatness of the area where the predetermined landing position is located is predicted, including: constructing a fitting model based on the pixel position mapping relationship; acquiring two consecutive frames of second images, the two consecutive frames of second images including the front image and the rear image; based on the fitting model, predicting the relative position of the pixel points of the front image in the rear image; and calculating the flatness based on the relative position of the pixel points of the front image in the rear image and the position of the corresponding pixel points in the rear image.

[0065] In some embodiments, the unmanned aerial vehicle can obtain multiple pairs of two consecutive frames of second images, use a dense optical flow algorithm to calculate the position change of each pixel point in the two consecutive frames of second images, and construct a pixel position mapping relationship by analyzing the position change of each pixel point in the two consecutive frames of second images.

[0066] In some embodiments, the unmanned aerial vehicle constructs a fitting model by performing statistical analysis on the pixel position mapping relationship, such as using the least squares method for analysis. For example, the fitting model can be a homography matrix model. The homography matrix model is used to describe the pixel position mapping relationship between two consecutive frames of second images. In the process of calculating the flatness of the area where the predetermined landing position is located, the unmanned aerial vehicle can obtain two consecutive frames of second images, which are included in the front image and the rear image, and transform the pixel points in the front image to the coordinate system of the rear image through the homography matrix model. The unmanned aerial vehicle predicts the flatness by calculating the predicted distance between the position of the pixel point in the front image mapped to the coordinate system of the rear image and the position of the actual corresponding pixel point in the rear frame image.

[0067] In some embodiments, the UAV may pre-establish a relationship between predicted distance and flatness, whereby the smaller the predicted distance, the greater the flatness, and vice versa. After calculating the predicted distance, the UAV may determine the flatness of the ground in the area of the intended landing location based on the relationship between the predicted distance and flatness.

[0068] In some embodiments, if the flatness is less than a preset flatness threshold, it indicates that the ground in the area of the predetermined landing location may be significantly undulating or irregular in shape, making it unsuitable for the UAV to land. In this case, the UAV determines that the environmental characteristics do not meet the preset landing requirements.

[0069] In some embodiments, by identifying the flatness of the area where the object feature in the second image is located, it can be determined whether the area where the predetermined landing location is located contains objects that may affect the landing safety of the UAV, or other objects that are unsuitable for the UAV to land. By estimating the flatness of the ground in the area where the predetermined landing location is located, it can be determined whether the ground in the area where the predetermined landing location is located is suitable for landing.

[0070] In some embodiments of the present application, if the object characteristics include preset characteristics, it is determined that the environmental characteristics do not meet the preset landing requirements, and the preset characteristics include characteristics of the object that affect the landing safety of the unmanned aerial vehicle.

[0071] In some embodiments, the preset features can be customized. For example, the preset features can include features such as people, vehicles, animals, grass, and water bodies.

[0072] In some embodiments, if the object characteristics include preset characteristics, this indicates that the area of the scheduled landing location contains object characteristics that could affect the UAV's landing safety. Controlling the UAV to land at the scheduled landing location could be detrimental to the safety of the UAV and the safety of people and objects in the area. Therefore, if the object characteristics include preset characteristics, the UAV determines that the environmental characteristics do not meet the preset landing requirements and may prompt an adjustment to the scheduled landing location.

[0073] In some embodiments of the present application, if the flatness is less than a preset flatness threshold, it is determined that the environmental characteristics do not meet the preset landing requirements.

[0074] S14: If the environmental characteristics do not meet the preset landing requirements, a first prompt message is generated and sent to the control terminal to prompt the user to adjust the predetermined landing position.

[0075] In some embodiments, the preset landing requirements include but are not limited to the flatness of the ground in the area where the predetermined landing position is located being less than a preset flatness threshold, and the object features in the area not containing preset features.

[0076] In some embodiments, if the environmental characteristics do not meet the preset landing requirements, a video image of the area where the predetermined landing location is located is obtained, and a first prompt message is generated based on the video image and the environmental characteristics.

[0077] In some embodiments, the image transmission image represents real-time images of the area surrounding the predetermined landing location captured by the UAV and transmitted via an image transmission system (image transmission system) to a display device on a control terminal. The image transmission image may include image information captured by a preset camera device, as well as additional data such as flight parameters, battery charge, and signal strength.

[0078] In some embodiments, the UAV may mark the predetermined landing location in the image transmission screen based on the coordinate information of the predetermined landing location, and determine the representation of the predetermined landing location in the image transmission screen as a landing point icon.

[0079] In some embodiments, the control terminal receives the first prompt information, parses the first prompt information, obtains the image transmission image and environmental characteristics of the area where the predetermined landing location is located, and displays information related to the image transmission image and environmental characteristics on a display device of the control terminal.

[0080] In some embodiments, the predetermined landing position can be adjusted by adjusting the position of the landing point icon in the image transmission screen. The user can adjust the position of the landing point icon in the image transmission screen according to the environmental characteristics of the area where the predetermined landing position is located, thereby reducing the situation where the adjusted landing position falls into a dangerous area and improving the safety of the UAV landing. In some embodiments, such as Figure 3 As shown, the image transmission screen 20 includes an icon 21 corresponding to the UAV, an icon 22 corresponding to the control terminal, and a landing point icon 23 corresponding to the predetermined landing location. The user can adjust the predetermined landing location by manipulating the joystick of the control terminal, or move the landing point icon 23, such as by dragging the landing point icon 23 or selecting another location as the location of the landing point icon 23, thereby adjusting the position of the landing point icon 23. In response to the user's operation of adjusting the predetermined landing location via the joystick or the user's operation of moving the landing point icon 23, the control terminal updates the position of the landing point icon 23 in the image transmission screen, generates an adjustment instruction carrying the updated location of the landing point icon, and sends the adjustment instruction to the UAV.

[0081] S15: Responding to the instruction to adjust the predetermined landing position, taking the adjusted landing position as the target landing position, and controlling the UAV to land at the target landing position.

[0082] In some embodiments, the target landing location represents a location or area where the UAV determines it will land.

[0083] In a landing control method provided by an embodiment of the present application, during the return process of the unmanned aerial vehicle, the predetermined landing position is photographed by controlling the preset shooting device multiple times to obtain an image containing the surrounding environment of the predetermined landing position. When it is determined that the image does not contain the apron sign, the environmental characteristics of the area where the predetermined landing position is located are analyzed based on the captured image. If the environmental characteristics do not meet the preset landing requirements, a first prompt message is generated and sent to the control terminal to prompt the user to adjust the predetermined landing position. In response to the adjustment instruction for the predetermined landing position, the adjusted landing position is used as the target landing position, and the unmanned aerial vehicle is controlled to land at the target landing position. By detecting the environmental characteristics around the predetermined landing position during the return process of the unmanned aerial vehicle, and prompting the user to adjust the predetermined landing position when the environmental characteristics do not meet the preset landing requirements, the probability of the unmanned aerial vehicle landing in a dangerous area can be effectively reduced, thereby improving the landing safety of the unmanned aerial vehicle.

[0084] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0085] See also Figure 4 , is a structural diagram of the landing control device provided in the embodiment of the present application, which can implement the details of the landing control method in the above embodiment and achieve the same effect. Figure 4 As shown, a landing control device 10 can be applied to an unmanned aerial vehicle with a data processing function. The landing control device 10 includes: a first shooting module 11, which is used to control a preset shooting device to shoot a predetermined landing position to obtain a first image if the altitude of the unmanned aerial vehicle above the ground is less than or equal to a first altitude threshold during the return process of the unmanned aerial vehicle; a second shooting module 12, which is used to control the preset shooting device to shoot the predetermined landing position to obtain a second image if the feature information in the first image does not include the apron mark and the altitude of the unmanned aerial vehicle above the ground is less than or equal to a second altitude threshold, wherein the second altitude threshold is less than the first altitude threshold; a feature determination module 13, which is used to determine the environmental features of the area where the predetermined landing position is located based on the second image; an information prompt module 14, which is used to generate a first prompt message if the environmental features do not meet the preset landing requirements, and send the first prompt message to the control terminal to prompt the user to adjust the predetermined landing position; and a landing control module 15, which is used to respond to an adjustment instruction for the predetermined landing position, use the adjusted landing position as the target landing position, and control the unmanned aerial vehicle to land at the target landing position.

[0086] The specific definitions of the landing control device 10 can be found in the above-mentioned definitions of the landing control method and will not be elaborated here. Each module within the landing control device 10 can be implemented in whole or in part via software, hardware, or a combination thereof. Each module can be embedded in or independent of the UAV's processor in hardware form, or stored in the UAV's memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0087] See also Figure 5 , Figure 5 FIG2 is a schematic diagram of the structure of an unmanned aerial vehicle (UAV) according to an embodiment of the present application. UAV 100 includes, but is not limited to, any of a multirotor aircraft and an unmanned aerial vehicle (UAV). The network in which UAV 100 resides includes, but is not limited to, the Internet, a wide area network (WAN), a metropolitan area network (MAN), a local area network (LAN), a virtual private network (VPN), and the like.

[0088] like Figure 5 As shown, the UAV 100 includes a communication module 101, a memory 102, a processor 103, an input / output interface 104, and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102, and the input / output interface 104 via the bus 105.

[0089] The communication module 101 can be a wireless communication module or a mobile communication module. The wireless communication module can provide wireless communication solutions for the UAV 100, including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), and the like. The mobile communication module can provide wireless communication solutions for the UAV 100, including 2G / 3G / 4G / 5G.

[0090] Memory 102 may include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). RAM can be directly read and written by processor 103 and can be used to store executable programs (e.g., machine instructions) for the operating system or other running programs, as well as user and application data. RAM may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0091] The non-volatile memory can also store executable programs and user and application data, etc., and can be pre-loaded into the random access memory for direct reading and writing by the processor 103. The non-volatile memory can include disk storage devices and flash memory.

[0092] The memory 102 is configured to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include a plurality of instructions that, when executed by the processor 103, can implement a landing control method executed on the UAV 100.

[0093] In other embodiments, the UAV 100 further includes an external memory interface for connecting to an external memory to expand the storage capacity of the UAV 100 .

[0094] The processor 103 may include one or more processing units. For example, the processor 103 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0095] The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute the computer program stored in the memory 102 to implement the above-mentioned landing control method.

[0096] The input / output interface 104 is used to provide a channel for user input or output. For example, the input / output interface 104 can be used to connect various input and output devices, such as a mouse, keyboard, touch device, display screen, etc., so that the user can enter information or visualize information.

[0097] The bus 105 is at least used to provide a communication channel between the communication module 101 , the memory 102 , the processor 103 , and the input / output interface 104 in the UAV 100 .

[0098] It should be understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on UAV 100. In other embodiments of this application, UAV 100 may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of hardware and software.

[0099] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the landing control method in the above-mentioned embodiments of the present application.

[0100] The computer-readable storage medium may be the internal memory of the UAV in the above-described embodiments, such as the UAV's hard drive or memory. Alternatively, the computer-readable storage medium may be an external storage device of the UAV, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, or a flash memory card.

[0101] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the unmanned aerial vehicle, etc.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A landing control method, applied to an unmanned aerial vehicle, characterized in that: The unmanned aerial vehicle is in communication with a control terminal, and the landing control method includes: During the return of the unmanned aerial vehicle, if the altitude of the unmanned aerial vehicle above the ground is less than or equal to a first altitude threshold, controlling a preset photographing device to photograph a predetermined landing position to obtain a first image; If the feature information in the first image does not include a landing pad mark, and if the altitude of the UAV above the ground is less than or equal to a second altitude threshold, controlling the preset camera to capture the predetermined landing location to obtain a second image, the second altitude threshold being less than the first altitude threshold; Determining, based on the second image, environmental characteristics of the area where the predetermined landing position is located, including: predicting the flatness of the area where the predetermined landing position is located based on a pixel position mapping relationship of pixel points in a plurality of second images, including: constructing a fitting model based on the pixel position mapping relationship; acquiring two consecutive frames of second images, the two consecutive frames of second images comprising a front image and a rear image; predicting, based on the fitting model, the relative positions of pixels of the front image in the rear image; calculating, based on the relative positions of pixels of the front image in the rear image and the positions of corresponding pixels in the rear image, a predicted distance between positions of pixels in the front image mapped to a coordinate system of the rear image and actual positions of corresponding pixels in the rear image; and determining the flatness of the area where the predetermined landing position is located based on the corresponding relationship between the predicted distance and the flatness, wherein the fitting model comprises a homography matrix model; If the environmental characteristics do not meet the preset landing requirements, generating a first prompt message, and sending the first prompt message to the control terminal, prompting the user to adjust the predetermined landing position; In response to the adjustment instruction for the predetermined landing position, the adjusted landing position is used as the target landing position, and the UAV is controlled to land at the target landing position.

2. The landing control method according to claim 1, wherein: The method further comprises: If the characteristic information includes the apron identifier, generating second prompt information according to the apron identifier, and sending the second prompt information to the control terminal to prompt the user to select whether to land at the apron corresponding to the apron identifier; In response to a control instruction to land on the helipad, the unmanned aerial vehicle is controlled to land on the helipad.

3. The landing control method according to claim 1, wherein: The determining, based on the second image, environmental characteristics of the area where the predetermined landing position is located includes: Identifying features of objects in the second image using a preset classification model; The object features and the flatness are used as the environment features.

4. The landing control method according to claim 3, wherein: The method further comprises: If the object characteristics include preset characteristics, it is determined that the environmental characteristics do not meet the preset landing requirements, and the preset characteristics include characteristics of the object that affects the landing safety of the unmanned aerial vehicle.

5. The landing control method according to claim 3, wherein: The method further comprises: If the flatness is less than a preset flatness threshold, it is determined that the environmental characteristics do not meet the preset landing requirements.

6. The landing control method according to claim 1, wherein: The unmanned aerial vehicle includes a gimbal, the gimbal is connected to the preset shooting device, and controlling the preset shooting device to shoot a first image of a predetermined landing position includes: By adjusting the posture of the pan / tilt platform, the posture of the preset shooting device is adjusted so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is acquired; and / or By adjusting the posture of the unmanned aerial vehicle, the posture of the preset shooting device is adjusted so that the shooting direction of the preset shooting device is directed toward the predetermined landing position and the first image shot by the preset shooting device is acquired.

7. The landing control method according to claim 1, wherein: The method further comprises: If the environmental characteristics do not meet the preset landing requirements, obtaining a video image of the area where the predetermined landing location is located; The first prompt information is generated according to the image transmission image and the environmental characteristics.

8. An aircraft control system, characterized in that: include: An unmanned aerial vehicle, the unmanned aerial vehicle being configured to execute the landing control method according to any one of claims 1 to 7; A control terminal is communicatively connected to the unmanned aerial vehicle, and the control terminal includes a display device, and the display device is used to display the first prompt information sent by the unmanned aerial vehicle.

9. The aircraft control system according to claim 8, wherein: The first prompt information includes a picture transmission screen, wherein a landing point icon corresponding to a predetermined landing position is marked on the picture transmission screen. The control terminal further includes a joystick, and the control terminal is used to: In response to a user's operation of adjusting the predetermined landing position through the joystick, or in response to a user's operation of moving the landing point icon, the position of the landing point icon in the image transmission screen is updated, and an adjustment instruction carrying the updated position of the landing point icon is generated, and the adjustment instruction is sent to the unmanned aerial vehicle, so that the unmanned aerial vehicle determines the target landing position according to the updated position of the landing point icon.

Images