Landing control method and aircraft control system

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and provides a landing control method and an aircraft control system.The method comprises the steps that in the return flight process of an unmanned aerial vehicle, if the height to the ground is smaller than or equal to a first height threshold value, a preset landing position is shot to obtain a first image; under the condition that the feature information in the first image does not contain the parking apron identifier, if the height to the ground is smaller than or equal to a second height threshold value, shooting a preset landing position to obtain a second image; based on the second image, determining environment characteristics of an area where the predetermined landing position is located; if the environment feature does not meet the preset landing requirement, generating first prompt information, and sending the first prompt information to a control terminal to prompt to adjust a preset landing position; and responding to an adjustment instruction for a predetermined landing position, taking the adjusted landing position as a target landing position, and controlling the unmanned aerial vehicle to land to the target landing position. By using the method, the landing safety of the unmanned aerial vehicle can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to a landing control method and an aircraft control system. Background Art

[0002] During the return process of the UAV, the position information of the UAV is usually located through the Global Positioning System (GPS), and the distance and direction between the UAV and the return point are calculated. The UAV generates a flight path based on the distance and direction between the UAV and the return point and lands to the return point along the flight path.

[0003] However, due to the certain error in GPS positioning accuracy, the actual landing position of the UAV may differ from the predetermined return point by tens of centimeters to several meters. If the actual landing position is unknown, it may have an adverse impact on the landing safety of the UAV and the safety of people and objects in the landing area of ​​the UAV. Summary of the invention

[0004] In view of this, the present application provides a landing control method and an aircraft control system to solve the problem that the landing control method of the unmanned aerial vehicle has errors, resulting in the actual landing position being unknown, which may have an adverse impact on the landing safety of the unmanned aerial vehicle and the safety of people and objects in the landing area of ​​the unmanned aerial vehicle.

[0005] A first aspect of an embodiment of the present application provides a landing control method, which is applied to an unmanned aerial vehicle, wherein the unmanned aerial vehicle is communicatively connected to a control terminal, and the landing control method includes: during the return process of the unmanned aerial vehicle, if the height of the unmanned aerial vehicle above the ground is less than or equal to a first height threshold, controlling a preset shooting device to shoot a predetermined landing position to obtain a first image; if the feature information in the first image does not include a helipad mark, when the height of the unmanned aerial vehicle above the ground is less than or equal to a second height threshold, controlling the preset shooting device to shoot the predetermined landing position to obtain a second image, and the second height threshold is less than the first height threshold; based on the second image, determining the environmental characteristics of the area where the predetermined landing position is located; if the environmental characteristics do not meet the preset landing requirements, generating a first prompt message, and sending the first prompt message to the control terminal to prompt a user to adjust the predetermined landing position; in response to an adjustment instruction for the predetermined landing position, using the adjusted landing position as a target landing position, and controlling the unmanned aerial vehicle to land at the target landing position.

[0006] In some embodiments, if the characteristic information includes the apron identification, a second prompt message is generated according to the apron identification, and the second prompt message is sent to the control terminal, prompting the user to select whether to land at the apron corresponding to the apron identification; in response to a control command to land at the apron, the unmanned aerial vehicle is controlled to land at the apron.

[0007] In some embodiments, determining the environmental features of the area where the predetermined landing position is located based on the second image includes: using a preset classification model to identify object features in the second image; predicting the flatness of the area where the predetermined landing position is located based on a pixel position mapping relationship between pixel points in multiple second images; and using the object features and the flatness as the environmental features.

[0008] In some embodiments, predicting the flatness of the area where the predetermined landing position is located based on the pixel position mapping relationship of pixel points in multiple second images includes: constructing a fitting model based on the pixel position mapping relationship; acquiring two consecutive frames of second images, the two consecutive frames of second images include a front image and a rear image; predicting the relative position of the pixel points of the front image in the rear image based on the fitting model; calculating the flatness based on the relative position of the pixel points of the front image in the rear image and the position of the corresponding pixel points in the rear image.

[0009] In some embodiments, the method further includes: if the object characteristics include preset characteristics, determining that the environmental characteristics do not meet the preset landing requirements, and the preset characteristics include characteristics of the object that affect the landing safety of the unmanned aerial vehicle.

[0010] In some embodiments, the method further includes: if the flatness is less than a preset flatness threshold, determining that the environmental characteristics do not meet the preset landing requirements.

[0011] In some embodiments, the unmanned aerial vehicle includes a gimbal, which is connected to the preset shooting device, and controlling the preset shooting device to shoot a first image of a predetermined landing position includes: adjusting the posture of the preset shooting device by adjusting the posture of the gimbal, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained; and / or adjusting the posture of the preset shooting device by adjusting the posture of the unmanned aerial vehicle, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained.

[0012] In some embodiments, if the environmental characteristics do not meet the preset landing requirements, a video image of the area where the predetermined landing position is located is obtained; and the first prompt information is generated based on the video image and the environmental characteristics.

[0013] A second aspect of an embodiment of the present application provides an aircraft control system, comprising: an unmanned aerial vehicle, the unmanned aerial vehicle being used to execute the above-mentioned landing control method; a control terminal, the control terminal being communicatively connected to the unmanned aerial vehicle, the control terminal comprising a display device, the display device being used to display a first prompt information sent by the unmanned aerial vehicle.

[0014] In some embodiments, the first prompt information includes the image transmission screen, in which a landing point icon corresponding to a predetermined landing position is marked, and the control terminal also includes a joystick, and the control terminal is used to: respond to a user's operation of adjusting the predetermined landing position through the joystick, or respond to a user's operation of moving the landing point icon, update the position of the landing point icon in the image transmission screen, generate an adjustment instruction carrying the position of the updated landing point icon, and send the adjusted instruction to the unmanned aerial vehicle, so that the unmanned aerial vehicle determines the target landing position according to the position of the updated landing point icon.

[0015] A third aspect of an embodiment of the present application provides an unmanned aerial vehicle, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned landing control method when executing the computer-readable instructions.

[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions implement the above-mentioned landing control method when executed by a processor.

[0017] In a landing control method provided by an embodiment of the present application, during the return process of the unmanned aerial vehicle, the predetermined landing position is photographed by controlling the preset shooting device multiple times to obtain an image containing the surrounding environment of the predetermined landing position. When it is determined that the image does not contain the apron mark, the environmental characteristics of the area where the predetermined landing position is located are analyzed according to the photographed image. If the environmental characteristics do not meet the preset landing requirements, a first prompt message is generated, and the first prompt message is sent to the control terminal to prompt the user to adjust the predetermined landing position. In response to the adjustment instruction for the predetermined landing position, the adjusted landing position is used as the target landing position, and the unmanned aerial vehicle is controlled to land at the target landing position. By detecting the environmental characteristics around the predetermined landing position during the return process of the unmanned aerial vehicle, and prompting the user to adjust the predetermined landing position when the environmental characteristics do not meet the preset landing requirements, the probability of the unmanned aerial vehicle landing in a dangerous area can be effectively reduced, thereby improving the landing safety of the unmanned aerial vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 It is a schematic diagram of the structure of an aircraft control system provided in an embodiment of the present application.

[0020] Figure 2 It is a flow chart for implementing the landing control method provided in an embodiment of the present application.

[0021] Figure 3 This is an example diagram of the interface of the image transmission screen provided in an embodiment of the present application.

[0022] Figure 4 It is a schematic diagram of the structure of the landing control device provided in an embodiment of the present application.

[0023] Figure 5 It is a schematic diagram of the structure of the unmanned aerial vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as suggesting or implying relative importance or implicitly indicating the number of suggested technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field in this application. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can represent A or B. "And / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more. "Multiple" means two or more than two. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c seven situations.

[0027] See also Figure 1 As shown, it is a schematic diagram of the structure of the aircraft control system provided in the embodiment of the present application, such as Figure 1 As shown, the aircraft control system 1000 includes an unmanned aerial vehicle 100 and a control terminal 200, and the unmanned aerial vehicle 100 is in communication with the control terminal 200. The landing control method provided in the embodiment of the present application can be applied to the unmanned aerial vehicle 100 or the control device in the unmanned aerial vehicle 100. The control terminal 200 can be used to receive data sent by the unmanned aerial vehicle 100, such as video, pictures, sensor data, and status information of the unmanned aerial vehicle 100.

[0028] In the process of implementing landing control, the unmanned aerial vehicle 100 performing the return mission detects the height above the ground. If the height above the ground of the unmanned aerial vehicle 100 is less than or equal to the first height threshold, the preset shooting device is controlled to shoot the predetermined landing position to obtain a first image. The unmanned aerial vehicle 100 analyzes the characteristic information in the first image. In the case where the characteristic information in the first image does not include the apron mark, if the unmanned aerial vehicle 100 detects that the height above the ground of the unmanned aerial vehicle 100 is less than or equal to the second height threshold, the preset shooting device is controlled to shoot the predetermined landing position to obtain a second image. Among them, the second height threshold is less than the first height threshold. Based on the second image, the unmanned aerial vehicle 100 determines the environmental characteristics of the area where the predetermined landing position is located. If the environmental characteristics do not meet the preset landing requirements, the unmanned aerial vehicle 100 generates a first prompt information based on the second image and the environmental characteristics, and sends the first prompt information to the control terminal 200 to prompt the user to adjust the predetermined landing position.

[0029] The control terminal 200 receives the first prompt information and displays the first prompt information on the display device. The user adjusts the predetermined landing position according to the first prompt information, and the control terminal 200 generates an adjustment instruction in response to the user's adjustment operation, and sends the adjustment instruction to the unmanned aerial vehicle 100. The unmanned aerial vehicle 100 responds to the adjustment instruction for the predetermined landing position, uses the adjusted landing position as the target landing position, and controls the unmanned aerial vehicle 100 to land at the target landing position. The landing control method provided in the embodiment of the present application, if the apron is not detected in the area where the predetermined landing position is located, detects whether the environmental characteristics of the area where the predetermined landing position is located meet the landing requirements. If the environmental characteristics do not meet the preset landing requirements, a prompt information is generated and sent to the on-board control terminal to prompt the user to adjust the predetermined landing position, thereby effectively reducing the probability of the unmanned aerial vehicle landing in a dangerous area and improving the landing safety of the unmanned aerial vehicle.

[0030] In some embodiments, the first prompt information includes a picture of the area where the predetermined landing position is located, and a landing point icon corresponding to the predetermined landing position is marked in the picture. The user can adjust the predetermined landing position by manipulating the joystick of the control terminal, or move the landing point icon (for example, drag the landing point icon, or select other positions as the position of the landing point icon, etc.) to adjust the position of the landing point icon, thereby adjusting the predetermined landing position. The control terminal responds to the user's operation of adjusting the predetermined landing position through the joystick, or responds to the user's operation of moving the landing point icon, updates the position of the landing point icon in the picture, generates an adjustment instruction carrying the updated position of the landing point icon, and sends the adjusted instruction to the unmanned aerial vehicle, so that the unmanned aerial vehicle determines the target landing position according to the updated position of the landing point icon.

[0031] In some embodiments, the unmanned aerial vehicle 100 may be a multi-rotor drone, an unmanned aircraft, etc. The control terminal 200 may be a remote controller, or a user terminal, such as any one or more of a mobile phone, a tablet computer, a wearable device, etc. The embodiments of the present application do not limit the specific device types of the unmanned aerial vehicle 100 and the control terminal 200.

[0032] In some embodiments, the communication connection method includes but is not limited to wireless communication connection. For example, the wireless channel from the unmanned aerial vehicle 100 to the control terminal 200 can be used to transmit data collected by the unmanned aerial vehicle 100, such as videos, pictures, sensor data, and status information of the unmanned aerial vehicle 100. The wireless channel from the control terminal 200 to the unmanned aerial vehicle 100 can be used to transmit remote control data, such as flight control instructions and control instructions such as return instructions, photo instructions, and video recording instructions.

[0033] Figure 1 The illustrated scenarios are merely illustrative examples, and the return control method provided in the present application may also be applied in other scenarios. The embodiments of the present application do not limit the specific application scenarios of the return control method.

[0034] See also Figure 2 The figure is a flowchart of the implementation of the landing control method provided in the embodiment of the present application. The method is applied to an unmanned aerial vehicle. The embodiment of the present application is applied to the method in Figure 1 The method is described by taking the unmanned aerial vehicle 100 in FIG.

[0035] S11: During the return process of the unmanned aerial vehicle, if the height of the unmanned aerial vehicle above the ground is less than or equal to a first height threshold, control a preset shooting device to shoot a predetermined landing position to obtain a first image.

[0036] In some embodiments, the predetermined landing location represents a preset location or area where the UAV will land. The predetermined landing location can be determined based on the location of the take-off point of the UAV or can be specified by the user, or can be a location determined during autonomous return, such as low battery return. The embodiments of the present application do not limit the method for determining the predetermined landing location.

[0037] In some embodiments, the first height threshold can be customized, for example, it can be set to 10 meters, 9 meters, etc. The first height threshold is not limited in the embodiment of the present application.

[0038] In some embodiments, the first image represents an image of an area where the predetermined landing position is located, and is used to indicate environmental conditions of the area where the predetermined landing position is located.

[0039] In some embodiments, when it is determined that the altitude of the unmanned aerial vehicle above the ground is less than or equal to a first altitude threshold, the unmanned aerial vehicle controls a preset shooting device to shoot a predetermined landing position to obtain a first image to ensure that the captured first image is relatively clear, so as to improve the accuracy of subsequent identification of apron signs based on the first image.

[0040] In some embodiments, the method for detecting the height of the UAV above the ground can be selected according to the specific application scenario, accuracy requirements, and cost budget of the UAV. For example, the height of the UAV above the ground can be detected by any one or more technologies such as sensor technology (such as a height sensor above the ground), radar technology, and global positioning system (GPS) technology. The specific detection method is not limited in the embodiments of the present application.

[0041] In some embodiments of the present application, the unmanned aerial vehicle includes a gimbal, which is connected to a preset shooting device, and controls the preset shooting device to shoot a first image of a predetermined landing position, including: adjusting the posture of the preset shooting device by adjusting the posture of the gimbal, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained; and / or adjusting the posture of the preset shooting device by adjusting the posture of the unmanned aerial vehicle, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained.

[0042] In some embodiments, the preset shooting device may be one or more of a camera, a video camera, etc. The present application embodiment is not limited thereto.

[0043] In some embodiments, the gimbal is connected to a preset shooting device to stabilize and support the preset shooting device. The gimbal is adjustable. The unmanned aerial vehicle can adjust the posture (such as angle and direction) of the gimbal through a built-in control device. This adjustment can be horizontal, vertical, or tilted.

[0044] In some embodiments, the unmanned aerial vehicle can adjust the shooting direction of the preset shooting device by adjusting the posture of the gimbal, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image shot by the preset shooting device is obtained.

[0045] For example, when returning, the UAV usually flies to a designated location (such as a predetermined landing location) according to a preset flight path, and then lands vertically to ensure the safe landing of the UAV. When the UAV flies to the bottom of the predetermined landing location, the UAV can adjust the posture of the gimbal so that the preset shooting device shoots the predetermined landing location vertically downward.

[0046] In other embodiments, the UAV may also adjust the posture of the preset camera device by adjusting the posture of the UAV, so that the shooting direction of the preset camera device is toward the predetermined landing position and the first image captured by the preset camera device is obtained.

[0047] In other embodiments, the unmanned aerial vehicle can also adjust the posture of the preset shooting device by adjusting the posture of the gimbal and the posture of the unmanned aerial vehicle, so that the shooting direction of the preset shooting device is toward the predetermined landing position and the first image taken by the preset shooting device is obtained.

[0048] S12: If the feature information in the first image does not include the apron mark, when the height of the UAV above the ground is less than or equal to a second height threshold, control the preset shooting device to shoot the predetermined landing position to obtain a second image, wherein the second height threshold is less than the first height threshold.

[0049] In some embodiments, the feature information may include geometric features (e.g., outlines, edge outlines, pixel values), topological features (e.g., surface features, moment features), and landmark information (e.g., specific patterns, building features, and terrain features) of objects in the area where the predetermined landing location is located. Objects include, but are not limited to, elements in nature (e.g., trees, vegetation, rocks, and water bodies), as well as various types of artificially constructed or set signs (e.g., buildings, landmark signs, etc.).

[0050] In some embodiments, the unmanned aerial vehicle may obtain feature information by preprocessing, feature extraction, feature matching, etc. of the first image. Preprocessing may be image denoising, image enhancement, image correction, etc. of the first image. The unmanned aerial vehicle may extract features by one or more of local features, deep learning, optical flow, etc. The unmanned aerial vehicle may match features by one or more of similarity measurement, machine learning, geometric constraint relationships, etc.

[0051] In some embodiments, the apron sign can be customized to identify the apron. For example, the apron sign can be set to an "H" sign. This is not limited in the embodiments of the present application.

[0052] In some embodiments, if the feature information in the first image does not include the apron mark, the UAV continues to land, and when the height of the UAV above the ground is less than or equal to the second height threshold, the UAV is relatively close to the predetermined landing position. In this case, the UAV controls the preset shooting device to shoot the predetermined landing position to obtain the second image.

[0053] In some embodiments, the UAV can more accurately obtain the features of objects in the area where the predetermined landing position is located by analyzing the second image, so as to more accurately determine whether the area where the predetermined landing position is located is suitable for landing.

[0054] In some embodiments of the present application, if the characteristic information includes an apron identification, a second prompt message is generated based on the apron identification, and the second prompt message is sent to the control terminal, prompting the user to select whether to land at the apron corresponding to the apron identification; in response to a control command to land at the apron, the unmanned aerial vehicle is controlled to land at the apron.

[0055] In some embodiments, if the characteristic information includes a helipad mark, it means that a helipad is provided in the area where the predetermined landing position is located. At this time, the unmanned aerial vehicle can generate a second prompt information based on the helipad mark, and send the second prompt information to the control terminal. The control terminal receives the second prompt information and can display the second prompt information on the display device. For example, the second prompt information can be "There is a helipad with an apron mark of "H" in the area where the predetermined landing position is located. Please confirm whether to choose the helipad landing." If the user chooses to land on the helipad, the control terminal generates a control instruction for landing on the helipad and sends the control instruction to the unmanned aerial vehicle. In response to the control instruction for landing on the helipad, the unmanned aerial vehicle adjusts the predetermined landing position, uses the location of the helipad as the target landing position, and controls the unmanned aerial vehicle to land on the helipad.

[0056] S13: Based on the second image, determine the environmental characteristics of the area where the predetermined landing position is located.

[0057] In some embodiments, environmental features include, but are not limited to, geometric features of objects in the area where the predetermined landing location is located (e.g., the object's outer contour, edge contour, pixel value), topological features (e.g., the object's texture features, moment features), and landmark information (e.g., specific patterns, building features, terrain features).

[0058] In some embodiments, the unmanned aerial vehicle may obtain feature information by preprocessing, feature extraction, feature matching, etc. of the second image. Preprocessing may be image denoising, image enhancement, image correction, etc. of the second image. The unmanned aerial vehicle may extract features by one or more of local features, deep learning, optical flow, etc. The unmanned aerial vehicle may match features by one or more of similarity measurement, machine learning, geometric constraint relationships, etc.

[0059] In some embodiments of the present application, environmental features of the area where the predetermined landing position is located are determined based on the second image, including: using a preset classification model to identify object features in the second image; predicting the flatness of the area where the predetermined landing position is located based on a pixel position mapping relationship between pixel points in multiple second images; and using the object features and the flatness as environmental features.

[0060] In some embodiments, the object features include but are not limited to the category, location, shape, size, etc. of the object. Flatness indicates the flatness of the ground in the area where the predetermined landing position is located.

[0061] In some embodiments, the preset classification model is a deep learning model, for example, a convolutional neural network model, a support vector machine, a random forest, etc.

[0062] In some embodiments, the unmanned aerial vehicle can train the preset classification model in the following manner: the unmanned aerial vehicle obtains multiple second images as training samples, and preprocesses the collected sample images, such as image denoising, enhancement, geometric correction, and distortion correction, etc., to improve image quality and recognition accuracy. Then, the unmanned aerial vehicle extracts features from the preprocessed sample images, such as color feature extraction, texture feature extraction, shape feature extraction, etc. After extracting multiple features, the unmanned aerial vehicle performs feature selection and fusion to improve recognition accuracy and efficiency. The unmanned aerial vehicle selects and fuses the features to train the initial classification model to obtain a preset classification model.

[0063] In the process of using the preset classification model to identify the features of the object in the second image, the unmanned aerial vehicle inputs the second image into the preset classification model, and identifies the features of the object in the second image through comparison and classification.

[0064] In some embodiments of the present application, based on the pixel position mapping relationship of pixel points in multiple second images, the flatness of the area where the predetermined landing position is located is predicted, including: constructing a fitting model based on the pixel position mapping relationship; acquiring two consecutive frames of second images, the two consecutive frames of second images include a front image and a rear image; based on the fitting model, predicting the relative position of the pixel points in the front image in the rear image; based on the relative position of the pixel points in the front image in the rear image and the position of the corresponding pixel points in the rear image, calculating the flatness.

[0065] In some embodiments, the unmanned aerial vehicle can obtain multiple pairs of two consecutive frames of second images, use a dense optical flow algorithm to calculate the position change of each pixel in the two consecutive frames of second images, and construct a pixel position mapping relationship by analyzing the position change of each pixel in the two consecutive frames of second images.

[0066] In some embodiments, the unmanned aerial vehicle constructs a fitting model by performing statistical analysis on the pixel position mapping relationship, such as using the least squares method for analysis. For example, the fitting model may be a homography matrix model. The homography matrix model is used to describe the pixel position mapping relationship between two consecutive frames of the second image. In the process of calculating the flatness of the area where the predetermined landing position is located, the unmanned aerial vehicle may obtain two consecutive frames of the second image, the two consecutive frames of the second image being included in the front image and the rear image, and transform the pixel points in the front image into the coordinate system of the rear image through the homography matrix model. The unmanned aerial vehicle predicts the flatness by calculating the predicted distance between the position of the pixel point in the front image mapped to the coordinate system of the rear image and the position of the actual corresponding pixel point in the rear frame image.

[0067] In some embodiments, the UAV may pre-establish a corresponding relationship between the predicted distance and the flatness, where the smaller the predicted distance, the greater the flatness, and the larger the predicted distance, the smaller the flatness. After calculating the predicted distance, the UAV may determine the flatness of the ground in the area where the predetermined landing position is located based on the corresponding relationship between the predicted distance and the flatness.

[0068] In some embodiments, if the flatness is less than a preset flatness threshold, it means that the ground in the area where the predetermined landing position is located may have large undulations or irregular shapes, which is not suitable for the UAV to land. In this case, the UAV determines that the environmental characteristics do not meet the preset landing requirements.

[0069] In some embodiments, by identifying the flatness of the area where the object feature in the second image is located, it can be determined whether there are objects that affect the landing safety of the unmanned aerial vehicle in the area where the predetermined landing position is located, or there are other objects that are not suitable for the unmanned aerial vehicle to land. By predicting the flatness of the ground in the area where the predetermined landing position is located, it can be determined whether the ground in the area where the predetermined landing position is located is suitable for landing.

[0070] In some embodiments of the present application, if the object characteristics include preset characteristics, it is determined that the environmental characteristics do not meet the preset landing requirements, and the preset characteristics include characteristics of the object that affect the landing safety of the unmanned aerial vehicle.

[0071] In some embodiments, the preset features can be customized, for example, the preset features can include features such as people, vehicles, animals, grass, and water bodies.

[0072] In some embodiments, if the object feature includes a preset feature, it means that the area where the predetermined landing position is located has the feature of the object that may affect the landing safety of the unmanned aerial vehicle. If the unmanned aerial vehicle is controlled to land at the predetermined landing position, it may be detrimental to the safety of the unmanned aerial vehicle and the safety of people and objects in the area where the predetermined landing position is located. Therefore, if the object feature includes a preset feature, the unmanned aerial vehicle determines that the environmental feature does not meet the preset landing requirements, and may prompt to adjust the predetermined landing position.

[0073] In some embodiments of the present application, if the flatness is less than a preset flatness threshold, it is determined that the environmental characteristics do not meet the preset landing requirements.

[0074] S14: If the environmental characteristics do not meet the preset landing requirements, a first prompt message is generated and sent to the control terminal to prompt the user to adjust the predetermined landing position.

[0075] In some embodiments, the preset landing requirements include but are not limited to the flatness of the ground in the area where the predetermined landing position is located being less than a preset flatness threshold, and the object features in the area do not contain preset features.

[0076] In some embodiments, if the environmental characteristics do not meet the preset landing requirements, an image transmission image of the area where the predetermined landing position is located is obtained, and a first prompt information is generated based on the image transmission image and the environmental characteristics.

[0077] In some embodiments, the image transmission picture represents the real-time picture of the area where the predetermined landing position is located that is captured by the unmanned aerial vehicle and sent to the display device of the control terminal through the image transmission system (image transmission system). The image transmission picture may include image information captured by the preset camera device, and may also include some additional data, such as flight parameters, battery power, signal strength, etc.

[0078] In some embodiments, the unmanned aerial vehicle can mark the predetermined landing position in the image transmission screen according to the coordinate information of the predetermined landing position, and determine the representation of the predetermined landing position in the image transmission screen as a landing point icon.

[0079] In some embodiments, the control terminal receives the first prompt information and parses the first prompt information to obtain the image transmission picture and environmental features of the area where the predetermined landing position is located, and displays the image transmission picture and information related to the environmental features on a display device of the control terminal.

[0080] In some embodiments, the predetermined landing position is adjusted by adjusting the position of the landing point icon in the image transmission screen. The user can adjust the position of the landing point icon in the image transmission screen according to the environmental characteristics of the area where the predetermined landing position is located, thereby reducing the situation where the adjusted landing position falls into a dangerous area and improving the safety of the UAV landing. Figure 3 As shown, the image transmission screen 20 includes an icon 21 corresponding to the unmanned aerial vehicle, an icon 22 corresponding to the control terminal, and a landing point icon 23 corresponding to the predetermined landing position. The user can adjust the predetermined landing position by manipulating the joystick of the control terminal, or move the landing point icon 23, such as dragging the landing point icon 23, or selecting other positions as the position of the landing point icon 23, etc., to adjust the position of the landing point icon 23, thereby adjusting the predetermined landing position. The control terminal responds to the user's operation of adjusting the predetermined landing position through the joystick, or responds to the user's operation of moving the landing point icon 23, updates the position of the landing point icon 23 in the image transmission screen, generates an adjustment instruction carrying the updated position of the landing point icon, and sends the adjustment instruction to the unmanned aerial vehicle.

[0081] S15: Responding to the adjustment instruction for the predetermined landing position, taking the adjusted landing position as the target landing position, and controlling the unmanned aerial vehicle to land at the target landing position.

[0082] In some embodiments, the target landing location represents a location or area where the UAV determines it will land.

[0083] In a landing control method provided by an embodiment of the present application, during the return process of the unmanned aerial vehicle, the predetermined landing position is photographed by controlling the preset shooting device multiple times to obtain an image containing the surrounding environment of the predetermined landing position. When it is determined that the image does not contain the apron mark, the environmental characteristics of the area where the predetermined landing position is located are analyzed according to the photographed image. If the environmental characteristics do not meet the preset landing requirements, a first prompt message is generated, and the first prompt message is sent to the control terminal to prompt the user to adjust the predetermined landing position. In response to the adjustment instruction for the predetermined landing position, the adjusted landing position is used as the target landing position, and the unmanned aerial vehicle is controlled to land at the target landing position. By detecting the environmental characteristics around the predetermined landing position during the return process of the unmanned aerial vehicle, and prompting the user to adjust the predetermined landing position when the environmental characteristics do not meet the preset landing requirements, the probability of the unmanned aerial vehicle landing in a dangerous area can be effectively reduced, thereby improving the landing safety of the unmanned aerial vehicle.

[0084] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0085] See also Figure 4 , is a structural diagram of a landing control device provided in an embodiment of the present application, which can implement the details of the landing control method in the above embodiment and achieve the same effect. Figure 4 As shown, the landing control device 10 can be applied to an unmanned aerial vehicle with a data processing function, and the landing control device 10 includes: a first shooting module 11, which is used to control a preset shooting device to shoot a predetermined landing position to obtain a first image if the height of the unmanned aerial vehicle above the ground is less than or equal to a first height threshold during the return process of the unmanned aerial vehicle; a second shooting module 12, which is used to control a preset shooting device to shoot a predetermined landing position to obtain a second image if the feature information in the first image does not contain the apron mark and the height of the unmanned aerial vehicle above the ground is less than or equal to a second height threshold, and the second height threshold is less than the first height threshold; a feature determination module 13, which is used to determine the environmental features of the area where the predetermined landing position is located based on the second image; an information prompt module 14, which is used to generate a first prompt message if the environmental features do not meet the preset landing requirements, and send the first prompt message to the control terminal to prompt the user to adjust the predetermined landing position; a landing control module 15, which is used to respond to the adjustment instruction of the predetermined landing position, use the adjusted landing position as the target landing position, and control the unmanned aerial vehicle to land at the target landing position.

[0086] For the specific definition of the landing control device 10, please refer to the definition of the landing control method above, which will not be repeated here. Each module in the above landing control device 10 can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the unmanned aerial vehicle in the form of hardware, or can be stored in the memory in the unmanned aerial vehicle in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0087] See also Figure 5 , Figure 5 The figure shows a schematic diagram of the structure of an unmanned aerial vehicle provided in an embodiment of the present application. The unmanned aerial vehicle 100 includes but is not limited to any one of a multi-rotor aircraft, an unmanned aerial vehicle, etc. The network where the unmanned aerial vehicle 100 is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0088] like Figure 5 As shown, the UAV 100 includes a communication module 101, a memory 102, a processor 103, an input / output interface 104 and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102 and the input / output interface 104 through the bus 105. The communication module 101 can be a wireless communication module or a mobile communication module. The wireless communication module can provide wireless communication solutions including wireless local area networks (WLAN) (for example, wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., which are applied to the unmanned aerial vehicle 100. The mobile communication module can provide wireless communication solutions including 2G / 3G / 4G / 5G, etc., which are applied to the unmanned aerial vehicle 100. The memory 102 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 103, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data. The random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally called DDR5 SDRAM), etc. The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 103. The non-volatile memory may include a disk storage device and a flash memory. The memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include a plurality of instructions, and when the plurality of instructions are executed by the processor 103, a landing control method executed on the unmanned aerial vehicle 100 can be implemented. In other embodiments, the UAV 100 further includes an external memory interface for connecting to an external memory to expand the storage capacity of the UAV 100 . The processor 103 may include one or more processing units, for example, the processor 103 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute a computer program stored in the memory 102 to implement the above-mentioned landing control method. The input / output interface 104 is used to provide a channel for user input or output. For example, the input / output interface 104 can be used to connect various input and output devices, such as a mouse, keyboard, touch device, display screen, etc., so that the user can enter information or visualize information. The bus 105 is at least used to provide a channel for mutual communication among the communication module 101 , the memory 102 , the processor 103 , and the input / output interface 104 in the UAV 100 . It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the unmanned aerial vehicle 100. In other embodiments of the present application, the unmanned aerial vehicle 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0089] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the landing control method in the above-mentioned embodiments of the present application. The computer-readable storage medium may be an internal memory of the UAV of the above-mentioned embodiment, such as a hard disk or memory of the UAV. The computer-readable storage medium may also be an external storage device of the UAV, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the UAV. Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the unmanned aerial vehicle, etc.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A landing control method, applied to an unmanned aerial vehicle, characterized in that: The unmanned aerial vehicle is in communication connection with a control terminal, and the landing control method comprises: During the return process of the unmanned aerial vehicle, if the height of the unmanned aerial vehicle above the ground is less than or equal to a first height threshold, controlling a preset shooting device to shoot a predetermined landing position to obtain a first image; If the feature information in the first image does not include the apron mark, when the height of the unmanned aerial vehicle above the ground is less than or equal to a second height threshold, controlling the preset shooting device to shoot the predetermined landing position to obtain a second image, and the second height threshold is less than the first height threshold; determining, based on the second image, environmental characteristics of an area where the predetermined landing position is located; If the environmental characteristics do not meet the preset landing requirements, generating a first prompt message, and sending the first prompt message to the control terminal to prompt the user to adjust the predetermined landing position; In response to the adjustment instruction for the predetermined landing position, the adjusted landing position is used as the target landing position, and the unmanned aerial vehicle is controlled to land at the target landing position.

2. The landing control method according to claim 1, characterized in that: The method further comprises: If the characteristic information includes the apron mark, generating second prompt information according to the apron mark, and sending the second prompt information to the control terminal to prompt the user to select whether to land at the apron corresponding to the apron mark; In response to a control instruction to land on the helipad, the unmanned aerial vehicle is controlled to land on the helipad.

3. The landing control method according to claim 1, characterized in that: The determining, based on the second image, environmental features of the area where the predetermined landing position is located, includes: Identifying features of an object in the second image using a preset classification model; Predicting the flatness of the area where the predetermined landing position is located based on a pixel position mapping relationship of pixel points in the plurality of second images; The object features and the flatness are used as the environment features.

4. The landing control method according to claim 3, characterized in that: The predicting the flatness of the area where the predetermined landing position is located based on the pixel position mapping relationship of the pixel points in the plurality of second images includes: Based on the pixel position mapping relationship, construct a fitting model; Acquire two consecutive frames of second images, wherein the two consecutive frames of second images include a front image and a rear image; Based on the fitting model, predicting the relative position of the pixel points of the previous image in the subsequent image; The flatness is calculated based on the relative positions of the pixels of the preceding image in the succeeding image and the positions of the corresponding pixels in the succeeding image.

5. The landing control method according to claim 3, characterized in that: The method further comprises: If the object feature includes a preset feature, it is determined that the environmental feature does not meet the preset landing requirement, and the preset feature includes a feature of an object that affects the landing safety of the unmanned aerial vehicle.

6. The landing control method according to claim 3, characterized in that: The method further comprises: If the flatness is less than a preset flatness threshold, it is determined that the environmental characteristics do not meet the preset landing requirements.

7. The landing control method according to claim 1, characterized in that: The unmanned aerial vehicle includes a gimbal, the gimbal is connected to the preset shooting device, and the controlling the preset shooting device to shoot a first image of a predetermined landing position includes: By adjusting the posture of the gimbal, the posture of the preset shooting device is adjusted so that the shooting direction of the preset shooting device faces the predetermined landing position and the first image shot by the preset shooting device is acquired; and / or By adjusting the posture of the unmanned aerial vehicle, the posture of the preset shooting device is adjusted so that the shooting direction of the preset shooting device is directed toward the predetermined landing position and the first image shot by the preset shooting device is acquired.

8. The landing control method according to claim 1, characterized in that: The method further comprises: If the environmental characteristics do not meet the preset landing requirements, obtaining a video transmission image of the area where the predetermined landing position is located; The first prompt information is generated according to the image transmission picture and the environmental characteristics.

9. An aircraft control system, characterized in that: include: An unmanned aerial vehicle, the unmanned aerial vehicle being used to perform the landing control method according to any one of claims 1 to 8; A control terminal is communicatively connected to the unmanned aerial vehicle, and the control terminal includes a display device, and the display device is used to display the first prompt information sent by the unmanned aerial vehicle.

10. The aircraft control system according to claim 9, characterized in that: The first prompt information includes a picture transmission screen, and a landing point icon corresponding to a predetermined landing position is marked in the picture transmission screen. The control terminal also includes a joystick, and the control terminal is used to: In response to a user's operation of adjusting the predetermined landing position through the joystick, or in response to a user's operation of moving the landing point icon, the position of the landing point icon in the image transmission screen is updated, and an adjustment instruction carrying the updated position of the landing point icon is generated, and the adjustment instruction is sent to the unmanned aerial vehicle, so that the unmanned aerial vehicle determines the target landing position according to the updated position of the landing point icon.

Images