A PAPI flight calibration method and system based on drone

Abstract

The present invention relates to a PAPI flight verification method and system based on an unmanned aerial vehicle (UAV). The method comprises: generating flight mission data based on PAPI signal light equipment data; starting a UAV equipped with a camera to fly according to the flight mission data, receiving a video image of the PAPI signal light taken by the camera and the positioning data of the UAV from the UAV; calculating the azimuth and elevation data of the UAV relative to the ground PAPI signal light based on the received video image of the PAPI signal light and the positioning data of the UAV, identifying the position and light color of the PAPI signal light, and verifying the PAPI signal light. The embodiment of the present invention realizes the automatic generation and uploading of flight mission data based on PAPI signal light equipment data, enables the UAV to fly automatically during the flight verification process, improves the verification accuracy, simplifies the control difficulty of the UAV, and reduces safety risks.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned aerial vehicle (UAV) flight verification, and relates to a PAPI flight verification method and system based on an UAV. Background Art

[0002] The Precision Approach Path Indicator (PAPI) is installed on one side of the airport runway and consists of four independent signal lights that provide pilots with information for controlling the aircraft's approach altitude. The lampshade of the Precision Approach Path Indicator uses a Fresnel lens, allowing pilots to observe the light from a distance. The working principle of the Precision Approach Path Indicator (PAPI) is to indicate the aircraft's altitude during the approach by the color of the light. If two red and two white lights are displayed (from left to right), it means that the aircraft is at the correct altitude during the approach and its descent angle is 3°; if one white and three red lights are displayed, it means that the aircraft's descent angle is 2.8°, which is lower than the normal descent angle; if four red lights are displayed, it means that the aircraft's descent angle is already lower than 2.5°, which is significantly lower than the normal descent angle; if four white lights are displayed, it means that the aircraft's descent angle is already higher than 3.5°, which is significantly higher than the normal descent angle.

[0003] In the prior art, the calibration of the precision approach bank indicator (PAPI) is typically performed through an in-flight calibration method. During the in-flight calibration, the aircraft is leveled, and the pilot observes the moment when the light changes from red to white during flight, triggering an event. The in-flight calibration device calculates the current elevation angle based on the aircraft's position at the time of the triggering event and the ground coordinates of the PAPI light. The in-flight calibration device then calibrates the PAPI light angle accordingly, guiding the ground crew to adjust the PAPI light elevation angle to meet standard specifications.

[0004] However, due to the high speed of manned aircraft, pilots experience a certain time delay in judging the color of the lights, which affects the accuracy of the precision approach angle indicator (PAPI) lighting calibration. Furthermore, the flight calibration aircraft are large, resulting in high calibration costs. Summary of the Invention

[0005] In order to solve the above-mentioned technical problems of low accuracy and high cost of manned aircraft during the PAPI flight verification process, the present invention proposes a PAPI flight verification method and system based on unmanned aerial vehicles.

[0006] The PAPI flight verification method based on a drone includes:

[0007] Generate flight mission data based on PAPI signal light equipment data;

[0008] Initiating a drone equipped with a camera to fly according to the flight mission data, and receiving from the drone a video image of a PAPI signal light on the ground captured by the camera and positioning data of the drone;

[0009] Calculating the azimuth and elevation angle of the PAPI signal light of the drone relative to the ground based on the received video image of the PAPI signal light and the positioning data of the drone, and identifying the position and light color of the PAPI signal light;

[0010] The PAPI signal light is verified based on the azimuth and elevation data of the UAV relative to the PAPI signal light, as well as the position and light color of the PAPI signal light.

[0011] In one embodiment, identifying the position of the PAPI signal light includes: analyzing the received video image of the PAPI signal light frame by frame, and automatically identifying the position of the PAPI signal light in the video frame of the video image using a contour detection method of computer vision.

[0012] In one embodiment, the method of automatically identifying the position of the PAPI signal light in the video frame of the video image using the contour detection method of computer vision includes: for each video frame,

[0013] Detect the brightness distribution of pixels in the video frame, take the average of the brightness mean and the maximum brightness value as the brightness threshold, and perform binarization on the video frame based on the brightness threshold, wherein pixels below the brightness threshold are marked as black, and pixels above the brightness threshold are marked as white;

[0014] Perform morphological opening operation on video frames to remove noise;

[0015] The contour set of bright spots in the video frame is obtained by searching for contours, and the circular pattern in the video frame is obtained by using the Hough circle detection algorithm;

[0016] Based on the fact that four PAPI signal lights are equidistant and arranged in parallel, the Euclidean distance algorithm of the image is used to calculate the distance features of the circular pattern in the image, thereby automatically identifying the position of the PAPI signal lights in the video frame.

[0017] In one embodiment, a convolutional neural network model for PAPI signal light color recognition is used to automatically recognize the color of the PAPI signal light, extract video frames of the PAPI signal light color change, obtain the PAPI signal light elevation angle corresponding to the PAPI signal light color change, and verify the PAPI signal light. The convolutional neural network model for PAPI signal light color recognition includes a convolution layer, a pooling layer, a normalization layer, a flattening layer, and a fully connected layer.

[0018] In one embodiment, the method further includes the following steps: constructing a convolutional neural network model for PAPI signal light color recognition:

[0019] Collect PAPI signal light images under different lighting conditions;

[0020] Identify the position of the PAPI signal light in the collected PAPI signal light image, and extract a single PAPI signal light image sample;

[0021] In the extracted single PAPI signal light image sample, marking the light color of the PAPI signal light;

[0022] The labeled single PAPI traffic light image sample is randomly divided into training data, test data and verification data. The convolutional neural network model is trained using the training data, and tested and verified using the test data and verification data to obtain a convolutional neural network model for PAPI traffic light color recognition.

[0023] In one embodiment, the PAPI signal light equipment data includes PAPI signal light coordinates, runway threshold coordinates, and light angles;

[0024] The flight mission data includes one or more flight check operation positions, and the climbing altitude and speed at the flight check operation positions;

[0025] The positioning data of the UAV includes the longitude, latitude and flight altitude of the UAV.

[0026] In one embodiment, generating flight mission data based on PAPI signal light device data includes:

[0027] By inputting the distance from the PAPI signal light to the runway's five-sided extension line, the waypoint information from the UAV's current position to the calibration operation position is automatically generated. In addition, the climbing height of the UAV at the calibration operation position is generated based on the distance from the PAPI signal light to the runway's five-sided extension line and the light angle information in the PAPI signal light equipment.

[0028] The five-side extension line is the center line from the exit to the entrance of the runway.

[0029] In one embodiment, the method further includes: sending the calculated azimuth and elevation data to the drone in real time to adjust the drone's onboard camera gimbal so that the PAPI signal light on the ground always remains at the center position of the video image of the PAPI signal light captured by the camera device.

[0030] The present invention also provides a UAV-based PAPI flight verification system, which executes the method described above.

[0031] The present invention further provides a PAPI flight verification system based on a UAV, the PAPI flight verification system comprising a UAV and a ground control unit, the UAV having a camera device, the ground control unit comprising a flight controller, a flight verification device and a PAPI signal light equipment database,

[0032] The UAV receives flight mission data and operation instructions from the flight controller of the ground control unit, and flies according to the flight mission data. At the flight verification operation position, the camera device of the UAV captures a video image of the PAPI signal light, and transmits the video image and the positioning data of the UAV to the ground control unit;

[0033] The flight controller transmits the flight mission data and operation instructions received from the flight verification device to the UAV, receives the video image of the PAPI signal light captured by the camera device and the positioning data of the UAV from the UAV, and transmits the received video image and positioning data of the UAV to the flight verification device;

[0034] The flight verification device generates the flight mission data based on the PAPI signal light equipment database, transmits the flight mission data to the flight controller, and calculates the azimuth and elevation data of the UAV relative to the ground PAPI signal light based on the video image and the positioning data of the UAV received by the flight controller, identifies the position and light color of the PAPI signal light, and verifies the PAPI signal light.

[0035] The advantages of the drone-based PAPI flight verification method and system of the embodiments of the present invention over the prior art are:

[0036] (1) The embodiments of the present invention realize the automatic generation and uploading of flight mission data based on PAPI signal light equipment data, which enables the UAV to fly automatically during the flight verification process, improves the verification accuracy, simplifies the control difficulty of the UAV, and reduces safety risks;

[0037] (2) The embodiment of the present invention realizes real-time automatic control of the azimuth and elevation angle of the airborne camera device, ensuring that the image captured by the drone always takes the PAPI signal light as the image center, thereby improving operation efficiency;

[0038] (3) The embodiment of the present invention uses image processing technology in computer vision to extract the image of the PAPI signal light from the background complex shooting image based on the brightness, equidistant, and parallel characteristics of the PAPI signal light;

[0039] (4) Based on machine learning technology, the embodiment of the present invention utilizes a large amount of labeled image data to train a convolutional neural network data model that can accurately and sensitively identify the color of PAPI traffic lights. This model is then used to achieve automated recognition of the color of PAPI traffic lights, thereby improving the efficiency of PAPI traffic light calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a structural block diagram of the UAV-based PAPI flight verification system proposed in an embodiment of the present invention;

[0041] Figure 2 This is a flow chart of the UAV-based PAPI flight verification method proposed in an embodiment of the present invention;

[0042] Figure 3 This is a flow chart of video image processing of PAPI signal lights in a UAV-based PAPI flight verification system proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments and accompanying drawings. However, those skilled in the art will appreciate that the present invention is not limited to the accompanying drawings and the following embodiments.

[0044] The embodiment of the present invention proposes a PAPI flight verification system based on a drone, such as Figure 1 As shown, it includes a UAV and a ground control unit, the UAV has a camera device, and the ground control unit includes a flight controller, a flight verification device and a PAPI signal light equipment database.

[0045] Among them, the embodiment of the present invention batch exports the PAPI light equipment database in a specific area from the manned machine verification system. Based on the PAPI signal light equipment database, it can automatically retrieve the airport's PAPI equipment information according to the airport identification information, including PAPI signal light coordinates, runway entrance and exit coordinates, lighting angle and other information.

[0046] The UAV receives flight mission data and operating instructions from the flight controller of the ground control unit. The flight mission data includes one or more flight verification operation locations, the climb altitude and speed at the flight verification operation locations, and the operating instructions include, for example, pan / tilt control instructions for the camera. The UAV flies according to the flight mission data. At the flight verification operation location, the UAV's camera captures a video image of the PAPI signal light and transmits the video image and positioning data such as the UAV's longitude, latitude, and altitude to the ground control unit.

[0047] Preferably, the drone is a rotary-wing drone, and the DJI M300RTK drone with excellent performance and easy control is preferably selected. A fixed-point straight-up and straight-down method is adopted, which can reduce the flight time, improve the verification efficiency, and complete the video capture of the PAPI signal light with the shortest flight time; the camera device uses a Z30 gimbal camera with 30x zoom.

[0048] The flight controller is used for a remote ground-to-air digital and image transmission data link, transmitting the flight mission data and operation instructions received from the flight verification device to the UAV, and receiving the video image of the PAPI signal light taken by the camera device and the positioning data of the UAV from the UAV, and transmitting the received video image and positioning data of the UAV to the flight verification device.

[0049] The flight verification device is preferably the flight verification device, which generates the flight mission data based on the PAPI signal light equipment database and transmits the flight mission data to the flight controller. Specifically, by inputting the distance between the PAPI signal light under test and the five-sided extension line of the runway, waypoint information from the current position of the UAV to the verification operation position is automatically generated, and the climbing height of the UAV at the verification operation position is generated based on the five-sided extension line distance between the PAPI signal light under test and the runway and the light angle information in the PAPI signal light equipment, wherein the five-sided extension line is the centerline from the exit to the entrance of the runway.

[0050] The flight verification device further receives the video image and the positioning data of the UAV based on the flight controller, calculates the azimuth and elevation data of the UAV relative to the ground PAPI signal light, identifies the position and light color of the PAPI signal light, and verifies the PAPI signal light.

[0051] In a preferred embodiment, when the drone automatically flies to the calibration location, it begins capturing PAPI signal light video and transmits it to the flight calibration device for data processing. During the camera's capture process, capturing the changing angle of the PAPI signal light requires continuous capture at different heights. Without adjusting the camera's angle, the PAPI signal light can easily slip out of the video content. Manual adjustment often results in a significant loss of focus due to the large magnification and low control precision, resulting in discontinuous and inefficient capture. Therefore, in a preferred embodiment of the present invention, the flight calibration device calculates the drone's azimuth and elevation data relative to the ground PAPI signal light by acquiring real-time positioning data from the drone. This data is then transmitted to the drone, automatically adjusting the camera's pan / tilt position to ensure the PAPI signal light remains centered in the video image captured by the camera. This improves calibration efficiency and video quality, thereby enhancing the accuracy of PAPI signal light angle calibration.

[0052] In order to enable the verification personnel to make intuitive observations, in the flight verification device, the calculated elevation angle data of the drone relative to the ground PAPI signal light is superimposed on the image captured by the camera device, so that the captured image superimposed with the elevation angle information is displayed in real time on the display screen of the flight verification device.

[0053] In an embodiment of the present invention, the flight verification device performs frame-by-frame analysis on the received video images, automatically identifies the position of the PAPI signal light in the video frame using a contour detection method of computer vision, and automatically identifies the light color using a model trained based on a convolutional neural network machine learning algorithm using a single PAPI signal light image sample, thereby performing fully automatic verification of the PAPI signal light.

[0054] Among them, machine learning is a general term for a class of algorithms that mine implicit rules from a large amount of historical data and use them for prediction or classification. More specifically, machine learning can be seen as finding a function, where the input is sample data and the output is the desired result, but this function is too complex to be easily expressed formally. The goal of machine learning is to make the learned function well applicable to "new samples." Convolutional Neural Networks (CNN) is a type of feedforward neural network (Feedforward Neural Networks) that includes convolution calculations and has a deep structure. It is one of the commonly used algorithms for machine learning and is often used in fields such as automatic image recognition and speech recognition. The present invention uses convolutional neural networks to perform intelligent recognition of the light color of PAPI traffic lights.

[0055] Specifically, the position of the PAPI signal light in the video frame is automatically identified by using a contour detection method of computer vision, and a single PAPI signal light image sample is extracted.

[0056] In order to automatically identify the color of the PAPI signal light, it is first necessary to extract the position and image of the PAPI signal light from the video frame. The embodiment of the present invention uses the contour detection method of computer vision to perform image recognition on PAPI and the like. Since the PAPI signal light is an active light source, its brightness in the video frame is medium to high. Therefore, the method includes: first, detecting the brightness distribution of each pixel in the video frame, taking the average of the brightness mean and the maximum brightness value as the brightness threshold to binarize the video frame, marking pixels below the brightness threshold as black, and pixels above the threshold as white; second, performing a morphological opening operation on the video frame to remove noise; then, by searching for contours, obtaining the contour set of the bright spots in the video frame, and using the Hough circle detection algorithm to obtain the circular pattern in the video frame; finally, based on the positional attributes of the four PAPI signal lights that are equidistant and arranged in parallel, using the image Euclidean distance algorithm to calculate the distance features of the circular pattern in the image, the position of the PAPI signal light in the video frame and its image coordinates are detected.

[0057] In this embodiment, a convolutional neural network model for PAPI signal light color recognition was constructed and trained using a large number of labeled individual PAPI signal light image samples. This machine-learning-based convolutional neural network, using a specially designed artificial neural network modeled after the human cerebral cortex, can extract image features that are difficult for the human eye to detect. This significantly improves the accuracy of PAPI signal light color recognition, making it unaffected by factors such as exposure time, ambient light, and light halo. This convolutional neural network model automatically identifies the light color at the PAPI signal light location within a video frame, accurately identifying the light color, particularly during the transition zone between red and white light colors.

[0058] This embodiment of the present invention constructs a multi-layer convolutional neural network model consisting of a convolutional layer, a pooling layer, a normalization layer, a flattening layer, and a fully connected layer. The convolutional layer extracts feature parameters from the input image and uses the ReLU function as its activation function. The pooling layer reduces the feature map. The normalization layer avoids the vanishing gradient problem. After the flattening layer, the parameters are input into the fully connected layer for decision making.

[0059] Using a drone-mounted camera device, 5,000 PAPI signal light images were acquired under different lighting conditions, including backlight, low light, and strong light. The position of the PAPI signal light in the image was identified, and individual PAPI signal light image samples were extracted, and the light color of the PAPI signal light was manually labeled as white or red. Then, the multiple extracted individual PAPI signal light image samples were randomly divided into training data, test data, and verification data. The convolutional neural network model was trained using the training data, and tested and verified using the test data and verification data. A convolutional neural network model for PAPI signal light color recognition with an accuracy rate of up to 0.9983 was obtained, and the model was saved.

[0060] During the actual verification process, a trained convolutional neural network model was loaded and the PAPI traffic light image extracted from the captured image was used as the model input to predict the light color. Testing showed that its recognition accuracy and sensitivity were superior to manual interpretation.

[0061] The following describes the verification method performed by the ground control unit of the UAV-based PAPI flight verification system according to an embodiment of the present invention. Figure 2 As shown, the UAV-based PAPI flight verification method includes the following steps:

[0062] S1: Build a convolutional neural network model for PAPI traffic light color recognition.

[0063] Capturing PAPI signal light images under different lighting conditions. As previously mentioned, using a drone-mounted camera, we captured 5,000 images of the airport's PAPI signal lights under various lighting conditions, including backlight, low light, and strong light.

[0064] For example, the contour detection method of computer vision is used to identify the position of the PAPI signal light in the collected PAPI signal light image, and extract a single PAPI signal light image sample.

[0065] In the extracted single PAPI signal light image sample, mark the light color of the PAPI signal light - white or red.

[0066] The labeled single PAPI traffic light image samples were randomly divided into three data groups: training data, test data, and verification data. The convolutional neural network model was trained using the training data, and tested and verified using the test data and verification data. A convolutional neural network model for PAPI traffic light color recognition with an accuracy of up to 0.9983 was obtained and saved.

[0067] S2: Generate flight mission data based on PAPI signal light equipment data.

[0068] In an embodiment of the present invention, the flight mission data is generated based on a PAPI signal light equipment database. The PAPI signal light equipment database for a specific area is exported in batches from a manned aircraft verification system. The flight mission data includes one or more flight verification operation locations, as well as the climb altitude and speed at the flight verification operation locations. The UAV flies according to the flight mission data. At the flight verification operation locations, the UAV's camera captures video images of the PAPI signal lights and transmits the video images and positioning data, such as the UAV's longitude, latitude, and altitude, to the ground control unit.

[0069] Based on the PAPI signal light equipment database, the airport's PAPI equipment information can be automatically retrieved according to the airport identification information, including PAPI signal light coordinates, runway entrance and exit coordinates, lighting angles and other information.

[0070] Specifically, by inputting the distance from the PAPI signal light under test to the five-sided extension line of the runway, waypoint information from the UAV's current position to the verification operation position is automatically generated. Furthermore, based on the five-sided extension line distance from the PAPI signal light under test to the runway and the light angle information in the PAPI signal light equipment, the climbing altitude of the UAV at the verification operation position is generated, where the five-sided extension line is the centerline from the runway exit to the entrance.

[0071] For safety reasons, the default altitude for flight to the calibration location is 50 meters. In daily operations, this altitude effectively avoids airport obstacles and allows for safe arrival at the work location. For PAPI beacon light angle calibration, the automatically generated waypoint is not located on the runway extension line, but on the PAPI beacon light extension line parallel to the runway. This allows the drone to face the PAPI beacon light directly, achieving a more optimal shooting effect.

[0072] S3: Start the drone with the camera device to fly according to the flight mission data, and receive the video image of the PAPI signal light captured by the camera device and the positioning data of the drone from the drone.

[0073] The positioning data of the UAV includes the longitude, latitude and flight altitude of the UAV.

[0074] Start the UAV for flight verification. The UAV automatically flies to the verification operation position, captures the video image of the PAPI signal light through the camera device, and transmits the video image and the UAV's positioning data to the ground control unit. The ground control unit receives the video image and the UAV's positioning data.

[0075] S4: Based on the received video image of the PAPI signal light and the positioning data of the UAV, the azimuth and elevation data of the UAV relative to the ground PAPI signal light are calculated, the position and light color of the PAPI signal light are identified, and the PAPI signal light is verified.

[0076] Specifically, the received video image of the PAPI signal light is analyzed frame by frame, and the position of the PAPI signal light in the video frame is automatically identified using a contour detection method based on computer vision. The computer vision contour detection method includes: first, detecting the brightness distribution of each pixel in the video frame, taking the average of the brightness mean and the maximum brightness value as the brightness threshold to binarize the video frame, marking pixels below the brightness threshold as black and pixels above the threshold as white; second, performing a morphological opening operation on the video frame to remove noise; then, by searching for contours, obtaining a set of contours of bright spots in the video frame, and using the Hough circle detection algorithm to obtain a circular pattern in the video frame; finally, based on the positional properties of the four PAPI signal lights arranged equidistantly and in parallel, using the image Euclidean distance algorithm to calculate the distance features of the circular pattern in the image, and detecting the position of the PAPI signal light in the video frame and its image coordinates.

[0077] The convolutional neural network model for PAPI signal light color recognition is used to automatically recognize the light color of the PAPI signal light, extract video frames of the PAPI signal light color change, obtain the PAPI signal light elevation angle corresponding to the PAPI signal light color change, verify the PAPI signal light, and save the video frames marked with the light color as a post-processed video file for viewing and use.

[0078] Preferably, in this step, the calculated azimuth and elevation parameters are sent to the drone in real time to adjust the drone's onboard camera gimbal so that the ground PAPI signal light always remains in the center of the video image of the PAPI signal light captured by the camera device, thereby improving the shooting accuracy and image quality.

[0079] The video image processing flow of the PAPI signal light of the UAV PAPI flight verification system according to the embodiment of the present invention is as follows: Figure 3As shown in the figure, the source video file of the PAPI traffic light is opened and the images are read frame by frame from the video file. Based on the brightness distribution in the image, the image is binarized by taking the average of the mean and maximum brightness values ​​as the brightness threshold. Values ​​below the threshold are marked black, and values ​​above the threshold are marked white. Next, a morphological opening operation is performed on the image to remove noise. Then, by searching for contours, the contour set of bright spots in the image is obtained. The Hough circle detection algorithm is used to detect the circular pattern in the image. Finally, based on the geometric properties of the four PAPI traffic lights being equidistant and parallel, the Euclidean distance algorithm is used to calculate the distance features of the circular pattern in the image, detecting the location of the PAPI traffic lights in the image and their image coordinates. The four light images are extracted from the original image and sequentially fed into the trained convolutional neural network model obtained in step 1 for intelligent light color recognition. If the color of the light in the current video frame is different from that in the previous frame, it indicates that the light has changed color. The current video frame is extracted as the color-changed image of the light, and the corresponding elevation angle in the image is the elevation angle of the light. If the number or distance constraints of circular patterns are not met during detection or the color of the detected light does not change, the current frame is skipped and the next frame is read until the end of the video file.

[0080] The embodiment of the present invention can automatically identify the color of lights. The convolutional neural network model obtained through training with a large amount of data can accurately determine the frames in which the color of the PAPI signal light changes. In addition, the light color is marked in the video captured after processing by this method and archived to form a historical record of the light calibration. The video images of the PAPI signal light captured during each flight calibration can be added to the sample library of the PAPI signal light after being marked, and the convolutional neural network model can be retrained using the method of step S1 to continuously iterate to produce a more intelligent light color recognition model with higher recognition accuracy and robustness, thereby achieving continuous optimization of the machine learning model.

[0081] The embodiment of the present invention further proposes a PAPI flight verification system based on a drone, which executes the method described above.

[0082] An embodiment of the present invention further provides a storage medium storing a computer program for executing the above-mentioned method.

[0083] An embodiment of the present invention further provides a processor, which runs a computer program that executes the method described above.

[0084] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0085] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement the hardware: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0086] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0087] The above describes the embodiments of the present invention. However, the present invention is not limited to the above embodiments. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A PAPI flight verification method based on drone, characterized in that: include: Generate flight mission data based on PAPI signal light equipment data; Initiating a drone equipped with a camera to fly according to the flight mission data, and receiving from the drone a video image of a PAPI signal light on the ground captured by the camera and positioning data of the drone; Based on the received video image of the PAPI signal light and the positioning data of the UAV, the azimuth and elevation data of the PAPI signal light of the UAV relative to the ground are calculated, and the position and light color of the PAPI signal light are identified, wherein the identifying the position of the PAPI signal light includes: analyzing the received video image of the PAPI signal light frame by frame, detecting the brightness distribution of pixels in the video frame for each video frame, taking the average of the brightness mean and the maximum brightness value as the brightness threshold, and binarizing the video frame based on the brightness threshold, marking pixels below the brightness threshold as black and pixels above the brightness threshold as white; performing a morphological opening operation on the video frame to remove noise; obtaining a contour set of bright spots in the video frame by searching for contours, and obtaining a circular pattern in the video frame using a Hough circle detection algorithm; based on the four PAPI signal lights being equidistant and arranged in parallel, using an image Euclidean distance algorithm to calculate the distance feature of the circular pattern in the image, thereby automatically identifying the position of the PAPI signal light in the video frame; The PAPI signal light is verified based on the azimuth and elevation data of the UAV relative to the PAPI signal light, as well as the position and light color of the PAPI signal light.

2. The PAPI flight verification method according to claim 1, wherein: A convolutional neural network model for PAPI signal light color recognition is used to automatically identify the color of the PAPI signal light, extract video frames of the PAPI signal light color change, obtain the PAPI signal light elevation angle corresponding to the PAPI signal light color change, and verify the PAPI signal light. The convolutional neural network model for PAPI signal light color recognition includes a convolution layer, a pooling layer, a normalization layer, a flattening layer, and a fully connected layer.

3. The PAPI flight verification method according to claim 2, characterized in that: It also includes the steps for building a convolutional neural network model for PAPI signal light color recognition, as follows: Collect PAPI signal light images under different lighting conditions; Identify the position of the PAPI signal light in the collected PAPI signal light image, and extract a single PAPI signal light image sample; In the extracted single PAPI signal light image sample, marking the light color of the PAPI signal light; The labeled single PAPI traffic light image sample is randomly divided into training data, test data and verification data. The convolutional neural network model is trained using the training data, and tested and verified using the test data and verification data to obtain a convolutional neural network model for PAPI traffic light color recognition.

4. The PAPI flight verification method according to claim 1, wherein: The PAPI signal light equipment data includes PAPI signal light coordinates, runway entrance and exit coordinates and light angles; The flight mission data includes one or more flight check operation positions, and the climbing altitude and speed at the flight check operation positions; The positioning data of the UAV includes the longitude, latitude and flight altitude of the UAV.

5. The PAPI flight verification method according to claim 4, characterized in that: Generating flight mission data based on PAPI signal light equipment data includes: By inputting the distance from the PAPI signal light to the runway's five-sided extension line, the waypoint information from the UAV's current position to the calibration operation position is automatically generated. In addition, the climbing height of the UAV at the calibration operation position is generated based on the distance from the PAPI signal light to the runway's five-sided extension line and the light angle information in the PAPI signal light equipment. The five-side extension line is the center line from the exit to the entrance of the runway.

6. The PAPI flight verification method according to claim 1, wherein: Also includes: The calculated azimuth and elevation data are sent to the drone in real time to adjust the drone's onboard camera gimbal so that the ground PAPI signal light always remains at the center of the video image of the PAPI signal light captured by the camera device.

7. A PAPI flight verification system based on drones, characterized by: Execute the method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium stores a computer program for executing the method according to any one of claims 1 to 6.

9. A processor, characterized in that: The processor runs a computer program that performs the method according to any one of claims 1 to 6.

10. A PAPI flight verification system based on drones, characterized by: The PAPI flight verification system includes an unmanned aerial vehicle (UAV) with a camera and a ground control unit, wherein the UAV includes a flight controller, a flight verification device, and a PAPI signal light equipment database. The UAV receives flight mission data and operation instructions from the flight controller of the ground control unit, and flies according to the flight mission data. At the flight verification operation position, the camera device of the UAV captures a video image of the PAPI signal light, and transmits the video image and the positioning data of the UAV to the ground control unit; The flight controller transmits the flight mission data and operation instructions received from the flight verification device to the UAV, receives the video image of the PAPI signal light captured by the camera device and the positioning data of the UAV from the UAV, and transmits the received video image and positioning data of the UAV to the flight verification device; The flight verification device generates the flight mission data based on the PAPI signal light equipment database, transmits the flight mission data to the flight controller, and calculates the azimuth and elevation data of the UAV relative to the ground PAPI signal light according to the video image and the positioning data of the UAV received by the flight controller, identifies the position and light color of the PAPI signal light, and verifies the PAPI signal light. Among them, the identifying the position of the PAPI signal light includes: analyzing the received video image of the PAPI signal light frame by frame, detecting the brightness distribution of pixels in the video frame for each video frame, taking the average of the brightness mean and the maximum brightness value as the brightness threshold, and binarizing the video frame based on the brightness threshold, marking pixels below the brightness threshold as black, and pixels above the brightness threshold as white; performing a morphological opening operation on the video frame to remove noise; obtaining a set of contours of bright spots in the video frame by searching for contours, and obtaining a circular pattern in the video frame using a Hough circle detection algorithm; based on the four PAPI signal lights being equidistant and arranged in parallel, using an image Euclidean distance algorithm to calculate the distance features of the circular pattern in the image, thereby automatically identifying the position of the PAPI signal light in the video frame.

Images