Unmanned aerial vehicle runway target detection method, device, equipment and medium

Abstract

The application discloses a UAV runway target detection method and device, equipment and medium, and relates to the field of UAV visual navigation, which comprises the following steps: if the flight parameters of a target UAV meet preset conditions, runway target detection is performed on the collected UAV image; based on the detection result, the image parameters of the UAV image and the UAV parameters, the angle parameters of the UAV image are calculated, and whether there is false detection in the UAV image is determined according to the angle parameters; according to the false detection result and the historical effective frame count, the current effective frame count is determined, and whether the prediction initialization is completed is determined according to the current effective frame count; if the prediction initialization is completed, the UAV image is subjected to prediction consistency verification to determine whether the runway target detection result is taken as the current detection result; if the prediction initialization is not completed, the UAV image is subjected to target tracking, and the current detection result is determined based on the target detection result and the obtained tracking result. Thus, the accuracy of UAV runway target detection can be improved.

Description

Technical Field

[0001] This invention relates to the field of visual navigation for unmanned aerial vehicles (UAVs), and in particular to a method, apparatus, equipment, and medium for UAV runway target detection. Background Technology

[0002] With the rapid development of artificial intelligence and computer vision technologies, vision-based navigation and guidance have become an important technological direction for autonomous flight of unmanned aerial vehicles (UAVs). During critical mission phases such as approach and landing, the UAV's pose information relative to the runway target area is the core basis for flight control and guidance. Common engineering vision navigation processes typically include: extracting the detection box or key features of the runway target from the image; calculating the UAV's relative pose by combining camera imaging geometry; and thus providing input for flight control.

[0003] Currently, target detection technology is mainly divided into traditional image processing methods and deep learning-based detection methods. The latter has gradually become the mainstream in practical applications due to its data-driven adaptive capabilities and high detection accuracy.

[0004] However, existing technologies still have some problems in UAV visual navigation applications. For example, visual perception inevitably produces false detections and failures when targeting runways in complex environments. Existing systems often lack a system-level method that can combine flight status and visual results for consistency verification, real-time elimination of false detections, and adaptive fusion between detection and tracking. How to leverage the richness of visual information while introducing flight parameter constraints to construct a reliability judgment mechanism, and achieve controlled fusion with tracking when detection fails, thereby ensuring the continuity, stability, and reliability of visual navigation output, has become a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for detecting unmanned aerial vehicle (UAV) runway targets, which can reduce the impact of false detections, improve output continuity and stability, and avoid navigation / guidance interruptions or jumps. The specific solution is as follows: In a first aspect, this application discloses a method for detecting runway targets on unmanned aerial vehicles (UAVs), applied to a UAV visual navigation system, comprising: If the target flight parameters of the target UAV meet the preset parameter conditions, then the target UAV image collected by the target UAV is subjected to runway target detection through the preset detection network to obtain the corresponding runway target detection result. Based on the runway target detection result, the image parameters corresponding to the target UAV image, and the UAV parameters of the target UAV, the angle parameters corresponding to the target UAV image are calculated, and the presence of false detections in the target UAV image is determined according to the angle parameters to obtain the corresponding false detection results; the angle parameters are the parameters of the runway angle in the target UAV image. The current valid frame count is determined based on the false detection results and the historical valid frame count, and the prediction initialization is determined based on the current valid frame count. If prediction initialization has been completed, then the prediction consistency check is performed on the target UAV image, and the runway target detection result is used as the current detection result based on the prediction consistency check result. If prediction initialization is not completed, target tracking is performed based on the target UAV image using a preset tracker, and the current detection result is determined based on the runway target detection result and the obtained tracking result.

[0006] Wherein, if the target flight parameters of the target UAV meet the preset parameter conditions, then runway target detection is performed on the target UAV image acquired by the target UAV through a preset detection network to obtain the corresponding runway target detection results, including: The system acquires the target drone's currently captured image from its onboard camera, as well as the target drone's flight parameters at the time the image was acquired. The target flight parameters include attitude angles, runway heading angles, and inertial navigation status words. The attitude angles include pitch angle, roll angle, and yaw angle. Determine whether the inertial navigation status word is valid. If the inertial navigation status word is valid, then perform runway target detection on the target UAV image collected by the target UAV through a preset detection network to determine the target bounding box coordinates, detection confidence, and target category label corresponding to the target UAV image. Based on the ratio between the preset detection network input size and the original image size, the target bounding box is mapped from the target detection network input size coordinate system to the original image coordinate system to obtain the mapped bounding box; The validity of the mapped bounding box is determined based on the target category label and the detection confidence. If the target category label is a runway and the detection confidence is greater than or equal to a preset confidence threshold, then the mapped bounding box is determined to be valid. The valid bounding box result after mapping, the target category label, and the detection confidence score are used as the runway target detection result.

[0007] The step of calculating the angle parameters corresponding to the target drone image based on the runway target detection result, the image parameters corresponding to the target drone image, and the drone parameters of the target drone, and determining whether there are false detections in the target drone image based on the angle parameters to obtain the corresponding false detection result, includes: Determine the rotation matrix, bottom pixel coordinates, and intrinsic parameter matrix corresponding to the target drone image; the rotation matrix is ​​a matrix describing the orientation of the target drone; the bottom pixel coordinates are the coordinates of the center point of the bottom edge of the bounding box corresponding to the target drone image in the target drone image; The target homogeneous coordinates are constructed based on the base pixel coordinates, and the product of the target homogeneous coordinates, the transpose of the rotation matrix, and the inverse of the intrinsic parameter matrix is ​​used as an intermediate quantity of the target UAV image. The numerical parameters corresponding to the intermediate quantity are determined, and the first angle index corresponding to the target UAV image is calculated based on the numerical parameters through tangent operation. The second angle index corresponding to the target UAV image is calculated based on the numerical parameters through inverse tangent operation. The first angle index and the second angle index are indices representing directional deviation. If the first angle indicator falls within the first preset angle range and the second angle indicator falls within the second preset angle range, then it is determined that there is no false detection in the target drone image, so as to generate a first false detection result. If the first angle indicator does not fall within the first preset angle range, or the second angle indicator does not fall within the second preset angle range, then it is determined that the target drone image has a false detection, the target drone image is removed, and a second false detection result is generated.

[0008] The step of determining the current valid frame count based on the false detection result and the historical valid frame count, and determining whether the prediction initialization is complete based on the current valid frame count, includes: If the false detection result is the first false detection result, then the sum of the historical valid frame count and the preset valid frame increment is used as the current valid frame count; If the current valid frame count is not less than the preset valid frame count threshold, then the prediction initialization is determined to be complete.

[0009] The step of performing a prediction consistency check on the target UAV image if prediction initialization has been completed, and determining whether to use the runway target detection result as the current detection result based on the prediction consistency check result, includes: If prediction initialization has been completed, the historical rotation matrix of the previous valid frame is determined, and the historical intermediate vector is calculated based on the transpose of the historical rotation matrix, the inverse of the intrinsic parameter matrix, and the homogeneous coordinates of the previous valid frame. A parameter vector is constructed based on the ratios between the components of the historical intermediate vector, and pixel position prediction is performed based on the parameter vector, the intrinsic parameter matrix, and the rotation matrix to obtain the predicted pixel position. A consistency comparison is performed between the predicted pixel position and the bottom pixel coordinates. If the deviation between the predicted pixel position and the bottom pixel coordinates is greater than a preset deviation threshold, the target drone image is removed. If the deviation between the predicted pixel position and the bottom pixel coordinates is not greater than the preset deviation threshold, then the target UAV image is determined to have passed the prediction consistency check, and the runway target detection result is taken as the current detection result.

[0010] If prediction initialization is not completed, target tracking is performed based on the target UAV image using a preset tracker, and the current detection result is determined based on the runway target detection result and the obtained tracking result, including: If prediction initialization is not completed, the historical bounding box of the previous valid frame is converted into the tracker initialization box of the preset tracker, and the preset tracker is initialized based on the target UAV image to obtain the target tracker; The target tracker performs target tracking on the target UAV image and determines false detections based on the tracking results. If the tracking result passes the false detection judgment, then the tracking result is taken as the current detection result; If the tracking result fails the false detection judgment, the current tracking is terminated.

[0011] The unmanned aerial vehicle (UAV) runway target detection method further includes: The current detection result is output, and the target flight parameters are updated based on the current detection result, so as to perform target detection in the next frame based on the updated flight parameters.

[0012] Secondly, this application discloses a UAV runway target detection device, applied to a UAV visual navigation system, comprising: The target detection module is used to perform runway target detection on the target drone image collected by the target drone through a preset detection network if the target flight parameters of the target drone meet the preset parameter conditions, so as to obtain the corresponding runway target detection result. The false detection judgment module is used to calculate the angle parameters corresponding to the target UAV image based on the runway target detection result, the image parameters corresponding to the target UAV image, and the UAV parameters of the target UAV, and to determine whether there is a false detection in the target UAV image based on the angle parameters, so as to obtain the corresponding false detection result; the angle parameters are the parameters of the runway angle in the target UAV image; The prediction initialization judgment module is used to determine the current valid frame count based on the false detection result and the historical valid frame count, and to determine whether the prediction initialization is completed based on the current valid frame count. The first detection result determination module is used to perform prediction consistency verification on the target UAV image if prediction initialization has been completed, and determine whether to take the runway target detection result as the current detection result based on the prediction consistency verification result. The second detection result determination module is used to perform target tracking based on the target UAV image using a preset tracker if prediction initialization has not been completed, and to determine the current detection result based on the runway target detection result and the obtained tracking result.

[0013] Thirdly, this application discloses an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the unmanned aerial vehicle runway target detection method as described above.

[0014] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned unmanned aerial vehicle runway target detection method.

[0015] In this application, if the target flight parameters of the target UAV meet preset parameter conditions, runway target detection is performed on the target UAV image collected by the target UAV through a preset detection network to obtain the corresponding runway target detection result; based on the runway target detection result, the image parameters corresponding to the target UAV image, and the UAV parameters of the target UAV, the angle parameters corresponding to the target UAV image are calculated, and the presence of false detections in the target UAV image is determined according to the angle parameters to obtain the corresponding false detection result; the angle parameters are the parameters of the runway angle in the target UAV image; the current valid frame count is determined according to the false detection result and the historical valid frame count, and the prediction initialization is determined according to the current valid frame count; if the prediction initialization has been completed, the prediction consistency check is performed on the target UAV image, and the runway target detection result is determined as the current detection result according to the prediction consistency check result; if the prediction initialization has not been completed, the target is tracked based on the target UAV image through a preset tracker, and the current detection result is determined based on the runway target detection result and the obtained tracking result.

[0016] Therefore, the method of this application allows runway target detection on the airborne camera image via a preset detection network only when the target flight parameters of the target UAV meet the preset parameter conditions, outputting the runway target detection result. Then, based on the runway target detection result, image parameters, and UAV parameters, the angle parameters corresponding to the target UAV image are calculated. Whether the angle parameters fall within a reasonable range is used to determine if there are false detections, and the false detection result is output. The historical valid frame count is accumulated based on the false detection results, and this is used to determine whether prediction initialization is complete. If prediction initialization is complete, the pixel position of the target bottom point in the current frame is predicted using the historical rotation matrix of the previous valid frame, the historical bottom point coordinates, and the rotation matrix of the current frame, and is compared with the bottom point position detected in the current frame for consistency. If prediction initialization is not yet complete, the target is tracked in the current image using a preset tracker. The tracking result is combined with the detection result to determine the current detection result, maintaining output continuity even when detection is temporarily unstable. In this way, by introducing flight attitude, imaging geometry, and runway heading priors into the detection result discrimination, false detections that are similar in appearance but geometrically unreasonable can be eliminated in real time, rather than relying solely on network confidence. This significantly reduces the probability of false detections entering the downstream pose calculation or control guidance link, avoiding the risk of pose estimation jumps or misguided navigation. On the other hand, when detection experiences a phase failure, the target tracking path provides continuous position estimation within a limited range, avoiding interruption of visual navigation output. After detection recovers, the detection link can be switched back, achieving adaptive fusion between detection and tracking, balancing accuracy and continuity. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for detecting unmanned aerial vehicle (UAV) runway targets disclosed in this application; Figure 2 This application discloses a data input and preprocessing flowchart; Figure 3 This application discloses a flowchart for judging false detection. Figure 4 This application discloses a logic flowchart for initializing and enabling prediction verification of a detection process. Figure 5 This is a flowchart of a false detection prediction and verification algorithm disclosed in this application; Figure 6 This is a schematic diagram illustrating the generation relationship of an output field as disclosed in this application; Figure 7 This is a schematic diagram illustrating the module composition and data flow of a visual navigation system for unmanned aerial vehicles (UAVs) disclosed in this application. Figure 8 This is a schematic diagram of the structure of a UAV runway target detection device disclosed in this application; Figure 9 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Currently, existing technologies still have some problems in the application of visual navigation in UAVs. For example, visual perception inevitably produces false detections and failures when targeting runway targets in complex environments. Existing systems often lack a system-level method that can combine flight status and visual results for consistency verification, eliminate false detections in real time, and achieve adaptive fusion between detection and tracking.

[0021] To overcome the aforementioned technical problems, this application discloses a method, apparatus, equipment, and medium for detecting unmanned aerial vehicle (UAV) runway targets, which can reduce the impact of false detections, improve output continuity and stability, and avoid navigation / guidance interruptions or jumps.

[0022] See Figure 1 As shown, this embodiment of the invention discloses a method for detecting unmanned aerial vehicle (UAV) runway targets, applied to an UAV visual navigation system, including: Step S11: If the target flight parameters of the target UAV meet the preset parameter conditions, then the target UAV image collected by the target UAV is subjected to runway target detection through the preset detection network to obtain the corresponding runway target detection result.

[0023] In this embodiment, before determining the flight parameters of the target UAV, data input and preprocessing are required. For example, Figure 2 As shown, the first step is to acquire the target UAV's currently captured image from its onboard camera, along with the target's flight parameters at the time of image acquisition. These flight parameters include attitude angles, runway heading angle, and inertial navigation status word (lins_state). Attitude angles include pitch, roll, and yaw. Specifically, real-time image frames are acquired from the target UAV's onboard camera to form the current frame's target UAV image data input. To adapt to subsequent neural network inference, a preprocessing procedure, such as size transformation, is performed on the image within the visual navigation system to output image data that meets the input size requirements of the detection network. Then, the current frame's flight parameters (i.e., target flight parameters) need to be acquired from the flight control system. These parameters include at least pitch, yaw, roll, runway heading angle, altitude, and inertial navigation status word (lins_state). Pitch, roll, and yaw angles are defined in the right-forward-lower coordinate system, in degrees (°). Among them, pitch angle, yaw angle, and roll angle are used for attitude rotation matrix calculation and prediction verification, runway heading angle is used as a priori parameter related to mission / scenario for system use, altitude can participate in scenario judgment and subsequent processing, and inertial navigation status words are used for flight parameter validity judgment and process gating.

[0024] Furthermore, such as Figure 2As shown, to ensure reliable input for subsequent flight parameter-based prediction verification, the validity of the inertial navigation status word needs to be determined, including whether it is aligned and whether the IMU (Inertial Measurement Unit) data is valid, to confirm whether the target flight parameters meet the preset parameter conditions. When the flight parameter is valid (i.e., meets the preset parameter conditions), target detection and subsequent false detection verification are allowed; when the flight parameter is invalid, the current frame does not enter the core false detection judgment link of this invention, and the system can switch to state machine update or other safety strategy processing. Specifically, if the inertial navigation status word is valid, runway target detection is performed on the target UAV image acquired by the target UAV through the preset detection network to determine the target bounding box coordinates, detection confidence, and target category label corresponding to the target UAV image. Among them, the target bounding box coordinates are bounds_lt and bounds_rb; the detection confidence is confidence; and the target category label is class_id.

[0025] Next, based on the predefined ratio between the input size of the detection network and the original image size, the target bounding box needs to be mapped from the target detection network input size coordinate system to the original image coordinate system to obtain the mapped bounding box. It should be noted that the bounding box coordinates output by the detection network are usually defined in the network input size coordinate system (denoted as img_size × img_size). To maintain consistency with the original image coordinate system, the detection box coordinates need to be mapped back to the original image size (src_w × src_h).

[0026] In this embodiment, each detection box (l,r,t,b) is mapped as follows: info.l=(info.l / img_size)×src_w; info.r=(info.r / img_size)×src_w; info.t=(info.t / img_size)×src_h; info.b=(info.b / img_size)×src_h; Where img_size is the input size configuration for the detection network, and src_w / src_h are the width and height of the original image, respectively.

[0027] Furthermore, the validity of the mapped bounding box needs to be determined based on the target category label and detection confidence. If the target category label is "runway" and the detection confidence is greater than or equal to a preset confidence threshold, the mapped bounding box is considered valid. Before proceeding to more stringent false positive prediction verification, the detection results are initially screened. The preliminary validity conditions in this embodiment are: runway target detected: class_id=0, confidence threshold: confidence≥0.60. Here, class_id=0 confirms that the detected target is a runway, and confidence≥0.60 filters low-confidence candidate boxes, reducing the probability of false positives. Finally, the valid mapped bounding box result, the target category label, and the detection confidence are taken as the runway target detection result.

[0028] In this way, by accurately mapping the bounding box coordinates output by the detection network back to the original image size from the network input size, the pixel positions used for subsequent geometric verification and false detection determination are ensured to have physical authenticity, avoiding scale deviation caused by inconsistency in coordinate systems. Secondly, by introducing a dual screening mechanism of category determination and confidence gating, the runway target category is locked at the semantic level, while low-quality detection boxes are filtered out in advance, effectively reducing the invalid computational load of subsequent flight parameter prediction verification and angle constraint determination, and ensuring the system's real-time processing capability under high frame rate video streams. Finally, when there are valid candidate detections, they are sent to the subsequent false detection elimination link for geometric and physical verification; when there are no valid candidates, the verification stage is skipped and the continuity guarantee mechanism is triggered, thereby realizing the clarity and controllability of the logical connection between visual perception and flight parameter constraints, providing a stable and reliable candidate input premise for the entire adaptive fusion framework.

[0029] Step S12: Calculate the angle parameters corresponding to the target drone image based on the runway target detection result, the image parameters corresponding to the target drone image, and the drone parameters of the target drone, and determine whether there are false detections in the target drone image based on the angle parameters to obtain the corresponding false detection results; the angle parameters are the parameters of the runway angle in the target drone image.

[0030] In this embodiment, it is necessary to determine whether false detections have occurred. Specifically, such as... Figure 3As shown, the first step is to determine the rotation matrix, bottom pixel coordinates, and intrinsic parameter matrix corresponding to the target drone image and its onboard camera. The rotation matrix describes the target drone's orientation. It should be noted that the bottom pixel coordinates are the coordinates of the center point of the bottom edge of the bounding box corresponding to the target drone image within the image itself. Furthermore, the intrinsic parameter matrix K is provided by the system calibration, while the rotation matrix R needs to be determined based on the target drone's flight parameters: pitch angle, yaw angle, and roll angle. The formula for calculating the rotation matrix is ​​shown below: ; Furthermore, it is necessary to construct homogeneous coordinates and calculate intermediate quantities. Specifically, the target homogeneous coordinates need to be constructed based on the base pixel coordinates, and the product of the target homogeneous coordinates, the transpose of the rotation matrix, and the inverse of the intrinsic parameter matrix is ​​used as the intermediate quantity A of the target UAV image. Specifically, the current candidate base pixel coordinates point=(x,y) are used to construct... Then calculate .

[0031] The next step is to determine the numerical parameters corresponding to the intermediate quantities, and calculate the first angle index corresponding to the target UAV image based on the numerical parameters through tangent operation, and calculate the second angle index corresponding to the target UAV image based on the numerical parameters through inverse tangent operation; the first angle index and the second angle index are indicators representing directional deviation. Specifically, it is necessary to calculate the intermediate quantities a and b. When |A(1)|>0.000001, a=A(2) / A(1), b=A(3) / A(1), otherwise, a=A(1) / A(2), b=A(3) / A(2), and then calculate the first angle index ty / tx and the second angle index tz / tx. Among them, it is necessary to set Where runway_yaw is the runway yaw angle, then the first angle index corresponding to the target UAV image is calculated based on the numerical parameters through tangent operation, and its expression is as follows: ; The second angle index corresponding to the target UAV image is calculated based on the numerical parameters using the inverse tangent operation, and its expression is as follows: ; Specifically, if the first angle indicator falls within the first preset angle range and the second angle indicator falls within the second preset angle range, it is determined that there are no false detections in the target drone image, and a first false detection result is generated. If the first angle indicator does not fall within the first preset angle range, or the second angle indicator does not fall within the second preset angle range, it is determined that there are false detections in the target drone image, and the target drone image is removed, generating a second false detection result. The angle range corresponding to the first preset angle range is [-5°, +5°], and the radian range is [-5π / 180, +5π / 180]. If it exceeds the range, the geometric relationship is considered unreasonable. The angle range corresponding to the second preset angle range is [0°, 6°], and the radian range is [0, 6π / 180]. If it exceeds the range, the deviation is considered too large or unreasonable. When the determination fails, the system will mark the candidate result as "excluded by the false detection algorithm" and use it for subsequent gating / output status flags.

[0032] Step S13: Determine the current valid frame count based on the false detection results and the historical valid frame count, and determine whether the prediction initialization is complete based on the current valid frame count.

[0033] In this embodiment, as Figure 4 As shown, it is necessary to determine whether the prediction initialization is complete based on the false detection results and the historical valid frame count. Specifically, if the false detection result is the first false detection result, the sum of the historical valid frame count and the preset valid frame increment is used as the current valid frame count. It should be noted that when performing the current frame detection determination `cur_obj_det_valid_`, if the false detection result is the first false detection result, it indicates that there are no false detections in the target UAV image, meaning that the current detection link is valid. Therefore, the consecutive valid frame count can be incremented, i.e., `cur_valid_times_++` is incremented; otherwise, the count is cleared to zero, `cur_valid_times_=0`. Here, `cur_obj_det_valid_` indicates whether the current frame detection link is valid.

[0034] If the current valid frame count is not less than the preset valid frame count threshold, then prediction initialization is considered complete (target_tracking_begin_flag_=true). That is, when the consecutive valid frame count is greater than or equal to 3, cur_valid_times_>=3, and detection initialization is considered complete. At this point, prediction verification is enabled (obj_forecast_valid_=true), and the valid UAV flight parameters obj_lats_valid_flight_param_ and valid bottom coordinates last_bottom_ of the previous frame corresponding to the target UAV image are recorded. Simultaneously, a reference frame is prepared for subsequent tracking initialization; that is, one frame is saved as input for tracker initialization.

[0035] Furthermore, when the initialization conditions are not met, obj_forecast_valid_=false, prediction validation is disabled, and prediction-related temporary values ​​are cleared to zero, such as setting the prediction point cache to zero to avoid misusing old prediction results.

[0036] Step S14: If prediction initialization has been completed, perform prediction consistency verification on the target UAV image, and determine whether to use the runway target detection result as the current detection result based on the prediction consistency verification result.

[0037] In this embodiment, if prediction initialization has been completed, false detection prediction verification is required. For example... Figure 5 As shown, if prediction initialization has been completed, the historical rotation matrix of the previous valid frame is determined, and the historical intermediate vector is calculated based on the transpose of the historical rotation matrix, the inverse of the intrinsic parameter matrix, and the homogeneous coordinates of the previous valid frame. Specifically, the target bottom pixel coordinates need to be selected as the stable alignment object. The target bottom pixel coordinates (homogeneous coordinates) of the previous frame are W=[last_bottom_.x, last_bottom_.y,1]^T, and the target bottom pixel coordinates detected in the current frame are point=(point.x, point.y). Then, the historical intermediate vector A is calculated, and its expression is as follows: ; Where last_beat_R is the rotation matrix of the previous valid frame corresponding to the target UAV image, K is the camera's intrinsic parameter matrix, and W is the homogeneous coordinate of the previous valid frame corresponding to the target UAV image.

[0038] Further, position prediction calculation is required. Specifically, a parameter vector needs to be constructed based on the ratios between the components of the historical intermediate vector. Then, pixel position prediction is performed based on the parameter vector, the intrinsic parameter matrix, and the rotation matrix to obtain the predicted pixel position. The expression for the parameter vector is as follows: when hour: ,otherwise, The above branches are used to avoid A(1) being too small, which would lead to numerical instability.

[0039] Then, we need to take the third row of the current rotation matrix R to construct it: Calculate the denominator: Final predicted pixel position: Predicted The predicted pixel coordinates of the target bottom point in the current frame.

[0040] Then, a consistency comparison is performed between the predicted pixel position and the bottom pixel coordinates. If the deviation between the predicted pixel position and the bottom pixel coordinates is greater than a preset deviation threshold, the target UAV image is removed. If the deviation is not greater than the preset deviation threshold, the target UAV image passes the prediction consistency check, and the runway target detection result is used as the current detection result. Specifically, the predicted pixel position... A consistency comparison is performed with the current frame's detected bottom point. The determination condition is: if... ,or (Unit: pixels) If a detection is determined to be a false detection, return false and discard the detection result; otherwise, if a detection is determined to be valid, return true and allow the detection to proceed to the next step of false detection determination and fusion.

[0041] In this way, by using the predictive consistency check and the angle range constraint in step S12, detection results that clearly do not conform to the flight attitude / geometric constraints can be eliminated in real time, avoiding false detections from directly entering the downstream control / guidance, thereby improving the accuracy of UAV runway target detection.

[0042] Step S15: If prediction initialization is not completed, target tracking is performed based on the target UAV image using a preset tracker, and the current detection result is determined based on the runway target detection result and the obtained tracking result.

[0043] In this embodiment, if the prediction initialization is not completed, it is necessary to clear the count when the prediction initialization is not completed, so it is necessary to enter a tracking link. The tracking link attempts to maintain the update of the target bounding box within a limited number of times, and then performs false detection elimination determination on the tracking result. Only when the tracking is valid and passes the false detection elimination can it be allowed to be a valid output candidate. Specifically, if the prediction initialization is not completed, the historical bounding box of the last valid frame is converted into a tracker initialization bounding box of a preset tracker, and the preset tracker is initialized based on the target UAV image to obtain the target tracker. It should be noted that the initialization of the preset tracker needs to meet certain initialization conditions, where the initialization conditions include: the detection initialization has been completed, target_tracking_begin_flag_=true; the current detection is invalid: cur_obj_det_valid_=false; the number of valid tracking times does not exceed the limit: cur_obj_tracker_valid_times_<TRACKER_MAX_TIMES (TRACKER_MAX_TIMES=20 in implementation); the tracker has not been initialized yet, target_tracking_flag_=false. When initializing the tracker, it is necessary to calculate the tracker initialization bounding box runway_bbox_ based on the runway bounding box information saved in the valid detection result of the previous frame, such as last_runway_info_.bounds_lt / bounds_rb. Since the tracker operates in a specific frame size coordinate system, such as the input frame inputframe, it is necessary to map the saved bounding box coordinates to the tracking frame coordinate system according to the width-height ratio. Then call the tracker initialization interface, and the MEDIANFLOW tracking algorithm is used by default: Init_Tracker("MEDIANFLOW", runway_bbox_), and finally set target_tracking_flag_=true, indicating that the tracker has been initialized.

[0044] Furthermore, the target UAV image is tracked by the target tracker, and the tracking results are judged for false detection. If the tracking result passes the false detection judgment, it is used as the current detection result; if the tracking result fails the false detection judgment, the current tracking is terminated. Specifically, after entering the tracking state, the following updates are performed on each frame: Image format conversion: The current frame is converted from YUV (NV12) to BGR (Blue Green Red) to obtain a frame that can be processed by the tracker. Then, the update function is called to update the tracking box: target_valid_flag_=tracker_->update(frame,runway_bbox_), where target_valid_flag_ is the tracking validity flag. Then, the tracking result is written back: When target_valid_flag_=true, the updated runway_bbox_ is mapped back to the result structure stored in the system according to the ratio, for example, updating the bounds_lt / bounds_rb of last_runway_info_, and thereby updating the bottom point coordinates. The mid_line.bottom is calculated based on the center and bottom edge of the box to provide candidate input for subsequent false detection verification and fusion. If tracking fails, when target_valid_flag_=false, cur_obj_tracker_valid_times_ is set to the upper limit, which is equivalent to terminating the tracking link and avoiding infinite attempts.

[0045] It should be noted that the candidate bottom points obtained in this embodiment also undergo false detection rejection. The false detection judgment in step S12 needs to be called. Only if target_valid_flag_=true and the false detection judgment passes is the current frame considered a usable tracking result. Otherwise, the tracking result is marked as rejected and its subsequent use is stopped.

[0046] In this embodiment, runway target detection is only allowed through the preset detection network on the airborne camera image when the target flight parameters of the target UAV meet the preset parameter conditions, and the runway target detection result is output. Then, based on the runway target detection result, image parameters, and UAV parameters, the angle parameters corresponding to the target UAV image are calculated, and whether there is a false detection is determined based on whether the angle parameter falls within a reasonable range, and the false detection result is output. The historical valid frame count is accumulated based on the false detection result, and the prediction initialization is determined accordingly. If the prediction initialization is completed, the pixel position of the target bottom point in the current frame is predicted using the historical rotation matrix of the previous valid frame, the historical bottom point coordinates, and the current frame rotation matrix, and is compared with the bottom point position detected in the current frame. If the prediction initialization is not completed, the target is tracked in the current image through the preset tracker, and the tracking result is combined with the detection result to determine the current detection result, so as to maintain the continuity of output when the detection is unstable for a short time. In this way, on the one hand, by introducing the flight attitude, imaging geometry, and runway heading prior into the discrimination of detection results, false detections that are similar in appearance but have unreasonable geometry can be eliminated in real time, rather than relying solely on network confidence. This significantly reduces the probability of false detections entering the downstream pose calculation or control guidance link, avoiding the risk of pose estimation jumps or misguided navigation. On the other hand, when detection experiences a phase failure, the target tracking path provides continuous position estimation within a limited range, avoiding interruption of visual navigation output. After detection recovers, the detection link can be switched back, achieving adaptive fusion between detection and tracking, balancing accuracy and continuity.

[0047] As a preferred embodiment, this embodiment provides a detailed explanation of how to output the current detection result and update the parameters based on the current detection result. Specifically, it is necessary to output the current detection result and update the target flight parameters based on the current detection result, so that target detection in the next frame can be performed based on the updated flight parameters. It should be noted that after completing target detection, false detection removal, comprehensive judgment of detection effectiveness, and target tracking, an adaptive selection can be made based on the fusion decision to ultimately use the detection result or the tracking result for the target UAV image, and multiple sets of coordinate results including original / corrected / limited coordinates and corresponding status flags are output for subsequent guidance / control or upper-layer services.

[0048] The fusion decision is mainly based on the state variables and intermediate results shown in Table 1, and Table 1 is as follows: Table 1. Key Inputs and Meanings for Fusion Decision Making ; It should be noted that the fusion decision is not a numerically weighted average of detection and tracking, but rather employs a gated adaptive selection strategy. Specifically, if a detection exists in the current frame and passes false detection rejection, and if necessary, a prediction consistency check is performed, then the detection link result is output. If the detection is invalid, but initialization has been completed and tracking is still within the allowed number of attempts, then the tracking link result is output, and false detection rejection is also required. When both detection and tracking are unavailable, `valid=false` is output, and critical outputs are set to zero or a safety value to prevent erroneous information from entering the downstream link. Furthermore, the selection rules for the fusion output are shown in Table 2, and Table 2 is as follows: Table 2. Merging Output Selection Rules ; It should be further explained that in this embodiment, several status fields need to be written into the result structure to indicate the source of the current frame output and the reason for its rejection. Typical fields include: detection_status, false_detection_status, forecast_status, limiting_flag, etc. The specific output status fields and their corresponding meanings are shown in Table 3, as follows: Table 3 Output Status Fields and Typical Meanings ; Furthermore, once the target drone image is finally output as valid, regardless of whether it comes from detection or tracking, the system generates two types of "enhanced coordinate output" to improve the stability of downstream guidance / control. Figure 6 The diagram shows the relationship between the output fields. The first category is pixel correction, which corrects the bounding box and bottom pixel coordinates when the inertial navigation state word is valid. Its core idea is to utilize the relationship between the actual pose (yaw / pitch / roll) and the "theoretical pose" (the ideal pose constructed using runway_yaw) to map the observed pixels to a more consistent reference pose, thereby reducing the impact of pose perturbations on pixel measurements. It also outputs bounds_rectify_. Fields such as `mid_line_rectify.bottom` are used in the implementation. ; Where K is the camera intrinsic parameter matrix, R is the rotation matrix of the actual flight attitude in the current frame, Rtheory is the theoretical attitude rotation matrix, W is the homogeneous pixel coordinates of the runway bottom point actually observed in the current frame, A_calc is an intermediate direction vector representing the direction of the bottom point observed from the actual attitude in the body coordinate system, and Attitude is... TRefers to a specific row vector of the actual attitude rotation matrix, Attitude_Theory T This is the row vector corresponding to the theoretical attitude rotation matrix, used to extract the projection component of the optical axis direction under the theoretical attitude, representing the depth scaling reference under the theoretical attitude.

[0049] The second type is optional amplitude limiting. When conditions such as valid target detection, successful gating, valid inertial navigation status word, and valid visual guidance are met, the amplitude limiting value of the centerline bottom point is corrected to avoid large jumps in the bottom point y-value due to short-term anomalies. During processing, a historical buffer is maintained and amplitude limiting is triggered at appropriate times to make the output smoother.

[0050] Furthermore, historical state updates and safety outputs are also required. Historical state updates are used for prediction in the next frame. When detection initialization is complete or the current frame output is valid, the system updates: `last_runway_info_`: saves the latest available runway bounding box and bottom point information (tracking also updates this structure); `obj_lats_valid_flight_param_` and `last_bottom_`: serve as baseline inputs for stage three prediction verification; `last_input_frame_`: is the reference frame used for tracker initialization. Safety outputs refer to the safe handling of invalid outputs. When `result->valid=false`, critical outputs (such as bottom point coordinates, angles, etc.) are set to zero or their default values ​​and output via callback to prevent downstream misuse of abnormal information.

[0051] As a preferred embodiment, this embodiment describes the specific process framework of the UAV runway target detection method. The system uses image frames and current frame flight parameters as input data for each frame. The image data is pre-processed to meet the input requirements of the detection network; the flight parameters include pitch angle, roll angle, yaw angle, runway heading angle, altitude, and inertial navigation status words.

[0052] The validity determination of the inertial status word serves as a pre-gate control for the entire process: if the inertial navigation status word is invalid, it indicates that the current flight parameters are unreliable, and the system will not enter the core false detection judgment link of this invention, but can switch to state machine update or other safety strategy processing; only when the inertial navigation status word is valid is it allowed to enter the subsequent target detection and false detection verification link.

[0053] After gating, image preprocessing is performed to transform the original image into a data format that meets the input size requirements of the detection network. Then, target detection is performed, with the detection neural network inferring and outputting candidate runway target boxes along with their confidence and category information. Each detection result undergoes preliminary screening: a candidate result is qualified to proceed to the subsequent false positive verification stage if and only if the category identifier class_id=0 (runway target) and the confidence score conf≥0.60; candidate results that do not meet the conditions are directly filtered out.

[0054] Based on the initial screening of candidate detection results, the system proceeds to a new type of false detection judgment (angle constraint verification). This step uses the current frame's flight attitude and runway heading angle to calculate the angle indices ty_tx and tz_tx, and checks whether they fall within a preset threshold range (ty_tx∈[-5°,+5°], tz_tx∈[0°,6°]). If they exceed the range, they are judged as false detections and are removed; if they are within the range, this level of verification is passed.

[0055] Meanwhile, the system determines whether to enable prediction consistency verification based on the completion status of detection initialization. Verification is enabled when the number of consecutive valid detection frames reaches 3 or more (detection initialization is complete); otherwise, prediction consistency verification is skipped. Prediction consistency verification uses the target bottom pixel position and flight attitude from the previous valid frame, combined with the current frame's attitude and camera intrinsic parameters, to calculate the predicted pixel position of the bottom point in the current frame, and compares the deviation with the actual detection position (a deviation ≥ 20 pixels is considered a false detection). Only detection results that pass this level of verification are considered "detection link valid".

[0056] The consecutive valid frame count is then updated: if the current frame detects a valid link, the consecutive valid frame count is incremented; if invalid, the count is reset to zero. When the count reaches 3 frames, the detection initialization is considered complete, and subsequent frames can then enable prediction consistency verification and be allowed to enter the tracking link.

[0057] The fusion selection is the core decision-making process in the entire process. The fusion strategy adopts the "detection priority" principle: when the detection link is valid, the detection result is used as the output of the current frame; when the detection link is invalid, it is determined whether the tracking link is available. If it is available, the tracking result is used as a fallback; if both detection and tracking are invalid, an invalid state is output.

[0058] Tracking updates are triggered when the detection link is invalid and detection initialization is complete. If the tracker is not yet initialized, initialization is performed based on the most recent valid detection result (using the MedianFlow algorithm by default); once initialized, tracking updates are executed. Tracking results must be re-evaluated using a new type of false detection (angle constraint verification). Only those that pass the verification are considered valid tracking candidates for fusion selection. The tracking module is limited by a maximum number of continuous frames (20 frames in this embodiment); exceeding this limit terminates the tracking link.

[0059] When both detection and tracking fail, the system performs a safety zeroing / default value process, setting the key output coordinates to zero or to a preset safety value to prevent erroneous information from entering the downstream link.

[0060] For all candidate results deemed valid output (regardless of whether they originate from detection or tracking), the system sequentially performs output enhancement processing: First, pixel correction is performed, which uses the relationship between theoretical and actual attitudes to map the observed pixel coordinates to a standard attitude based on the runway heading, eliminating pixel measurement fluctuations caused by attitude disturbances; then, optional data limiting is performed, which uses historical buffers to limit and correct the bottom point, suppressing short-term abnormal jumps and making the output smoother and more stable.

[0061] Finally, the system outputs the complete processing results through the output interface, including a validity flag, status fields (output source, reason for rejection, prediction verification status, etc.) and multiple sets of coordinate results (original coordinates, corrected coordinates, and amplitude-limited coordinates), for use by downstream navigation guidance or flight control modules.

[0062] As a preferred embodiment, such as Figure 7 The diagram illustrates the module composition and data flow of a UAV visual navigation system. This embodiment provides a detailed description of the system. The system input side includes two parallel data paths: an image acquisition / input path and a flight parameter input path. The image acquisition path uses the UAV's onboard camera to acquire real-time image frames, which are then digitized by the video acquisition module and sent to the pre-processing module. The flight parameter input path obtains real-time flight parameters (including pitch angle, roll angle, yaw angle, runway heading angle, altitude, and inertial navigation status words) from the flight control system, providing attitude and prior information input for subsequent processing stages.

[0063] After the image data undergoes preprocessing operations such as size transformation by the preprocessing module, it enters the flight parameter validity gating stage. This gating stage determines whether the flight parameters of the current frame are valid based on the inertial navigation status word: if the flight parameters are invalid, the current frame does not enter the core false detection judgment link of this invention, and the system switches to safety policy processing; if the flight parameters are valid, it is allowed to enter the subsequent target detection and false detection verification link.

[0064] Gated image data is fed into the target detection module, where the detection neural network infers and outputs candidate runway target detection boxes along with their confidence and category information. After being mapped back to the original image size, preliminary validity screening is performed (the category is runway and the confidence level is not lower than 0.60). The candidate detection results that pass the screening are fed into two false detection rejection paths in parallel: The first path is the prediction consistency verification module, which uses the coordinates of the flyby point and the bottom point of the previous valid frame, combined with the flyby point and camera intrinsic parameters of the current frame, to calculate the predicted pixel position of the target bottom point in the current frame, and compares the deviation with the actual detection position. If the deviation exceeds the limit, it is judged as a false detection and removed. The second path is the false detection judgment (angle constraint) module, which uses the flight attitude and runway heading angle of the current frame to calculate the angle indexes ty_tx and tz_tx, and checks whether they fall within the preset threshold range. If they exceed the limit, they are judged as false detections and removed.

[0065] After the two-way verification is completed, the system enters the detection validity comprehensive judgment module. It controls the timing of the prediction verification and the access conditions of the tracking link by counting consecutive valid frames (≥3 frames are considered as initialization completion). At the same time, it maintains the flight participation baseline of the previous valid frame to provide historical basis for the prediction verification of subsequent frames.

[0066] When the detection fails but the initialization completion condition has been met, the system starts the tracking module and initializes the tracker with the most recent valid detection result (the MedianFlow algorithm is used by default). The target position is updated within a limited number of frames, and the candidate bottom points obtained by tracking are also sent to the angle constraint module of stage four for false detection and elimination, so as to ensure that the tracking results also meet the geometric rationality requirements.

[0067] Subsequently, the fusion decision module (gating selection) integrates the detection validity status, prediction verification results, false detection judgment results, and tracking validity status, adopting a gating adaptive selection strategy of "detection first, tracking as a fallback, and output invalid if all are invalid" to determine whether the detection result or the tracking result is ultimately used as the valid output candidate for this frame.

[0068] Before the final output, the system performs two layers of post-processing enhancement on valid candidates: pixel correction module and amplitude limiting module (optional). Pixel correction uses the relationship between theoretical attitude and actual attitude to eliminate the influence of attitude disturbance on pixel measurement and outputs the corrected coordinates. Amplitude limiting module performs amplitude limiting correction on the bottom point through historical buffer to suppress short-term abnormal jumps.

[0069] The system ultimately outputs the processing results through the output interface / callback, including a validity flag, status fields (including output source, reason for rejection, prediction verification status, etc.) and multiple sets of coordinate results (original / corrected / limited), for use by downstream navigation guidance or flight control modules.

[0070] See Figure 8 As shown, this embodiment of the invention discloses a UAV runway target detection device, applied to a UAV visual navigation system, comprising: The target detection module 11 is used to perform runway target detection on the target drone image collected by the target drone through a preset detection network if the target flight parameters of the target drone meet the preset parameter conditions, so as to obtain the corresponding runway target detection result. The false detection judgment module 12 is used to calculate the angle parameters corresponding to the target drone image based on the runway target detection result, the image parameters corresponding to the target drone image, and the drone parameters of the target drone, and to determine whether there is a false detection in the target drone image based on the angle parameters, so as to obtain the corresponding false detection result; the angle parameters are the parameters of the runway angle in the target drone image; The prediction initialization judgment module 13 is used to determine the current valid frame count based on the false detection result and the historical valid frame count, and to determine whether the prediction initialization is completed based on the current valid frame count. The first detection result determination module 14 is used to perform prediction consistency verification on the target UAV image if prediction initialization has been completed, and determine whether to take the runway target detection result as the current detection result based on the prediction consistency verification result. The second detection result determination module 15 is used to perform target tracking based on the target UAV image using a preset tracker if prediction initialization has not been completed, and to determine the current detection result based on the runway target detection result and the obtained tracking result.

[0071] In some embodiments, the target detection module 11 may specifically include: The data acquisition unit is used to acquire the target UAV image currently captured by the airborne camera of the target UAV and the target flight parameters when the target UAV image was acquired; the target flight parameters include attitude angles, runway heading angles and inertial navigation status words; the attitude angles include pitch angle, roll angle and yaw angle; An image detection unit is used to determine whether the inertial navigation status word is valid. If the inertial navigation status word is valid, the target drone image collected by the target drone is subjected to runway target detection through a preset detection network to determine the target bounding box coordinates, detection confidence and target category label corresponding to the target drone image. The bounding box mapping unit is used to map the target bounding box from the target detection network input size coordinate system to the original image coordinate system based on the ratio between the preset detection network input size and the original image size, so as to obtain the mapped bounding box; The bounding box validity determination unit is used to determine the validity of the mapped bounding box based on the target category label and the detection confidence. If the target category label is a runway and the detection confidence is greater than or equal to a preset confidence threshold, then the mapped bounding box is determined to be valid. The detection result determination unit is used to take the valid result of the mapped bounding box, the target category label, and the detection confidence as the runway target detection result.

[0072] In some embodiments, the false detection judgment module 12 may specifically include: The image parameter determination unit is used to determine the rotation matrix, bottom pixel coordinates, and intrinsic parameter matrix corresponding to the airborne camera of the target drone image; the rotation matrix is ​​a matrix describing the orientation of the target drone; the bottom pixel coordinates are the coordinates of the center point of the bottom edge of the bounding box corresponding to the target drone image in the target drone image; The intermediate quantity determination unit is used to construct the target homogeneous coordinates based on the bottom pixel coordinates, and use the product of the target homogeneous coordinates, the transpose of the rotation matrix, and the inverse of the intrinsic parameter matrix as the intermediate quantity of the target UAV image. An angle index calculation unit is used to determine the numerical parameters corresponding to the intermediate quantity, and calculate the first angle index corresponding to the target UAV image based on the numerical parameters through tangent operation, and calculate the second angle index corresponding to the target UAV image based on the numerical parameters through inverse tangent operation; the first angle index and the second angle index are indices representing directional deviation; The first false detection result determination unit is used to determine that there is no false detection in the target UAV image if the first angle index falls within the first preset angle range and the second angle index falls within the second preset angle range, so as to generate a first false detection result. The second false detection result determination unit is used to determine that the target drone image has a false detection if the first angle indicator does not fall within the first preset angle range or the second angle indicator does not fall within the second preset angle range, remove the target drone image, and generate a second false detection result.

[0073] In some embodiments, the prediction initialization judgment module 13 may specifically include: The effective frame count determination unit is used to take the sum of the historical effective frame count and the preset effective frame increment as the current effective frame count if the false detection result is the first false detection result. The initialization completion judgment unit is used to determine that the prediction initialization has been completed if the current valid frame count is not less than the preset valid frame count threshold.

[0074] In some embodiments, the first detection result determination module 14 may specifically include: The historical intermediate vector calculation unit is used to determine the historical rotation matrix of the previous valid frame if prediction initialization has been completed, and to calculate the historical intermediate vector based on the transpose of the historical rotation matrix, the inverse of the intrinsic parameter matrix, and the homogeneous coordinates of the previous valid frame. A pixel position prediction unit is used to construct a parameter vector based on the ratio between the components of the historical intermediate vector, and to predict the pixel position based on the parameter vector, the intrinsic parameter matrix, and the rotation matrix to obtain the predicted pixel position. The image removal unit is used to compare the predicted pixel position and the bottom pixel coordinates for consistency. If the deviation between the predicted pixel position and the bottom pixel coordinates is greater than a preset deviation threshold, the target drone image is removed. The first detection result determination unit is used to determine that the target UAV image has passed the prediction consistency check if the deviation between the predicted pixel position and the bottom pixel coordinates is not greater than the preset deviation threshold, and to take the runway target detection result as the current detection result.

[0075] In some embodiments, the second detection result determining module 15 may specifically include: The tracker determination unit is used to convert the historical bounding box of the previous valid frame into the tracker initialization box of the preset tracker if the prediction initialization has not been completed, and to initialize the preset tracker based on the target UAV image to obtain the target tracker. The false detection judgment unit is used to perform target tracking on the target UAV image based on the target tracker, and to judge the false detection of the obtained tracking result; The second detection result determination unit is used to take the tracking result as the current detection result if the tracking result passes the false detection judgment. The tracking termination unit is used to terminate the current tracking if the tracking result fails the false detection judgment.

[0076] In some embodiments, the unmanned aerial vehicle (UAV) runway target detection device may further include: The flight parameter update unit is used to output the current detection result and update the target flight parameters according to the current detection result, so as to perform target detection in the next frame based on the updated flight parameters.

[0077] Furthermore, embodiments of this application also disclose an electronic device, Figure 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0078] Figure 9This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the UAV runway target detection method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0079] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0080] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0081] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the UAV runway target detection method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0082] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned unmanned aerial vehicle (UAV) runway target detection method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0083] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0084] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0085] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0086] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0087] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting targets on a UAV runway, characterized in that, Applications in UAV visual navigation systems include: If the target flight parameters of the target UAV meet the preset parameter conditions, then the target UAV image collected by the target UAV is subjected to runway target detection through the preset detection network to obtain the corresponding runway target detection result. Based on the runway target detection result, the image parameters corresponding to the target UAV image, and the UAV parameters of the target UAV, the angle parameters corresponding to the target UAV image are calculated, and the presence of false detections in the target UAV image is determined according to the angle parameters to obtain the corresponding false detection results; the angle parameters are the parameters of the runway angle in the target UAV image. The current valid frame count is determined based on the false detection results and the historical valid frame count, and the prediction initialization is determined based on the current valid frame count. If prediction initialization has been completed, then the prediction consistency check is performed on the target UAV image, and the runway target detection result is used as the current detection result based on the prediction consistency check result. If prediction initialization is not completed, target tracking is performed based on the target UAV image using a preset tracker, and the current detection result is determined based on the runway target detection result and the obtained tracking result.

2. The unmanned aerial vehicle (UAV) runway target detection method according to claim 1, characterized in that, If the target flight parameters of the target UAV meet the preset parameter conditions, then runway target detection is performed on the target UAV image acquired by the target UAV through a preset detection network to obtain the corresponding runway target detection results, including: The system acquires the target drone's currently captured image from its onboard camera, as well as the target drone's flight parameters at the time the image was acquired. The target flight parameters include attitude angles, runway heading angles, and inertial navigation status words. The attitude angles include pitch angle, roll angle, and yaw angle. Determine whether the inertial navigation status word is valid. If the inertial navigation status word is valid, then perform runway target detection on the target UAV image collected by the target UAV through a preset detection network to determine the target bounding box coordinates, detection confidence, and target category label corresponding to the target UAV image. Based on the ratio between the preset detection network input size and the original image size, the target bounding box is mapped from the target detection network input size coordinate system to the original image coordinate system to obtain the mapped bounding box; The validity of the mapped bounding box is determined based on the target category label and the detection confidence. If the target category label is a runway and the detection confidence is greater than or equal to a preset confidence threshold, then the mapped bounding box is determined to be valid. The valid bounding box result after mapping, the target category label, and the detection confidence score are used as the runway target detection result.

3. The unmanned aerial vehicle (UAV) runway target detection method according to claim 1, characterized in that, The step of calculating the angle parameters corresponding to the target drone image based on the runway target detection result, the image parameters corresponding to the target drone image, and the drone parameters of the target drone, and determining whether there are false detections in the target drone image based on the angle parameters, to obtain the corresponding false detection result, includes: Determine the rotation matrix, bottom pixel coordinates, and intrinsic parameter matrix corresponding to the target drone image; the rotation matrix is ​​a matrix describing the orientation of the target drone; the bottom pixel coordinates are the coordinates of the center point of the bottom edge of the bounding box corresponding to the target drone image in the target drone image; The target homogeneous coordinates are constructed based on the base pixel coordinates, and the product of the target homogeneous coordinates, the transpose of the rotation matrix, and the inverse of the intrinsic parameter matrix is ​​used as an intermediate quantity of the target UAV image. The numerical parameters corresponding to the intermediate quantity are determined, and the first angle index corresponding to the target UAV image is calculated based on the numerical parameters through tangent operation. The second angle index corresponding to the target UAV image is calculated based on the numerical parameters through inverse tangent operation. The first angle index and the second angle index are indices representing directional deviation. If the first angle indicator falls within the first preset angle range and the second angle indicator falls within the second preset angle range, then it is determined that there is no false detection in the target drone image, so as to generate a first false detection result. If the first angle indicator does not fall within the first preset angle range, or the second angle indicator does not fall within the second preset angle range, then it is determined that the target drone image has a false detection, the target drone image is removed, and a second false detection result is generated.

4. The unmanned aerial vehicle (UAV) runway target detection method according to claim 3, characterized in that, The step of determining the current valid frame count based on the false detection result and the historical valid frame count, and determining whether the prediction initialization is complete based on the current valid frame count, includes: If the false detection result is the first false detection result, then the sum of the historical valid frame count and the preset valid frame increment is used as the current valid frame count; If the current valid frame count is not less than the preset valid frame count threshold, then the prediction initialization is determined to be complete.

5. The unmanned aerial vehicle (UAV) runway target detection method according to claim 3, characterized in that, If prediction initialization has been completed, then a prediction consistency check is performed on the target UAV image, and based on the prediction consistency check result, it is determined whether to use the runway target detection result as the current detection result, including: If prediction initialization has been completed, the historical rotation matrix of the previous valid frame is determined, and the historical intermediate vector is calculated based on the transpose of the historical rotation matrix, the inverse of the intrinsic parameter matrix, and the homogeneous coordinates of the previous valid frame. A parameter vector is constructed based on the ratios between the components of the historical intermediate vector, and pixel position prediction is performed based on the parameter vector, the intrinsic parameter matrix, and the rotation matrix to obtain the predicted pixel position. A consistency comparison is performed between the predicted pixel position and the bottom pixel coordinates. If the deviation between the predicted pixel position and the bottom pixel coordinates is greater than a preset deviation threshold, the target drone image is removed. If the deviation between the predicted pixel position and the bottom pixel coordinates is not greater than the preset deviation threshold, then the target UAV image is determined to have passed the prediction consistency check, and the runway target detection result is taken as the current detection result.

6. The unmanned aerial vehicle (UAV) runway target detection method according to claim 1, characterized in that, If prediction initialization is not completed, target tracking is performed based on the target UAV image using a preset tracker, and the current detection result is determined based on the runway target detection result and the obtained tracking result, including: If prediction initialization is not completed, the historical bounding box of the previous valid frame is converted into the tracker initialization box of the preset tracker, and the preset tracker is initialized based on the target UAV image to obtain the target tracker; The target tracker performs target tracking on the target UAV image and determines false detections based on the tracking results. If the tracking result passes the false detection judgment, then the tracking result is taken as the current detection result; If the tracking result fails the false detection judgment, the current tracking is terminated.

7. The unmanned aerial vehicle runway target detection method according to any one of claims 1 to 6, characterized in that, Also includes: The current detection result is output, and the target flight parameters are updated based on the current detection result, so as to perform target detection in the next frame based on the updated flight parameters.

8. A runway target detection device for unmanned aerial vehicles (UAVs), characterized in that, Applications in UAV visual navigation systems include: The target detection module is used to perform runway target detection on the target drone image collected by the target drone through a preset detection network if the target flight parameters of the target drone meet the preset parameter conditions, so as to obtain the corresponding runway target detection result. The false detection judgment module is used to calculate the angle parameters corresponding to the target UAV image based on the runway target detection result, the image parameters corresponding to the target UAV image, and the UAV parameters of the target UAV, and to determine whether there is a false detection in the target UAV image based on the angle parameters, so as to obtain the corresponding false detection result; the angle parameters are the parameters of the runway angle in the target UAV image; The prediction initialization judgment module is used to determine the current valid frame count based on the false detection result and the historical valid frame count, and to determine whether the prediction initialization is completed based on the current valid frame count. The first detection result determination module is used to perform prediction consistency verification on the target UAV image if prediction initialization has been completed, and determine whether to take the runway target detection result as the current detection result based on the prediction consistency verification result. The second detection result determination module is used to perform target tracking based on the target UAV image using a preset tracker if prediction initialization has not been completed, and to determine the current detection result based on the runway target detection result and the obtained tracking result.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the unmanned aerial vehicle runway target detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the unmanned aerial vehicle runway target detection method as described in any one of claims 1 to 7.

Images