Unmanned aerial vehicle accurate landing control system based on industrial vision

Abstract

The invention discloses an unmanned aerial vehicle accurate landing control system based on industrial vision, and relates to the technical field of unmanned aerial vehicle visual navigation and control, and the system comprises the steps: collecting a landing field image sequence through an optical industrial camera array carried on an unmanned aerial vehicle, and carrying out the dynamic background subtraction and multi-channel image registration through an image preprocessing module; and generating a fused differential image which highlights the specific optical identification. The feature recognition module performs morphological skeleton extraction and angular point group detection on the image, acquires sub-pixel-level coordinates of identification feature points, and forms a real-time image feature point set. And the pose resolving module carries out distortion correction on the point set according to camera calibration parameters, and solves real-time pose parameters of the unmanned aerial vehicle relative to the identifier by using a three-dimensional space geometric model of the identifier through a perspective point resolving algorithm. The control instruction module generates a landing control instruction according to the parameters. According to the method, the robustness and the pose calculation precision of visual landing in a complex environment are improved.

Description

Technical Field

[0001] This invention belongs to the field of UAV visual navigation and control technology, specifically a UAV precision landing control system based on industrial vision. Background Technology

[0002] Unmanned aerial vehicle (UAV) visual landing technology widely employs schemes that identify pre-defined planar cooperative markers. Existing image preprocessing techniques largely rely on general filtering or static background modeling. In real outdoor scenarios, dynamic changes in natural lighting and unstructured interference from the background environment make it difficult to distinguish markers from the background, challenging the stability of feature extraction. Current feature recognition methods typically estimate the pose of the marker by relying on a two-dimensional planar model after obtaining the pixel coordinates of its corner points. This method is limited by pixel-level positioning accuracy, and the assumption of an angle between the marker and the camera's imaging plane in the two-dimensional planar model introduces significant pose calculation errors, making it difficult to directly output accurate and complete three-dimensional spatial pose.

[0003] A technical solution is needed that can effectively overcome dynamic environmental interference, achieve highly robust extraction of identifying features, and break through the limitations of two-dimensional models to directly perform high-precision six-DOF pose calculation based on accurate three-dimensional geometric information. This requires the preprocessing stage to effectively fuse multi-view information and suppress dynamic background noise, while the pose calculation stage possesses a computational framework with sub-pixel-level feature localization capabilities and direct matching with the three-dimensional spatial model. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; To this end, the present invention proposes a precision landing control system for unmanned aerial vehicles based on industrial vision, comprising: The image acquisition module is used to acquire raw optical image sequences of the landing site area at a fixed sampling frequency using an optical industrial camera array mounted on the UAV platform. The image preprocessing module is used to input the original optical image sequence into the image feature preprocessor, perform dynamic background subtraction and multi-channel image registration, and generate a fused difference image. The fused difference image is used to highlight the difference between a preset specific optical marker in the landing site and the background environment. The feature recognition module is used to perform morphological skeleton extraction and corner group detection on the fused differential image, identify all feature points that constitute the specific optical identifier, calculate the sub-pixel level coordinates of each feature point in the image coordinate system, and combine them to generate a real-time image feature point set of the landing identifier. The pose calculation module is used to perform distortion correction on the real-time image feature point set of the landing mark based on the pre-calibrated intrinsic parameter matrix and distortion coefficient of the optical industrial camera array, and to solve the real-time pose parameters of the UAV relative to the specific optical mark by using the preset three-dimensional spatial geometric model of the specific optical mark and the perspective point calculation algorithm. The control command module is used to generate UAV landing control commands based on the real-time pose parameters.

[0005] Further, the original optical image sequence is input into an image feature preprocessor to perform dynamic background subtraction and multi-channel image registration, generating a fused difference image, including: The original optical image sequence contains synchronous imaging data of the infrared spectral channel and the visible spectral channel; The infrared spectral channel image and the visible spectral channel image are separated from the original optical image sequence to form a synchronous but independent spectral image pair; Perform grayscale difference operation on the infrared spectral channel image of the current frame and the infrared spectral channel image of the historical frame to generate an infrared difference image; Perform a vector norm difference operation based on the color space on the visible spectral channel image of the current frame and the visible spectral channel image of the historical frame to generate a visible light difference image; The feature descriptors of the infrared differential image and the visible light differential image are extracted using the scale-invariant feature transformation algorithm, and the homography transformation matrix between the images is calculated based on the matching relationship of the feature descriptors. The spatial coordinate system of the visible light differential image is then aligned with the spatial coordinate system of the infrared differential image. The infrared differential image and the visible light differential image, which are based on the spatial coordinate system one, are fused pixel-level according to a preset weighting coefficient, and the fused differential image is output.

[0006] Furthermore, morphological skeleton extraction and corner cluster detection are performed on the fused difference image to identify all feature points constituting the specific optical identifier, including: An adaptive threshold segmentation algorithm is applied to the fused difference image to obtain a binarized image; Perform continuous multi-round iterative erosion and dilation operations on the binarized image until the width of all connected regions in the image converges to a single pixel to obtain the skeleton image. In the skeleton image, all pixels are traversed and the number of foreground pixels in their eight neighborhoods is counted. Pixels with one or more or more foreground pixels in their neighborhoods are marked as skeleton endpoints or branch points. Within the neighborhood of the marked skeleton endpoints and branch points, a corner response function is applied to the original fused difference image to calculate and filter out pixels with local response values ​​exceeding a threshold as candidate corner points. Based on the preset topology of the specific optical marker, the candidate corner points are grouped and geometrically constrained for verification. Outliers that do not conform to the preset topology are eliminated, and the verified candidate corner points are confirmed as the feature points.

[0007] Furthermore, using a preset three-dimensional spatial geometric model of the specific optical marker, and through a perspective point calculation algorithm, the real-time pose parameters of the UAV relative to the specific optical marker are solved, including: The real-time pose parameters include a position vector and a pose quaternion; Establish a ground coordinate system with the center of the specific optical mark as the origin, and obtain the three-dimensional coordinates of each key feature point in the three-dimensional spatial geometric model of the specific optical mark in the ground coordinate system; The two-dimensional image coordinates of each feature point in the real-time image feature point set of the distortion-corrected landing mark are paired with the three-dimensional coordinates of the corresponding key feature points in the three-dimensional spatial geometric model of the specific optical mark. Based on the pinhole imaging model of an optical industrial camera array, an overdetermined set of equations defined by a paired point set is constructed, which describes the mathematical relationship between three-dimensional spatial points projected onto a two-dimensional image plane. An optimization algorithm based on orthogonal iteration is used to solve the overdetermined system of equations, minimize the image projection error, and directly calculate the rotation matrix and translation vector of the optical industrial camera array coordinate system relative to the ground coordinate system. The calculated rotation matrix is ​​converted into attitude quaternion form, and the translation vector is used as the position vector to form the real-time pose parameters.

[0008] Further, based on the real-time pose parameters, a UAV landing control command is generated, including: The position vector in the real-time pose parameters is input into the position controller, which calculates the velocity control components along the three coordinate axes in the three-dimensional ground coordinate system based on the deviation between the current position of the UAV and the target landing point. The attitude quaternion in the real-time pose parameters is input into the attitude controller. The attitude controller calculates the angular velocity control components around the three coordinate axes in the UAV body coordinate system based on the deviation between the current attitude of the UAV and the target landing attitude. By fusing the velocity control component and the angular velocity control component, and superimposing a time-domain filter determined by the sampling frequency of the optical industrial camera array, a smooth basic command for UAV flight control is generated. Acceleration and angular velocity feedback provided by the UAV inertial measurement unit are introduced to dynamically compensate and correct the basic flight control commands of the UAV, generating the final UAV control surface and power system execution commands.

[0009] Furthermore, the method for constructing the preset three-dimensional spatial geometric model of the specific optical identifier includes: Obtain initial three-dimensional model data of the specific optical sign, wherein the initial three-dimensional model data describes the physical shape of the specific optical sign using the coordinates of multiple three-dimensional spatial points and the geometric topology formed by connecting the three-dimensional spatial points; Place the physical object containing the specific optical mark on a ground calibration platform with a known location and orientation; The drone equipped with the aforementioned optical industrial camera array is driven to cruise along multiple preset calibration flight paths. During the cruise, the optical industrial camera array captures multiple high-resolution calibration images containing the complete physical object of the specific optical identifier from multiple different spatial angles and distances. Extract two-dimensional image feature points of the specific optical identifier from each frame of the high-resolution calibration image; Based on the pre-calibrated intrinsic parameter matrix and distortion coefficients of the optical industrial camera array, as well as the pose of the UAV when acquiring each frame of the high-resolution calibration image, and combined with the feature points of the two-dimensional image, the coordinates of all three-dimensional spatial points in the initial three-dimensional model data are iteratively optimized using the bundle adjustment optimization algorithm. Based on the optimized three-dimensional spatial point coordinates, a fine mesh model of the specific optical identifier is generated through a three-dimensional reconstruction algorithm, and the three-dimensional spatial geometric model is defined using the coordinates of all vertices in the fine mesh model and the mesh patches formed by connecting the vertices.

[0010] Furthermore, it also includes a prediction and risk assessment module, used for: The real-time pose parameters for multiple consecutive sampling periods are stored in a circular queue to form a real-time pose parameter time series. Polynomial fitting is performed on the position vector components in the real-time pose parameter time series to predict the estimated motion trajectory of the UAV in three-dimensional space in the next few sampling periods. Differentiate the attitude quaternion components in the real-time pose parameter time series to obtain the current attitude angular velocity and attitude angular acceleration of the UAV. Based on the estimated motion trajectory and current attitude dynamic parameters, combined with the preset three-dimensional geofence model of the landing site, it is determined whether the UAV is at risk of spatial interference with obstacles or deviating from the preset safe landing corridor in the future prediction period. When a risk is identified, a risk type code and a risk level signal are generated. These risk type code and risk level signal are used to trigger subsequent instruction correction logic.

[0011] Furthermore, when a risk is determined to exist, a risk type code and a risk level signal are generated. These risk type code and risk level signal are used to trigger subsequent instruction correction logic, including: If the risk assessment result indicates a risk of spatial interference with an obstacle, the distance between the estimated trajectory and the nearest obstacle surface in the three-dimensional geofence model of the landing site is calculated, and a first-class risk level signal is generated based on the ratio of the distance to a preset safety threshold. If the risk assessment result indicates that there is a risk of deviating from the preset safe landing corridor, the average lateral deviation distance between the centerline of the estimated trajectory and the centerline of the preset safe landing corridor is calculated, and a second type of risk level signal is generated based on the ratio of the average lateral deviation distance to the corridor width. The generated first-class risk level signal or second-class risk level signal, together with the corresponding risk type code, are encapsulated into a risk warning data packet; The risk warning data packet is sent to the position controller and attitude controller, and participates in the recalculation of the velocity control component and angular velocity control component as an external input condition.

[0012] Furthermore, it also includes an identifier switching module for: During the continuous processing of the original optical image sequence, the number of feature points successfully identified and used for calculation in each sampling period is counted in real time. When the number of successfully identified feature points is lower than a preset threshold, it is determined that the visibility or integrity of the current specific optical tag is insufficient, and a tag switching request is triggered. The tag switching request is sent to the optical tag display device at the landing site, instructing the optical tag display device to turn off the currently displayed specific optical tag pattern and activate a pre-stored alternative optical tag pattern; The optical industrial camera array continues to acquire the original optical image sequence containing the candidate optical marker pattern, and the subsequent process switches to the recognition and pose calculation of the candidate optical marker pattern.

[0013] Furthermore, it also includes a fusion positioning module, used for: Within a single sampling period, a set of real-time pose parameters are independently solved using the specific optical marker or the alternative optical marker pattern, which is denoted as the single-cycle solved pose. Accumulate multiple sets of single-cycle pose calculations across multiple consecutive sampling periods within a time window; Kalman filtering is performed on the position vectors in the multiple sets of single-cycle solved poses accumulated within the time window to estimate the optimal smooth position of the UAV within the filtering period. The attitude quaternions accumulated in the multiple single-cycle solved poses within the time window are fused based on quaternion averaging to estimate the optimal smooth attitude of the UAV within the filtering period. The optimal smooth position and optimal smooth attitude are output as the final real-time pose parameters for generating UAV landing control commands, replacing the single-cycle pose calculation with a single sampling period.

[0014] Compared with the prior art, the beneficial effects of the present invention are: Dynamic background subtraction and multi-channel image registration techniques are employed to generate fused differential images. Dynamic background subtraction can adapt to slow or periodic changes in lighting and background elements in a scene, effectively suppressing interference caused by global illumination fluctuations or local dynamic textures. Multi-channel image registration spatially aligns and fuses images synchronously acquired from different perspectives by an optical camera array, compensating for the loss or distortion of target information caused by the limitations of a single camera's field of view and parallax. This results in the stable generation of a high signal-to-noise ratio fused image that highlights specific optical features in complex, unstructured outdoor environments, improving the quality and reliability of input data in the feature recognition stage.

[0015] By performing morphological skeleton extraction and corner group detection on the fused image and calculating sub-pixel level coordinates, the geometric topology of the marker can be accurately reconstructed, and all real feature points constituting the marker can be identified. Its positioning accuracy exceeds the integer unit of pixels, reaching the sub-pixel level. On this basis, a pre-established precise three-dimensional spatial geometric model of a specific optical marker is directly used. Through a perspective point solution algorithm, the acquired two-dimensional sub-pixel feature point set is matched and solved with the three-dimensional model points. This method abandons the solution method that relies on the approximation assumption of a two-dimensional plane. It can directly and once output the real-time pose parameters of the UAV relative to the marker origin, which include three translations and three rotations, with centimeter-angle level accuracy. This provides the flight controller with higher-precision state feedback without intermediate conversion links. Attached Figure Description

[0016] Figure 1 This is a timing diagram of the industrial vision-based UAV precision landing control system described in this invention. Figure 2 Flowchart for feature point recognition; Figure 3 A flowchart for real-time pose parameter calculation; Figure 4 Technical performance analysis diagram for the construction phase of 3D model of UAV optical identifiers; Figure 5A core status monitoring curve for the entire process of precise drone landing. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 The image acquisition module acquires raw optical image sequences of the landing area at a fixed sampling frequency using an optical industrial camera array mounted on the UAV platform. The image preprocessing module inputs the raw optical image sequence into the image feature preprocessor, performs dynamic background subtraction and multi-channel image registration, and generates a fused difference image. This fused difference image highlights the difference between a pre-defined specific optical marker in the landing area and the background environment. The feature recognition module performs morphological skeleton extraction and corner group detection on the fused difference image, identifies all feature points constituting the specific optical marker, and calculates the sub-pixel level coordinates of each feature point in the image coordinate system, combining them to generate a real-time image feature point set for the landing marker. The pose calculation module performs distortion correction on the real-time image feature point set of the landing marker based on the pre-calibrated intrinsic parameter matrix and distortion coefficients of the optical industrial camera array. Using the pre-defined three-dimensional spatial geometric model of the specific optical marker, it solves for the real-time pose parameters of the UAV relative to the specific optical marker using a perspective point calculation algorithm. The control command module generates UAV landing control commands based on the real-time pose parameters.

[0019] See Figure 2 In one embodiment of the present invention, the image preprocessing module receives a raw optical image sequence from an optical industrial camera array. The raw optical image sequence contains imaging data of synchronously acquired infrared and visible spectral channels. The image preprocessing module internally includes an image feature preprocessor. The function of the image feature preprocessor is to perform dynamic background subtraction and multi-channel image registration on the raw optical image sequence, thereby generating a fused difference image. The first step is to separate the infrared spectral channel image and the visible spectral channel image from the raw optical image sequence, forming a synchronous but independent spectral image pair. Gray-level difference operations are performed between the infrared spectral channel image of the current frame and the infrared spectral channel image of a historical frame to generate an infrared difference image. Vector norm-based difference operations are performed between the visible spectral channel image of the current frame and the visible spectral channel image of a historical frame to generate a visible light difference image. The vector norm is defined as the three-dimensional vector magnitude corresponding to each pixel of the visible spectral channel image in the RGB color space, and the difference operation is performed between the vector magnitudes of corresponding pixels.

[0020] In some embodiments, feature descriptors are extracted from the infrared differential image and the visible light differential image using a scale-invariant feature transform algorithm. Based on the matching relationship of the feature descriptors, a homography transformation matrix is ​​calculated between the infrared and visible light differential images. The calculated homography transformation matrix is ​​then used to map the spatial coordinate system of the visible light differential image to the spatial coordinate system of the infrared differential image. The infrared and visible light differential images, after the spatial coordinate system mapping is completed, are then fused pixel-wise according to preset weighting coefficients. These weighting coefficients are, for example, set as the infrared differential image weighting coefficient α and the visible light differential image weighting coefficient β, satisfying α + β = 1 and α > 0, β > 0. The fusion operation outputs a fused differential image.

[0021] Optionally, the image feature preprocessor outputs the fused difference image to the feature recognition module. The feature recognition module performs morphological skeleton extraction and corner cluster detection on the fused difference image to identify all feature points constituting a specific optical identifier. The morphological skeleton extraction process first applies an adaptive threshold segmentation algorithm to the fused difference image to obtain a binarized image. Then, iterative erosion and dilation operations are performed on the binarized image until the width of all connected regions in the image converges to a single pixel, thus obtaining the skeleton image.

[0022] The process involves traversing all pixels in the skeleton image and counting the number of foreground pixels in their eight-neighborhood. Pixels with one or more foreground pixels in their neighborhood are marked as skeleton endpoints or branch points. Within the neighborhood of the marked skeleton endpoints and branch points, a corner response function is applied to the original fused difference image to calculate and select pixels with local response values ​​exceeding a preset threshold as candidate corner points.

[0023] In some embodiments, candidate corner points are grouped and geometrically constrained according to a preset topology of a specific optical marker. The preset topology defines the relative distance and angular relationships between feature points. The geometric constraint verification process calculates the actual distances and angles between candidate corner points and compares them with the preset topology, eliminating outliers that do not conform to the preset topology, and confirming the verified candidate corner points as feature points constituting the specific optical marker. The feature recognition module calculates the sub-pixel level coordinates of each feature point in the image coordinate system and combines the sub-pixel level coordinates of all feature points to generate a real-time image feature point set for the landing marker.

[0024] See Figure 3In one embodiment of the present invention, the pose calculation module receives a real-time image feature point set of the landing marker from the feature recognition module. The real-time image feature point set contains sub-pixel level coordinates of the feature points in the image coordinate system. The pose calculation module performs distortion correction on the real-time image feature point set of the landing marker based on the pre-calibrated intrinsic parameter matrix and distortion coefficients of the optical industrial camera array. The correction process applies pre-calibrated radial and tangential distortion models to transform the coordinates of the image feature points from the distorted image plane to the ideal pinhole imaging plane. A ground coordinate system is established with the center of the specific optical marker as the origin, and the three-dimensional coordinates of each key feature point in the preset three-dimensional spatial geometric model of the specific optical marker in the ground coordinate system are obtained. The ground coordinate system is typically defined as having the X and Y axes located in the marker plane, and the Z axis perpendicular to the marker plane and pointing upwards.

[0025] In some embodiments, the pose calculation module pairs the two-dimensional image coordinates of each feature point in the real-time image feature point set of the distortion-corrected landing marker with the three-dimensional coordinates of the corresponding key feature points in the three-dimensional spatial geometric model of a specific optical marker. The pairing relationship is predefined based on the marker topology of the feature points. Based on the pinhole imaging model of an optical industrial camera array, an overdetermined set of equations defined by these paired point sets is constructed. This overdetermined set of equations describes the mathematical relationship between the projection of three-dimensional spatial points onto the two-dimensional image plane. It can be understood that for each pair of matched feature points, there exists a projection equation: in: and This represents the two-dimensional pixel coordinates of the feature point on the camera image plane. , , This represents the three-dimensional coordinates of the feature point in the ground coordinate system. Represents a non-zero scale factor. This represents the intrinsic parameter matrix of an optical industrial camera array. This represents the rotation matrix from the ground coordinate system to the camera coordinate system. This represents the translation vector from the ground coordinate system to the camera coordinate system. A system of equations containing multiple equations can be established using multiple pairs of matching points. Since the number of unknowns is less than the number of equations, this system of equations constitutes an overdetermined system.

[0026] Optionally, an optimization algorithm based on orthogonal iteration can be used to solve this overdetermined system of equations. The orthogonal iteration algorithm optimizes the rotation matrix iteratively. With translation vector The goal is to minimize the error between the coordinates of two-dimensional image points obtained by projecting three-dimensional points and the coordinates of the actually detected two-dimensional image feature points; this error is called image projection error. An orthogonal iterative algorithm directly calculates the rotation matrix of the optical industrial camera array coordinate system relative to the ground coordinate system. With translation vector The pose calculation module will calculate the rotation matrix. Convert to attitude quaternion form and convert the translation vector As a position vector, the position vector, together with the attitude quaternion, constitutes the real-time pose parameters of the UAV relative to a specific optical marker.

[0027] In some embodiments, the transformation of attitude quaternions follows the standard mathematical transformation relationship from rotation matrix to quaternion, assuming the rotation matrix... It is known that the corresponding quaternion can be solved by calculating its trace and matrix elements. The position vector in the real-time pose parameters is the translation vector. Its weight This directly represents the coordinates of the origin of the optical industrial camera array coordinate system in the ground coordinate system, which also represents the position of the UAV relative to the center of a specific optical marker. The pose calculation module outputs real-time pose parameters, including the position vector and attitude quaternions, to the control command module to generate UAV landing control commands.

[0028] In one embodiment of the present invention, the control command module receives real-time pose parameters from the pose calculation module. These real-time pose parameters include a position vector and an attitude quaternion. The control command module internally includes a position controller and an attitude controller. The position controller processes the position vector, and the attitude controller processes the attitude quaternion. The position vector from the real-time pose parameters is input to the position controller. The position controller calculates the velocity control components along the three coordinate axes in a three-dimensional ground coordinate system based on the deviation between the current position of the UAV and the target landing point position. The target landing point position is a predefined fixed coordinate point in the ground coordinate system. The position vector represents the current coordinates of the UAV, and the difference between the two constitutes the position deviation vector. Based on this position deviation vector, the position controller calculates the desired compensation velocity using a proportional-integral-derivative control law. This desired compensation velocity is decomposed into velocity control components along the X, Y, and Z axes of the ground coordinate system.

[0029] In some embodiments, the attitude quaternion from the real-time pose parameters is input to the attitude controller. The attitude controller calculates the angular velocity control components around the three coordinate axes in the UAV body coordinate system based on the deviation between the current attitude of the UAV and the target landing attitude. The target landing attitude is usually defined as the attitude in which the UAV body coordinate system is aligned with the ground coordinate system. The attitude quaternion represents the current attitude of the UAV. The attitude error quaternion from the current attitude to the target attitude can be obtained through quaternion algebra operations. Based on this attitude error quaternion, the attitude controller calculates the desired compensation angular velocity using a proportional-integral-derivative control law. This desired compensation angular velocity is decomposed into angular velocity control components around the X-axis, Y-axis, and Z-axis of the UAV body coordinate system.

[0030] It is understandable that the control command module needs to fuse the velocity control components from the position controller and the angular velocity control components from the attitude controller. The fusion operation transforms the velocity control components in the ground coordinate system to the UAV body coordinate system, and combines them with the angular velocity control components in the body coordinate system to form the original vector of the UAV flight control basic commands. Optionally, during the formation of the UAV flight control basic commands, a time-domain filter determined by the sampling frequency of the optical industrial camera array is superimposed. The time-domain filter uses a first-order low-pass filter or a moving average filter to smooth the fused original command vector, suppressing high-frequency fluctuations introduced by image processing noise or pose calculation jitter, and generating smooth UAV flight control basic commands.

[0031] In some embodiments, acceleration and angular velocity feedback provided by the UAV's inertial measurement unit (IMU) are introduced to dynamically compensate and correct the UAV's basic flight control commands. The IMU provides real-time three-axis acceleration and three-axis angular velocity data of the UAV at a frequency higher than the sampling frequency of the optical industrial camera array. The dynamic compensation and correction process compares the UAV's basic flight control commands with the feedback data from the IMU, calculates the difference, and generates a correction value through a feedforward or feedback control loop. The correction value is superimposed on the UAV's basic flight control commands to generate the final UAV control surface and propulsion system execution commands. The UAV control surface and propulsion system execution commands are output in the form of pulse width modulation signals or analog voltage signals, directly driving the UAV's servos, motors, or ESCs to control the UAV's flight attitude and position, achieving precise landing.

[0032] In one embodiment of the present invention, the construction of a pre-defined three-dimensional spatial geometric model of a specific optical marker is the foundation for system operation. Initial three-dimensional model data of the specific optical marker is acquired. This initial three-dimensional model data describes the physical shape of the specific optical marker using the coordinates of multiple three-dimensional spatial points and the geometric topology formed by connecting these points. The physical object containing the specific optical marker is placed on a ground calibration platform with a known position and orientation. The position and orientation of the ground calibration platform are precisely measured and known. A drone equipped with an optical industrial camera array is driven to cruise along multiple pre-defined calibration flight paths. These paths cover multiple observation positions at different heights and tilt angles centered on the ground calibration platform. During the cruise, multiple high-resolution calibration images containing the complete physical object of the specific optical marker are acquired from multiple different spatial angles and distances using the optical industrial camera array. These high-resolution calibration images are used for subsequent feature extraction (see Table 1).

[0033] Table 1: Data Collection Table for Calibration Flight Path Two-dimensional image feature points of specific optical markers are extracted from each frame of high-resolution calibration image. Methods for extracting these feature points can include scale-invariant feature transformation and accelerated robust feature detection algorithms. Based on the pre-calibrated intrinsic parameter matrix and distortion coefficients of the optical industrial camera array, as well as the pose of the UAV during the acquisition of each frame of high-resolution calibration image, and combined with the extracted two-dimensional image feature points, the coordinates of all three-dimensional spatial points in the initial three-dimensional model data are iteratively optimized using a bundle adjustment optimization algorithm. The goal of the bundle adjustment optimization algorithm is to minimize the projection-reprojection error of all two-dimensional image feature points. Based on the optimized three-dimensional spatial point coordinates, a fine mesh model of the specific optical markers is generated using a three-dimensional reconstruction algorithm. The three-dimensional reconstruction algorithm can be Delaunay triangulation or Poisson surface reconstruction. A three-dimensional spatial geometric model for pose calculation is defined using the coordinates of all vertices in the fine mesh model and the mesh patches formed by connecting the vertices.

[0034] The system also includes a prediction and risk assessment module. This module stores real-time pose parameters from multiple consecutive sampling periods in a circular queue, forming a real-time pose parameter time series. Polynomial fitting is performed on the position vector components of the real-time pose parameter time series, using second- or third-order polynomials to predict the estimated motion trajectory of the UAV in three-dimensional space over several future sampling periods. Differential operations are performed on the attitude quaternion components of the real-time pose parameter time series to obtain the UAV's current attitude angular velocity and attitude angular acceleration. The differential operation can use the central difference method. Based on the estimated motion trajectory and current attitude dynamic parameters, combined with a pre-set 3D geofence model of the landing field, the system determines whether the UAV faces the risk of spatial interference with obstacles or deviation from the preset safe landing corridor during the predicted future period. The 3D geofence model defines the location, shape, and size of obstacles within the landing field, as well as the spatial boundaries of the safe landing corridor. When a risk is identified, a risk type code and a risk level signal are generated, which trigger subsequent instruction correction logic.

[0035] Optionally, if the risk assessment result indicates a risk of spatial interference with obstacles, the distance between the estimated trajectory and the nearest obstacle surface in the 3D geofence model of the landing site is calculated. A first-type risk level signal is generated based on the ratio of this distance to a preset safety threshold; a smaller ratio indicates a higher risk level. If the risk assessment result indicates a risk of deviating from a preset safe landing corridor, the average lateral deviation distance between the centerline of the estimated trajectory and the centerline of the preset safe landing corridor is calculated. A second-type risk level signal is generated based on the ratio of this average lateral deviation distance to the corridor width; a larger ratio indicates a higher risk level. In some embodiments, the calculation formulas for the first-type or second-type risk level signal are defined as follows: in: This represents the calculated first-class risk level signal. This indicates a preset safety threshold. This represents the minimum distance between the estimated trajectory and the nearest obstacle surface in the 3D geofence model of the landing site. A value greater than 1 indicates safety, while a value less than 1 indicates risk; the smaller the value, the higher the risk. The generated first-class or second-class risk level signal, along with the corresponding risk type code, is encapsulated into a risk warning data packet. This risk warning data packet is sent to the position controller and attitude controller, serving as external input conditions for the recalculation of velocity and angular velocity control components. For example, upon receiving a high-risk warning, the position controller may adjust its control gain or introduce additional obstacle avoidance correction commands.

[0036] See Figure 4This is a technical performance analysis chart of the 3D model construction stage of UAV optical markers, clearly showing the dynamic relationship between feature point recognition and reprojection error with the sampling period. The number of feature points in the 1-15 period rapidly increases from 15 to a peak (about 33), indicating that the UAV gradually aligns with the optical markers during calibration flight, and the imaging quality and feature extraction effect are continuously optimized. The number of feature points in the 10-15 period stabilizes above 30, which is within the safe range of the threshold (20), providing sufficient geometric constraints for pose calculation. The number of feature points in the 15-20 period rapidly drops back to 16, which is below the threshold, indicating that the visibility of the markers decreases in this stage, and the marker switching module needs to be triggered to enable the alternative pattern. Within the 10-15 sampling period, the number of feature points is sufficient and the reprojection error is the lowest, which is the optimal data segment for 3D model construction. It is recommended to prioritize the use of data in this range for the reconstruction of the fine mesh model of the optical markers.

[0037] In one embodiment of the present invention, the marker switching module continuously counts the number of feature points successfully identified and used for calculation in each sampling period during the continuous processing of the original optical image sequence. The marker switching module compares the number of successfully identified feature points with a preset threshold, which is set as a proportion of the total number of theoretical feature points for a specific optical marker. When the number of successfully identified feature points is lower than the preset threshold, the marker switching module determines that the visibility or integrity of the current specific optical marker is insufficient and triggers a marker switching request. The marker switching request is sent to the optical marker display device at the landing site, which is a light-emitting display panel or projection device located on the landing site ground capable of dynamically displaying different patterns.

[0038] In some embodiments, the sign switching request instructs the optical sign display device to turn off the currently displayed specific optical sign pattern and activate a candidate optical sign pattern pre-stored in the control memory of the optical sign display device. The candidate optical sign pattern differs significantly from the currently displayed specific optical sign pattern in size, shape, color, or topology. The optical industrial camera array continues to acquire raw optical image sequences containing the newly activated candidate optical sign pattern. Subsequent image preprocessing, feature recognition, and pose calculation processes of the system are switched to the recognition and pose calculation of the candidate optical sign pattern. The index of the three-dimensional spatial geometric model within the system is also switched accordingly to the model data corresponding to the candidate optical sign pattern.

[0039] The system also includes a fusion positioning module, which independently solves for a set of real-time pose parameters using specific or alternative optical marker patterns within a single sampling period, denoted as the single-period solved pose. The fusion positioning module accumulates multiple sets of single-period solved poses across several consecutive sampling periods within a time window. The length of the time window is configurable, for example, it can cover the most recent 10 sampling periods. Kalman filtering is applied to the position vectors in the accumulated single-period solved poses within the time window. The state vector of the Kalman filter contains both position and velocity, and the observed values ​​are the position vectors in each single-period solved pose. The optimal smoothed position of the UAV within the filtering period is estimated through the prediction and update steps of the Kalman filter.

[0040] It is understandable that the attitude quaternions accumulated within multiple single-cycle pose calculations over a time window are fused using quaternion averaging. The quaternion averaging algorithm is used to calculate the average rotation of a set of quaternions. It is understandable that, assuming there are [missing information] within the time window... The quaternion for each attitude is denoted as Find a mean quaternion. To minimize the sum of the angular distances from the quaternion to all sample quaternions, the solution process can be achieved by constructing a matrix and calculating its principal eigenvectors. Finally, the optimal smoothed attitude of the UAV within the filtering period is estimated. The optimal smoothed position and optimal smoothed attitude are output as the final real-time pose parameters for generating UAV landing control commands, replacing the single-cycle pose calculation for a single sampling period.

[0041] Optionally, the fusion positioning module outputs the optimal smooth position and optimal smooth attitude to the control command module. The control command module generates more stable and lower-noise UAV landing control commands based on the optimal smooth position and optimal smooth attitude output by the fusion positioning module. In some embodiments, when the identifier switching module triggers a switch, the time window accumulated data of the fusion positioning module is reset or processed with a forgetting factor to avoid interference from the solution data of the old identifier to the fusion of the solution data based on the new identifier, ensuring that the optimal smooth position and optimal smooth attitude output by the fusion positioning module accurately reflect the UAV's pose state relative to the current valid optical identifier.

[0042] See Figure 5This is a core state monitoring curve of the entire process of a drone's precise landing, fully recording the dynamic response of the drone's altitude, horizontal positioning error, and yaw angle over 60 seconds. The altitude shows a smooth downward trend overall, accompanied by small, high-frequency fluctuations, with significant deceleration at the end of landing. The horizontal error initially increases and then decreases, peaking at 15 seconds and converging into the negative range after 40 seconds. The yaw angle reaches a positive peak at 15 seconds, then continuously decays, exhibiting reverse overshoot at the end. Throughout the landing process, the altitude curve is smooth without abrupt changes. Although the horizontal error and yaw angle fluctuate, they converge effectively without showing signs of loss of control or divergence, proving the effectiveness of the industrial vision pose calculation and fusion control strategy. The relatively large peak values ​​of the horizontal error and yaw angle around 15 seconds indicate a lag in the controller's initial response. It is recommended to optimize the parameter gain of the control command module or introduce trajectory prediction in advance into the prediction and risk identification module to reduce initial overshoot.

[0043] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A precision landing control system for unmanned aerial vehicles (UAVs) based on industrial vision, characterized in that, The system includes: The image acquisition module is used to acquire raw optical image sequences of the landing site area at a fixed sampling frequency using an optical industrial camera array mounted on the UAV platform. The image preprocessing module is used to input the original optical image sequence into the image feature preprocessor, perform dynamic background subtraction and multi-channel image registration, and generate a fused difference image. The fused difference image is used to highlight the difference between a preset specific optical marker in the landing site and the background environment. The feature recognition module is used to perform morphological skeleton extraction and corner group detection on the fused differential image, identify all feature points that constitute the specific optical identifier, calculate the sub-pixel level coordinates of each feature point in the image coordinate system, and combine them to generate a real-time image feature point set of the landing identifier. The pose calculation module is used to perform distortion correction on the real-time image feature point set of the landing mark based on the pre-calibrated intrinsic parameter matrix and distortion coefficient of the optical industrial camera array, and to solve the real-time pose parameters of the UAV relative to the specific optical mark by using the preset three-dimensional spatial geometric model of the specific optical mark and the perspective point calculation algorithm. The control command module is used to generate UAV landing control commands based on the real-time pose parameters.

2. The UAV precision landing control system based on industrial vision according to claim 1, characterized in that, The original optical image sequence is input into an image feature preprocessor, where dynamic background subtraction and multi-channel image registration are performed to generate a fused difference image, including: The original optical image sequence contains synchronous imaging data of the infrared spectral channel and the visible spectral channel; The infrared spectral channel image and the visible spectral channel image are separated from the original optical image sequence to form a synchronous but independent spectral image pair; Perform grayscale difference operation on the infrared spectral channel image of the current frame and the infrared spectral channel image of the historical frame to generate an infrared difference image; Perform a vector norm difference operation based on the color space on the visible spectral channel image of the current frame and the visible spectral channel image of the historical frame to generate a visible light difference image; The feature descriptors of the infrared differential image and the visible light differential image are extracted using the scale-invariant feature transformation algorithm, and the homography transformation matrix between the images is calculated based on the matching relationship of the feature descriptors. The spatial coordinate system of the visible light differential image is then aligned with the spatial coordinate system of the infrared differential image. The infrared differential image and the visible light differential image, which are based on the spatial coordinate system one, are fused pixel-level according to a preset weighting coefficient, and the fused differential image is output.

3. The UAV precision landing control system based on industrial vision according to claim 2, characterized in that, Morphological skeleton extraction and corner group detection are performed on the fused difference image to identify all feature points constituting the specific optical identifier, including: An adaptive threshold segmentation algorithm is applied to the fused difference image to obtain a binarized image; Perform continuous multi-round iterative erosion and dilation operations on the binarized image until the width of all connected regions in the image converges to a single pixel to obtain the skeleton image. In the skeleton image, all pixels are traversed and the number of foreground pixels in their eight neighborhoods is counted. Pixels with one or more or more foreground pixels in their neighborhoods are marked as skeleton endpoints or branch points. Within the neighborhood of the marked skeleton endpoints and branch points, a corner response function is applied to the original fused difference image to calculate and filter out pixels with local response values ​​exceeding a threshold as candidate corner points. Based on the preset topology of the specific optical marker, the candidate corner points are grouped and geometrically constrained for verification. Outliers that do not conform to the preset topology are eliminated, and the verified candidate corner points are confirmed as the feature points.

4. The industrial vision-based UAV precision landing control system according to claim 3, characterized in that, Using a preset three-dimensional spatial geometric model of the specific optical marker, and through a perspective point calculation algorithm, the real-time pose parameters of the UAV relative to the specific optical marker are solved, including: The real-time pose parameters include a position vector and a pose quaternion; Establish a ground coordinate system with the center of the specific optical mark as the origin, and obtain the three-dimensional coordinates of each key feature point in the three-dimensional spatial geometric model of the specific optical mark in the ground coordinate system; The two-dimensional image coordinates of each feature point in the real-time image feature point set of the distortion-corrected landing mark are paired with the three-dimensional coordinates of the corresponding key feature points in the three-dimensional spatial geometric model of the specific optical mark. Based on the pinhole imaging model of an optical industrial camera array, an overdetermined set of equations defined by a paired point set is constructed, which describes the mathematical relationship between three-dimensional spatial points projected onto a two-dimensional image plane. An optimization algorithm based on orthogonal iteration is used to solve the overdetermined system of equations, minimize the image projection error, and directly calculate the rotation matrix and translation vector of the optical industrial camera array coordinate system relative to the ground coordinate system. The calculated rotation matrix is ​​converted into attitude quaternion form, and the translation vector is used as the position vector to form the real-time pose parameters.

5. The industrial vision-based UAV precision landing control system according to claim 4, characterized in that, Based on the real-time pose parameters, unmanned aerial vehicle (UAV) landing control commands are generated, including: The position vector in the real-time pose parameters is input into the position controller, which calculates the velocity control components along the three coordinate axes in the three-dimensional ground coordinate system based on the deviation between the current position of the UAV and the target landing point. The attitude quaternion in the real-time pose parameters is input into the attitude controller. The attitude controller calculates the angular velocity control components around the three coordinate axes in the UAV body coordinate system based on the deviation between the current attitude of the UAV and the target landing attitude. By fusing the velocity control component and the angular velocity control component, and superimposing a time-domain filter determined by the sampling frequency of the optical industrial camera array, a smooth basic command for UAV flight control is generated. Acceleration and angular velocity feedback provided by the UAV inertial measurement unit are introduced to dynamically compensate and correct the basic flight control commands of the UAV, generating the final UAV control surface and power system execution commands.

6. The industrial vision-based UAV precision landing control system according to claim 5, characterized in that, The construction methods for the preset three-dimensional spatial geometric model of the specific optical mark include: Obtain initial three-dimensional model data of the specific optical sign, wherein the initial three-dimensional model data describes the physical shape of the specific optical sign using the coordinates of multiple three-dimensional spatial points and the geometric topology formed by connecting the three-dimensional spatial points; Place the physical object containing the specific optical mark on a ground calibration platform with a known location and orientation; The drone equipped with the aforementioned optical industrial camera array is driven to cruise along multiple preset calibration flight paths. During the cruise, the optical industrial camera array captures multiple high-resolution calibration images containing the complete physical object of the specific optical identifier from multiple different spatial angles and distances. Extract two-dimensional image feature points of the specific optical identifier from each frame of the high-resolution calibration image; Based on the pre-calibrated intrinsic parameter matrix and distortion coefficients of the optical industrial camera array, as well as the pose of the UAV when acquiring each frame of the high-resolution calibration image, and combined with the feature points of the two-dimensional image, the coordinates of all three-dimensional spatial points in the initial three-dimensional model data are iteratively optimized using the bundle adjustment optimization algorithm. Based on the optimized three-dimensional spatial point coordinates, a fine mesh model of the specific optical identifier is generated through a three-dimensional reconstruction algorithm, and the three-dimensional spatial geometric model is defined using the coordinates of all vertices in the fine mesh model and the mesh patches formed by connecting the vertices.

7. The UAV precision landing control system based on industrial vision according to claim 6, characterized in that, It also includes a prediction and risk assessment module, used for: The real-time pose parameters for multiple consecutive sampling periods are stored in a circular queue to form a real-time pose parameter time series. Polynomial fitting is performed on the position vector components in the real-time pose parameter time series to predict the estimated motion trajectory of the UAV in three-dimensional space in the next few sampling periods. Differentiate the attitude quaternion components in the real-time pose parameter time series to obtain the current attitude angular velocity and attitude angular acceleration of the UAV. Based on the estimated motion trajectory and current attitude dynamic parameters, combined with the preset three-dimensional geofence model of the landing site, it is determined whether the UAV is at risk of spatial interference with obstacles or deviating from the preset safe landing corridor in the future prediction period. When a risk is identified, a risk type code and a risk level signal are generated. These risk type code and risk level signal are used to trigger subsequent instruction correction logic.

8. The industrial vision-based UAV precision landing control system according to claim 7, characterized in that, When a risk is determined to exist, a risk type code and a risk level signal are generated. These risk type code and risk level signal are used to trigger subsequent instruction correction logic, including: If the risk assessment result indicates a risk of spatial interference with an obstacle, the distance between the estimated trajectory and the nearest obstacle surface in the three-dimensional geofence model of the landing site is calculated, and a first-class risk level signal is generated based on the ratio of the distance to a preset safety threshold. If the risk assessment result indicates that there is a risk of deviating from the preset safe landing corridor, the average lateral deviation distance between the centerline of the estimated trajectory and the centerline of the preset safe landing corridor is calculated, and a second type of risk level signal is generated based on the ratio of the average lateral deviation distance to the corridor width. The generated first-class risk level signal or second-class risk level signal, together with the corresponding risk type code, are encapsulated into a risk warning data packet; The risk warning data packet is sent to the position controller and attitude controller, and participates in the recalculation of the velocity control component and angular velocity control component as an external input condition.

9. The industrial vision-based UAV precision landing control system according to claim 8, characterized in that, It also includes an identifier switching module, used for: During the continuous processing of the original optical image sequence, the number of feature points successfully identified and used for calculation in each sampling period is counted in real time. When the number of successfully identified feature points is lower than a preset threshold, it is determined that the visibility or integrity of the current specific optical tag is insufficient, and a tag switching request is triggered. The tag switching request is sent to the optical tag display device at the landing site, instructing the optical tag display device to turn off the currently displayed specific optical tag pattern and activate a pre-stored alternative optical tag pattern; The optical industrial camera array continues to acquire the original optical image sequence containing the candidate optical marker pattern, and the subsequent process switches to the recognition and pose calculation of the candidate optical marker pattern.

10. The industrial vision-based UAV precision landing control system according to claim 9, characterized in that, It also includes a fusion positioning module, used for: Within a single sampling period, a set of real-time pose parameters are independently solved using the specific optical marker or the alternative optical marker pattern, which is denoted as the single-cycle solved pose. Accumulate multiple sets of single-cycle pose calculations across multiple consecutive sampling periods within a time window; Kalman filtering is performed on the position vectors in the multiple sets of single-cycle solved poses accumulated within the time window to estimate the optimal smooth position of the UAV within the filtering period. The attitude quaternions accumulated in the multiple single-cycle solved poses within the time window are fused based on quaternion averaging to estimate the optimal smooth attitude of the UAV within the filtering period. The optimal smooth position and optimal smooth attitude are output as the final real-time pose parameters for generating UAV landing control commands, replacing the single-cycle pose calculation with a single sampling period.

Images