Unmanned aerial vehicle intelligent image recognition and autonomous flight control system

Abstract

The invention discloses an intelligent image recognition and autonomous flight control system for an unmanned aerial vehicle, belongs to the technical field of information technologies, and solves the problem that the unmanned aerial vehicle still needs to fly autonomously for patrol under the condition that the unmanned aerial vehicle is not explored in advance and cannot be connected to a network, so that a current unmanned aerial vehicle system needs to be improved. Comprising a feature collection module, a flight path processing module, a path storage module, a wireless communication module and an interaction module, the feature collection module collects features of the surrounding environment of the unmanned aerial vehicle with the unmanned aerial vehicle as a reference and collects flight features in the flight process of the unmanned aerial vehicle, and the flight path processing module forms a flight path through the flight features. According to the method, the autonomous positioning system based on the local coordinate system is established, and multi-source data fusion of vision, the IMU and the barometer is combined, so that high-precision positioning without satellite signals is realized, and the unmanned aerial vehicle gets rid of dependence of a preset route and a network in a dangerous scene and autonomously completes a cyclic patrol task in a complex environment.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and particularly to an intelligent image recognition and autonomous flight control system for unmanned aerial vehicles. Background Art

[0002] An unmanned aerial vehicle (UAV) is an unpiloted aircraft remotely controlled by radio or autonomously programmed, consisting of core components such as a power system, a navigation module, and a mission payload. Since its application in the military field in the early 20th century, with technological advancements, its form and functions have been continuously innovated, and now it has formed three major application branches: consumer, industrial, and military. In the civilian field, UAVs have reshaped traditional operation modes with their mobility and low cost: Aerial photography equipment provides a new perspective for film and surveying; agricultural UAVs achieve precise pesticide application with an efficiency increase of over 60%; logistics UAVs break through terrain limitations to assist in the delivery of emergency supplies in remote areas; in fire-fighting and rescue scenarios, thermal imaging technology can quickly locate signs of life. In terms of technical classification, fixed-wing, multi-rotor, and composite are the three major mainstream architectures. Among them, multi-rotor UAVs dominate the consumer market due to their simple structure and stable hovering, while new power systems such as hydrogen energy are driving industrial-grade models towards longer endurance.

[0003] Currently, UAVs are also used in area inspections. Specifically, UAV inspection is a technology that uses UAVs equipped with high-definition cameras, infrared sensors, and other devices to conduct automated or semi-automated inspections of target areas. Through preset flight paths or real-time control, it can efficiently cover areas that are difficult for humans to reach, such as high-voltage transmission lines, oil and gas pipelines, forest fire prevention belts, and large-scale infrastructure construction sites. Compared with traditional manual inspections, UAVs have the advantages of strong flexibility, low cost, and real-time data transmission, especially in dangerous environments (such as landslides, polluted areas), which can significantly reduce the safety risks of personnel.

[0004] At the technical level of UAV inspections, the processor of the UAV combines image recognition and wireless transmission, enabling the UAV to automatically identify equipment defects (such as tower corrosion, pipeline leaks) or abnormal phenomena (such as illegal logging, fire hazards), and generate a structured report. Currently, this technology has been widely applied in the fields of power, transportation, environmental protection, and urban management.

[0005] In general, the route of drone inspection often needs to cooperate with the wireless communication module and the satellite positioning module for real-time positioning of the drone. At the same time, its flight route also needs to cooperate with the satellite positioning module to preset the flight route of the drone in advance. Such an operation not only requires the drone to always maintain a connection through the wireless communication module and the satellite positioning module to refresh the position in real time, but also requires the operator who operates the drone inspection to be very familiar with the environment of the inspected area. If the operator who operates the drone inspection is not familiar with the environment of the inspected area, a large amount of energy and time are required to conduct exploration of the inspected area environment in advance to ensure that the drone will not collide with environmental objects during the autonomous inspection process. Therefore, if the drone still needs to be able to fly autonomously for inspection without prior exploration and without being able to connect to the network, the current drone system needs to be improved.

[0006] Therefore, a drone intelligent image recognition and autonomous flight control system is proposed to solve or alleviate the above problems. Summary of the Invention

[0007] The purpose of the present invention is to solve the deficiencies in the prior art and propose a drone intelligent image recognition and autonomous flight control system.

[0008] To achieve the above purpose, the present invention adopts the following technical solutions:

[0009] A drone intelligent image recognition and autonomous flight control system, including a feature acquisition module, a flight trajectory processing module, a trajectory storage module, a wireless communication module, and an interaction module. The feature acquisition module collects the environmental features around the drone based on the drone and collects flight features during the flight of the drone. The flight trajectory processing module forms a flight route through the flight features, extracts specific marker nodes through the environmental features around the drone to correct the flight route and determine the initial position range, and controls the drone to fly in a loop through the flight route, specific marker nodes, and initial position range. The trajectory storage module stores the flight route of the drone. The flight trajectory processing module is communicatively connected to the interaction module through the wireless communication module. The interaction module enables the operator to interact with the flight trajectory processing module.

[0010] Preferably, the feature acquisition module includes an image acquisition unit, an inertial measurement unit, and a barometer. The image acquisition unit collects the environmental features around the drone based on the drone. The inertial measurement unit and the barometer collect flight features during the flight of the drone.

[0011] Preferably, the flight trajectory processing module forms a flight route through the flight features, including the following steps

[0012] Obtain flight characteristics, where the flight characteristics include three-axis acceleration (a xk , a yk , a zk ), heading angle ψ, pitch angle θ, roll angle φ, three-axis magnetic field components (m xk , m yk , m zk ), temperature T, and time t;

[0013] Set the initial position P 0 =(x 0 , y 0 , z 0 ), initial velocity v 0 =(v x0 , v y0 , v z0 ), and initial attitude (ψ 0 , θ 0 , φ 0 );

[0014] Update the attitude of the UAV where ω k is the angular velocity, ω k =(ω xk , ω yk , ω zk );

[0015] Correct the heading angle ψ k =arctan2(m yk , m xk ), where arctan2(y, x) is the four-quadrant arctangent function;

[0016] Calculate and update the current altitude of the UAV where R is the gas constant, M is the molecular mass of air, g is the acceleration due to gravity, p 0 is the standard atmospheric pressure at sea level, p k is the current atmospheric pressure, and γ is the gas constant;

[0017] Calculate the acceleration a through the three-axis acceleration k =(a xk , a yk , a zk ), convert the acceleration to the ground coordinate system a global,k =R(θ k , φ k , ψ k )·a k , and update the velocity v k+1 =v k +a global,k t and the position P k+1 =P k +vk t;, where R(θ k , φ k , ψ k ) is a rotation matrix composed of pitch angle, roll angle and heading angle;

[0018] Continuously update the above content, and use the take-off point of the UAV as the origin O l and use the horizontal projection of the forward direction at take-off as the X-axis X l and use the direction perpendicular to the ground as the Z-axis Z l and use the Y-axis Y determined by the right-hand rule l to establish a local coordinate system and obtain the flight path of the UAV.

[0019] Preferably, the flight trajectory processing module includes a processor. The processor forms a flight path through flight characteristics, extracts specific marker nodes through the environmental characteristics around the UAV to correct the flight path and determine the initial position range, and controls the UAV to fly in a loop through the flight path, specific marker nodes, and initial position range.

[0020] Preferably, the extracting specific marker nodes through the environmental characteristics around the UAV to correct the flight path and determine the initial position range includes the following steps

[0021] Extract the environmental characteristics around the UAV, and the environmental characteristics around the UAV include four-direction images;

[0022] Judge whether the motion state of the UAV is stationary or moving;

[0023] If the motion state of the UAV is moving, then extract flight characteristics to correct the four-direction images and remove the dynamic content therein to obtain corrected four-direction images, and then perform subsequent processing;

[0024] If the motion state of the UAV is stationary, then directly perform subsequent processing on the four-direction images;

[0025] Extract specific features from the corrected four-direction images or four-direction images through a cross-scene feature model as specific marker nodes or determine the initial position range, and correct the flight path of the UAV through the specific marker nodes.

[0026] Preferably, the extracting flight characteristics to correct the four-direction images and remove the dynamic content therein to obtain corrected four-direction images includes the following steps

[0027] Map the pixel coordinates (u t , v t ) of the current frame of the four-direction image to the global coordinate system (X, Y, Z) where K is the internal parameter matrix of the image acquisition unit, V is the speed of the UAV, and Δt is the frame interval time difference;

[0028] Align adjacent frames using perspective transformation

[0029] Calculate the residual optical flow field (Δu, Δv) after motion compensation using the Lucas-Kanade optical flow method, where I x and I y are spatial gradients, and I t is the temporal gradient;

[0030] Calculate the motion energy E and binarize it

[0031] Generate a dynamic region mask

[0032] Remove the dynamic region mask M from the four-direction image d , obtaining a corrected four-direction image.

[0033] Preferably, extracting specific features as specific marker nodes or determining the initial position range from the corrected four-direction image or the four-direction image through a cross-scene feature model includes the following steps,

[0034] Perform illumination-invariant feature extraction on the corrected four-direction image or the four-direction image, and use a multi-band Gabor filter bank for phase consistency edge detection to generate an initial feature map resistant to illumination interference where M so is the multi-directional Gabor filter response, W o is the frequency weighting matrix, T o is the threshold, and ε is a small constant;

[0035] Extract high-level semantic features and spatial gradient features Φ sem (I) = f conv5 (ReLU(f conv4 (...ReLU(f conv1 (I))))) for each initial feature map respectively, and perform channel dimension fusion to form a geometric-semantic composite feature tensor

[0036] Establish the epipolar geometry matrix E of the corrected four-direction image or the four-direction image ij = K -T [t] × RK -1 , and remove the mismatches through the RANSAC algorithm;

[0037] Project the feature tensors of each perspective onto a virtual spherical coordinate system centered on the drone and perform spherical parameterization transformation where (x c , y c , z c ) is the origin of the drone coordinate system and f is the focal length;

[0038] Adopt geodesic distance weighting of the spherical convolution kernel to perform spatial context fusion on adjacent feature nodes;

[0039] Use a graph attention network to model spatio-temporal context relationships through an attention mechanism where the node h j represents the feature subgraph of each perspective;

[0040] Construct a multi-perspective bundle adjustment optimization model where T is the pose transformation matrix to be solved, X is the three-dimensional road marking point, and ρ is the robust loss function;

[0041] Fuse the data of the inertial measurement unit to construct a tightly coupled optimization objective function Adopt the L-M algorithm to iteratively solve the 6-degree-of-freedom pose parameters;

[0042] Calculate the final pose confidence Select the valid solutions with a confidence greater than the threshold.

[0043] Preferably, the correction of the flight route of the drone by the specific marked node includes the following steps

[0044] Determine the state vector x = [p l , v l , q lb , b a , b g , Δh] T ∈R 16 , where p l is the position in the flight route, v l is the speed of the drone, q lb is the rotation quaternion of the drone, b a and b g are the zero biases of the inertial measurement unit, and Δh is the barometric altitude drift compensation amount;

[0045] Establish the IMU kinematic equation where g l = [0, 0, -g]T Gravity vector in the local coordinate system, a m Measurement value of the accelerometer, ω m Measurement value of the gyroscope Quaternion multiplication

[0046] Perform discrete pre-integration improvement to calculate the position change Δp ij Velocity change Δv ij And rotation change ΔR ij , Where Δt is the interval time

[0047] Pose observation of the visual output relative to the origin And construct the residual term Complete visual positioning constraint

[0048] Establish a barometric observation model to calculate the altitude value z measured by the barometer baro = z l + h 0 + Δh + η baro , where η baro Is the barometer measurement noise, and construct the altitude residual Where Is the Z-axis component of the visual positioning output, complete barometer altitude fusion

[0049] Combine the IMU pre-integration residual, visual residual and barometric residual to construct the objective function Introduce the local system gravity constraint term r gravity = R lb · [0, 0, 1] T - [0, 0, 1] T Ensure that the Z-axis is aligned with the gravity direction

[0050] Implement the zero-velocity correction mechanism. When it is detected to be stationary (||a k - b a || < ∈), perform zero-velocity correction r static = [v l , ω m - b g ;

[0051] Add closed-loop detection constraint when the vision revisits a known position

[0052] Calculate the initial parameters through a 5-second stationary period when the UAV takes off Adopt the proportional-integral compensation mechanism to suppress the planar position drift compensation And altitude drift compensation Through the Jacobian matrix J gl Implementing covariance propagation in a local system

[0053] In the initialization phase, a local coordinate system is established and the zero bias of the inertial measurement unit is calibrated. In the online phase, the IMU, vision and air pressure data are optimized and fused through a sliding window. The online phase includes a front end, a middle end and a back end. The front end includes IMU pre-integration and visual pose solution correction. The middle end includes joint IMU pre-integration residual, visual residual and air pressure residual correction. The back end includes sliding window optimization, and the window size is 5-10 frames. When vision is lost, IMU and barometer dead reckoning are used.

[0054] Preferably, the trajectory storage module comprises a memory.

[0055] The present invention has the following beneficial effects:

[0056] The intelligent image recognition and autonomous flight control system for unmanned aerial vehicles proposed in the present invention establishes an autonomous positioning system based on a local coordinate system, constructs a three-dimensional orthogonal coordinate system with the take-off point as the origin, and combines multi-source data fusion of vision, IMU and barometer to achieve high-precision positioning without relying on satellite signals. In dangerous scenarios such as high-voltage power transmission line inspection, oil and gas pipeline monitoring, and forest fire prevention and control, the unmanned aerial vehicle can get rid of its dependence on preset routes and network connections, and independently complete cyclic inspection tasks in complex environments, reducing the need for manual intervention by more than 90%, while improving the efficiency of traditional manual inspections by more than 60%. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 It is a structural block diagram of the present invention.

[0059] 1. Feature collection module; 2. Flight trajectory processing module; 3. Trajectory storage module; 4. Wireless communication module; 5. Interaction module. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. The components of the embodiments of the present invention usually described and illustrated in the drawings here can be arranged and designed in various different configurations.

[0061] Therefore, the detailed description of the embodiments of the present invention provided in the drawings below is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0062] It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0063] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the invention is normally placed, or the orientation or positional relationship commonly understood by those skilled in the art. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0064] In addition, the terms "first", "second", "third", etc. are only used for descriptive distinction and should not be construed as indicating or implying relative importance.

[0065] In the description of the present invention, it should also be noted that unless otherwise clearly specified and defined, the terms "set", "installed", "connected", "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0066] The intelligent image recognition and autonomous flight control system of the unmanned aerial vehicle, as Figure 1 shown, includes a feature acquisition module 1, a flight trajectory processing module 2, a trajectory storage module 3, a wireless communication module 4, and an interaction module 5;

[0067] The feature acquisition module 1 acquires the environmental features around the UAV based on the UAV and acquires flight features during the flight of the UAV. The feature acquisition module 1 includes an image acquisition unit, an inertial measurement unit, and a barometer. The image acquisition unit acquires the environmental features around the UAV based on the UAV, and the inertial measurement unit and the barometer acquire flight features during the flight of the UAV.

[0068] The flight trajectory processing module 2 forms a flight route through the flight features, extracts specific marker nodes through the environmental features around the UAV to correct the flight route and determine the initial position range, and controls the UAV to fly in a loop through the flight route, specific marker nodes, and initial position range. The flight trajectory processing module 2 includes a processor. The processor forms a flight route through the flight features, extracts specific marker nodes through the environmental features around the UAV to correct the flight route and determine the initial position range, and controls the UAV to fly in a loop through the flight route, specific marker nodes, and initial position range.

[0069] The trajectory storage module 3 stores the flight route of the UAV. The trajectory storage module 3 includes a memory.

[0070] The flight trajectory processing module 2 is communicatively connected to the interaction module 5 through the wireless communication module 4. The interaction module 5 enables an operator to interact with the flight trajectory processing module 2.

[0071] Specifically,

[0072] The flight trajectory processing module 2 forms a flight route through the flight features, including the following steps:

[0073] Obtain flight features, where the flight features include three-axis acceleration (a xk , a yk , a zk ), heading angle ψ, pitch angle θ, roll angle φ, three-axis magnetic field components (m xk , m yk , m zk ), temperature T, and time t;

[0074] Set the initial position P 0 =(x 0 , y 0 , z 0 ), initial velocity v 0 =(v x0 , v y0 , v z0 ), and initial attitude (ψ 0 , θ 0 , φ 0 );

[0075] Update the attitude of the UAV where ω kis the angular velocity, ω k =(ω xk , ω yk , ω zk );

[0076] The corrected heading angle ψ k = arctan2(m yk , m xk ), where arctan2(y, x) is the four - quadrant arctangent function;

[0077] Calculate and update the current altitude of the UAV where R is the gas constant, M is the molecular mass of air, g is the acceleration due to gravity, p 0 is the standard atmospheric pressure at sea level, p k is the current atmospheric pressure, and γ is the gas constant;

[0078] Calculate the acceleration a through the triaxial acceleration k =(a xk , a yk , a zk ), convert the acceleration to the ground coordinate system a global,k = R(θ k , φ k , ψ k )·a k , and update the velocity v k+1 = v k + a global,k t and the position P k+1 = P k + v k t; where R(θ k , φ k , ψ k ) is the rotation matrix composed of the pitch angle, roll angle, and heading angle;

[0079] Continuously update the above content, and take the take - off point of the UAV as the origin O l , the horizontal projection of the forward direction at take - off as the X - axis X l , the direction perpendicular to the ground as the Z - axis Z l , and determine the Y - axis Y through the right - hand rule l to establish a local coordinate system and obtain the flight path of the UAV.

[0080] Preferably, extract specific marker nodes through the environmental characteristics around the UAV to correct the flight path and determine the initial position range, including the following steps,

[0081] Extract the environmental characteristics around the UAV, and the environmental characteristics around the UAV include four - direction images;

[0082] Determine whether the motion state of the drone is stationary or moving;

[0083] If the motion state of the drone is moving, extract flight features to correct the four-direction images and remove the dynamic content therein to obtain corrected four-direction images, and then perform subsequent processing;

[0084] If the motion state of the drone is stationary, directly perform subsequent processing on the four-direction images;

[0085] Extract specific features from the corrected four-direction images or the four-direction images through a cross-scene feature model as specific marker nodes or determine the initial position range, and correct the flight route of the drone through the specific marker nodes.

[0086] Preferably, extracting flight features to correct the four-direction images and removing the dynamic content therein to obtain corrected four-direction images includes the following steps,

[0087] Map the pixel coordinates (u t , v t ) of the current frame of the four-direction image to the global coordinate system (X, Y, Z) where K is the internal parameter matrix of the image acquisition unit, V is the speed of the drone, and Δt is the frame interval time difference;

[0088] Adopt perspective transformation to align adjacent frames

[0089] Use the Lucas-Kanade optical flow method to calculate the residual optical flow field (Δu, Δv) after motion compensation, where I x and I y are the spatial gradients, and I t is the temporal gradient;

[0090] Calculate the motion energy E and binarize it

[0091] Generate a dynamic region mask

[0092] Remove the dynamic region mask M in the four-direction images d , and obtain the corrected four-direction images.

[0093] Preferably, extracting specific features from the corrected four-direction images or the four-direction images through a cross-scene feature model as specific marker nodes or determining the initial position range includes the following steps,

[0094] Perform illumination-invariant feature extraction on the corrected four-direction images or the four-direction images, and use a multi-band Gabor filter bank for phase consistency edge detection to generate an initial feature map resistant to illumination interference Among them, M so is the multi-directional Gabor filter response, W o is the frequency weighting matrix, T o is the threshold, and ε is a small constant;

[0095] For each initial feature map, use a dual-path convolutional network to extract high-level semantic features and spatial gradient features Φ sem (I) = f conv5 (ReLU(f conv4 (...ReLU(f conv1 (I))))) and perform channel dimension fusion to form a geometric-semantic composite feature tensor

[0096] Establish the epipolar geometry matrix E of the corrected four-way graph or four-directional image ij = K -T [t] × RK -1 , and eliminate the wrong matches through the RANSAC algorithm;

[0097] Project each perspective feature tensor onto a virtual spherical coordinate system centered on the UAV and perform spherical parameterization conversion Among them, (x c , y c , z c ) is the origin of the UAV coordinate system, and f is the focal length;

[0098] Adopt geodesic distance weighting of the spherical convolution kernel Perform spatial context fusion on adjacent feature nodes;

[0099] Use a graph attention network to model the spatio-temporal context relationship through the attention mechanism Among them, the node h j represents each perspective feature subgraph;

[0100] Construct a multi-perspective bundle adjustment optimization model Among them, T is the pose transformation matrix to be solved, X is the three-dimensional road point, and ρ is the robust loss function;

[0101] Fuse the inertial measurement unit data to construct a tightly coupled optimization objective function Adopt the L-M algorithm to iteratively solve the 6-degree-of-freedom pose parameters;

[0102] Calculate the final pose confidence Select valid solutions with confidence greater than the threshold.

[0103] Preferably, the flight route of the drone is corrected by specific marker nodes, including the following steps:

[0104] Determine the state vector x = [p l , v l , q lb , b a , b g , Δh] T ∈R 16 , where p l is the position in the flight route, v l is the speed of the drone, q lb is the rotation quaternion of the drone, b a and b g are the zero biases of the inertial measurement unit, and Δh is the barometric altitude drift compensation amount;

[0105] Establish the IMU kinematic equation where g l = [0, 0, -g] T is the gravity vector in the local coordinate system, a m is the measurement value of the accelerometer, ω m is the measurement value of the gyroscope, is the quaternion multiplication;

[0106] Perform discrete pre-integration improvement and calculate the position change Δp ij , the speed change Δv ij and the rotation change ΔR ij , where Δt is the interval time;

[0107] Pose observation of the visual output relative to the origin and construct the residual term to complete the visual positioning constraint;

[0108] Establish a barometric observation model to calculate the altitude value z measured by the barometer baro = z l + h 0 + Δh + η baro , where η baro is the barometer measurement noise, and construct the altitude residual where is the Z-axis component of the visual positioning output, and complete the barometer altitude fusion;

[0109] Combine the IMU pre-integration residual, visual residual, and barometric residual to construct the objective function Introduce the local system gravity constraint term r gravity = R lb ·[0, 0, 1 ]T - [0, 0, 1] T Ensure that the Z-axis is aligned with the gravity direction;

[0110] Implement the zero-velocity correction mechanism. When it is detected that the object is stationary (||a k - b a || < ∈), perform zero-velocity correction r static = [v l , ω m - b g ;

[0111] Add closed-loop detection constraints when the vision revisits a known location

[0112] Calculate the initial parameters through a 5-second stationary period when the UAV takes off Adopt a proportional-integral compensation mechanism to suppress the planar position drift compensation and altitude drift compensation Implement covariance propagation in the local system through the Jacobian matrix J gl

[0113] In the initialization stage, establish a local coordinate system and calibrate the zero bias of the inertial measurement unit. In the online stage, optimize and fuse IMU, vision, and barometric data through a sliding window. The online stage includes the front end, middle end, and back end. The front end includes IMU pre-integration and vision pose solution correction. The middle end includes joint IMU pre-integration residual, vision residual, and barometric residual correction. The back end includes sliding window optimization, and the window size is 5 - 10 frames. And when the vision is lost, use IMU and barometer dead reckoning

[0114] The UAV intelligent image recognition and autonomous flight control system proposed by the present invention significantly improves the autonomous inspection ability of the UAV in an environment without network support and without prior exploration by combining the technical architecture with a multi-modal fusion mechanism, and can be applied to tasks such as industrial inspection and emergency rescue in complex scenarios.

[0115] During specific operation, by establishing an autonomous positioning system based on the local coordinate system, constructing a three-dimensional orthogonal coordinate system with the takeoff point as the origin, and combining multi-source data fusion of vision, IMU, and barometer, high-precision positioning without relying on satellite signals is achieved.

[0116] ​The gravity direction locking and Z-axis stability constraint in the local coordinate system, combined with the improved discrete pre-integration model, effectively suppress the cumulative error of inertial navigation, control the horizontal positioning error within the flight distance, and solve the problem of serious drift of traditional inertial navigation in long-duration missions.

[0117] Secondly, the dynamic environment perception and intelligent correction mechanism greatly enhances the system adaptability. The four-direction image acquisition, combined with the Lucas-Kanade optical flow method and motion energy analysis, can remove the dynamic interference area in real time and generate a stable corrected four-direction map.

[0118] The cross-scene feature model extracts geometric-semantic composite features resistant to light interference through multi-band Gabor filtering, spherical convolutional network and graph attention mechanism. Even under conditions of drastic light changes or partial occlusion, it can accurately identify specific marker nodes and correct the flight route.

[0119] At the same time, a two-way drift compensation strategy is adopted. The proportional-integral algorithm is used to dynamically calibrate the plane position deviation and barometric altitude drift of vision and inertial navigation, making the altitude error less than 0.3 meters in the static environment and not exceeding 1.2 meters during dynamic flight. At the same time, the heading angle drift is suppressed within 2° / min, significantly improving the reliability of three-dimensional space positioning.

[0120] And when the visual signal is lost, the system can seamlessly switch to the dead reckoning mode that fuses the IMU and barometer, and tightly couples and processes the data of the nearest 5-10 frames in combination with the sliding window optimization algorithm to ensure positioning continuity.

[0121] As a result, the UAV can get rid of the dependence on the preset route and network connection in dangerous scenarios such as high-voltage transmission line inspection, oil and gas pipeline monitoring, and forest fire prevention, autonomously complete the cyclic inspection task in complex environments, reduce the demand for manual intervention by more than 90%, and at the same time improve the efficiency of traditional manual inspection by more than 60%.

[0122] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. UAV intelligent image recognition and autonomous flight control system, characterized by: The invention comprises a feature acquisition module (1), a flight trajectory processing module (2), a trajectory storage module (3), a wireless communication module (4), and an interaction module (5). The feature acquisition module (1) acquires the features of the environment around the unmanned aerial vehicle based on the unmanned aerial vehicle, and acquires the flight features during the flight of the unmanned aerial vehicle. The flight trajectory processing module (2) forms a flight route through the flight features, extracts specific marking nodes through the environment features around the unmanned aerial vehicle to correct the flight route and determine the initial position range, and controls the unmanned aerial vehicle to fly cyclically through the flight route, the specific marking nodes, and the initial position range. The trajectory storage module (3) stores the flight route of the unmanned aerial vehicle. The flight trajectory processing module (2) is connected to the interaction module (5) through the wireless communication module (4), and the interaction module (5) allows an operator to interact with the flight trajectory processing module (2).

2. The UAV intelligent image recognition and autonomous flight control system according to claim 1 is characterized in that: The feature acquisition module (1) comprises an image acquisition unit, an inertial measurement unit, and a barometer. The image acquisition unit acquires features of the environment surrounding the drone based on the drone, and the inertial measurement unit and the barometer acquire flight features during the flight of the drone.

3. The UAV intelligent image recognition and autonomous flight control system according to claim 2 is characterized in that: The flight trajectory processing module (2) forms a flight route through flight characteristics, comprising the following steps: Acquire flight characteristics, including three-axis acceleration (a xk ,a yk ,a zk ), heading angle ψ, pitch angle θ, roll angle φ, three-axis magnetic field components (m xk ,m yk ,m zk ), temperature T, time t; Set the initial position P0 = (x0, y0, z0), the initial velocity v0 = (v x0 ,v y0 ,v z0 ), and the initial posture (ψ0,θ0,φ0); Update the drone's attitude Among them, ω k is the angular velocity, ω k =(ω xk ,ω yk ,ω zk ); Corrected heading angle ψ k =arctan2(m yk ,m xk ), where arctan2(y,x) is the four-quadrant inverse tangent function; Calculate and update the current altitude of the drone Among them, R is the gas constant, M is the molecular mass of air, g is the acceleration of gravity, p0 is the standard atmospheric pressure at sea level, and p k is the current air pressure, γ is the gas constant; Calculate the acceleration a by using the three-axis acceleration k =(a xk ,a yk ,a zk ), convert the acceleration to the ground coordinate system a global,k =R(θ k ,φ k ,ψ k )·a k , and update the velocity v k+1 =v k +a global,k t and position P k+1 =P k +v k t; where R(θ k ,φ k ,ψ k ) is the rotation matrix consisting of the pitch angle, roll angle and heading angle; Continue to update the above content, and use the drone take-off point as the origin O l , take the horizontal projection in the forward direction at takeoff as the X-axis l , take the vertical direction of the ground as the Z axis l , to determine the Y axis Y by the right-hand rule l Establish a local coordinate system and obtain the flight path of the UAV.

4. The UAV intelligent image recognition and autonomous flight control system according to claim 1 is characterized in that: The flight trajectory processing module (2) comprises a processor, which forms a flight route through flight characteristics, extracts specific marking nodes through characteristics of the surrounding environment of the UAV to correct the flight route and determine an initial position range, and controls the UAV to fly cyclically through the flight route, specific marking nodes and initial position range.

5. The UAV intelligent image recognition and autonomous flight control system according to claim 3 is characterized in that: The method of extracting specific marked nodes through the surrounding environment features of the drone to correct the flight route and determine the initial position range includes the following steps: Extracting the features of the environment around the drone, wherein the features of the environment around the drone include four-directional images; Determine whether the drone's motion state is stationary or moving; If the UAV is in motion, the flight features are extracted to correct the four-directional image and remove the dynamic content to obtain a corrected four-directional image for subsequent processing; If the UAV motion state is stationary, the four-directional images are directly processed; The corrected four-way graph or four-way image is used to extract specific features as specific marking nodes or determine the initial position range through a cross-scene feature model, and the flight path of the UAV is corrected through the specific marking nodes.

6. The UAV intelligent image recognition and autonomous flight control system according to claim 5 is characterized in that: The method of extracting flight features to correct the four-directional image and remove dynamic content therein to obtain a corrected four-directional image includes the following steps: The pixel coordinates of the current frame of the four-directional image are 9u t ,v t ) is mapped to the global coordinate system (X, Y, Z) Among them, K is the internal parameter matrix of the image acquisition unit, V is the speed of the drone, and Δt is the time difference between frames; Align adjacent frames using perspective transformation The Lucas-Kanade optical flow method is used to calculate the residual optical flow field (Δu, Δv) after motion compensation. Among them, I x and I y is the spatial gradient, I t is the time gradient; Calculate the motion energy E and binarize it Generating dynamic region masks Remove the dynamic area mask M from the four-directional image d , and obtain the modified four-way graph.

7. The UAV intelligent image recognition and autonomous flight control system according to claim 6 is characterized in that: The method of extracting specific features from the modified four-directional graph or the four-directional image as specific marking nodes or determining the initial position range through a cross-scene feature model comprises the following steps: The modified four-directional graph or four-directional image is subjected to illumination invariant feature extraction, and a multi-band Gabor filter bank is used for phase consistency edge detection to generate an initial feature graph that is resistant to illumination interference. Among them, M so is the multi-directional Gabor filter response, W o is the frequency weighting matrix, T o is the threshold, ε is a small constant; For each initial feature map, a dual-path convolutional network is used to extract high-level semantic features and spatial gradient features Φ sem (I) = f conv5 (ReLU(f conv4 (...ReLU(f conv1 (I))))) and perform channel dimension fusion to form a geometric-semantic composite feature tensor Establish the epipolar geometry matrix E of the modified four-way graph or four-way image ij =K -T [t] × R -1 , remove false matches through the RANSAC algorithm; Project each view feature tensor to a virtual spherical coordinate system centered on the drone and perform spherical parameterization transformation Among them, (x c ,y c ,z c ) is the origin of the drone coordinate system, and f is the focal length; Geodesic distance weighting Spherical convolution kernel Perform spatial context fusion on adjacent feature nodes; Modeling spatiotemporal contextual relationships using graph attention networks via attention mechanisms Among them, node h j Represents the feature sub-image of each view; Constructing a multi-view bundle adjustment optimization model Among them, T is the pose transformation matrix to be determined, X is the three-dimensional landmark point, and ρ is the robust loss function; Fusing IMU data to build a tightly coupled optimization objective function The LM algorithm is used to iteratively solve the 6-DOF posture parameters; Calculate the final pose confidence, Select valid solutions with confidence greater than the threshold.

8. The UAV intelligent image recognition and autonomous flight control system according to claim 7, characterized in that: The method of correcting the flight path of the UAV by using a specific marking node includes the following steps: Determine the state vector x = [p l ,v l ,q lb ,b a ,b g ,Δh] T ∈R 16 , where p l is the position in the flight path, v l is the speed of the drone, q lb is the rotation quaternion of the drone, b a and b g is the zero bias of the inertial measurement unit, Δh is the compensation for the atmospheric pressure altitude drift; Establishing IMU kinematic equations Among them, g l =[0,0,-g] T The gravity vector in the local coordinate system, a m is the measurement value of the accelerometer, ω m is the measurement value of the gyroscope, is quaternion multiplication; Perform discrete pre-integration improvement to calculate the position change Δp ij , speed change Δv ij and the rotation change ΔR ij , Among them, Δt is the interval time; Visual output relative to the origin of the pose observation And construct the residual term Complete visual positioning constraints; Establish an air pressure observation model to calculate the height value z measured by the barometer baro =z l +h0+Δh+η baro , where η baro is the barometer measurement noise, and constructs the height residual r baro = in, The Z-axis component of the visual positioning output is used to complete the barometer height fusion; The objective function is constructed by combining IMU pre-integration residual, visual residual and air pressure residual. Introducing the local system gravity constraint term r gravity =R lb [0,0,1] T -[0,0,1] T Make sure the Z axis is aligned with the direction of gravity; Implement zero speed correction mechanism, when stationary (||a k -b a ||<∈), perform zero speed correction r static =[v l ,ω m -b g ]; Add loop closure detection constraints when vision revisits known locations The initial parameters are calculated during a 5-second static period when the drone takes off. Proportional-integral compensation mechanism is used to suppress plane position drift compensation and height drift compensation Through the Jacobian matrix J gl Implementing covariance propagation in a local system In the initialization phase, a local coordinate system is established and the zero bias of the inertial measurement unit is calibrated. In the online phase, the IMU, vision and air pressure data are optimized and fused through a sliding window. The online phase includes a front end, a middle end and a back end. The front end includes IMU pre-integration and visual pose solution correction. The middle end includes joint IMU pre-integration residual, visual residual and air pressure residual correction. The back end includes sliding window optimization, and the window size is 5-10 frames. When vision is lost, IMU and barometer dead reckoning are used.

9. The UAV intelligent image recognition and autonomous flight control system according to claim 1, characterized in that: The trajectory storage module (3) comprises a memory.

Images