A drone inspection method and system based on AI visual control

Abstract

The present invention discloses a drone inspection method and system based on AI visual control, which belongs to the field of drone automatic control technology. The method includes extracting and compressing feature components based on discontinuous frame image data collected by an onboard camera and preset inertial navigation data to obtain a compressed feature stream, performing multimodal obstacle safety level parameter analysis, generating an obstacle avoidance decision priority map and constructing a safety corridor, and generating a real-time self-correcting route; performing semantic analysis on the deviation between the real-time flight trajectory and the real-time self-correcting route to generate target area abnormality marking data; dynamically adjusting the bandwidth allocation of the dual-channel communication network, and transmitting the target area abnormality marking data to the ground control terminal. The present invention adopts a fusion modeling mechanism of discontinuous frame images and inertial navigation, combined with multimodal obstacle safety classification and dual-channel dynamic bandwidth allocation strategy, which can achieve real-time response of dynamic obstacle avoidance, accurate judgment of abnormality detection and efficient transmission of key data.

Description

Technical Field

[0001] The present invention relates to the field of drone automatic control technology, and in particular to a drone inspection method and system based on AI vision control. Background Art

[0002] Drone inspection technology is widely used in scenarios such as power testing and facility monitoring. Its core technology relies on flight path planning and dynamic environment perception. Existing technologies typically use GPS and inertial navigation systems to construct flight trajectories, combined with visible light or infrared sensors for obstacle detection.

[0003] The current mainstream solution uses continuous video streaming to collect environmental information and generates a 3D map using SLAM algorithms. Some systems use LiDAR point cloud data to compare with pre-set routes for obstacle avoidance, transmitting detection data to a ground terminal via a single communication link.

[0004] However, continuous frame data processing leads to excessive computing load on edge devices, and model updates are significantly delayed in complex scenarios; lidar point cloud analysis has a lag in responding to dynamic obstacles and is unable to cope with sudden threats from moving targets; single-channel communication is susceptible to environmental interference, high-priority data transmission is unreliable, and there is a risk of data loss under extreme conditions. Summary of the Invention

[0005] To solve the above problems, the present invention provides a drone inspection method and system based on AI vision control. It adopts a fusion modeling mechanism of discontinuous frame images and inertial navigation, combined with multimodal obstacle safety classification and dual-channel dynamic bandwidth allocation strategy, which can achieve real-time response of dynamic obstacle avoidance, accurate judgment of anomaly detection and efficient transmission of key data.

[0006] The above objectives can be achieved through the following solutions:

[0007] A drone inspection method and system based on AI visual control includes generating a dynamic three-dimensional space vector model based on non-continuous frame image data collected by an onboard camera and preset inertial navigation data; extracting and compressing feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream; performing multimodal obstacle safety level parameter analysis based on the compressed feature stream to generate an obstacle avoidance decision priority map; constructing a safety corridor based on the obstacle avoidance decision priority map, and using the safety corridor to generate a real-time self-correcting route; performing semantic analysis on the deviation between the real-time flight trajectory and the real-time self-correcting route to generate target area anomaly marking data; and dynamically adjusting the bandwidth allocation of a dual-channel communication network to transmit the target area anomaly marking data to a ground control terminal.

[0008] Optionally, generating a dynamic three-dimensional space vector model based on the discontinuous frame image data collected by the airborne camera and the preset inertial navigation data includes: obtaining a scene image sequence and real-time inertial posture parameters under a preset discontinuous frame sampling period; clustering key feature points of the scene image sequence to generate spatially distributed vector nodes; and spatially aligning the spatially distributed vector nodes with the real-time inertial posture parameters to form a dynamic three-dimensional space vector model.

[0009] Optionally, the feature component extraction and compression of the dynamic three-dimensional space vector model to generate a compressed feature stream includes: performing feature layered encoding on the dynamic three-dimensional space vector model to generate an environmental multi-dimensional feature matrix; and filtering the principal components in the environmental multi-dimensional feature matrix through a preset feature entropy value threshold to generate a compressed feature stream.

[0010] Optionally, the multimodal obstacle safety level parameter analysis based on the compressed feature stream to generate an obstacle avoidance decision priority map includes: performing semantic segmentation on the compressed feature stream to generate an obstacle contour feature vector; performing safety factor assessment on the obstacle contour feature vector in combination with a preset obstacle type database to generate an obstacle avoidance decision priority map.

[0011] Optionally, constructing a safety corridor according to the obstacle avoidance decision priority map and generating a real-time self-correcting route using the safety corridor includes: establishing a safety corridor according to the obstacle avoidance decision priority map; searching for feasible paths using the safety corridor, and screening feasible paths that meet preset route continuity rules to obtain a candidate path set; setting a path weight coefficient for each candidate path in the candidate path set to generate an emergency path library; screening a target path from the emergency path library according to the size of the path weight coefficient, and performing secondary optimization on the target path to generate a real-time self-correcting route.

[0012] Optionally, the method further includes: obtaining the inertial motion parameters of the UAV in the current flight state, calculating the future trajectory collision probability through the inertial motion parameters, and generating a risk distribution heat map; superimposing and analyzing the risk distribution heat map with the obstacle avoidance decision priority map to generate a path weight coefficient.

[0013] Optionally, the semantic analysis of the deviation between the real-time flight trajectory and the real-time self-correcting route to generate target area abnormality marking data includes: obtaining geometric topological difference parameters between the preset standard inspection path and the real-time flight trajectory; using the geometric topological difference parameters to generate multi-layer semantic abnormality marking data; using the multi-layer semantic abnormality marking data to identify the abnormality type and generate target area abnormality marking data.

[0014] Optionally, the use of the geometric topology difference parameters to generate multi-layer semantic anomaly labeling data includes: jointly analyzing the geometric topology difference parameters to generate a confidence score; judging the size of the confidence score and a preset abnormal area confidence threshold; and generating multi-layer semantic anomaly labeling data based on the judgment result.

[0015] Optionally, the dynamic adjustment of the dual-channel communication network bandwidth allocation and the transmission of the target area abnormality mark data to the ground control terminal include: real-time monitoring of the signal quality fluctuation parameters and the transmission load status parameters of the primary and backup channels; using the signal quality fluctuation parameters and the transmission load status parameters to calculate the channel score; allocating bandwidth resources according to the channel score and generating a channel switching decision instruction; adjusting the dual-channel communication network bandwidth allocation according to the channel switching decision instruction, and transmitting the target area abnormality mark data to the ground control terminal.

[0016] Based on the same inventive concept, the present invention also provides a drone inspection system based on AI visual control, which includes: a dynamic three-dimensional modeling module, which is used to generate a dynamic three-dimensional space vector model based on non-continuous frame image data collected by an onboard camera and preset inertial navigation data; a feature compression module, which is used to extract and compress feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream; an obstacle avoidance decision module, which is used to perform multimodal obstacle safety level parameter analysis based on the compressed feature stream to generate an obstacle avoidance decision priority map; a route self-correction module, which is used to construct a safety corridor based on the obstacle avoidance decision priority map, and use the safety corridor to generate a real-time self-correcting route; an anomaly detection module, which is used to perform semantic analysis of the deviation between the real-time flight trajectory and the real-time self-correcting route to generate target area anomaly marking data; a communication optimization module, which is used to dynamically adjust the bandwidth allocation of the dual-channel communication network and transmit the target area anomaly marking data to the ground control terminal.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] 1. This invention constructs a dynamic three-dimensional vector model by fusing discontinuous frame images with inertial navigation data, effectively reducing data processing redundancy. The discontinuous sampling mechanism reduces the computational load on the edge computing end, and combined with spatiotemporal alignment technology, improves environmental modeling accuracy, significantly improving modeling efficiency and anti-interference capabilities in complex scenarios.

[0019] 2. Use feature entropy thresholds to screen principal components and combine them with multimodal obstacle safety level analysis to achieve efficient obstacle avoidance decisions. Through semantic segmentation and dynamic feature extraction, obstacle types and threat levels are accurately identified. Combined with a safe corridor path optimization mechanism, this ensures the real-time and safe obstacle avoidance response of the drone in dynamic environments.

[0020] 3. Anomaly detection methods based on risk distribution heat maps and semantic analysis enhance the reliability of inspection target identification. Through geometric topology difference analysis and multi-layer semantic labeling, accurate positioning and classification of abnormal areas are achieved, addressing the lack of adaptability of traditional methods to sudden environmental changes.

[0021] 4. Dynamic dual-channel bandwidth allocation strategy ensures the integrity of critical data transmission; by integrating channel quality assessment with the priority matrix, it optimizes the network resource allocation mechanism, effectively responds to communication fluctuations in complex electromagnetic environments, and improves the real-time and robustness of abnormal data return.

[0022] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is a flow chart of a drone inspection method based on AI vision control according to an embodiment of the present invention.

[0025] Figure 2 3D coordinate diagram of feature point distribution in non-continuous frames according to an embodiment of the present invention.

[0026] Figure 3 3D space vector model diagram of an embodiment of the present invention.

[0027] Figure 4 4 is a characteristic information entropy distribution diagram of an embodiment of the present invention.

[0028] Figure 5 2 is a comparison diagram of the self-correcting routes according to an embodiment of the present invention.

[0029] Figure 6 This is a graph showing changes in the quality scores of the primary and backup channels according to an embodiment of the present invention.

[0030] Figure 7 This is a structural diagram of an AI-based vision-controlled drone inspection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0032] Reference Figure 1 One embodiment of the present invention proposes a drone inspection method based on AI visual control, which adopts a fusion modeling mechanism of non-continuous frame images and inertial navigation, combined with multimodal obstacle safety classification and dual-channel dynamic bandwidth allocation strategy, and can achieve real-time response of dynamic obstacle avoidance, accurate judgment of anomaly detection and efficient transmission of key data.

[0033] The method of this embodiment specifically includes:

[0034] Generate a dynamic three-dimensional space vector model based on the non-continuous frame image data collected by the onboard camera and the preset inertial navigation data;

[0035] Extracting and compressing feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream;

[0036] Performing multimodal obstacle safety level parameter analysis based on the compressed feature stream to generate an obstacle avoidance decision priority map;

[0037] Constructing a safety corridor based on the obstacle avoidance decision priority map, and generating a real-time self-correcting route using the safety corridor;

[0038] Performing semantic analysis on the deviation between the real-time flight trajectory and the real-time self-correction route to generate target area abnormality marking data;

[0039] Dynamically adjust the bandwidth allocation of the dual-channel communication network and transmit the target area anomaly mark data to the ground control terminal.

[0040] Specifically, based on the fusion modeling principle of discrete frame image sampling and inertial navigation data, the system collects image sequences at preset discrete frame intervals, significantly reducing the computational complexity of redundant data. Simultaneously, it integrates the real-time attitude parameters of inertial navigation to construct a spatiotemporally correlated vector node network. Key feature point clustering and a spatiotemporal alignment algorithm dynamically bind the spatial information of discrete image frames to continuous inertial data, generating a three-dimensional spatial vector model that can be updated in real time. This method overcomes the computing power bottleneck of traditional continuous frame processing and achieves a balanced optimization of dynamic environment representation capabilities. Through heterogeneous sensor data fusion and dynamic modeling, it significantly improves computational efficiency while maintaining environmental perception accuracy, effectively adapting to complex weather or motion interference scenarios. The combination of discrete sampling and inertial compensation reduces edge resource consumption, enabling the drone to maintain high-frequency model iterations despite limited computing power, supporting high-precision obstacle avoidance decision-making. The spatiotemporal alignment algorithm enhances the stability and scalability of the three-dimensional vector model in dynamic scenarios, providing interference-resistant spatial information support for subsequent path planning.

[0041] Optionally, generating a dynamic three-dimensional space vector model based on the non-continuous frame image data collected by the airborne camera and preset inertial navigation data includes:

[0042] Obtain scene image sequences and real-time inertial attitude parameters under a preset discontinuous frame sampling period;

[0043] Clustering key feature points of the scene image sequence to generate spatially distributed vector nodes;

[0044] The spatially distributed vector nodes are aligned with the real-time inertial attitude parameters in time and space to form a dynamic three-dimensional space vector model.

[0045] Specifically, the distribution of feature points in non-continuous frames is as follows: Figure 2 As shown, the camera carried by the drone collects a sequence of scene images according to a preset sampling period, and the preset period is a non-continuous frame capture mode with an interval of 5-30 seconds. At the same time, the real-time inertial attitude parameters are obtained by the onboard IMU inertial measurement unit. The real-time inertial attitude parameters include the pitch angle, roll angle, yaw angle and three-axis acceleration components of the drone. The scene image sequence is clustered with key feature points. The SURF feature detection algorithm is used to extract a set of stable feature points for each image frame, and the feature points whose spatial positions appear more than a preset threshold in a continuous preset number of sampling frames are marked as key feature points. For the spatial distribution vector node coordinate v, there is:

[0046] ;

[0047] In the formula represents the frequency weight of the i-th key feature point, is the three-dimensional coordinate of the i-th key feature point in the image coordinate system. Finally, a three-dimensional point cloud structure composed of multiple vector nodes is generated. The time-space stamps corresponding to each spatially distributed vector node are aligned with the inertial attitude parameter time series. The Kalman filter algorithm is used to compensate the time-space displacement vector to eliminate the coordinate offset caused by the drone movement. The vector nodes after displacement compensation are transformed and mapped to the IMU coordinate system to form a dynamic three-dimensional space vector model including the time-space alignment vector network, such as Figure 3 shown.

[0048] For example, the above steps are implemented in a high-voltage line inspection scenario in a mountainous area. The drone captures a set of non-continuous image sequences every 15 seconds and simultaneously records the flight attitude parameters. In the feature point clustering process, it is found that the corner feature of the transmission tower appears stably in three consecutive samplings, and it is given a high weight. The floating object feature was filtered out because its frequency of occurrence fell below the threshold of 2 times per cycle. In the vector model created after spatiotemporal alignment, the transmission tower structure coordinate accuracy was improved to ±0.3 meters, reducing the computational load by 60% compared to traditional continuous-frame modeling methods. Discontinuous sampling reduces redundant data processing and improves resource utilization in edge computing modules. A frequency-weighted mechanism provides dynamic interference resistance in the modeling process, enabling accurate identification of the spatial location characteristics of infrastructure even under complex operating conditions.

[0049] Optionally, extracting and compressing feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream includes:

[0050] Performing feature hierarchical coding on the dynamic three-dimensional space vector model to generate a multi-dimensional feature matrix of the environment;

[0051] The principal components in the multi-dimensional feature matrix of the environment are screened by a preset feature entropy value threshold to generate a compressed feature stream.

[0052] Specifically, the dynamic three-dimensional space vector model is firstly subjected to feature hierarchical encoding. The dynamic three-dimensional space vector model is composed of several spatially distributed vector nodes, each of which contains three-dimensional coordinate information and an associated acquisition timestamp. The edge computing end uses a multi-layer convolutional network to extract local features from the spatial distribution of vector nodes, and generates a primary feature matrix containing geometric structure information, texture information, and dynamic change information. The cloud computing end receives the primary feature matrix, and uses a global feature association algorithm to vertically fuse features at each level to generate a multi-dimensional feature matrix of the environment, where the rows of the matrix correspond to spatially distributed vector nodes, and the columns represent feature dimensions at different levels. The principal components in the multi-dimensional feature matrix of the environment are screened by a preset feature entropy value threshold, and the feature entropy value is calculated by the information entropy formula:

[0053] ,

[0054] Where, is the information entropy of the feature dimension, is the normalized probability distribution value of the ith eigenvalue in all space vector nodes. In the specific calculation, the eigenvalue distribution frequency of each column in the multi-dimensional feature matrix of the statistical environment is normalized to obtain . Set the entropy threshold It is the system preset experience value. When , it is determined that the feature dimension contains valid environmental information and is retained as a key component; when When , it is determined to be a redundant component and deleted. Finally, the filtered feature dimensions are reorganized according to the order of space vector nodes to generate a compressed feature stream with reduced dimensions. In the above process, feature hierarchical coding realizes the efficient integration of local and global features, reducing the amount of redundant data of the original three-dimensional model; the feature entropy value screening is as follows Figure 4 As shown in the figure, the feature entropy threshold screening removes low-value noise features by quantifying the information entropy and retains the principal components that have a significant contribution to obstacle recognition.

[0055] For example, in the transmission line inspection scenario, the edge computing end extracts the structural characteristics of the transmission tower insulator string and the dynamic change characteristics of the conductor sag to form a primary feature matrix containing geometric shape, surface corrosion level, and motion trajectory. Each feature dimension corresponds to a 32-bit floating-point value. The cloud computing end vertically fuses the feature matrix collected for 5 consecutive minutes to generate a 1000-row by 50-column environmental multidimensional feature matrix. By calculating the entropy value of each column, it is found that the conductor vibration frequency dimension H=2.3, which is higher than ; The background cloud layer displacement dimension H=0.5 is below the threshold and is eliminated. The 30 columns of key features finally retained are reorganized to generate a compressed feature stream, reducing the data volume by 40%. Through the layered coding mechanism, local detail features and global context information are organically integrated, so that the compressed feature stream can not only characterize the microscopic state of the drone inspection target, but also reflect the macroscopic change law of the environment. The screening method based on information entropy effectively filters out invalid feature dimensions introduced by weather interference or sensor noise, reducing the computational complexity of the subsequent obstacle avoidance decision algorithm. The compressed feature stream retains the core features of the key components of the transmission line, such as high-frequency vibration and structural deformation, to ensure the accuracy of the safety level analysis, while significantly reducing the transmission bandwidth requirements between the edge and the cloud.

[0056] Optionally, performing multimodal obstacle safety level parameter analysis based on the compressed feature stream to generate an obstacle avoidance decision priority map includes:

[0057] Performing semantic segmentation on the compressed feature stream to generate an obstacle contour feature vector;

[0058] The obstacle profile feature vector is evaluated for safety factor in combination with a preset obstacle type database to generate an obstacle avoidance decision priority map.

[0059] Specifically, the compressed feature stream is first subjected to semantic segmentation processing. The compressed feature stream is matrix data with reduced dimensions, in which each row corresponds to a spatially distributed vector node and each column is a retained key feature dimension. A deformable convolutional neural network is used to divide the compressed feature stream into spatial domains, and a semantic segmentation mask is constructed based on the correlation between adjacent nodes in the feature matrix to identify potential obstacle blocks in the environmental scene. The dynamic change characteristics of the obstacle blocks, including the motion speed vector and the shape change trend parameters, are extracted through a bidirectional LSTM network and encoded into an obstacle contour feature vector. The classification benchmark features are then called from the preset obstacle type database. The database includes standard feature templates for fixed obstacle categories and mobile obstacle categories, such as the geometric shape, reflectivity and motion pattern characteristics of typical obstacles such as insulator strings, tower structures, hanging objects, and bird flocks in transmission line scenes. The similarity matching calculation is performed on the obstacle contour feature vector and the classification benchmark feature:

[0060] ,

[0061] Where, is the safety factor, ranging from 0 to 1; Represents the obstacle contour feature vector; Represents the standard feature vector of a certain type of obstacle in the database; is the number of dimensions of the feature vector. By traversing the obstacle categories in the database, calculate all The obstacle type corresponding to the maximum value is selected as the current obstacle classification result. Safety level parameters are then assigned based on the classification results, and a mapping rule for safety level L is set: when a fixed infrastructure obstacle is identified, L = 1, indicating no avoidance is required; when a low-dynamic obstacle is identified, L = 2, indicating that path optimization is required; and when a high-risk moving obstacle is identified, L = 3, indicating that emergency avoidance is necessary. The spatial coordinates of each obstacle and its safety level parameters are integrated into a dynamic grid map to generate an obstacle avoidance decision priority map that includes obstacle distribution density, movement direction, and safety level.

[0062] For example, during an inspection mission in an industrial park's power pipeline corridor, a drone detected an unusually moving target in a certain area using compressed feature streams. Semantic segmentation separated the target area from the background, and contour feature vector extraction revealed a horizontal displacement of 0.2 meters per second and an elliptical shape. After matching the target with the database, the target was determined to be a patrol robot with a matching degree of S = 0.8 for similar features and S = 0.3 for flocking birds. Based on this, the target was classified as a low-dynamic obstacle and L = 2 was set. Based on its location information, an orange warning area was generated in the priority map, triggering smoothing and offset correction of the drone's flight path. The combination of semantic segmentation and dynamic feature extraction accurately identifies the physical characteristics and motion patterns of obstacles, avoiding misjudgments caused by ambient noise. A safety factor-based classification mechanism enables the system to differentiate between obstacles of varying threat levels, balancing inspection efficiency and flight safety. The priority map provides intuitive decision-making through visual spatial annotation. This significantly reduces the complexity of obstacle avoidance path planning, particularly in mixed obstacle scenarios, ensuring the drone's precise obstacle avoidance and continuous operation in narrow pipeline corridors.

[0063] Optionally, constructing a safety corridor according to the obstacle avoidance decision priority map and generating a real-time self-correcting route using the safety corridor includes:

[0064] establishing a safe corridor according to the obstacle avoidance decision priority map;

[0065] Searching for feasible paths using the safety corridor and screening feasible paths that meet preset route continuity rules to obtain a set of candidate paths;

[0066] Setting a path weight coefficient for each candidate path in the candidate path set to generate an emergency path library;

[0067] A target path is selected from the emergency path library according to the size of the path weight coefficient, and the target path is optimized twice to generate a real-time self-correction route.

[0068] Specifically, the self-correcting route is as follows Figure 5As shown in the figure, first, a three-dimensional safety bounding box is established based on the spatial coordinates and safety level parameters of each obstacle marked in the obstacle avoidance decision priority map. For each high-risk mobile obstacle with a safety level of L=3, a spherical safety boundary is generated by expanding the center point of the obstacle according to the preset expansion coefficient. The expansion coefficient is set according to 1.2 times the wingspan of the drone. For low-dynamic obstacles with a safety level of L=2, a cylindrical safety boundary is generated by extending along its movement direction. The length of the column is the product of the obstacle movement speed and the preset reaction time. A Boolean union operation is performed on all safety boundaries to generate a three-dimensional safety corridor model containing multiple layers of protection areas. The construction of the emergency path library searches for feasible paths in the safety corridor through the random sampling tree RRT algorithm. A dynamic weight adjustment mechanism is introduced in the sampling process, and the node selection probability The calculation formula is:

[0069] ,

[0070] Where d represents the Euclidean distance between the sampling node and the target waypoint, k is the path continuity weight factor, and C is the normalization constant. The k value is dynamically updated through the current flight speed and attitude angle parameters of the drone. When the drone is in a high-speed flight state, the k value is increased to give priority to the path segment with smaller route curvature as the feasible path. The feasible paths that meet the route continuity rule are marked as candidate paths and the path weight coefficient is set. The candidate paths and the corresponding path weight coefficients are used to construct an emergency path library. The route continuity rule requires that the heading angle change between adjacent path nodes does not exceed 5 degrees and the acceleration fluctuation rate is lower than the preset threshold. The candidate path with the largest path weight coefficient is selected from the emergency path library as the target path. The generation of real-time self-correcting routes uses a nonlinear optimization algorithm to perform secondary optimization on the target path. Establish the objective function:

[0071] ,

[0072] Where, is the heading angle change value of the adjacent path segment, is the acceleration requirement of the path point, 、 The heading smoothing weight and motion smoothing weight are configured based on the UAV's model performance parameters. The objective function's minimum value is iteratively solved using the gradient descent method, ultimately generating a self-correcting route that balances safety and flight stability.

[0073] Optionally, the method further includes:

[0074] Obtain the inertial motion parameters of the drone in its current flight state, calculate the future trajectory collision probability based on the inertial motion parameters, and generate a risk distribution heat map;

[0075] The risk distribution heat map and the obstacle avoidance decision priority map are superimposed and analyzed to generate a path weight coefficient.

[0076] Specifically, the inertial motion parameters of the drone in the current flight state include the real-time velocity vector, acceleration vector and attitude angle change rate of the drone in three-dimensional space. Set the future trajectory prediction time window in the onboard processor ,Pick The reaction time corresponding to the braking distance of 2 times the current speed of the drone. The trajectory prediction model is established based on the inertial motion parameters, and the model input is the current speed With acceleration , the output is the predicted position coordinates at time t in the future :

[0077] ,

[0078] In the formula , is the vector consisting of three axial velocity components, is the real-time acceleration vector. Based on the predicted trajectory and the obstacle distribution in the obstacle avoidance decision priority map, a spatial correlation analysis is performed. For each predicted moment , calculate the bounding box of each obstacle and The shortest Euclidean distance and preset dynamic safety thresholds Compare, Set to 1.5 times the wingspan length according to the drone type. When statistics are potential collision events, the collision probability The calculation formula is:

[0079] ,

[0080] In the formula To adjust the coefficient, the UAV dynamic response parameters are calibrated. of Mapping to three-dimensional space generates a risk distribution heat map, with the risk density value of each voxel in the map For all within its scope The risk distribution heat map and the obstacle avoidance decision priority map are spatially aligned, and multi-dimensional parameters are superimposed at the corresponding coordinate points to generate a comprehensive influencing factor matrix. Path weight coefficient The calculation formula is:

[0081] ,

[0082] In the formula is the obstacle avoidance priority weight coefficient, is the risk thermal weight coefficient, is the safety level parameter of all obstacles covered by the path segment, The number of grid cells to cover for the path segment. 、 Dynamically adjust according to the UAV operation mode, increase when the speed exceeds the threshold value to increase risk avoidance priority.

[0083] By integrating real-time kinematic predictions with prior obstacle avoidance data, the path weight allocation mechanism proactively identifies dynamic risks and effectively mitigates the threat of sudden moving obstacles. Direct calculation of inertial motion parameters combined with a distance decay function ensures that the risk probability model conforms to real-world physical laws. The multi-dimensional overlay of heat maps and priority maps significantly enhances decision-making robustness in complex scenarios, particularly in densely populated areas, preventing route conflicts caused by inertial navigation errors or sudden obstacle changes.

[0084] Optionally, performing semantic analysis on the deviation between the real-time flight trajectory and the real-time self-correction route to generate target area abnormality marking data includes:

[0085] Obtain the geometric topology difference parameters between the preset standard inspection path and the real-time flight trajectory;

[0086] generating multi-layer semantic anomaly labeling data using the geometric topology difference parameters;

[0087] The multi-layer semantic anomaly labeling data is used to identify anomaly types and generate target area anomaly labeling data.

[0088] Optionally, generating multi-layer semantic anomaly labeling data using the geometric topology difference parameters includes:

[0089] performing a joint analysis on the geometric topological difference parameters to generate a confidence score;

[0090] Determining the magnitude of the confidence score and a preset abnormal area confidence threshold;

[0091] Based on the judgment results, multi-layer semantic anomaly labeling data is generated.

[0092] Specifically, the standard inspection path and real-time flight trajectory data preset by the drone are obtained. The standard inspection path contains a serialized GPS coordinate point set and the corresponding flight altitude constraint parameters. The real-time flight trajectory consists of the latitude and longitude, altitude and timestamp output by the onboard integrated navigation system. The spatial topological relationship of the path nodes synchronized with each timestamp is compared to calculate the three-dimensional Euclidean distance error. Angle deviation from the path Specifically, Use the following formula to calculate point by point:

[0093] ,

[0094] Where, 、 、 Represents the three-dimensional coordinates of the real-time flight trajectory node, 、 、 is the three-dimensional coordinate of the standard inspection path node at the corresponding time. It is obtained by calculating the angle between the real-time heading vector and the standard path vector. The vector direction is determined by the line connecting the adjacent path nodes. , path direction angle deviation Combined with the abnormal area parameter Construct geometric topological difference parameters. Use geometric topological difference parameters to calculate joint eigenvectors :

[0095] ,

[0096] In the formula represents the joint eigenvalue of the jth abnormal region; 、 、 It is a preset weighting coefficient, which is determined through training of historical abnormal data; is the cumulative sum of distance errors of consecutive waypoints; Indicates the time rate of change of the angular deviation; The confidence score is calculated by the minimum circumscribed rectangle area of ​​the abnormal area projected on the horizontal plane. :

[0097] ,

[0098] In the formula is the confidence level, is the attenuation coefficient. and the preset confidence threshold of abnormal area (Take the range of 0.7-0.9) for comparison, when Valid detection targets are identified when an error occurs; otherwise, they are considered false alarm noise. The joint feature vectors of valid detection targets are analyzed hierarchically. The first level extracts the error distribution pattern and determines whether it is a device offset or environmental interference type. The second level analyzes the spatial extension characteristics of the abnormal area, classifying local point anomalies and continuous surface anomalies. The third level combines historical inspection data to verify the persistence of the anomaly, ultimately generating multi-layer semantic anomaly labeling data that includes the anomaly category, impact range, and confidence level. The anomaly type is identified based on the information contained in the multi-layer semantic anomaly labeling data, and the target area anomaly labeling data is aggregated.

[0099] Optionally, the dynamically adjusting bandwidth allocation of the dual-channel communication network to transmit the target area abnormality mark data to the ground control terminal includes:

[0100] Real-time monitoring of signal quality fluctuation parameters and transmission load status parameters of the primary and backup channels;

[0101] Calculating a channel score using the signal quality fluctuation parameter and the transmission load state parameter;

[0102] Allocate bandwidth resources according to the channel scores and generate channel switching decision instructions;

[0103] The dual-channel communication network bandwidth allocation is adjusted according to the channel switching decision instruction, and the target area abnormality mark data is transmitted to the ground control terminal.

[0104] Specifically, the quality scores of the primary and backup channels are as follows: Figure 6 As shown, the communication status parameters of the main channel and the backup channel are collected in real time. The signal quality fluctuation parameters are obtained through the physical layer detection of the airborne communication module, including the received signal strength indication RSSI, the bit error rate BER and the channel delay fluctuation value. RSSI is obtained by converting the wireless signal receiving power, BER is the ratio of the number of error bits per unit time to the total number of transmitted bits, and the channel delay fluctuation value measures the standard deviation of the round-trip time RTD of the data packet. The transmission load status parameters include the current channel bandwidth occupancy rate and data queue backlog . The bandwidth utilization rate is calculated by dividing the currently used bandwidth by the theoretical maximum bandwidth of the channel, and the data queue backlog counts the total number of bytes of data packets to be transmitted. Construct a transmission data priority matrix, define the target area abnormal marking data as the highest priority category P1, and divide other inspection data into levels P2 to P4 according to the information type. The rows of the priority matrix correspond to the data type, and the columns include priority weight, minimum bandwidth requirement and maximum allowed delay. The priority weight is preset according to the urgency of the exception handling, and the P1 category has the highest weight. The bandwidth allocation decision function calculates the comprehensive channel quality score of each channel:

[0105] ,

[0106] In the formula It is the comprehensive score of the channel, used to quantify the transmission capacity of the channel. is the normalized received signal strength, BER is the current bit error rate value, is the bandwidth utilization rate, It is the normalized value of the data queue backlog. 、 、 、 is the weight coefficient, which is dynamically adjusted according to the UAV's flight altitude and the intensity of environmental interference. The coefficient is used to increase the weight of the bit error rate. The values ​​are calculated and compared independently. Below the backup channel And the difference exceeds the preset threshold The bandwidth allocation algorithm performs a two-stage operation based on the priority matrix: the first stage is hard allocation, ensuring that all P1 data obtains its minimum bandwidth requirement. The second stage is dynamic allocation, which allocates bandwidth to other data types in the remaining bandwidth according to the priority weight ratio. When the P1 data is transmitted, the dual-channel parallel transmission mode is immediately started, so that the P1 data fragments are transmitted simultaneously through the primary and backup channels, and the data integrity is ensured through redundancy verification.

[0107] For example, during an offshore oil and gas platform inspection, a drone detected pipeline corrosion anomalies and generated P1-level anomaly flag data. At this point, the RSSI of the primary channel decreased due to wave reflections, while the BER increased. The backup channel remained unchanged through the satellite link. The calculated score for the primary channel was 0.65, and the score for the backup channel was 0.82, a difference of more than Threshold-triggered switching. The system prioritizes 2Mbps bandwidth to ensure abnormal data transmission, and the remaining 4Mbps bandwidth is allocated to the visible light inspection video stream. When strong wind disturbances cause a brief interruption of the satellite link, the algorithm automatically enables dual-channel parallel transmission, splits the abnormal data into encrypted data blocks and transmits them through the remaining available maritime radio channels. After the satellite is restored, the integrity check shows no packet loss. The dual-channel management mechanism based on the comprehensive channel quality score and priority matrix can autonomously select the optimal transmission path in complex electromagnetic environments to ensure real-time and reliable transmission of critical abnormal data. The dynamic bandwidth allocation strategy takes into account the transmission requirements of different data types, giving priority to high-priority services when channel conditions deteriorate, and avoiding the loss of detection information caused by congestion of a single channel. The dual-channel parallel transmission and redundancy check mechanism effectively deal with link instability problems in extreme environments, significantly improving the success rate of drone inspection data return in severe weather and complex terrain areas.

[0108] Based on the same inventive concept, Figure 7 As shown, the present invention also provides a drone inspection system based on AI visual control, the system comprising:

[0109] A dynamic 3D modeling module is used to generate a dynamic 3D space vector model based on the non-continuous frame image data collected by the onboard camera and the preset inertial navigation data;

[0110] A feature compression module, configured to extract and compress feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream;

[0111] An obstacle avoidance decision module, configured to perform multimodal obstacle safety level parameter analysis based on the compressed feature stream and generate an obstacle avoidance decision priority map;

[0112] a route self-correction module, configured to construct a safety corridor based on the obstacle avoidance decision priority map and generate a real-time self-correction route using the safety corridor;

[0113] An anomaly detection module, configured to perform semantic analysis on deviations between the real-time flight trajectory and the real-time self-correcting route, and generate target area anomaly marking data;

[0114] The communication optimization module is used to dynamically adjust the bandwidth allocation of the dual-channel communication network and transmit the abnormal marking data of the target area to the ground control terminal.

[0115] It should be noted that the electrical connections between the above-mentioned units do not necessarily mean direct connections of lines. Indirect connections are applicable to the embodiments of the present invention as long as the purpose of the present invention is achieved. The above description is only an exemplary embodiment of the present invention and is not intended to limit the scope of the present invention.

[0116] That is, any equivalent changes and modifications made according to the teachings of the present invention are still within the scope of the present invention. Those skilled in the art will readily conceive of other embodiments of the present invention after considering the disclosure of the specification and practical truths. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary technical means in the art not described herein.

Claims

1. A drone inspection method based on AI visual control, characterized in that: The method comprises: Generate a dynamic three-dimensional space vector model based on the non-continuous frame image data collected by the onboard camera and the preset inertial navigation data; Extracting and compressing feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream; Performing a multimodal obstacle safety level parameter analysis based on the compressed feature stream to generate an obstacle avoidance decision priority map; the method includes: performing semantic segmentation on the compressed feature stream to generate an obstacle contour feature vector; and performing a safety factor assessment on the obstacle contour feature vector in combination with a preset obstacle type database to generate an obstacle avoidance decision priority map. Constructing a safety corridor based on the obstacle avoidance decision priority map, and generating a real-time self-correcting route using the safety corridor; Performing semantic analysis on the deviation between the real-time flight trajectory and the real-time self-correcting route to generate target area anomaly marking data; this includes: obtaining geometric topological difference parameters between a preset standard inspection path and the real-time flight trajectory; generating multi-layer semantic anomaly marking data using the geometric topological difference parameters; and identifying anomaly types using the multi-layer semantic anomaly marking data to generate target area anomaly marking data; Dynamically adjust the bandwidth allocation of the dual-channel communication network to transmit the target area anomaly mark data to the ground control terminal; Among them, the use of the geometric topological difference parameters to generate multi-layer semantic anomaly labeling data includes: jointly analyzing the geometric topological difference parameters to generate a confidence score; judging the size of the confidence score and a preset abnormal area confidence threshold; and generating multi-layer semantic anomaly labeling data based on the judgment result.

2. The AI ​​vision-controlled drone inspection method according to claim 1 is characterized in that: Generating a dynamic three-dimensional space vector model based on the non-continuous frame image data collected by the airborne camera and the preset inertial navigation data includes: Obtain scene image sequences and real-time inertial attitude parameters under a preset discontinuous frame sampling period; Clustering key feature points of the scene image sequence to generate spatially distributed vector nodes; The spatially distributed vector nodes are aligned with the real-time inertial attitude parameters in time and space to form a dynamic three-dimensional space vector model.

3. The AI ​​vision-controlled drone inspection method according to claim 1 is characterized in that: Extracting and compressing the feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream includes: Performing feature hierarchical coding on the dynamic three-dimensional space vector model to generate a multi-dimensional feature matrix of the environment; The principal components in the multi-dimensional feature matrix of the environment are screened by a preset feature entropy value threshold to generate a compressed feature stream.

4. The AI ​​vision-controlled drone inspection method according to claim 1, characterized in that: The step of constructing a safety corridor according to the obstacle avoidance decision priority map and generating a real-time self-correcting route using the safety corridor includes: establishing a safe corridor according to the obstacle avoidance decision priority map; Searching for feasible paths using the safety corridor and screening feasible paths that meet preset route continuity rules to obtain a set of candidate paths; Setting a path weight coefficient for each candidate path in the candidate path set to generate an emergency path library; A target path is selected from the emergency path library according to the size of the path weight coefficient, and the target path is optimized twice to generate a real-time self-correction route.

5. The AI ​​vision-controlled drone inspection method according to claim 4 is characterized in that: The method further comprises: Obtain the inertial motion parameters of the drone in its current flight state, calculate the future trajectory collision probability based on the inertial motion parameters, and generate a risk distribution heat map; The risk distribution heat map and the obstacle avoidance decision priority map are superimposed and analyzed to generate a path weight coefficient.

6. The AI ​​vision-controlled drone inspection method according to claim 1, characterized in that: The dynamically adjusting the bandwidth allocation of the dual-channel communication network and transmitting the target area abnormality mark data to the ground control terminal includes: Real-time monitoring of signal quality fluctuation parameters and transmission load status parameters of the primary and backup channels; Calculating a channel score using the signal quality fluctuation parameter and the transmission load state parameter; Allocate bandwidth resources according to the channel scores and generate channel switching decision instructions; The dual-channel communication network bandwidth allocation is adjusted according to the channel switching decision instruction, and the target area abnormality mark data is transmitted to the ground control terminal.

7. A drone inspection system based on AI vision control, applied to the drone inspection method based on AI vision control according to any one of claims 1 to 6, characterized in that: The system comprises: A dynamic 3D modeling module is used to generate a dynamic 3D space vector model based on the non-continuous frame image data collected by the onboard camera and the preset inertial navigation data; A feature compression module, configured to extract and compress feature components of the dynamic three-dimensional space vector model to generate a compressed feature stream; An obstacle avoidance decision module is configured to perform multimodal obstacle safety level parameter analysis based on the compressed feature stream to generate an obstacle avoidance decision priority map. This includes: performing semantic segmentation on the compressed feature stream to generate obstacle contour feature vectors; and performing safety factor assessment on the obstacle contour feature vectors in conjunction with a preset obstacle type database to generate an obstacle avoidance decision priority map. a route self-correction module, configured to construct a safety corridor based on the obstacle avoidance decision priority map and generate a real-time self-correction route using the safety corridor; An anomaly detection module is used to perform semantic analysis on the deviation between the real-time flight trajectory and the real-time self-correcting route to generate target area anomaly labeling data; this includes: obtaining geometric topological difference parameters between the preset standard inspection path and the real-time flight trajectory; using the geometric topological difference parameters to generate multi-layer semantic anomaly labeling data; using the multi-layer semantic anomaly labeling data to identify the anomaly type and generate target area anomaly labeling data; A communication optimization module is used to dynamically adjust the bandwidth allocation of the dual-channel communication network and transmit the target area anomaly mark data to the ground control terminal; Among them, the use of the geometric topological difference parameters to generate multi-layer semantic anomaly labeling data includes: jointly analyzing the geometric topological difference parameters to generate a confidence score; judging the size of the confidence score and a preset abnormal area confidence threshold; and generating multi-layer semantic anomaly labeling data based on the judgment result.

Images