Unmanned aerial vehicle inspection method and system combining skeleton feature extraction and scene generation technology

Abstract

The invention discloses an unmanned aerial vehicle inspection method and system combining skeleton feature extraction and scene generation technology, and the method comprises the steps: S1, enabling dual laser radars to be carried on an unmanned aerial vehicle, and obtaining the three-dimensional point cloud data of a closed space through the dual laser radars; s2, performing feature extraction on the three-dimensional point cloud data by using a skeleton feature extraction method to obtain 3D skeleton features; s3, using a scene generation technology to construct an unknown simulation scene for the three-dimensional point cloud data; s4, modeling a closed scene based on the 3D skeleton features and the unknown simulation scene, and generating an unmanned aerial vehicle inspection track based on a closed space model; and S5, constructing a complete priori map based on the unmanned aerial vehicle inspection track, and completing unmanned aerial vehicle inspection based on the priori map.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and particularly to an unmanned aerial vehicle inspection method and system combining skeleton feature extraction and scene generation technologies. Background Art

[0002] Traditional unmanned aerial vehicle inspection technologies usually rely on prior maps, which limits their application in unknown environments. Existing inspection methods have poor adaptability in complex enclosed environments, and it is difficult to efficiently extract environmental features and conduct autonomous exploration. In addition, traditional methods lack real-time performance and robustness in dynamic environments, resulting in low inspection efficiency of unmanned aerial vehicles in complex environments. Therefore, there is a need for an unmanned aerial vehicle inspection technology that can perform high-precision feature extraction and efficient autonomous exploration in unknown environments. Summary of the Invention

[0003] To solve the above problems, the present invention proposes an unmanned aerial vehicle inspection method combining skeleton feature extraction and scene generation technologies, and the method includes:

[0004] Step S1: Mount dual lidars on the unmanned aerial vehicle, and use the dual lidars to obtain three-dimensional point cloud data of an enclosed space;

[0005] Step S2: Use a skeleton feature extraction method to extract features from the three-dimensional point cloud data to obtain 3D skeleton features;

[0006] Step S3: Use a scene generation technology to construct an unknown simulation scene for the three-dimensional point cloud data;

[0007] Step S4: Model the enclosed scene based on the 3D skeleton features and the unknown simulation scene, and generate an unmanned aerial vehicle inspection flight path based on the enclosed space model;

[0008] Step S5: Construct a complete prior map based on the unmanned aerial vehicle inspection flight path, and complete the unmanned aerial vehicle inspection based on the prior map.

[0009] Optionally, in step S2, the content of obtaining 3D skeleton features specifically includes: using fire simulation and maximum disk operation to extract 3D skeleton features from the environmental boundary points of the three-dimensional point cloud data.

[0010] Optionally, the process of the fire simulation includes:

[0011]

[0012] Wherein, is the level set function, F is the evolution speed function, κ is the curvature, α is the curvature weight coefficient, is the norm of the gradient of the level set function.

[0013] Optionally, the process of the maximum disk operation specifically includes:

[0014]

[0015] Among them, D(p) is the distance from point p to the boundary, A is the target area, S is the set of points that meet the conditions, that is, it represents the center positions of all the maximum inscribed circles in area A, and D(q) is the radius of the inscribed circle corresponding to the points in area A.

[0016] Optionally, the process of step S3, using the scenario generation technology to construct an unknown simulation scenario for the three-dimensional point cloud data specifically includes:

[0017] According to the existing images of the enclosed space, use the scenario generation technology to generate simulation enclosed scenario data and then perform preprocessing;

[0018] Construct a neural network model, use the preprocessed simulation enclosed scenario data for training, and obtain the trained neural network model;

[0019] Input the three-dimensional point cloud data into the trained neural network model to obtain an unknown simulation scenario.

[0020] Optionally, the preprocessing process specifically includes: performing binarization processing and gradual erosion on the simulation enclosed scenario data to generate a single-pixel frame with size invariance and rotation invariance, and extracting environmental features based on the single-pixel frame.

[0021] Optionally, the content of the gradual erosion specifically includes:

[0022]

[0023] Among them, A is the set of original images, B is the structuring element, z is the translation vector or position, and B z is the position of the structuring element B after translation by z.

[0024] Optionally, the process of the rotation invariance specifically includes:

[0025]

[0026] Among them, V nm (r,θ) is the Zernike polynomial; f(r,θ) is the input function in the polar coordinate system, defined within the unit circle, r is the radial distance, n is the radial order, m is the azimuthal order, and V n * m (r,θ) is the complex conjugate of the Zernike polynomial.

[0027] The present invention also discloses a drone inspection system that combines skeleton feature extraction and scene generation technologies. The system includes:

[0028] A point cloud acquisition module, which is used to mount dual lidar on a drone and use the dual lidar to acquire three-dimensional point cloud data of an enclosed space;

[0029] A skeleton feature extraction module, which is used to extract features from the three-dimensional point cloud data using a skeleton feature extraction method to obtain 3D skeleton features;

[0030] A scene simulation module, which is used to construct an unknown simulation scene for the three-dimensional point cloud data using scene generation technologies;

[0031] An inspection flight path generation module, which is used to model an enclosed scene based on the 3D skeleton features and the unknown simulation scene, and generate a drone inspection flight path based on the enclosed space model;

[0032] A drone inspection module, which is used to construct a complete prior map based on the drone inspection flight path and complete drone inspection based on the prior map.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] By combining skeleton feature extraction and scene generation technologies, the present invention significantly improves the autonomous exploration and inspection capabilities of drones in unknown environments. This method can improve the adaptability of drones in complex enclosed environments and achieve efficient inspections while ensuring reliability.

[0035] Through fire simulation and maximum disk operations, the skeleton feature extraction technology can accurately extract key structural features (such as boundary points, obstacle contours, etc.) from complex environments and generate a single-pixel framework with size invariance and rotation invariance. In an enclosed space environment, this method can effectively remove image noise, identify feasible areas of narrow channels, and the accuracy is greatly improved compared with traditional methods. Combined with morphological operations, drones can generate high-resolution environmental models in real time, providing a reliable data basis for subsequent path planning.

[0036] The scene generation technology constructs a virtual environment containing dynamic obstacles, light and shadow changes, and complex structures through procedural generation, reconstruction based on real data, and physical engine simulation. For example, in the inspection scenario of a petrochemical plant, the virtual environment can simulate sudden situations such as pipeline leaks and valve state changes, enabling drones to adapt to the dynamic challenges of the real environment in advance during training. According to industry reports, the first mission success rate of drones using virtual training technology in the real environment is increased by more than 40%. In addition, the real-time constructed prior map can predict the possible structures of unknown areas, significantly shortening the exploration time of drones in unfamiliar environments.

[0037] By combining deep learning algorithms with multi - fork tree traversal algorithms, drones can analyze the environmental topological structure in real - time and dynamically plan the optimal inspection path. For example, in the inspection of power facilities, the system can identify the break points of transmission lines or damaged insulators and ensure that all key nodes are detected without omission through the multi - fork tree traversal algorithm. The path planning efficiency of this method in complex environments is improved compared with the traditional A* algorithm, and the energy consumption is also reduced. Through the deep integration of skeleton feature extraction and scene generation technologies, the present invention solves the core problems of traditional drone inspections that rely on prior maps and have poor dynamic adaptability. On the premise of ensuring the reliability of the system, high - precision perception, virtual - real combination training, and dynamic path planning in complex enclosed environments are achieved, significantly improving the autonomous exploration ability and inspection efficiency of drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the technical solutions of the present invention, the following briefly introduces the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0039] Figure 1 It is a flowchart of the drone inspection method combining skeleton feature extraction and scene generation technology according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the above - mentioned objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0041] Embodiment 1

[0042] A drone inspection method combining skeleton feature extraction and scene generation technology, as Figure 1 shown, the method is specifically as follows:

[0043] Step S1: Mount two lidars on the drone and use the two lidars to obtain three - dimensional point cloud data of the enclosed space.

[0044] Step S2: Use the skeleton feature extraction method to extract features from the three - dimensional point cloud data to obtain 3D skeleton features.

[0045] Precisely track and record the environmental boundary points through fire simulation and maximum disk operation, and combine morphological operations to binarize and gradually erode the unknown environment image to generate a single - pixel framework with size invariance and rotation invariance to extract the key structural features in the environment.

[0046] The process of fire simulation includes:

[0047]

[0048] Among them, φ(x, y, t) is the level set function, F is the evolution speed function, κ is the curvature, and α is the curvature weight coefficient.

[0049] The process of the maximum disk operation specifically includes:

[0050]

[0051] Among them, D(p) is the distance from point p to the boundary, and A0 is the target area.

[0052] In this embodiment, the maximum disk operation identifies the maximum inscribed circle area in the environment, providing basic data support for path planning.

[0053] The process of image binarization specifically includes:

[0054]

[0055] Among them, T is the adaptive threshold.

[0056] The content of gradual erosion specifically includes:

[0057]

[0058] Among them, A is the initial three-dimensional point cloud set, B is the spherical structuring element, z is the three-dimensional translation vector or position, and B z is the position of the structuring element B translated by z.

[0059] This embodiment adopts the Hu invariant matrix: [[ID=XX]]

[0060] I1 = η 20 + η 02

[0061]

[0062] Among them, η pq is the normalized central moment, satisfying translation, rotation, and scale invariance.

[0063] This embodiment improves the above Hu into a three-dimensional improved invariant moment:

[0064] I1 = Ψ 200 + Ψ 020 + Ψ 002

[0065]

[0066] Among them, Ψ pqr It should be noted that there seems to be some incomplete or unclear parts in the original text, especially in the formulas where some parts are not fully presented. This translation is based on the available content as accurately as possible.The three-dimensional normalized central moment satisfies translational, rotational, and scaling invariance in space and is used to characterize the essential features of three-dimensional geometric structures.

[0067] Step S3: Use scene generation technology to construct an unknown simulation scene for the three-dimensional point cloud data.

[0068] Based on the existing images of the enclosed space, use scene generation technology to generate simulation enclosed scene data and then perform preprocessing; construct a neural network model, use the preprocessed simulation enclosed scene data for training to obtain a trained neural network model; input the three-dimensional point cloud data into the trained neural network model to obtain an unknown simulation scene.

[0069] The preprocessing process specifically includes: performing binary processing and progressive erosion on the simulation enclosed scene data to generate a single-pixel frame with size invariance and rotational invariance, and extracting environmental features based on the single-pixel frame.

[0070] The content of progressive erosion specifically includes:

[0071]

[0072] Among them, A is the initial three-dimensional point cloud set, B is the spherical structure element, z is the three-dimensional translation vector or position, and B z is the position of the spherical structure element B after translation by z.

[0073] The processing process of rotational invariance specifically includes:

[0074]

[0075] Among them, V nm (r,θ) is the Zernike polynomial; R nm (r) is the radial basis function, f(r,θ) is the input function in the polar coordinate system, defined within the unit circle, r is the radial distance, n is the radial order, m is the azimuthal order, is the complex conjugate of the Zernike polynomial.

[0076] The content of single-pixel frame generation specifically includes:

[0077]

[0078] Among them, B is the spherical structure element, is the hit-or-miss transform.

[0079] Construct a high-fidelity virtual environment through procedural generation, real-data-based generation, and physics-based simulation methods, enabling the UAV to be trained and tested in the virtual environment to improve its adaptability and inspection efficiency in the real environment.

[0080] Procedural generation------Perlin noise generation terrain formula:

[0081]

[0082] Among them, grad(i,j): the random gradient vector at the grid vertex (i,j); The vector from the point (x,y) to the grid vertex (i,j); fade(t): the easing function (such as 6t 5 -15t 4 +10t 3 ), smooth interpolation.

[0083] Vegetation can also be generated using procedural generation methods:

[0084] Axiom:F

[0085] Rule:F→F[+F][-F]

[0086] Among them, F is the initial symbol replaced recursively; each iteration adds branches ([+F] is the right branch, [-F] is the left branch).

[0087] Step S4, model the enclosed scene based on the 3D skeleton features and the unknown simulation scene, and generate the UAV inspection flight path based on the enclosed space model.

[0088] In this embodiment, a diverse range of complex scenes are quickly created through a procedural generation algorithm, covering typical unknown environmental features such as enclosed spaces, dynamic obstacles, and light and shadow changes; the generation based on real data reconstructs the virtual scene using real sensor data, enhancing the authenticity of the environmental simulation; the physics-based simulation technology dynamically simulates physical behaviors such as collisions and light interactions through the engine, enabling the UAV to perform immersive training in the virtual environment.

[0089] Real data generation------ICP point cloud registration formula (minimizing the registration error):

[0090]

[0091] Among them, R, t: rotation matrix and translation vector; P i , q j : corresponding points in the source point cloud and the target point cloud;

[0092] Use deep learning algorithms to extract high-precision image features and identify obstacles, analyze the environmental topological structure in combination with graph convolutional networks, use the multi-way tree traversal algorithm to plan the exploration path, and adjust the path according to the dynamically constructed prior map to achieve efficient and non-omissive inspection tasks.

[0093] The convolutional neural network extracts high-precision image features and identifies obstacles. The graph convolutional network analyzes the environmental topological structure. The multi-way tree traversal algorithm ensures systematic exploration of unknown areas and avoids missing key inspection points. The drone adjusts its path according to the dynamically constructed prior map and uses the fast inference ability of the deep learning model to cope with environmental mutations, such as dynamic obstacles or changes in lighting conditions.

[0094] Real data generation ------ Structured light three-dimensional construction formula (triangulation principle):

[0095]

[0096] Parameter description: B stereo : Baseline distance between the projector and the camera; f: Camera focal length; d: Disparity value of the encoded pattern.

[0097] Convolution operation (basis of feature extraction): Calculation of the convolutional layer in the convolutional neural network (CNN):

[0098]

[0099] Among them, x: Input image or feature map; w: Convolution kernel weight; b: Bias term; y: Value of the output feature map.

[0100] Activation function (introducing non-linearity) ReLU function:

[0101] f(x) = max(0, x)

[0102] Used to enhance the non-linear expression ability of the model and suppress negative value features.

[0103] Pooling operation (dimensionality reduction and translational invariance) Max pooling:

[0104]

[0105] Reduce the size of the feature map while retaining significant features.

[0106] Loss function (obstacle recognition optimization) Cross-entropy loss (classification task):

[0107]

[0108] Among them, y c : One-hot encoding of the true label. p c ]>: Class probability predicted by the model (generated by Softmax).

[0109] Object detection algorithm (obstacle recognition) YOLO loss function combined with classification and localization:

[0110] L = λ coord ∑Lloc +λ obj ∑L obj +λ noobj ∑L noobj +∑L cls

[0111] λ coord 、λ obj etc. are weight coefficients to balance the losses of different tasks.

[0112] Convolutional neural network: The multi - fork tree traversal algorithm plans to explore the path: Path search objective function (such as A* algorithm):

[0113] f(n) = g(n) + h(n)

[0114] g(n): The actual path cost from the starting point to node n; h(n): The remaining cost from node n to the target estimated by the heuristic function (such as Euclidean distance); Path optimization (dynamic programming): If it is necessary to minimize the total path length or risk, the objective function can be defined:

[0115]

[0116] Among them, xi: Path points; Pcollision: Collision probability; λ: Weight parameter.

[0117] Graph convolutional neural network: Graph convolutional operation (topological structure analysis), graph convolution of the normalized adjacency matrix:

[0118]

[0119] Among them, The adjacency matrix with self - loops (A 原 is the original adjacency matrix, and I is the identity matrix). Degree matrix H (l) : The node feature matrix of the ll - th layer. W (l) : Trainable weight matrix. σ: Activation function (such as ReLU). (2) Environmental topology modeling, graph representation:

[0120] G=(V, E, X)

[0121] Among them, V: Node set (such as regions in the environment). E: Edge set (indicating the connectivity between nodes). X: Node feature matrix (such as position, sensor data).

[0122] Information aggregation: Each node updates its own representation by aggregating neighbor features:

[0123]

[0124] Among them, N(v): the neighbor set of node v. AGGREGATE: aggregation function (such as mean, maximum).

[0125] Dynamically construct a prior map to adjust the path, dynamic Bayesian update (map probability modeling):

[0126] p(m t |z 1:t ,u 1:t ) = η·p(z t |m t )·∫p(m t |m t-1 ,u t )p(m t-1 )dm t-1

[0127] Among them, m t : the map state at time t. z 1:t : the observation data sequence. u 1:t : the control input sequence. η: normalization constant.

[0128] Key steps and formulas for skeleton feature extraction: Topological preservation and Euler characteristic invariance:

[0129] A. Euler's formula:

[0130] χ(s) = O(s) + H(s) + C(s)

[0131] Among them: O(S) is the number of connected objects; H(S) is the number of holes; C(S) is the number of cavities.

[0132] B. Local Euler characteristic calculation: Calculate through the number of vertices (v), edges (e), faces (f), and octants (oct):

[0133] G6(s) = v - e + f - oct

[0134] Quickly calculate the change of local Euler characteristics through the configuration table of octants (3×3×3 neighborhood).

[0135] 2. Simple determination.

[0136] Definition: Deleting this point does not change the topology (number of connected components, holes, cavities).

[0137] Condition: The change of Euler characteristic is zero:

[0138] δG(S∩N(v)) = 0

[0139] The number of connected objects remains unchanged:

[0140] δO(S∩N(v)) = 0

[0141] The number of holes remains unchanged:

[0142] δH(S∩N(v)) = 0

[0143] 3. The steps of the parallel thinning algorithm are as follows: Iteratively delete boundary points in different directions (U, B, N, S, W, E). Use an octree data structure to check local connectivity. Maintain geometric conditions: Medial Surface: Retain surface points that satisfy specific octant configurations. Medial Axis: Retain arc endpoints and curve endpoints.

[0144] 4. Definition of surface points: Each octant of point v satisfies one of the following: Binary configurations are 240, 165, 170, 204 (corresponding to thin surface structures); The number of points within the octant is less than 3:

[0145]

[0146] 5. Preprocessing and postprocessing:

[0147] Preprocessing: Smooth noise using a 3D digital filter (such as maximum / minimum operations):

[0148] S max (Z 3 ) = {v|v ∈ S or v ∈ N(v′) for some v′ ∈ S}

[0149]

[0150] Postprocessing: Distance mapping: Calculate the Euclidean distance from the skeleton points to the boundary. Classification and thresholding: Remove short branches (length threshold ll).

[0151] Skeleton modeling: Construct a point table and an edge table to describe topological and geometric features.

[0152] Step S5: Construct a complete prior map based on the drone inspection flight path, and complete the drone inspection based on the prior map.

[0153] Example 2

[0154] A point cloud acquisition module, which is used to mount dual lidars on a drone and use the dual lidars to acquire three-dimensional point cloud data of a confined space.

[0155] A skeleton feature extraction module, which is used to extract features from the three-dimensional point cloud data using a skeleton feature extraction method to obtain 3D skeleton features.

[0156] Precisely track and record environmental boundary points through fire simulation and maximum disk operation, and combine morphological operations to binarize and gradually erode the unknown environmental image, generating a single-pixel framework with size and rotation invariance to extract key structural features in the environment.

[0157] The process of fire simulation includes:

[0158]

[0159] Among them, φ(x, y, t) is the level set function, F is the evolution speed function, κ is the curvature, and α is the curvature weight coefficient.

[0160] The process of the maximum disk operation specifically includes:

[0161]

[0162] Among them, D(p) is the distance from point p to the boundary, and A0 is the target area.

[0163] In this embodiment, the maximum disk operation identifies the largest inscribed circle area in the environment, providing basic data support for path planning.

[0164] The process of image binarization specifically includes:

[0165]

[0166] Among them, T is the adaptive threshold.

[0167] The content of gradual erosion specifically includes:

[0168]

[0169] Among them, A is the initial three-dimensional point cloud set, B is the spherical structuring element, z is the three-dimensional translation vector or position, and B z is the position of the structuring element B after translation by z.

[0170] This embodiment adopts the Hu invariant matrix:

[0171] I1 = η 20 +η 02

[0172]

[0173] Among them, η pq is the normalized central moment, satisfying translation, rotation, and scale invariance.

[0174] This embodiment improves the above Hu into a three-dimensional improved invariant moment:

[0175] I1 = Ψ200 +Ψ 020 +Ψ 002

[0176]

[0177] Among them, Ψ pqr is the three-dimensional normalized central moment, which satisfies spatial translation, rotation, and scaling invariance and is used to characterize the essential features of the three-dimensional geometric structure.

[0178] The scene simulation module is used to construct an unknown simulation scene for the three-dimensional point cloud data using scene generation technology.

[0179] According to the existing images of the enclosed space, use scene generation technology to generate simulation enclosed scene data and then perform preprocessing; construct a neural network model and use the preprocessed simulation enclosed scene data for training to obtain a trained neural network model; input the three-dimensional point cloud data into the trained neural network model to obtain an unknown simulation scene.

[0180] The preprocessing process specifically includes: performing binarization processing and gradual erosion on the simulation enclosed scene data to generate a single-pixel frame with size invariance and rotation invariance, and extracting environmental features based on the single-pixel frame.

[0181] The specific content of the gradual erosion includes:

[0182]

[0183] Among them, A is the initial three-dimensional point cloud set, B is the spherical structure element, z is the three-dimensional translation vector or position, and B z is the position of the spherical structure element B after translation by z.

[0184] The processing process of rotation invariance specifically includes:

[0185]

[0186] Among them, V nm (r,θ) is the Zernike polynomial; R nm (r) is the radial basis function, f(r,θ) is the input function in the polar coordinate system, defined within the unit circle, r is the radial distance, n is the radial order, m is the azimuthal order, and V n * m (r,θ) is the complex conjugate of the Zernike polynomial.

[0187] The specific content of the single-pixel frame generation includes:

[0188]

[0189] Among them, B is a spherical structural element, is the hit-or-miss transform.

[0190] Construct a high-fidelity virtual environment through procedural generation, real-data-based generation, and physics-based simulation methods, enabling the UAV to train and test in the virtual environment and improving its adaptability and inspection efficiency in the real environment.

[0191] Procedural generation ------ Perlin noise generation terrain formula:

[0192]

[0193] Among them, grad(i,j): the random gradient vector at the grid vertex (i,j); the vector from the point (x,y) to the grid vertex (i,j); fade(t): the easing function (such as 6t 5 - 15t 4 + 10t 3 ), smooth interpolation.

[0194] Vegetation can also be generated using the procedural generation method:

[0195] Axiom: F

[0196] Rule: F → F[+F][-F]

[0197] Among them, F is the initial symbol replaced recursively; a branch is added in each iteration ([+F] is the right branch, [-F] is the left branch).

[0198] The inspection flight path generation module is used to model the enclosed scene based on the 3D skeleton features and the unknown simulation scene, and generate the UAV inspection flight path based on the enclosed space model.

[0199] In this embodiment, a diverse and complex scene is quickly created through a procedural generation algorithm, covering typical unknown environmental features such as enclosed spaces, dynamic obstacles, and light and shadow changes; real-data-based generation reconstructs the virtual scene using real sensor data, enhancing the authenticity of the environmental simulation; physics-based simulation technology dynamically simulates physical behaviors such as collisions and light interactions through the engine, enabling the UAV to perform immersive training in the virtual environment.

[0200] Real-data generation ------ ICP point cloud registration formula (minimizing the registration error):

[0201]

[0202] Among them, R, t: rotation matrix and translation vector; P i [[ID=�2]]q j: Corresponding points in the source electric cloud and the target electric cloud;

[0203] Use deep learning algorithms to extract image features with high precision and identify obstacles, analyze the environmental topological structure using graph convolutional networks, plan exploration paths using multi-way tree traversal algorithms, and adjust the paths according to the dynamically constructed prior map to achieve efficient and non-omissive inspection tasks.

[0204] Convolutional neural networks extract image features with high precision and identify obstacles, graph convolutional networks analyze the environmental topological structure, multi-way tree traversal algorithms ensure systematic exploration of unknown areas and avoid missing key inspection points. The drone adjusts its path according to the dynamically constructed prior map and uses the fast inference ability of the deep learning model to cope with environmental mutations, such as dynamic obstacles or changes in lighting conditions.

[0205] Real data generation ------ Structured light three-dimensional construction formula (triangulation principle):

[0206]

[0207] Parameter description: B stereo : Baseline distance between the projector and the camera; f: Camera focal length; d: Parallax value of the encoded pattern.

[0208] Convolution operation (basis of feature extraction): Calculation of the convolutional layer in a convolutional neural network (CNN):

[0209]

[0210] Among them, x: Input image or feature map; w: Convolution kernel weight; b: Bias term; y: Value of the output feature map.

[0211] Activation function (introducing non-linearity) ReLU function:

[0212] f(x) = max(0, x)

[0213] Used to enhance the non-linear expression ability of the model and suppress negative value features.

[0214] Pooling operation (dimensionality reduction and translational invariance) Max pooling:

[0215]

[0216] Reduce the size of the feature map while retaining significant features.

[0217] Loss function (obstacle recognition optimization) Cross-entropy loss (classification task):

[0218]

[0219] Among them, yc : One-hot encoding of the true label. p c : Class probabilities predicted by the model (generated by Softmax).

[0220] Object detection algorithm (obstacle recognition) YOLO loss function combined with classification and localization:

[0221]

[0222] λ coord and λ obj etc. are weight coefficients to balance the losses of different tasks.

[0223] Convolutional neural network: Multi-way tree traversal algorithm to plan the exploration path: Path search objective function (such as A* algorithm):

[0224] f(n) = g(n) + h(n)

[0225] g(n): The actual path cost from the starting point to node n; h(n): The estimated remaining cost from n to the target by the heuristic function (such as Euclidean distance); Path optimization (dynamic programming): If it is necessary to minimize the total path length or risk, the objective function can be defined as:

[0226]

[0227] Among them, xi: Path points; Pcollision: Collision probability; λ: Weight parameter.

[0228] Graph convolutional neural network: Graph convolutional operation (topological structure analysis), graph convolution of the normalized adjacency matrix:

[0229]

[0230] Among them, Adjacency matrix with self-loops (A 原 is the original adjacency matrix, and I is the identity matrix). Degree matrix H (l) : Node feature matrix of the ll-th layer. W (l) : Trainable weight matrix. σ: Activation function (such as ReLU). (2) Environmental topology modeling, representation of the graph:

[0231] G = (V, E, X)

[0232] Among them, V: Set of nodes (such as regions in the environment). E: Set of edges (representing the connectivity between nodes). X: Node feature matrix (such as position, sensor data).

[0233] Information aggregation: Each node updates its own representation by aggregating neighbor features:

[0234]

[0235] Among them, N(v): the neighbor set of node v. AGGREGATE: aggregation function (such as mean, maximum).

[0236] Dynamically construct a prior map to adjust the path, dynamic Bayesian update (map probability modeling):

[0237] p(m t |z 1:t ,u 1:t ) = η · p(z t |m t ) · ∫p(m t |m t-1 ,u t )p(m t-1 )dm t-1

[0238] Among them, m t : the map state at time t. z 1:t : the observed data sequence. u 1:t : the control input sequence. η: normalization constant.

[0239] Key steps and formulas for skeleton feature extraction: Topological preservation and Euler characteristic invariance:

[0240] A. Euler's formula:

[0241] χ(s) = O(s) + H(s) + C(s)

[0242] Among them: O(S) is the number of connected objects; H(S) is the number of holes; C(S) is the number of cavities.

[0243] B. Local Euler characteristic calculation: Calculate through the number of vertices (v), edges (e), faces (f), and octants (oct):

[0244] G6(s) = v - e + f - oct

[0245] Quickly calculate the change of local Euler characteristics through the configuration table of octants (3×3×3 neighborhood).

[0246] 2. Simple point determination.

[0247] Definition: Deleting this point does not change the topology (number of connected components, holes, cavities).

[0248] Condition: The change in Euler characteristic is zero:

[0249] δG(S ∩ N(v)) = 0

[0250] The number of connected objects remains unchanged:

[0251] δO(S∩N(v)) = 0

[0252] The number of holes remains unchanged:

[0253] δH(S∩N(v)) = 0

[0254] 3. The steps of the parallel thinning algorithm are as follows: Iteratively delete boundary points in different directions (U, B, N, S, W, E). Use an octree data structure to check local connectivity. Maintain geometric conditions: Medial Surface: Retain surface points that satisfy a specific octant configuration. Medial Axis: Retain arc endpoints and curve endpoints.

[0255] 4. Definition of surface points: Each octant of point v satisfies one of the following: Binary configurations are 240, 165, 170, 204 (corresponding to thin surface structures); The number of points within the octant is less than 3:

[0256]

[0257] 5. Preprocessing and postprocessing:

[0258] Preprocessing: Smooth noise using a 3D digital filter (such as maximum / minimum operations):

[0259] S max (Z 3 ) = {v|v ∈ S or v ∈ N(v′) for some v′ ∈ S}

[0260]

[0261] Postprocessing: Distance mapping: Calculate the Euclidean distance from the skeleton points to the boundary. Classification and thresholding: Remove short branches (length threshold ll).

[0262] Skeleton modeling: Construct point tables and edge tables to describe topological and geometric features.

[0263] The UAV inspection module is used to construct a complete prior map based on the UAV inspection track and complete UAV inspection based on the prior map.

[0264] The above-described embodiments are only descriptions of the preferred embodiments of the present invention and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. An unmanned aerial vehicle inspection method combining skeleton feature extraction and scene generation technology, characterized in that, The method includes: Step S1: Mount dual lidars on a drone and use the dual lidars to obtain three-dimensional point cloud data of an enclosed space; Step S2: Use a skeleton feature extraction method to extract features from the three-dimensional point cloud data to obtain 3D skeleton features; Step S3: Use a scene generation technique to construct an unknown simulation scene from the three-dimensional point cloud data; Step S4: Model the enclosed scene based on the 3D skeleton features and the unknown simulation scene, and generate a drone inspection flight path based on the enclosed space model; Step S5: Construct a complete prior map based on the drone inspection flight path and complete drone inspection based on the prior map.

2. The UAV inspection method combining skeleton feature extraction and scene generation technology according to claim 1, characterized in that In step S2, the content of obtaining 3D skeleton features specifically includes: using fire simulation and maximum disk operation to extract 3D skeleton features from the environmental boundary points of the three-dimensional point cloud data.

3. The UAV inspection method combining skeleton feature extraction and scene generation technology according to claim 2, characterized in that The process of the fire simulation includes: wherein, is a level set function, F is an evolution speed function, κ is curvature, α is a curvature weight coefficient, is the magnitude of the gradient of the level set function.

4. The UAV inspection method combining skeleton feature extraction and scene generation technology according to claim 3, characterized in that The process of the maximum disk operation specifically includes: where D(p) is the distance from point p to the boundary, A is the target area, S is the set of points that meet the conditions, that is, it represents the center positions of all the maximum inscribed circles in area A, and D(q) is the radius of the inscribed circle corresponding to the points in area A.

5. The UAV inspection method combining skeleton feature extraction and scene generation technology according to claim 1, characterized in that, In step S3, the process of using a scene generation technique to construct an unknown simulation scene from the three-dimensional point cloud data specifically includes: According to the existing images of the enclosed space, use a scene generation technique to generate simulation enclosed scene data and then perform preprocessing; Construct a neural network model, use the preprocessed simulation enclosed scene data for training to obtain a trained neural network model; Input the three-dimensional point cloud data into the trained neural network model to obtain an unknown simulation scene.

6. The UAV inspection method combining skeleton feature extraction and scene generation technology according to claim 5, characterized in that, The preprocessing process specifically includes: performing binary processing and gradual erosion on the simulation enclosed scene data to generate a single-pixel framework with size invariance and rotation invariance, and extracting environmental features based on the single-pixel framework.

7. The UAV inspection method combining skeleton feature extraction and scene generation technology according to claim 6, characterized in that The content of the gradual erosion specifically includes: Among them, A is the set of original images, B is the structuring element, z is the translation vector or position, and B z is the position of the structuring element B after translation by z.

8. The UAV inspection method combining skeleton feature extraction and scene generation technology according to claim 7, characterized in that, The processing process of the rotation invariance specifically includes: where V nm (r, θ) is a Zernike polynomial; f(r, θ) is an input function in the polar coordinate system, defined within the unit circle, r is the radial distance, n is the radial order, m is the azimuthal order, is the complex conjugate of the Zernike polynomial.

9. A drone inspection system combining skeleton feature extraction and scene generation techniques for implementing the method according to any one of claims 1-8, the system includes: A point cloud acquisition module for mounting dual lidars on a drone and using the dual lidars to obtain three-dimensional point cloud data of an enclosed space; A skeleton feature extraction module for using a skeleton feature extraction method to extract features from the three-dimensional point cloud data to obtain 3D skeleton features; A scene simulation module for using a scene generation technique to construct an unknown simulation scene from the three-dimensional point cloud data; An inspection flight path generation module for modeling the enclosed scene based on the 3D skeleton features and the unknown simulation scene, and generating a drone inspection flight path based on the enclosed space model; A drone inspection module for constructing a complete prior map based on the drone inspection flight path and completing drone inspection based on the prior map.

Images