Three-dimensional modeling method and system for a substation

Abstract

The application discloses a substation three-dimensional modeling method, comprising the following steps: acquiring image data information of a target substation by using a UAV; extracting and matching feature points from the image data information; performing aerial triangulation to obtain three-dimensional data information of the feature points; performing multi-view image dense matching to obtain global dense point cloud data of the target substation, and obtaining a global rough model based on oblique photography; constructing a point cloud classification model and extracting local features of equipment; performing fine point cloud completion of the equipment based on a high-precision three-dimensional model of a neural radiation field; constructing an irregular triangular mesh to obtain a fine model of equipment of the target substation; and fusing the obtained models to complete three-dimensional modeling of the target substation. The application further discloses a system for implementing the substation three-dimensional modeling method. The application not only realizes fine three-dimensional modeling of the target substation, but also has higher reliability and better accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of electrical automation, and particularly relates to a substation three-dimensional modeling method and system. BACKGROUND

[0002] With the development of economy and technology and the improvement of people's living standards, electric energy has become an essential secondary energy in people's production and life, bringing endless convenience to people's production and life. Therefore, ensuring the stable and reliable supply of electric energy has become one of the most important tasks of the power system.

[0003] The substation is an important part of the power system; therefore, the inspection of the substation is of great significance to the power system. Previously, the substation inspection adopted a manual inspection mode, which was not only time-consuming and laborious, but also inefficient and had poor reliability. With the rapid development of unmanned aerial vehicle technology, at present, the inspection of the substation of the power system is basically carried out in the mode of unmanned aerial vehicle inspection.

[0004] The premise of using an unmanned aerial vehicle to inspect the substation is to build a high-precision three-dimensional model of the substation. At present, the commonly used substation three-dimensional modeling scheme mainly includes laser radar-based three-dimensional modeling and computer vision-based three-dimensional modeling. However, the laser radar-based three-dimensional modeling scheme alone has the problem of time-consuming and labor-consuming collection work, and the laser radar collection device is expensive, which is not suitable for scene data collection and reconstruction in a large area. The computer vision-based three-dimensional modeling scheme alone can basically only restore the geometric structure and appearance of the object to achieve high-precision three-dimensional modeling of a large-scale scene of the substation. SUMMARY

[0005] One of the purposes of the present application is to provide a substation three-dimensional modeling method with high reliability and good accuracy.

[0006] The second purpose of the present application is to provide a system for implementing the substation three-dimensional modeling method.

[0007] The substation three-dimensional modeling method provided by the present application comprises the following steps:

[0008] S1. Using an unmanned aerial vehicle to obtain image data information of a target substation;

[0009] S2. Extracting feature points and matching feature points from the image data information obtained in step S1;

[0010] S3. Performing aerial triangulation based on the feature points obtained in step S2 to obtain three-dimensional data information of the feature points;

[0011] S4. Multi-view image dense matching is performed on the data information obtained in step S3 to obtain global dense point cloud data of the target substation, and a global coarse model based on oblique photography is obtained;

[0012] S5. A point cloud classification model is constructed based on a Transformer, and local features of the equipment are extracted;

[0013] S6. Based on the local features obtained in step S5 and a high-precision three-dimensional model of a neural radiation field, fine point cloud completion of the equipment is performed;

[0014] S7. Based on the data information obtained in step S6, an irregular triangular mesh is constructed to obtain a fine model of the equipment in the target substation;

[0015] S8. The models obtained in steps S4 and S7 are fused to complete three-dimensional modeling of the target substation.

[0016] The feature point extraction and feature point matching on the image data information obtained in step S1 in step S2 include the following steps:

[0017] Based on a Gaussian function, a multi-scale Gaussian pyramid is constructed, and extreme points in a scale space are extracted;

[0018] Based on the constructed multi-scale Gaussian pyramid, a Gaussian difference pyramid is constructed, the obtained extreme points are filtered, and feature point extraction is realized;

[0019] Based on a convolutional neural network, feature point matching is performed.

[0020] The step S2 specifically includes the following steps:

[0021] A Gaussian convolution is used to construct a scale space L(x, y, sigma), which is represented as

[0022] L(x, y, sigma) = G(x, y, sigma) * I(x, y)

[0023] In the formula, G(x, y, sigma) is a Gaussian kernel function with (x, y) as the center; sigma is a scale parameter; * is a convolution operation; I(x, y) is a pixel value of an image at (x, y);

[0024] The difference between adjacent scale spaces is obtained to obtain a Gaussian difference pyramid D(x, y, sigma), which is represented as D(x, y, sigma) = L(x, y, ksigma) - L(x, y, sigma); wherein k is a multiple coefficient of adjacent scales;

[0025] The gradient direction features of the neighborhood pixels are used to obtain the direction parameters of the feature points, which are represented as

[0026]

[0027] Wherein m(x, y) is a gradient modulus value; L(x, y) is an image sample; and θ(x, y) is a phase angle.

[0028] Based on the convolutional neural network, the feature point matching is performed: the similarity learning module in the convolutional network is used to match the feature descriptors based on the cosine similarity or the Euclidean distance; the extracted feature description is used as a high-dimensional vector, and the output features of the neural network are used for matching calculation.

[0029] The step S4 is to perform multi-view image dense matching on the data information obtained in the step S3 to obtain the global dense point cloud data of the target transformer substation, and obtain the global rough model based on oblique photography, including the following steps:

[0030] The exterior orientation elements of the image and the ground encrypted point coordinates are calculated, and the encrypted point coordinates are used as the bundle block adjustment probability value;

[0031] The bundle formed by the images is used as a basic unit to construct the collinearity equation, the error equation of the undetermined encrypted point and the known control point of each image is constructed, and the exterior orientation elements in the image are calculated;

[0032] The actual ground coordinates of the undetermined point and the encrypted point are calculated through the space forward intersection formula of a plurality of images;

[0033] The depth information of each pixel point is calculated through the stereo matching of the sparse point cloud, the pixel point containing the depth information is mapped to the three-dimensional space, the obtained point cloud data is fused, the dense point cloud of the transformer substation is obtained, and finally the global rough model based on oblique photography is obtained.

[0034] The step S4 specifically includes the following steps:

[0035] The exterior orientation elements of the image and the ground encrypted point coordinates are calculated, and the encrypted point coordinates are used as the bundle block adjustment probability value;

[0036] The collinearity equation of the bundle block adjustment is represented as

[0037]

[0038] Wherein (x, y) is the coordinate of the image point on the image plane; f is the focal length of the camera; a1 is the first exterior orientation angle element The first direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; a2 is the first exterior orientation angle element The second direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; a3 is the first exterior orientation angle element The third direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; b1 is the first exterior orientation angle element a fourth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; b2 is the first exterior orientation angle element a fifth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; b3 is the first exterior orientation angle element a sixth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; c1 is the first exterior orientation angle element a seventh direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; c2 is the first exterior orientation angle element an eighth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; c3 is the first exterior orientation angle element a ninth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; (X, Y, Z) is the spatial coordinates of the object point; (X s ,Y s ,Z s ) is the spatial coordinates of the camera projection center;

[0039] The collinear equation of the bundle block adjustment is processed, and the error equation is obtained as

[0040] In the formula, (V x ,V y ) is the observation error in the x and y directions; is formed by the derivative (Jacobian matrix) of the collinear equation; (I x ,I y ) is the difference vector of the observation quantity, (x 0 ,y 0 ) is the image principal point coordinate, and I x =x-x 0 , I y =y-y 0 ;

[0041] The bundle composed of the image is taken as a basic unit to construct the collinear equation, the error equation of the undetermined encrypted point and the known control point of each image is constructed, and the exterior orientation element in the image is calculated;

[0042] The actual ground coordinates of the undetermined point and the encrypted point are calculated through the space forward intersection formula of a plurality of images;

[0043] The depth information of each pixel point is calculated through stereo matching of the sparse point cloud, and the pixel point containing the depth information is mapped to the three-dimensional space. The obtained point cloud data is fused to obtain the dense point cloud of the substation, and finally the global rough model based on oblique photography is obtained:

[0044] Aerial survey photos are selected from the photo set, and several photos are selected as neighboring photos for each photo. The results of sparse point clouds are used as the standard for judging whether there is a high degree of similarity, thereby ensuring the accuracy of dense 3D reconstruction.

[0045] Depth estimation is performed on the initial sparse point cloud feature points, and the sparse point cloud feature points are used as seed points;

[0046] The Patch Match algorithm is used for depth map estimation, based on the coordinates of pixel q (q x ,q y Describes parallax, denoted as

[0047] d = a q q x +b q q y +c q

[0048] In the formula, d represents parallax; a q b is the first constant coefficient in the pixel coordinate relationship. q c is the second constant coefficient in the pixel coordinate relationship. q The third constant coefficient in the pixel coordinate relationship is usually a q b q and c q Learned or estimated through optimization processes (such as Patch Match);

[0049] In each iteration, for each pixel (q) x ,q y ), calculate the matching cost of the pixel based on the current disparity d, and determine whether the disparity needs to be updated based on the magnitude of the cost: set the current disparity depth value to d. cur (q x ,q y ), in the matching cost function C(q) x ,q y The new depth d under ,d) new The update rule for (x,y) is:

[0050]

[0051] The depth information of the scene is obtained through continuous iteration; the mathematical model for optimizing the depth information is expressed as follows:

[0052]

[0053] In the formula, E is the optimized depth value; I R (i,j) represents the reference image depth value at position (i,j); c k I represents the depth scale factor for the k-th viewpoint;k (i,j) is a depth value of the kth view at position (i,j);

[0054] The depth maps are fused to obtain a dense point cloud of the substation, and finally a global coarse model based on oblique photography is obtained.

[0055] The local features obtained according to step S5 are used to complete the fine point cloud completion of the equipment based on the high-precision three-dimensional model of the neural radiance field, and specifically include the following steps:

[0056] The local features obtained in step S5 are represented as a point set;

[0057] The neural radiance field model is constructed to learn the local and overall information of the point cloud, so as to complete the fine point cloud completion of the equipment.

[0058] The neural radiance field model is constructed to learn the local and overall information of the point cloud, so as to complete the fine point cloud completion of the equipment, and specifically includes the following steps:

[0059] The neural radiance field model expresses the scene as a hidden function using a fully connected multi-layer perceptron, learns the color and density values of each pixel point in the 3D space, and uses ray tracing and volume rendering to complete the fine point cloud completion of the equipment;

[0060] The camera pose is estimated by COLMAP, and the pixel color value C(r) rendered along the ray direction is predicted using the ray r(t) = o + td emitted by the camera, wherein o is the optical center of the camera, d is the viewing direction, t is a set of samples from far to near on the ray, and is represented as

[0061]

[0062]

[0063] In the formula, t n is the minimum value; t f is the maximum value; T(t) is the cumulative transmittance from t1 to t; σ(r(t)) is the volume density, which is used to represent the probability of light termination at the point; c(r(t), d) is the color value of the sampling point on the ray; and s is the sampling point.

[0064] The model obtained by fusing step S4 and step S7 is completed to complete the three-dimensional modeling of the target substation, and specifically includes the following steps: first, feature point matching is used, and then the global coarse model and the fine model are aligned to the same coordinate system based on the homogeneous coordinate transformation of the calibration points.

[0065] The application further provides a system for realizing the substation three-dimensional modeling method, comprising a data acquisition module, a feature matching module, a triangulation module, a rough modeling module, a feature extraction module, a point cloud completion module, a fine modeling module and a three-dimensional modeling module; the data acquisition module, the feature matching module, the triangulation module, the rough modeling module, the feature extraction module, the point cloud completion module, the fine modeling module and the three-dimensional modeling module are connected in sequence; the data acquisition module is used for acquiring image data information of a target substation by using a UAV, and uploading the data information to the feature matching module; the feature matching module is used for performing feature point extraction and feature point matching on the acquired image data information according to the received data information, and uploading the data information to the triangulation module; the triangulation module is used for performing aerial triangulation according to the received data information and the obtained feature points, obtaining three-dimensional data information of the feature points, and uploading the data information to the rough modeling module; the rough modeling module is used for performing multi-view image dense matching on the obtained data information according to the received data information, obtaining global dense point cloud data of the target substation, obtaining a global rough model based on oblique photography, and uploading the data information to the feature extraction module; the feature extraction module is used for constructing a point cloud classification model based on a Transformer according to the received data information, extracting local features of equipment, and uploading the data information to the point cloud completion module; the point cloud completion module is used for performing fine point cloud completion of the equipment based on a high-precision three-dimensional model of a neural radiation field according to the received data information and the obtained local features, and uploading the data information to the fine modeling module; the fine modeling module is used for constructing an irregular triangular mesh according to the received data information and the obtained data information, obtaining a fine model of equipment of the target substation, and uploading the data information to the three-dimensional modeling module; and the three-dimensional modeling module is used for fusing the obtained model according to the received data information, and completing three-dimensional modeling of the target substation.

[0066] The substation three-dimensional modeling method and system provided by the application realize fine three-dimensional modeling of a target substation, have higher reliability and better accuracy, through acquisition, processing, feature extraction and matching of multi-dimensional data of the target substation, and point cloud completion and rough and fine modeling. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 It is a method flowchart of the method of the application.

[0068] Figure 2 It is a functional module schematic diagram of the system of the application. DETAILED DESCRIPTION

[0069] As Figure 1 It is a method flowchart of the method of the application: the substation three-dimensional modeling method disclosed by the application comprises the following steps:

[0070] S1. Adopting a UAV, obtaining image data information of a target substation;

[0071] S2. Feature point extraction and feature point matching are performed on the image data information obtained in step S1; including the following steps:

[0072] Based on the Gaussian function, a multi-scale Gaussian pyramid is constructed, and the extreme points of the scale space are extracted;

[0073] Based on the constructed multi-scale Gaussian pyramid, a Gaussian difference pyramid is constructed, and the obtained extreme points are filtered to realize the extraction of feature points;

[0074] Based on the convolutional neural network, feature point matching is performed;

[0075] In specific implementation, the following steps are included:

[0076] Gaussian convolution is used to construct a scale space L(x, y, σ), which is represented as

[0077] L(x, y, σ) = G(x, y, σ) * I(x, y)

[0078] In the formula, G(x, y, σ) is a Gaussian kernel function with (x, y) as the center; σ is a scale parameter; * is a convolution operation; I(x, y) is the pixel value of the image at (x, y);

[0079] The adjacent scale spaces are subtracted to obtain a Gaussian difference pyramid D(x, y, σ), which is D(x, y, σ) = L(x, y, kσ) - L(x, y, σ); wherein k is a multiple coefficient of adjacent scales;

[0080] The extracted feature points only contain position and scale information, and lack direction information, so the gradient direction features of the neighborhood pixels are used to obtain the direction parameters of the feature points, which are represented as

[0081]

[0082] In the formula, m(x, y) is the gradient modulus; L(x, y) is the image sample; θ(x, y) is the phase angle;

[0083] Based on the convolutional neural network, feature point matching is performed: using the similarity learning module in the convolutional network, the feature descriptors are matched based on the cosine similarity or Euclidean distance; the extracted feature description is used as a high-dimensional vector, and the output features of the neural network are used for matching calculation;

[0084] S3. According to the feature points obtained in step S2, aerial triangulation is performed to obtain three-dimensional data information of the feature points;

[0085] S4. Multi-view image dense matching is performed on the data information obtained in step S3 to obtain global dense point cloud data of the target transformer substation, and a global rough model based on oblique photography is obtained; the method comprises the following steps:

[0086] Calculate the exterior orientation elements of the image and the ground encrypted point coordinates, and take the encrypted point coordinates as the bundle block adjustment residuals;

[0087] Construct a collinearity equation with the bundle of images as a basic unit, construct an error equation of the undetermined encrypted point and the known control point of each image, and calculate the exterior orientation elements in the image;

[0088] Calculate the actual ground coordinates of the undetermined point and the encrypted point through the space forward intersection formula of several images;

[0089] Through stereo matching, the depth information of each pixel point is calculated, the pixel points containing the depth information are mapped to the three-dimensional space, the obtained point cloud data is fused, the dense point cloud of the transformer substation is obtained, and finally the global rough model based on oblique photography is obtained;

[0090] In specific implementation, the following steps are included:

[0091] Calculate the exterior orientation elements of the image and the ground encrypted point coordinates, and take the encrypted point coordinates as the bundle block adjustment residuals;

[0092] The collinearity equation of the bundle block adjustment is represented as

[0093]

[0094] In the formula, (x, y) is the coordinate of the image point on the image plane; f is the focal length of the camera; a1 is the first exterior orientation angle element The first direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; a2 is the first exterior orientation angle element The second direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; a3 is the first exterior orientation angle element The third direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; b1 is the first exterior orientation angle element The fourth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; b2 is the first exterior orientation angle element The fifth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; b3 is the first exterior orientation angle element The sixth direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; c1 is the first exterior orientation angle element The seventh direction cosine formed by the second exterior orientation angle element ω and the third exterior orientation angle element κ; c2 is the first exterior orientation angle element the eighth direction cosine formed by the second exterior orientation element omega and the third exterior orientation element kappa; c3 is the first exterior orientation element the ninth direction cosine formed by the second exterior orientation element omega and the third exterior orientation element kappa; (X, Y, Z) is the spatial coordinates of the object point; (X s ,Y s ,Z s ) is the spatial coordinates of the camera projection center;

[0095] The error equation is obtained by processing the collineation equation of the bundle adjustment area network

[0096] (V x ,V y ) is the observation error in the x, y direction; is the derivative (Jacobian matrix) of the collineation equation; (I x ,I y ) is the difference vector of the observation, (x 0 ,y 0 ) is the image principal point coordinates, and I x =x-x 0 , I y =y-y 0 ;

[0097] The collineation equation is constructed with the bundle of the image as the basic unit, the error equation of the undetermined encrypted point and the known control point of each image is constructed, and the exterior orientation element in the image is calculated;

[0098] The actual ground coordinates of the undetermined point and the encrypted point are calculated through the space forward intersection formula of several images;

[0099] The depth information of each pixel point is calculated by stereo matching of the sparse point cloud, and the pixel point containing the depth information is mapped to the three-dimensional space. The obtained point cloud data is fused to obtain the dense point cloud of the substation, and finally the global rough model based on oblique photography is obtained:

[0100] Select aerial photographs in the photo set, select several photographs as adjacent photographs for each photograph, and use the result of the sparse point cloud as the standard for judging whether there is high similarity, so as to ensure the accuracy of the dense three-dimensional reconstruction;

[0101] Depth estimation is performed on the initial sparse point cloud feature points, and the sparse point cloud feature points are used as seed points;

[0102] The Patch Match algorithm is used for depth map estimation, and the coordinates of the pixel point q (q x ,q y) describes the disparity, denoted as

[0103] d = a q q x +b q q y +c q

[0104] where d is the disparity; a q is the first constant coefficient of the pixel coordinate relationship, b q is the second constant coefficient of the pixel coordinate relationship, and c q is the third constant coefficient of the pixel coordinate relationship. Usually, a q , b q , and c q are learned or estimated by an optimization process (such as Patch Match);

[0105] In each iteration, for each pixel (q x , q y ), the matching cost of the pixel is calculated according to the current disparity d, and whether the disparity needs to be updated is determined according to the size of the cost: set the current disparity depth value as d cur (q x , q y ), the update rule of the new depth d new (x, y) under the matching cost function C(q x , q y , d) is:

[0106]

[0107] The depth information of the scene is obtained by continuously iterating; wherein the mathematical model of the depth information optimization is represented as

[0108]

[0109] where E is the optimized depth value; I R (i, j) is the reference image depth value at position (i, j); c k is the depth scale factor of the kth view; I k (i, j) is the depth value of the kth view at position (i, j);

[0110] The depth map is fused to obtain the dense point cloud of the substation, and finally the global rough model based on oblique photography is obtained;

[0111] S5. Constructing a point cloud classification model based on the Transformer, and extracting the local features of the equipment; applying the Transformer module with significant global feature learning capability to the local neighborhood of the point cloud, applying the offset attention block to the 3D point cloud, and extracting the local features of the equipment through the conversion module in layers to realize the feature mode and the dot product attention on the input, so as to realize accurate and effective processing;

[0112] S6. Based on the high-precision three-dimensional model of the neural radiation field, the fine point cloud completion of the equipment is carried out according to the local features obtained in step S5; specifically including the following steps:

[0113] The local features obtained in step S5 are expressed as a point set;

[0114] The neural radiation field model is constructed to learn the local and overall information of the point cloud, so as to carry out the fine point cloud completion of the equipment; specifically including the following steps:

[0115] The neural radiation field model expresses the scene as a hidden function with a fully connected multi-layer perceptron, learns the color and density values of each pixel point in the 3D space, and uses ray tracing and volume rendering to carry out the fine point cloud completion of the equipment;

[0116] The camera pose is estimated by COLMAP, and the pixel color value C(r) rendered along the ray direction is predicted by the ray r(t) = o + td emitted by the camera, wherein o is the optical center of the camera, d is the viewing direction, t is a set of samples from far to near on the ray, and is expressed as

[0117]

[0118]

[0119] In the formula, t n is the minimum value; t f is the maximum value; T(t) is the cumulative transmittance from t1 to t; σ(r(t)) is the volume density, which is used to represent the probability of light termination at the point; c(r(t), d) is the color value of the sampling point on the ray; and s is the sampling point.

[0120] S7. Constructing an irregular triangular mesh to obtain the fine model of the target substation equipment according to the data information obtained in step S6;

[0121] In specific implementation, the constructed triangular network is composed of a series of adjacent triangles, and each triangle is composed of three vertices and three edges;

[0122] S8. fusing the model obtained in step S4 and step S7, completing the three-dimensional modeling of the target substation; specifically comprising the following steps: firstly using feature point matching, and then aligning the global rough model and the fine model to the same coordinate system based on the homogeneous coordinate transformation of the calibration points.

[0123] As Figure 2 As shown in the functional module schematic diagram of the system of the application: the system for implementing the substation three-dimensional modeling method disclosed by the application comprises a data acquisition module, a feature matching module, a triangulation module, a rough model modeling module, a feature extraction module, a point cloud completion module, a fine modeling module and a three-dimensional modeling module; the data acquisition module, the feature matching module, the triangulation module, the rough model modeling module, the feature extraction module, the point cloud completion module, the fine modeling module and the three-dimensional modeling module are connected in series; the data acquisition module is used to acquire image data information of a target substation by using a UAV, and upload the data information to the feature matching module; the feature matching module is used to extract and match feature points from the acquired image data information according to the received data information, and upload the data information to the triangulation module; the triangulation module is used to perform aerial triangulation according to the received data information and the obtained feature points, obtain three-dimensional data information of the feature points, and upload the data information to the rough model modeling module; the rough model modeling module is used to obtain global dense point cloud data of the target substation by performing multi-view image dense matching on the obtained data information according to the received data information, obtain a global rough model based on oblique photography, and upload the data information to the feature extraction module; the feature extraction module is used to construct a point cloud classification model based on a Transformer according to the received data information, extract local features of equipment, and upload the data information to the point cloud completion module; the point cloud completion module is used to perform fine point cloud completion of equipment based on a neural radiation field high-precision three-dimensional model according to the received data information and the obtained local features, and upload the data information to the fine modeling module; the fine modeling module is used to construct an irregular triangle mesh according to the received data information and the obtained data information, obtain a fine model of equipment of the target substation, and upload the data information to the three-dimensional modeling module; and the three-dimensional modeling module is used to fuse the obtained model according to the received data information, and complete the three-dimensional modeling of the target substation.

Claims

1. A method for three-dimensional modeling of a substation, comprising the following steps: S1. Use drones to acquire image data of the target substation; S2. Perform feature point extraction and feature point matching on the image data information obtained in step S1; including the following steps: Based on the Gaussian function, a multi-scale Gaussian pyramid is constructed to extract the extreme points in the scale space; Based on the constructed multi-scale Gaussian pyramid, a Gaussian difference pyramid is constructed, and the obtained extreme points are filtered to extract feature points. Feature point matching is performed based on convolutional neural networks; S3. Based on the feature points obtained in step S2, perform aerial triangulation to obtain the three-dimensional data information of the feature points; S4. Perform multi-view image dense matching on the data information obtained in step S3 to obtain global dense point cloud data of the target substation, and obtain a global coarse model based on oblique photography; Includes the following steps: Calculate the exterior orientation elements of the image and the coordinates of the ground densification points, and use the densification point coordinates as the approximate values ​​for bundle adjustment of the regional network; Using the beams of light composed of images as basic units, collinearity equations are constructed, and error equations are constructed for the undetermined encryption points and known control points of each image. The exterior orientation elements in the image are then calculated. The actual ground coordinates of the undetermined point and the densification point are calculated using the spatial intersection formula of several images. The depth information of each pixel is calculated by stereo matching of sparse point cloud, and the pixels containing depth information are mapped to three-dimensional space. The obtained point cloud data is fused to obtain dense point cloud of substation, and finally a global coarse model based on oblique photography is obtained. S5. Construct a point cloud classification model based on Transformer and extract local features of the device; S6. Based on the local features obtained in step S5, perform refined point cloud completion of the device based on the high-precision three-dimensional model of the neural radiation field; S7. Based on the data obtained in step S6, construct an irregular triangular mesh to obtain a detailed model of the target substation equipment; S8. Integrate the models obtained in steps S4 and S7 to complete the 3D modeling of the target substation.

2. The three-dimensional modeling method for substations according to claim 1, characterized in that... Step S2 specifically includes the following steps: Scale space is constructed using Gaussian convolution. , represented as In the formula For A Gaussian kernel function centered at the center; For scale parameters; This is a convolution operation; for The pixel values ​​of the image at that location; These are the coordinates of the image point on the image plane; By subtracting adjacent scale spaces, we obtain the Gaussian difference pyramid. for ;in It is a multiple factor of adjacent scales; The orientation parameters of feature points are obtained by utilizing the gradient direction features of neighboring pixels, denoted as: In the formula The gradient magnitude; For image samples; Phase angle; Feature point matching is performed based on convolutional neural networks: the similarity learning module in the convolutional network is used to match feature descriptors based on cosine similarity or Euclidean distance; the extracted feature descriptors are treated as a high-dimensional vector, and the output features of the neural network are used for matching calculation.

3. The three-dimensional modeling method for substations according to claim 2, characterized in that... Step S4 specifically includes the following steps: Calculate the exterior orientation elements of the image and the coordinates of the ground densification points, and use the densification point coordinates as the approximate values ​​for bundled area network adjustment: The collinearity equation for bundle adjustment of a zone network is expressed as follows: In the formula These are the coordinates of the image point on the image plane; The focal length of the camera; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The first direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The second direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The third-direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The fourth direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The fifth direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The sixth direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The seventh direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The eighth direction cosine formed; The first outer azimuth element Second outer azimuth element and the third outer azimuth element The ninth direction cosine formed; The spatial coordinates of the object point; The spatial coordinates of the camera projection center; By processing the collinearity equations in bundle adjustment, the error equation is obtained as follows: In the formula In order to be in , Observational error in direction; The derivatives of collinear equations are formed; Let the difference vector of the observations be... Let the coordinates of the principal point be [image coordinates], and , ; Using the beams of light composed of images as basic units, collinearity equations are constructed, and error equations are constructed for the undetermined encryption points and known control points of each image. The exterior orientation elements in the image are then calculated. The actual ground coordinates of the undetermined point and the densification point are calculated using the spatial intersection formula of several images. By using stereo matching, the depth information of each pixel in the sparse point cloud is calculated, and the pixels containing the depth information are mapped to three-dimensional space. The resulting point cloud data is then fused to obtain a dense point cloud of the substation, ultimately yielding a global coarse model based on oblique photogrammetry. Aerial survey photos are selected from the photo set, and several photos are selected as neighboring photos for each photo. The results of sparse point clouds are used as the standard for judging whether there is a high degree of similarity, thereby ensuring the accuracy of dense 3D reconstruction. Depth estimation is performed on the initial sparse point cloud feature points, and the sparse point cloud feature points are used as seed points; Depth map estimation is performed using the Patch Match algorithm, based on pixel points. coordinates Describe parallax, denoted as In the formula For parallax; This is the first constant coefficient in the pixel coordinate relationship. This is the second constant coefficient in the pixel coordinate relationship. The third constant coefficient in the pixel coordinate relationship is usually... , and Obtained through learning or estimation during the optimization process; In each iteration, for each pixel According to the current parallax Calculate the matching cost for this pixel, and determine whether the disparity needs to be updated based on the magnitude of the cost: Set the current disparity depth value to... In the matching cost function New Depth The update rules are as follows: The depth information of the scene is obtained through continuous iteration; the mathematical model for optimizing the depth information is expressed as follows: In the formula The optimized depth value; For position Reference image depth value at the location; Let be the depth scale factor for the k-th viewpoint; For the k-th viewpoint at position The depth value at that location; By fusing the depth maps, a dense point cloud of the substation is obtained, ultimately yielding a global coarse model based on oblique photography.

4. The three-dimensional modeling method for substations according to claim 3, characterized in that... Step S6, which involves performing refined point cloud completion of the device based on the local features obtained in step S5 and a high-precision 3D model of the neural radiation field, specifically includes the following steps: The local features obtained in step S5 are represented as a set of points; A neural radiation field model is constructed to learn local and total information of the point cloud, thereby enabling refined point cloud completion for the device.

5. The three-dimensional modeling method for substations according to claim 4, characterized in that... The construction of the neural radiation field model, learning the local and total information of the point cloud, and thus performing refined point cloud completion for the device, specifically includes the following steps: The neural radiation field model represents the scene as a latent function using a fully connected multilayer perceptron, learns the color and density values ​​of each pixel in 3D space, and uses ray tracing and volumetric rendering to perform fine point cloud completion for the device. Camera pose is estimated using COLMAP, and the rays emitted by the camera are used. Predict the pixel color value to be rendered along the ray direction ,in For the optical center of the camera, The direction of the observation perspective; Let be a set of samples on the ray from far to near, and denoted as . In the formula It is the minimum value; It is the maximum value; for arrive The cumulative transmittance over time; is the volume density, used to represent the probability that a ray terminates at a point; The color value of the sampling point on the ray; These are the sampling points.

6. The three-dimensional modeling method for substations according to claim 5, characterized in that... The model obtained by fusing steps S4 and S7 in step S8 to complete the 3D modeling of the target substation includes the following steps: First, feature point matching is used, and then the global coarse model and the fine model are aligned to the same coordinate system based on the homogeneous coordinate transformation of the calibration points.

7. A system for implementing the three-dimensional modeling method for substations according to any one of claims 1 to 6, characterized in that... It includes a data acquisition module, a feature matching module, a triangulation module, a coarse modeling module, a feature extraction module, a point cloud completion module, a fine modeling module, and a 3D modeling module; these modules are connected in series. The data acquisition module is used to acquire image data of the target substation using a drone and upload the data to the feature matching module. The feature matching module extracts and matches feature points from the acquired image data based on the received data, and uploads the data to the triangulation module. The triangulation module performs aerial triangulation based on the received data and the obtained feature points to obtain the 3D data of the feature points, and uploads the data to the coarse modeling module. The coarse modeling module performs multi-view image dense matching based on the received data to obtain global dense point cloud data of the target substation, obtains a global coarse model based on oblique photography, and uploads the data to the feature extraction module. The feature extraction module is used to construct a point cloud classification model based on Transformer based on the received data information, extract the local features of the device, and upload the data information to the point cloud completion module; the point cloud completion module is used to perform fine point cloud completion of the device based on the received data information and the obtained local features, and based on the high-precision three-dimensional model of the neural radiation field, and upload the data information to the fine modeling module. The fine modeling module is used to construct an irregular triangular mesh based on the received data information to obtain a fine model of the target substation equipment, and upload the data information to the 3D modeling module; the 3D modeling module is used to fuse the obtained model based on the received data information to complete the 3D modeling of the target substation.

Images