Three-dimensional modeling method and system for transformer substation

Abstract

The invention discloses a three-dimensional modeling method for a transformer substation. The method comprises the following steps: acquiring image data information of a target transformer substation by adopting an unmanned aerial vehicle; performing feature point extraction and feature point matching on the image data information; performing aerial triangulation to obtain three-dimensional data information of the feature points; performing multi-view image dense matching to obtain global dense point cloud data of the target substation, and obtaining a global coarse model based on oblique photography; constructing a point cloud classification model and extracting local features of the equipment; carrying out refined point cloud completion on the equipment based on the high-precision three-dimensional model of the neural radiation field; constructing an irregular triangular grid to obtain a fine model of the target substation equipment; and three-dimensional modeling of the target transformer substation is completed through the model obtained through fusion. The invention also discloses a system for realizing the three-dimensional modeling method of the transformer substation. According to the method, refined three-dimensional modeling of the target transformer substation is realized, the reliability is higher, and the accuracy is better.

Description

Technical Field

[0001] The invention belongs to the field of electrical automation, and in particular relates to a three-dimensional modeling method and system for a substation. Background Art

[0002] With the development of economy and technology and the improvement of people's living standards, electricity has become an indispensable secondary energy source in people's production and life, bringing endless convenience to people's production and life. Therefore, ensuring the stable and reliable supply of electricity has become one of the most important tasks of the power system.

[0003] Substations are an important part of the power system; therefore, substation inspections are of great significance to the power system. In the past, substation inspections were carried out manually, which was not only time-consuming and labor-intensive, but also inefficient and unreliable. With the rapid development of drone technology, at this stage, the inspection of substations in power systems is basically carried out by drone inspections.

[0004] The premise of using drones to inspect substations is to build a high-precision 3D model of the substation. At present, the commonly used 3D modeling solutions for substations mainly include 3D modeling based on lidar and 3D modeling based on computer vision. However, the 3D modeling solution based solely on lidar has the problem of time-consuming and labor-intensive collection work, and the lidar collection equipment is expensive and not suitable for data collection and reconstruction of scenes with large areas. The 3D modeling solution based solely on computer vision can only restore the geometric structure and appearance of the object to achieve high-precision 3D modeling of large-scale substation scenes. Summary of the invention

[0005] One of the purposes of the present invention is to provide a three-dimensional modeling method for a substation with high reliability and good accuracy.

[0006] A second object of the present invention is to provide a system for implementing the substation three-dimensional modeling method.

[0007] The three-dimensional modeling method of a substation provided by the present invention comprises the following steps:

[0008] S1. Use drones to obtain image data information of the target substation;

[0009] S2. Extracting and matching feature points of the image data information obtained in step S1;

[0010] S3. Perform aerial triangulation based on the feature points obtained in step S2 to obtain three-dimensional data information of the feature points;

[0011] S4. Perform multi-view image dense matching on the data information obtained in step S3 to obtain global dense point cloud data of the target substation, and obtain a global rough model based on oblique photography;

[0012] S5. Build a point cloud classification model based on Transformer and extract local features of the device;

[0013] S6. Based on the local features obtained in step S5 and the high-precision three-dimensional model of the neural radiation field, the device is refined and point cloud completion is performed;

[0014] S7. According to the data information obtained in step S6, an irregular triangular mesh is constructed to obtain a fine model of the target substation equipment;

[0015] S8. The models obtained in step S4 and step S7 are integrated to complete the three-dimensional modeling of the target substation.

[0016] The feature point extraction and feature point matching of the image data information obtained in step S1 described in step S2 includes the following steps:

[0017] Based on the Gaussian function, a multi-scale Gaussian pyramid is constructed to extract the extreme points in the scale space;

[0018] Based on the constructed multi-scale Gaussian pyramid, a Gaussian difference pyramid is constructed, and the obtained extreme points are filtered to realize the extraction of feature points;

[0019] Feature point matching is performed based on convolutional neural network.

[0020] The step S2 specifically includes the following steps:

[0021] Gaussian convolution is used to construct the scale space L(x,y,σ), which can be expressed as

[0022] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0023] Where G(x,y,σ) is the Gaussian kernel function centered at (x,y); σ is the scale parameter; * is the convolution operation; I(x,y) is the pixel value of the image at (x,y);

[0024] Subtract the adjacent scale spaces to obtain the Gaussian difference pyramid D(x,y,σ) as D(x,y,σ)=L(x,y,kσ)-L(x,y,σ); where k is the multiple coefficient of the adjacent scale;

[0025] The gradient direction characteristics of the neighborhood pixels are used to obtain the direction parameters of the feature points, which can be expressed as

[0026]

[0027] Where m(x,y) is the gradient modulus; L(x,y) is the image sample; θ(x,y) is the phase angle;

[0028] Feature point matching based on convolutional neural network: Utilize the similarity learning module in the convolutional network to match feature descriptors based on cosine similarity or Euclidean distance; Treat the extracted feature description as a high-dimensional vector and use the output features of the neural network for matching calculations.

[0029] Step S4, in which the data information obtained in step S3 is densely matched with multi-view images to obtain global dense point cloud data of the target substation, and obtain a global rough model based on oblique photography, comprises the following steps:

[0030] Calculate the image exterior orientation elements and the coordinates of the ground encrypted points, and use the coordinates of the encrypted points as the estimated values ​​of the bundle method regional block adjustment;

[0031] The collinear equation is constructed with the beam composed of the image as the basic unit, and the error equation of the unknown encryption point and the known control point of each image is constructed to calculate the image's internal and external orientation elements;

[0032] The actual ground coordinates of the pending points and the encrypted points are calculated by using the spatial front intersection formula of several images;

[0033] The sparse point cloud is matched by stereo to calculate the depth information of each pixel, and the pixel containing the depth information is mapped to the three-dimensional space. The point cloud data is fused to obtain the dense point cloud of the substation, and finally a global coarse model based on oblique photography is obtained.

[0034] The step S4 specifically comprises the following steps:

[0035] Calculate the image exterior orientation elements and the coordinates of the ground encrypted points, and use the coordinates of the encrypted points as the estimated values ​​of the bundle method regional block adjustment:

[0036] The collinear equation of bundle block adjustment is expressed as

[0037]

[0038] Where (x, y) is the coordinate of the image point on the image plane; f is the focal length of the camera; a1 is the first exterior azimuth element The first direction cosines are formed by the second exterior azimuth element ω and the third exterior azimuth element κ; a2 is the first exterior azimuth element The second direction cosines are formed by the second exterior azimuth element ω and the third exterior azimuth element κ; a3 is the first exterior azimuth element The third direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b1 is the first exterior azimuth element The fourth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b2 is the first exterior azimuth element The fifth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b3 is the first exterior azimuth element The sixth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c1 is the first exterior azimuth element The seventh direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c2 is the first exterior azimuth element The eighth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c3 is the first exterior azimuth element The ninth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; (X, Y, Z) is the spatial coordinate of the object point; (X s ,Y s ,Z s ) is the spatial coordinate of the camera projection center;

[0039] The collinear equation of the bundle method block adjustment is processed to obtain the error equation:

[0040] Where (V x ,V y ) is the observation error in the x and y directions; is the derivative (Jacobian matrix) of the collinear equation; (I x ,I y ) is the difference vector of the observed quantity, (x 0 ,y 0 ) is the image principal point coordinate, and I x =xx 0 , I y =yy 0 ;

[0041] The collinear equation is constructed with the beam composed of the image as the basic unit, and the error equation of the unknown encryption point and the known control point of each image is constructed to calculate the image's internal and external orientation elements;

[0042] The actual ground coordinates of the pending points and the encrypted points are calculated by using the spatial front intersection formula of several images;

[0043] The depth information of each pixel is calculated through stereo matching of the sparse point cloud, and the pixel points containing the depth information are mapped to the three-dimensional space. The obtained point cloud data are fused to obtain the dense point cloud of the substation, and finally the global rough model based on oblique photography is obtained:

[0044] Aerial survey photos are selected from the photo collection, and several photos are selected as neighboring photos for each photo. The sparse point cloud results are used as the standard for judging whether they have a high similarity, so as to ensure the accuracy of dense 3D reconstruction.

[0045] Perform depth estimation on the initial sparse point cloud feature points and use the sparse point cloud feature points as seed points;

[0046] The Patch Match algorithm is used to estimate the depth map. x ,q y ) describes the parallax, expressed as

[0047] d=a q q x +b q q y +c q

[0048] Where d is the parallax; a q is the first constant coefficient of the pixel coordinate relationship, b q is the second constant coefficient of the pixel coordinate relationship, c q is the third constant coefficient of the pixel coordinate relationship. Usually, a q 、b q and c q Learned or estimated by an optimization process (such as Patch Match);

[0049] In each iteration, for each pixel (q x ,q y ), calculate the matching cost of the pixel according to the current disparity d, and determine whether the disparity needs to be updated based on the size of the cost: set the current disparity depth value to d cur (q x ,q y ), in the matching cost function C(q x ,q y ,d) under the new depth d new The update rule for (x,y) is:

[0050]

[0051] The depth information of the scene is obtained by continuous iteration; the mathematical model for depth information optimization is expressed as

[0052]

[0053] Where E is the optimized depth value; I R (i, j) is the reference image depth value at position (i, j); c k is the depth scale factor of the k-th perspective; Ik (i, j) is the depth value of the k-th view at position (i, j);

[0054] The depth maps are fused to obtain a dense point cloud of the substation, and finally a global coarse model based on oblique photography is obtained.

[0055] Step S6, based on the local features obtained in step S5 and the high-precision three-dimensional model of the neural radiation field, performs refined point cloud completion of the device, which specifically includes the following steps:

[0056] Represent the local features obtained in step S5 as a point set;

[0057] Build a neural radiation field model to learn the local and full information of the point cloud, so as to complete the refined point cloud of the equipment.

[0058] The construction of the neural radiation field model and learning of the local and full information of the point cloud, thereby completing the refined point cloud of the device, specifically includes the following steps:

[0059] The neural radiation field model expresses the scene as an implicit function using a fully connected multi-layer perceptron, learns the color and density value of each pixel in 3D space, and uses ray tracing and volume rendering to complete the device's refined point cloud.

[0060] The camera pose is estimated by COLMAP, and the pixel color value C(r) rendered along the ray direction is predicted using the ray r(t)=o+td emitted by the camera, where o is the optical center of the camera, d is the viewing angle direction; t is a set of samples from far to near on the ray, and is expressed as

[0061]

[0062]

[0063] Where t n is the minimum value; t f is the maximum value; T(t) is the cumulative transmittance from t1 to t; σ(r(t)) is the volume density, which is used to indicate the probability that the light terminates at a point; c(r(t), d) is the color value of the sampling point on the ray; s is the sampling point.

[0064] Step S8 integrates the models obtained in step S4 and step S7 to complete the three-dimensional modeling of the target substation, which specifically includes the following steps: first, feature point matching is used, and then the global coarse model and the fine model are aligned to the same coordinate system based on the homogeneous coordinate transformation of the calibration points.

[0065] The present invention also provides a system for implementing the substation 3D modeling method, comprising a data acquisition module, a feature matching module, a triangulation module, a coarse modeling module, a feature extraction module, a point cloud completion module, a fine modeling module and a 3D modeling module; the data acquisition module, the feature matching module, the triangulation module, the coarse modeling module, the feature extraction module, the point cloud completion module, the fine modeling module and the 3D modeling module are connected in series in sequence; the data acquisition module is used to use a drone to acquire image data information of a target substation, and upload the data information to the feature matching module; the feature matching module is used to extract and match feature points on the acquired image data information according to the received data information, and upload the data information to the triangulation module; the triangulation module is used to perform aerial triangulation according to the received data information and the obtained feature points, obtain 3D data information of the feature points, and upload the data information to the coarse modeling module; the coarse modeling module is used to perform aerial triangulation according to the received data information and the obtained feature points, obtain 3D data information of the feature points, and upload the data information to the coarse modeling module; the coarse modeling module is used to perform aerial triangulation according to the received data information and the obtained feature points, obtain 3D data information of the feature points, and upload the data information to the coarse modeling module The received data information is used to perform multi-view image intensive matching on the obtained data information to obtain the global dense point cloud data of the target substation, obtain the global coarse model based on oblique photography, and upload the data information to the feature extraction module; the feature extraction module is used to build a point cloud classification model based on Transformer according to the received data information, and extract the local features of the equipment, and upload the data information to the point cloud completion module; the point cloud completion module is used to perform refined point cloud completion of the equipment based on the high-precision three-dimensional model of the neural radiation field according to the received data information and the obtained local features, and upload the data information to the fine modeling module; the fine modeling module is used to build an irregular triangular mesh according to the received data information, obtain a fine model of the target substation equipment, and upload the data information to the three-dimensional modeling module; the three-dimensional modeling module is used to fuse the obtained model according to the received data information to complete the three-dimensional modeling of the target substation.

[0066] The substation 3D modeling method and system provided by the present invention not only realizes the refined 3D modeling of the target substation, but also has higher reliability and better accuracy by acquiring, processing, extracting and matching the multi-dimensional data of the target substation, as well as completing the point cloud and coarse and fine modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 The figure is a schematic diagram of the method flow of the present invention.

[0068] Figure 2 Schematic diagram of the functional modules of the system of the present invention. DETAILED DESCRIPTION

[0069] like Figure 1 The method flow chart of the method of the present invention is shown as follows: The three-dimensional modeling method of the substation disclosed in the present invention comprises the following steps:

[0070] S1. Use drones to obtain image data information of the target substation;

[0071] S2. Extracting and matching feature points of the image data information obtained in step S1; comprising the following steps:

[0072] Based on the Gaussian function, a multi-scale Gaussian pyramid is constructed to extract the extreme points in the scale space;

[0073] Based on the constructed multi-scale Gaussian pyramid, a Gaussian difference pyramid is constructed, and the obtained extreme points are filtered to realize the extraction of feature points;

[0074] Based on convolutional neural network, feature point matching is performed;

[0075] The specific implementation includes the following steps:

[0076] Gaussian convolution is used to construct the scale space L(x,y,σ), which can be expressed as

[0077] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0078] Where G(x,y,σ) is the Gaussian kernel function centered at (x,y); σ is the scale parameter; * is the convolution operation; I(x,y) is the pixel value of the image at (x,y);

[0079] Subtract the adjacent scale spaces to obtain the Gaussian difference pyramid D(x,y,σ) as D(x,y,σ)=L(x,y,kσ)-L(x,y,σ); where k is the multiple coefficient of the adjacent scale;

[0080] The extracted feature points only contain position and scale information, but lack direction information. Therefore, the gradient direction characteristics of the neighborhood pixels are used to obtain the direction parameters of the feature points, which can be expressed as

[0081]

[0082] Where m(x,y) is the gradient modulus; L(x,y) is the image sample; θ(x,y) is the phase angle;

[0083] Based on convolutional neural network, feature point matching is performed: the similarity learning module in the convolutional network is used to match feature descriptors based on cosine similarity or Euclidean distance; the extracted feature description is used as a high-dimensional vector, and the output features of the neural network are used for matching calculation;

[0084] S3. Perform aerial triangulation based on the feature points obtained in step S2 to obtain three-dimensional data information of the feature points;

[0085] S4. Perform multi-view image dense matching on the data information obtained in step S3 to obtain global dense point cloud data of the target substation, and obtain a global rough model based on oblique photography; comprising the following steps:

[0086] Calculate the image exterior orientation elements and the coordinates of the ground encrypted points, and use the coordinates of the encrypted points as the estimated values ​​of the bundle method regional block adjustment;

[0087] The collinear equation is constructed with the beam composed of the image as the basic unit, and the error equation of the unknown encryption point and the known control point of each image is constructed to calculate the image's internal and external orientation elements;

[0088] The actual ground coordinates of the pending points and the encrypted points are calculated by using the spatial front intersection formula of several images;

[0089] The depth information of each pixel is calculated through stereo matching for the sparse point cloud, and the pixel containing the depth information is mapped to the three-dimensional space. The obtained point cloud data is fused to obtain the dense point cloud of the substation, and finally the global rough model based on oblique photography is obtained;

[0090] The specific implementation includes the following steps:

[0091] Calculate the image exterior orientation elements and the coordinates of the ground encrypted points, and use the coordinates of the encrypted points as the estimated values ​​of the bundle method regional block adjustment:

[0092] The collinear equation of bundle block adjustment is expressed as

[0093]

[0094] Where (x, y) is the coordinate of the image point on the image plane; f is the focal length of the camera; a1 is the first exterior azimuth element The first direction cosines are formed by the second exterior azimuth element ω and the third exterior azimuth element κ; a2 is the first exterior azimuth element The second direction cosines are formed by the second exterior azimuth element ω and the third exterior azimuth element κ; a3 is the first exterior azimuth element The third direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b1 is the first exterior azimuth element The fourth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b2 is the first exterior azimuth element The fifth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b3 is the first exterior azimuth element The sixth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c1 is the first exterior azimuth element The seventh direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c2 is the first exterior azimuth element The eighth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c3 is the first exterior azimuth element The ninth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; (X, Y, Z) is the spatial coordinate of the object point; (X s ,Y s ,Z s ) is the spatial coordinate of the camera projection center;

[0095] The collinear equation of the bundle method block adjustment is processed to obtain the error equation:

[0096] Where (V x ,V y ) is the observation error in the x and y directions; is the derivative (Jacobian matrix) of the collinear equation; (I x ,I y ) is the difference vector of the observed quantity, (x 0 ,y 0 ) is the image principal point coordinate, and I x =xx 0 , I y =yy 0 ;

[0097] The collinear equation is constructed with the beam composed of the image as the basic unit, and the error equation of the unknown encryption point and the known control point of each image is constructed to calculate the image's internal and external orientation elements;

[0098] The actual ground coordinates of the pending points and the encrypted points are calculated by using the spatial front intersection formula of several images;

[0099] The depth information of each pixel is calculated through stereo matching of the sparse point cloud, and the pixel points containing the depth information are mapped to the three-dimensional space. The obtained point cloud data are fused to obtain the dense point cloud of the substation, and finally the global rough model based on oblique photography is obtained:

[0100] Aerial survey photos are selected from the photo collection, and several photos are selected as neighboring photos for each photo. The sparse point cloud results are used as the standard for judging whether they have a high similarity, so as to ensure the accuracy of dense 3D reconstruction.

[0101] Perform depth estimation on the initial sparse point cloud feature points and use the sparse point cloud feature points as seed points;

[0102] The Patch Match algorithm is used to estimate the depth map. x ,q y ) describes the parallax, expressed as

[0103] d=a q q x +b q q y +c q

[0104] Where d is the parallax; a q is the first constant coefficient of the pixel coordinate relationship, b q is the second constant coefficient of the pixel coordinate relationship, c q is the third constant coefficient of the pixel coordinate relationship. Usually, a q 、b q and c q Learned or estimated by an optimization process (such as Patch Match);

[0105] In each iteration, for each pixel (q x ,q y ), calculate the matching cost of the pixel according to the current disparity d, and determine whether the disparity needs to be updated based on the size of the cost: set the current disparity depth value to d cur (q x ,q y ), in the matching cost function C(q x ,q y ,d) under the new depth d new The update rule for (x,y) is:

[0106]

[0107] The depth information of the scene is obtained by continuous iteration; the mathematical model for depth information optimization is expressed as

[0108]

[0109] Where E is the optimized depth value; I R (i, j) is the reference image depth value at position (i, j); c k is the depth scale factor of the k-th perspective; I k (i, j) is the depth value of the k-th view at position (i, j);

[0110] The depth map is fused to obtain a dense point cloud of the substation, and finally a global rough model based on oblique photography is obtained;

[0111] S5. Build a point cloud classification model based on Transformer and extract local features of the device; apply the Transformer module with significant global feature learning ability to the local neighborhood of the point cloud, apply the offset attention block to the 3D point cloud, and extract the local features of the device hierarchically through the transformation module to implement feature-wise and dot-product attention on the input for accurate and efficient processing;

[0112] S6. Based on the local features obtained in step S5 and the high-precision three-dimensional model of the neural radiation field, the device is refined and point cloud completion is performed; specifically, the steps include:

[0113] Represent the local features obtained in step S5 as a point set;

[0114] Construct a neural radiation field model to learn the local and full information of the point cloud, so as to complete the device's refined point cloud. The specific steps include:

[0115] The neural radiation field model expresses the scene as an implicit function using a fully connected multi-layer perceptron, learns the color and density value of each pixel in 3D space, and uses ray tracing and volume rendering to complete the device's refined point cloud.

[0116] The camera pose is estimated by COLMAP, and the pixel color value C(r) rendered along the ray direction is predicted using the ray r(t)=o+td emitted by the camera, where o is the optical center of the camera, d is the viewing angle direction; t is a set of samples from far to near on the ray, and is expressed as

[0117]

[0118]

[0119] Where t n is the minimum value; t f is the maximum value; T(t) is the cumulative transmittance from t1 to t; σ(r(t)) is the volume density, which is used to indicate the probability that the light terminates at a point; c(r(t), d) is the color value of the sampling point on the ray; s is the sampling point;

[0120] S7. According to the data information obtained in step S6, an irregular triangular mesh is constructed to obtain a fine model of the target substation equipment;

[0121] In specific implementation, the constructed triangle network consists of a series of adjacent triangles, each of which consists of three vertices and three edges;

[0122] S8. The models obtained by step S4 and step S7 are integrated to complete the three-dimensional modeling of the target substation; specifically, the following steps are included: first, feature point matching is used, and then the global coarse model and the fine model are aligned to the same coordinate system based on the homogeneous coordinate transformation of the calibration points.

[0123] like Figure 2 The figure shows a schematic diagram of the functional modules of the system of the present invention: the system for realizing the three-dimensional modeling method of the substation disclosed in the present invention comprises a data acquisition module, a feature matching module, a triangulation module, a coarse modeling module, a feature extraction module, a point cloud completion module, a fine modeling module and a three-dimensional modeling module; the data acquisition module, the feature matching module, the triangulation module, the coarse modeling module, the feature extraction module, the point cloud completion module, the fine modeling module and the three-dimensional modeling module are connected in series in sequence; the data acquisition module is used to use a drone to acquire image data information of the target substation, and upload the data information to the feature matching module; the feature matching module is used to extract and match feature points of the acquired image data information according to the received data information, and upload the data information to the triangulation module; the triangulation module is used to perform aerial triangulation according to the received data information and the obtained feature points, obtain three-dimensional data information of the feature points, and upload the data information to the coarse modeling module; the coarse model building The model module is used to perform multi-view image intensive matching on the received data information to obtain the global dense point cloud data of the target substation, obtain the global coarse model based on oblique photography, and upload the data information to the feature extraction module; the feature extraction module is used to build a point cloud classification model based on Transformer according to the received data information, extract the local features of the equipment, and upload the data information to the point cloud completion module; the point cloud completion module is used to perform refined point cloud completion of the equipment based on the high-precision three-dimensional model of the neural radiation field according to the received data information and the obtained local features, and upload the data information to the fine modeling module; the fine modeling module is used to build an irregular triangular mesh according to the received data information, obtain a fine model of the target substation equipment, and upload the data information to the three-dimensional modeling module; the three-dimensional modeling module is used to fuse the obtained model according to the received data information to complete the three-dimensional modeling of the target substation.

Claims

1. A three-dimensional modeling method for a substation, comprising the following steps: S1. Use drones to obtain image data information of the target substation; S2. Extracting and matching feature points of the image data information obtained in step S1; S3. Perform aerial triangulation based on the feature points obtained in step S2 to obtain three-dimensional data information of the feature points; S4. Perform multi-view image dense matching on the data information obtained in step S3 to obtain global dense point cloud data of the target substation, and obtain a global rough model based on oblique photography; S5. Build a point cloud classification model based on Transformer and extract local features of the device; S6. Based on the local features obtained in step S5 and the high-precision three-dimensional model of the neural radiation field, the device is refined and point cloud completion is performed; S7. According to the data information obtained in step S6, an irregular triangular mesh is constructed to obtain a fine model of the target substation equipment; S8. The models obtained in step S4 and step S7 are integrated to complete the three-dimensional modeling of the target substation.

2. The three-dimensional modeling method of a substation according to claim 1 is characterized in that The feature point extraction and feature point matching of the image data information obtained in step S1 described in step S2 includes the following steps: Based on the Gaussian function, a multi-scale Gaussian pyramid is constructed to extract the extreme points in the scale space; Based on the constructed multi-scale Gaussian pyramid, a Gaussian difference pyramid is constructed, and the obtained extreme points are filtered to realize the extraction of feature points; Feature point matching is performed based on convolutional neural network.

3. The three-dimensional modeling method of a substation according to claim 2 is characterized in that The step S2 specifically includes the following steps: Gaussian convolution is used to construct the scale space L(x,y,σ), which can be expressed as L(x,y,σ)=G(x,y,σ)*I(x,y) Where G(x,y,σ) is the Gaussian kernel function centered at (x,y); σ is the scale parameter; * is the convolution operation; I(x,y) is the pixel value of the image at (x,y); Subtract the adjacent scale spaces to obtain the Gaussian difference pyramid D(x,y,σ) as D(x,y,σ)=L(x,y,kσ)-L(x,y,σ); where k is the multiple coefficient of the adjacent scale; The gradient direction characteristics of the neighborhood pixels are used to obtain the direction parameters of the feature points, which can be expressed as Where m(x,y) is the gradient modulus; L(x,y) is the image sample; θ(x,y) is the phase angle; Feature point matching based on convolutional neural network: Utilize the similarity learning module in the convolutional network to match feature descriptors based on cosine similarity or Euclidean distance; Treat the extracted feature description as a high-dimensional vector and use the output features of the neural network for matching calculations.

4. The three-dimensional modeling method of a substation according to claim 3 is characterized in that Step S4, in which the data information obtained in step S3 is densely matched with multi-view images to obtain global dense point cloud data of the target substation, and obtain a global rough model based on oblique photography, comprises the following steps: Calculate the image exterior orientation elements and the coordinates of the ground encrypted points, and use the coordinates of the encrypted points as the estimated values ​​of the bundle method regional block adjustment; The collinear equation is constructed with the beam composed of the image as the basic unit, and the error equation of the unknown encryption point and the known control point of each image is constructed to calculate the image's internal and external orientation elements; The actual ground coordinates of the pending points and the encrypted points are calculated by using the spatial front intersection formula of several images; The sparse point cloud is matched by stereo to calculate the depth information of each pixel, and the pixel containing the depth information is mapped to the three-dimensional space. The point cloud data is fused to obtain the dense point cloud of the substation, and finally a global coarse model based on oblique photography is obtained.

5. The three-dimensional modeling method of a substation according to claim 4 is characterized in that The step S4 specifically comprises the following steps: Calculate the image exterior orientation elements and the coordinates of the ground encrypted points, and use the coordinates of the encrypted points as the estimated values ​​of the bundle method regional block adjustment: The collinear equation of bundle block adjustment is expressed as Where (x, y) is the coordinate of the image point on the image plane; f is the focal length of the camera; a1 is the first exterior azimuth element The first direction cosines are formed by the second exterior azimuth element ω and the third exterior azimuth element κ; a2 is the first exterior azimuth element The second direction cosines are formed by the second exterior azimuth element ω and the third exterior azimuth element κ; a3 is the first exterior azimuth element The third direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b1 is the first exterior azimuth element The fourth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b2 is the first exterior azimuth element The fifth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; b3 is the first exterior azimuth element The sixth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c1 is the first exterior azimuth element The seventh direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c2 is the first exterior azimuth element The eighth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; c3 is the first exterior azimuth element The ninth direction cosine formed by the second exterior azimuth element ω and the third exterior azimuth element κ; (X, Y, Z) is the spatial coordinate of the object point; (X s ,Y s ,Z s ) is the spatial coordinate of the camera projection center; The collinear equation of the bundle method block adjustment is processed to obtain the error equation: Where (V x ,V y ) is the observation error in the x and y directions; is the derivative (Jacobian matrix) of the collinear equation; (I x ,I y ) is the difference vector of the observed quantity, (x 0 ,y 0 ) is the image principal point coordinate, and I x =xx 0 , I y =yy 0 ; The collinear equation is constructed with the beam composed of the image as the basic unit, and the error equation of the unknown encryption point and the known control point of each image is constructed to calculate the image's internal and external orientation elements; The actual ground coordinates of the pending points and the encrypted points are calculated by using the spatial front intersection formula of several images; The depth information of each pixel is calculated through stereo matching of the sparse point cloud, and the pixel points containing the depth information are mapped to the three-dimensional space. The obtained point cloud data are fused to obtain the dense point cloud of the substation, and finally the global rough model based on oblique photography is obtained: Aerial survey photos are selected from the photo collection, and several photos are selected as neighboring photos for each photo. The sparse point cloud results are used as the standard for judging whether they have a high similarity, so as to ensure the accuracy of dense 3D reconstruction. Perform depth estimation on the initial sparse point cloud feature points and use the sparse point cloud feature points as seed points; The Patch Match algorithm is used to estimate the depth map. x ,q y ) describes the disparity, expressed as d=a q q x +b q q y +c q Where d is the parallax; a q is the first constant coefficient of the pixel coordinate relationship, b q is the second constant coefficient of the pixel coordinate relationship, c q is the third constant coefficient of the pixel coordinate relationship. Usually, a q 、b q and c q Learned or estimated by an optimization process (such as Patch Match); In each iteration, for each pixel (q x ,q y ), calculate the matching cost of the pixel according to the current disparity d, and determine whether the disparity needs to be updated based on the size of the cost: set the current disparity depth value to d cur (q x ,q y ), in the matching cost function C(q x ,q y ,d) under the new depth d new The update rule for (x,y) is: The depth information of the scene is obtained by continuous iteration; the mathematical model for depth information optimization is expressed as Where E is the optimized depth value; I R (i, j) is the reference image depth value at position (i, j); c k is the depth scale factor of the k-th perspective; I k (i, j) is the depth value of the k-th view at position (i, j); The depth maps are fused to obtain a dense point cloud of the substation, and finally a global coarse model based on oblique photography is obtained.

6. The three-dimensional modeling method of a substation according to claim 5, characterized in that Step S6, based on the local features obtained in step S5 and the high-precision three-dimensional model of the neural radiation field, performs refined point cloud completion of the device, which specifically includes the following steps: Represent the local features obtained in step S5 as a point set; Build a neural radiation field model to learn the local and full information of the point cloud, so as to complete the refined point cloud of the equipment.

7. The three-dimensional modeling method of a substation according to claim 6, characterized in that The construction of the neural radiation field model and learning of the local and full information of the point cloud, thereby completing the refined point cloud of the device, specifically includes the following steps: The neural radiation field model expresses the scene as an implicit function using a fully connected multi-layer perceptron, learns the color and density value of each pixel in 3D space, and uses ray tracing and volume rendering to complete the device's refined point cloud. The camera pose is estimated by COLMAP, and the pixel color value C(r) rendered along the ray direction is predicted using the ray r(t)=o+td emitted by the camera, where o is the optical center of the camera, d is the viewing angle direction; t is a set of samples from far to near on the ray, and is expressed as Where t n is the minimum value; t f is the maximum value; T(t) is the cumulative transmittance from t1 to t; σ(r(t)) is the volume density, which is used to indicate the probability that the light terminates at a point; c(r(t), d) is the color value of the sampling point on the ray; s is the sampling point.

8. The three-dimensional modeling method of a substation according to claim 7, characterized in that Step S8 integrates the models obtained in step S4 and step S7 to complete the three-dimensional modeling of the target substation, which specifically includes the following steps: first, feature point matching is used, and then the global coarse model and the fine model are aligned to the same coordinate system based on the homogeneous coordinate transformation of the calibration points.

9. A system for implementing the substation three-dimensional modeling method according to any one of claims 1 to 8, characterized in that It includes a data acquisition module, a feature matching module, a triangulation module, a rough modeling module, a feature extraction module, a point cloud completion module, a fine modeling module and a three-dimensional modeling module; the data acquisition module, the feature matching module, the triangulation module, the rough modeling module, the feature extraction module, the point cloud completion module, the fine modeling module and the three-dimensional modeling module are connected in series in sequence; the data acquisition module is used to use a drone to acquire image data information of a target substation, and upload the data information to the feature matching module; The feature matching module is used to extract and match feature points of the acquired image data information according to the received data information, and upload the data information to the triangulation module; the triangulation module is used to perform aerial triangulation according to the received data information and the obtained feature points, obtain three-dimensional data information of the feature points, and upload the data information to the rough modeling module; the rough modeling module is used to perform multi-view image dense matching on the obtained data information according to the received data information to obtain global dense point cloud data of the target substation, obtain a global rough model based on oblique photography, and upload the data information to the feature extraction module; The feature extraction module is used to build a point cloud classification model based on the Transformer according to the received data information, extract the local features of the device, and upload the data information to the point cloud completion module; the point cloud completion module is used to perform refined point cloud completion of the device based on the high-precision three-dimensional model of the neural radiation field according to the received data information and the obtained local features, and upload the data information to the fine modeling module; The fine modeling module is used to construct an irregular triangular mesh based on the received data information, obtain a fine model of the target substation equipment, and upload the data information to the three-dimensional modeling module; the three-dimensional modeling module is used to integrate the obtained model based on the received data information to complete the three-dimensional modeling of the target substation.

Images