Method and system for reconstructing three-dimensional structure of dry-type transformer

Abstract

The invention discloses a three-dimensional structure reconstruction method and system for a dry-type transformer, and relates to the field of power equipment, and the method comprises the steps: obtaining an image of the dry-type transformer; the ORB feature, the SURF feature and the Harris feature of the image are extracted respectively; performing cascade fusion on the ORB feature, the SURF feature and the Harris feature; constructing a matching model based on Transform, and matching the fused feature point cloud by using the matching model to obtain a matched feature point pair; according to the matched feature point pairs, converting the matched feature point pairs into a three-dimensional coordinate system by resolving camera pose parameters, and constructing a three-dimensional feature point cloud; and obtaining a three-dimensional digital twin model of the dry-type transformer through three-dimensional point cloud reconstruction according to the three-dimensional feature point cloud. Aiming at the low three-dimensional reconstruction precision of the power equipment in the prior art, the invention provides the method for improving the three-dimensional reconstruction precision of the power equipment by respectively extracting the ORB feature, the SURF feature and the Harris feature of the image and carrying out cascade fusion, and constructing the three-dimensional feature point cloud through Transform by using the fusion feature.

Description

Technical Field

[0001] The present application relates to the field of power equipment, and more specifically, to a method and system for reconstructing the three-dimensional structure of a dry-type transformer. Background Art

[0002] With the development of artificial intelligence technology and the demand for new energy technology, the operation and maintenance mode of traditional power systems is undergoing fundamental changes. Focusing on the construction of new power systems and smart grids, exploring the application of artificial intelligence and digital twin technology in intelligent auxiliary decision-making of power grids, and promoting the transformation and upgrading of the power system to be intelligent, digital and flexible are important tasks facing the current power industry. Dry-type transformers are important components of power systems. Their functions include current conversion, voltage conversion and impedance conversion. Because they do not use insulating oil and adopt natural cooling or air cooling, they have the advantages of environmental protection and safety. They are widely used in high-rise buildings, airports, power stations and commercial centers. In the power grid system, dry-type transformers mainly play the role of boosting or reducing voltage, and can also play the role of safety isolation to prevent faults from spreading to other circuits and ensure that the remotely transmitted power energy can guarantee the normal operation of special electrical equipment at the terminal. Dry-type transformers have good heat dissipation performance and strong load overload capacity. They are generally equipped with intelligent signal temperature control systems, which can automatically detect and display the working temperature and have alarm and tripping functions. However, given the important role of dry-type transformers, their health status monitoring and intelligent operation and maintenance are still an important task to be focused on in the work of power systems. With the development and application of artificial intelligence technologies such as digital twins, it is of great significance to collect the operating data of dry-type transformers in real time, build their digital twin models, promote the digital presentation, simulation and decision-making of physical power grids, and explore the application of artificial intelligence and digital twins in intelligent auxiliary decision-making and regulation of power grids. The existing three-dimensional structure reconstruction of transformers based on digital twin technology requires the combination of its own structural parameter information with a large number of sensors and efficient computing resources. In addition, the three-dimensional model is affected by factors such as the accuracy limitations of on-site data acquisition equipment, errors in data transmission, and limitations of data processing algorithms. It is difficult to achieve ideal accuracy and processing efficiency, resulting in distortion of digital twin simulation results, thereby affecting the accuracy of digital twin model decision-making. Therefore, it is necessary to study an automatic, fast, accurate, and intelligent method for fast reconstruction of the three-dimensional structure of dry-type transformers based on efficient image processing.

[0003] With the development of artificial intelligence and digital twin technology, by constructing a three-dimensional digital structural model of a dry-type transformer and using a variety of sensors on site to collect its operating data in real time, including vibration signals, temperature heating signals, gas signals and other key information parameters, the dynamic display of the terminal three-dimensional structural model can be realized. Not only can the operating status of each part of the transformer be intuitively grasped through the visualization interface of the three-dimensional digital structure, but also the operation trend of the transformer equipment status can be timely judged through historical data comparison and analysis, and the prediction of artificial intelligence algorithms can be used to find possible faults, providing accurate maintenance decision support for the operation and maintenance team, realizing the transformation from "periodic repair" to real-time status monitoring and active prevention, thereby improving the reliability and stability of the equipment. The three-dimensional structural model of the transformer is the basis of intelligent monitoring and operation and maintenance. Due to the accuracy and completeness of on-site data collection and the complexity of algorithm design, the existing digital twin model often fails to achieve the ideal reconstruction accuracy, resulting in distortion of transformer simulation results and affecting the accuracy of subsequent operation and maintenance decisions. In the study of 3D reconstruction of equipment related to power systems, the paper "A CNN transformer fault diagnosis method based on digital twins for oil-immersed transformers" was proposed. By integrating the transformer mechanism model and data-driven model, a five-dimensional model of the digital twin of the oil-immersed transformer was constructed. The paper "A digital reconstruction scheme for power grid facilities based on the extraction of key frames of on-site video and image recognition" was proposed. By designing dual feature point matching error constraints, the 3D reconstruction of large-scale power grid scenes was realized. The paper [7] proposed a 3D digital reconstruction method for power grid facilities based on multi-view foreground segmentation. Through the foreground segmentation design based on deep learning network, the influence of feature mismatch caused by complex image background and increased noise in the model body was reduced, and the visual effect of reconstruction was improved. The paper "A new architecture of digital twin model of wind turbine blades was proposed". By adopting wind turbine blade data based on bionic tree for modeling, the status monitoring, intelligent operation and management of wind turbine blades were realized. The paper "For remote video inspection of drones" proposed a transformer binocular vision reconstruction method based on improved SIFT feature points, which reduced the matching time and improved the computational efficiency of 3D reconstruction. The document "Evaluation of Transformer Fault Status" proposes a multimodal information fusion method based on vibration signals and infrared image data, and uses multimodal data information to analyze transformer fault diagnosis problems.

[0004] Although the above research methods have made great progress in transformer fault diagnosis and digital twin model reconstruction, most of the above designs are limited by the integrity and accuracy of on-site image or video acquisition data, as well as the design complexity of the model algorithm. There are still certain deficiencies in computational efficiency, reconstruction accuracy, and model update and maintenance, which need to be further improved. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In response to the problem of low accuracy of three-dimensional reconstruction of power equipment in the prior art, the present application provides a method and system for reconstructing the three-dimensional structure of a dry-type transformer, which improves the accuracy of three-dimensional reconstruction of power equipment by extracting ORB features, SURF features and Harris features of the image respectively and performing cascade fusion, and using the fused features to construct a three-dimensional feature point cloud through Transformer.

[0007] 2. Technical solution

[0008] The purpose of this application is achieved through the following technical solutions.

[0009] One aspect of the present application provides a three-dimensional structure reconstruction method for a dry-type transformer, comprising: acquiring an image of the dry-type transformer; extracting ORB features, SURF features, and Harris features of the image respectively; cascadingly fusing the ORB features, SURF features, and Harris features to obtain a fused feature point cloud of the image; constructing a Transformer-based matching model, and using the matching model to match the fused feature point cloud to obtain matching feature point pairs; according to the matching feature point pairs, by solving camera pose parameters, converting the matching feature point pairs to a three-dimensional coordinate system, and constructing a three-dimensional feature point cloud; according to the three-dimensional feature point cloud, by reconstructing the three-dimensional point cloud, a three-dimensional digital twin model of the dry-type transformer is obtained.

[0010] Furthermore, the ORB features of the image are extracted, including: performing FAST corner point detection on the image to obtain a candidate corner point set A 1 ; According to the candidate corner point set A 1 , build an image pyramid, and obtain the corner point set A by performing FAST corner point detection on the image pyramid 2 ; Diagonal point set A 2 Each corner point in ORB p , with corner ORB p Take the neighborhood image block as the center and establish the matrix M; according to the matrix M, calculate the corner point ORB p The main direction θ; for the corner point set A 2 For each corner point p in the corner point ORB p Select N point pairs in the neighborhood of , compare the grayscale value of each point pair, and generate the corner point ORB according to the comparison results. p BRIEF descriptor; the corner point ORB p The main direction θ and BRIEF descriptor as the corresponding corner point ORB p ORB feature output.

[0011] Further, according to the matrix M, calculate the corner point ORBp The main direction θ is given by the following formula: Where x and y represent the pixel coordinates of the pixel point, and I(x,y) represents the grayscale value of the pixel at this point; p represents x to the power of p, x q represents x to the power of q, p and q can take the value of 0 or 1, m p,q represents the p+q moment of the image, m 01 and m 10 Represents the first-order moment of the image along the x-axis and y-axis respectively.

[0012] Furthermore, the SURF features of the image are extracted, including: constructing a Gaussian scale space of the image, and obtaining a set of candidate feature points B by detecting extreme points in the Gaussian scale space 1 ; For the candidate feature point set B 1 Each feature point in SURF p , by comparing with the feature point SURF p Interpolate neighboring pixel information to obtain feature point set B 2 ; For feature point set B 2 Each feature point in SURF p , with feature point SURF p As the center, at the feature point SURF p Calculate the wavelet response in multiple directions in the neighborhood of and construct a direction histogram; According to the direction histogram, obtain the feature point set B 3 ; The direction with the largest wavelet response value in the direction histogram is taken as the feature point SURF p The main direction of the feature point set B 3 Each feature point in SURF p , according to the feature point SURF p The neighborhood of the feature point is divided into multiple sub-regions according to the main direction of the feature point; the feature vector of each sub-region is calculated, and the feature vectors of multiple sub-regions are cascaded to obtain the SURF feature descriptor; the main direction of the feature point and the SURF feature descriptor are output as the SURF feature of the corresponding feature point.

[0013] Further, the Harris features of the image are extracted, including: setting the size of the local window of the image, and calculating the pixel gradient I in the horizontal and vertical directions for each pixel (x, y) in the local window x (x,y) and I y (x,y); according to the pixel gradient I x (x,y) and I y (x, y), calculate the autocorrelation matrix A of the pixel (x, y); calculate the eigenvalue of the autocorrelation matrix A; obtain the corner point set C based on the eigenvalue1 ; Calculate the corner point set C 1 Harris p Harris response function value; perform non-maximum suppression on the Harris response function value, extract the local maximum point as the Harris corner point; and output the Harris corner point as the detection result.

[0014] Furthermore, according to the pixel gradient I x (x,y) and I y (x, y), calculate the autocorrelation matrix A of the pixel (x, y), the specific steps include: Among them, w(x i ,y i ) is the weight of each pixel in the window; I x and I y are the gradients of the corresponding image in the x and y directions respectively; i Indicates the horizontal coordinate of the pixel at this point, y i Indicates the vertical coordinate of the pixel at that point.

[0015] Further, obtaining a matching feature point pair includes: obtaining an image to be matched I 1 and I 2 , respectively extract the corresponding fusion feature point cloud P 1_fused and P 2_fused ; The fused feature point cloud P 1_fused and P 2_fused Input the Transformer-based matching model, encode the feature points through the self-attention mechanism, and obtain the encoded feature points E 1 and E 2 ; Calculate feature point E 1 and E 2 According to the similarity matrix, the initial matching relationship M between the two sets of feature points is obtained by decoding through the Transformer-based matching model. init ; According to the initial matching relationship M init Calculate the depth information of the feature points and use the depth information to adjust the feature points E 1 and E 2 The size of the feature point E is obtained 1_refine and E 2_refine Repeat the above matching steps until the preset convergence condition is reached to obtain the final matching relationship M final ; According to the final matching relationship M final , for the fused feature point cloud P 1_fused and P 2_fused Perform matching to obtain a set of matching feature point pairs; repeat the above image matching steps to obtain all matching feature point pairs.

[0016] Further, according to the matched feature point pairs, the camera pose parameters are solved, including: obtaining image I 1 and I 2 , according to image I 1 and I 2 The homography matrix is ​​solved by the 8-point algorithm for the matching feature point pairs between the two cameras. The camera intrinsic parameters are obtained by decomposing the homography matrix. The relative pose equation between the left and right cameras is established according to the coordinate transformation relationship of the matching feature point pairs. The relative pose equation contains the rotation matrix R and the translation vector t. The relative pose equation is solved by the least squares method to obtain the rotation matrix R and the translation vector t between the left and right cameras. The coordinates of the feature points in the three-dimensional space are solved by the triangulation method using the obtained camera intrinsic parameters, the rotation matrix R and the translation vector t, and the pixel coordinates of the matching feature point pairs. The rotation matrix R and the translation vector t are optimized according to the coordinates of the feature points in the three-dimensional space by the back-projection algorithm. The optimized rotation matrix R and the translation vector t are globally optimized by the BA algorithm to obtain a three-dimensional feature point cloud.

[0017] Furthermore, according to the coordinate transformation relationship of the matching feature point pairs, the relative pose equation between the left and right cameras is established, and the expression is as follows: Among them, R l Represents the 3D point x in the left camera space w To the left camera coordinate x l The rotation matrix, T l Represents the 3D point x in the left camera space w To the left camera coordinate x l The translation matrix, R r Represents the 3D point x in the right camera space w To the right camera coordinate x r The rotation matrix, T r Represents the 3D point x in the right camera space w To the right camera coordinate x r The translation matrix of .

[0018] Another aspect of the present application also provides a three-dimensional structure reconstruction system for a dry-type transformer, including: an acquisition module, which acquires an image of the dry-type transformer; a feature module, which extracts ORB features, SURF features and Harris features of the image respectively; and cascades and fuses the extracted ORB features, SURF features and Harris features to obtain fused features of the image; a matching module, which constructs a Transformer-based matching model, and uses the matching model to match the fused features to obtain matching feature point pairs; a modeling module, which converts the matching feature point pairs into a three-dimensional coordinate system according to the matched feature point pairs by solving the camera pose parameters, and constructs a three-dimensional feature point cloud; and obtains a three-dimensional model of the dry-type transformer through three-dimensional point cloud reconstruction according to the three-dimensional feature point cloud.

[0019] 3. Beneficial effects

[0020] Compared with the prior art, the advantages of this application are:

[0021] This application extracts ORB features, SURF features and Harris features from an image and performs cascade fusion of these features to obtain a fused feature point cloud of the image. Different feature extraction algorithms show different adaptability in terms of scale, rotation and illumination changes. By fusing these complementary features, the adaptability of feature representation to environmental changes is enhanced.

[0022] This application builds a matching model based on Transformer and uses the model to match the fused feature point cloud. Traditional feature matching algorithms usually determine the matching relationship based on the similarity of local feature descriptors, and it is difficult to consider the global association and geometric constraints between feature points. The Transformer model can model the long-range dependency between feature points through the self-attention mechanism, and can find the optimal matching result in the global scope through iterative optimization strategies, which can better handle the difficulties such as occlusion and repeated patterns in dry transformer images.

[0023] Based on the three-dimensional feature point cloud and the three-dimensional point cloud reconstruction algorithm, this application obtains a high-precision three-dimensional digital twin model of the dry-type transformer. By fusing texture information under different perspectives, a realistic texture mapping effect can be obtained, so that the reconstructed digital twin model has a high degree of visual realism. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The digital reconstruction process of the three-dimensional structure of the dry-type transformer of this application;

[0025] Figure 2 This is a schematic diagram of the Transformer-based feature point matching modeling process of this application;

[0026] Figure 3 This is a schematic diagram of multi-view parameter solution based on feature point matching in this application;

[0027] Figure 4 This is the three-dimensional reconstruction effect diagram based on dense point cloud of this application;

[0028] Figure 5 This is a comparison chart of the extraction effects of feature points in this application. DETAILED DESCRIPTION

[0029] The present application is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0030] The three-dimensional reconstruction of transformers based on digital twin technology requires the detailed structural parameters of the transformer to be known and the physical object to be scanned in detail on site. It is highly dependent on the physical object and the degree of integration of complex data is not high, resulting in low accuracy and insufficient universality of the digital twin model. Based on multi-source feature extraction and fusion technology, this paper deconstructs the geometric structural parameters between multiple visual sensors through feature point matching without the need for detailed information on the physical structural parameters, and establishes a flexible and accurate model for the three-dimensional reconstruction of power equipment represented by dry-type transformers, thereby improving the intelligent operation and maintenance efficiency of power equipment. The process of the three-dimensional structural reconstruction model of dry-type transformers established in this paper is as follows: Figure 1 shown.

[0031] This paper mainly uses the multi-view image acquisition, processing and analysis of the transformer, extracts and matches multi-source feature points, combines the geometric information of feature points and a coarse-to-fine deep network model framework to realize the flexible solution of camera position parameters between multiple images, without the need for on-site camera calibration, that is, it can quickly realize the dense feature point three-dimensional structure restoration based on multi-view analysis, and realize the digital reconstruction of the transformer's three-dimensional structure. Most of the existing transformer digital twin or three-dimensional reconstruction methods are based on the acquisition and analysis of point cloud data, which are generally collected and processed by special laser point cloud or radar point cloud scanning equipment, and the original factory parameter information of the equipment is also required. The above acquisition equipment is expensive, complicated to operate, and requires specialized technicians to handle. In addition, the reconstructed model is not universal enough and is difficult to adapt to large-scale modeling of different types of electrical equipment. Therefore, this paper combines the target three-dimensional reconstruction method of multi-view geometry, and realizes the three-dimensional reconstruction of the target by extracting the dense point cloud of multi-source features of the target image. This paper uses ORB (oriented FAST and RotatedBRIEF), SURF (Speed ​​Up Robust Features) and Harris feature point extraction to start the analysis. ORB features are scale invariant and rotation invariant, SURF features are also well adaptable to scale and rotation changes, and Harris features are a local feature based on the Harris corner detection algorithm. It calculates the autocorrelation matrix of pixel gradients in the local window of the image and determines whether the pixel is a corner point based on the eigenvalue of the matrix. Harris corner points are well invariant to image translation, rotation and illumination changes, and are one of the significant features in the image. In this application, Harris features, as one of the fusion features, provide stable and reliable corner information for dry-type transformer images and enhance the robustness of feature representation.

[0032] First, it is necessary to collect digital images of dry-type transformers as input data for subsequent processing. When collecting, attention should be paid to factors such as image resolution and lighting conditions to ensure image quality. The obtained image can be represented as a two-dimensional matrix I, and the matrix elements are the grayscale values ​​of the image pixels. Extract the ORB features of the image, input the image to be processed I, and set the FAST detection parameters, such as threshold t, number of consecutive pixels n, etc. Traverse each pixel p of image I: obtain the grayscale values ​​of 16 pixels with a radius of 3 around p. If there are n consecutive pixels whose grayscale values ​​are greater than the grayscale value of p + t or less than the grayscale value of p - t, then p is added to the candidate corner point set A 1 Output candidate corner point set A 1 .

[0033] Construct an image pyramid: Set the number of pyramid layers L and the scaling factor s. Initialize the image pyramid to an empty list pyramid. Add the original image I to pyramid as the 0th layer. From the i-th layer to the L-1 layer, perform Gaussian smoothing on the i-1th layer image. Downsample the smoothed image by s times to get the i-th layer image, and add it to pyramid. Output the image pyramid pyramid.

[0034] Multi-scale FAST corner detection: Initialize corner point set A 2 For each layer of the pyramid, perform FAST detection to obtain the corner point set of that layer. Multiply the coordinates of the corner points in the corner point set by the scaling factor corresponding to that layer and add them to A 2 Output multi-scale corner point set A 2 . Calculate the main direction of the corner point, for A 2 For each corner point p in the image, take p as the center and take a neighborhood image block with a side length of 2w+1 (w is generally 15). Calculate the centroid coordinates C(x c ,y c ), Calculate the moment matrix M: The direction θ of the feature point is defined as: m 00 =Σ(I(x,y)),m 10 =Σ(I x (x,y)), m 01 =∑(I y (x,y)); take θ as the main direction of the corner point and append it to the corner point information.

[0035] Generate a BRIEF descriptor and set the descriptor length N (usually 128, 256 or 512). 2 For each corner point p in the p neighborhood, select N point pairs (p i ,q i ). Initialize the N-bit binary descriptor des to 0. From 1 to N, if p iThe gray value is greater than q i , then des[i]=1, otherwise des[i]=0. Des is used as the BRIEF descriptor of the corner point and attached to the corner point information. Output an ORB feature point set with position, scale, main direction and BRIEF descriptor. SURF (Speeded-Up Robust Features) feature extraction is a key point feature extraction algorithm that constructs a Gaussian scale space. A square box filter is used during feature extraction to improve the operation speed. It can be used to extract dense feature points on the surface structure of dry-type transformers.

[0036] Extreme point detection in Gaussian scale space: SURF feature point extraction uses the Hessian matrix to determine the feature points. The Hessian matrix is ​​a second-order derivative matrix in the x and y directions. The value of its determinant represents the amount of change around the pixel point. The feature point selects the extreme point of the determinant value as the potential key point. Specifically, construct a Gaussian scale space, set the number of groups O and the number of layers S in each group, and the initial scale σ 0 . Calculate the scale of the current layer Use the scale σ ij Perform Gaussian smoothing on the original image I to obtain the smoothed image L ij Calculate L ij The Hessian matrix of the scale layer is used to suppress non-maximum values ​​in a 3x3x3 cubic neighborhood to obtain the extreme point set V of the scale layer. ij . Set all V ij Merge into candidate feature point set B 1 .

[0037] Positioning of key feature points: The feature points are positioned by interpolating the neighboring information of the feature points. In this process, the square filter is used instead of the Gaussian filter, and the integral image is used for calculation, which greatly improves the calculation speed and enables the rapid extraction of feature points. Direction positioning of feature points: With the feature point determined by the above process as the center, the Harr wavelet transform of the pixels around the point in the x and y directions is calculated to determine the main direction of the feature point. The direction with the maximum value in all directions in the histogram is the main direction of the feature point, ensuring the rotation invariance of the feature point. Specifically, for B 1 For each feature point p in the scale layer L where p is located, ij As the center layer, perform quadratic interpolation in a 3x3x3 cube neighborhood. Calculate the exact position (x, y, σ) and Hessian determinant response value r after interpolation. If |r| < threshold T (T = 0.04 is acceptable), remove the point; otherwise, add it to the precise feature point set B 2 . 2For each feature point q in the image, take q as the center and take a square neighborhood R with a side length of 6σ on the image corresponding to the scale σ of q. Divide R into 4x4 sub-regions. For each sub-region, calculate the Harr wavelet response (dx, dy) of 25 sampling points. Multiply (dx, dy) with the Gaussian weight w to obtain the feature vector v = (∑wdx, ∑wdy) of the sub-region. The feature vectors v of the 16 sub-regions are combined into a 128-dimensional SURF descriptor d = [v 1 ,v 2 ,.....,v 16 ]. Perform L2 normalization on d to make it have unit length. Calculate the Harr wavelet response of each pixel in R, weight the sum of the responses in each direction, and construct the direction histogram h. Find the direction θ corresponding to the maximum value in h as the main direction of the feature point. Add (q, σ, θ, d) to the feature point set B 3 Output SURF feature point set B with position q, scale σ, main direction θ and 128-dimensional descriptor d 3 .

[0038] Extract the Harris features of the image, input the image to be processed I, calculate the pixel gradient, set the local window size w (w = 3, 5, 7 and other odd numbers). Perform Gaussian smoothing on the image I to obtain the smoothed image Is to reduce the influence of noise, and output the pixel gradient I x and I y . Calculate the autocorrelation matrix: Initialize the Harris response map of the image R = 0, the autocorrelation matrix map M xx =0,M xy =0,M yy = 0. Generate a Gaussian weight template W of size (w, w) with a standard deviation of σ = 0.5w. Take the pixel gradient I within the (w, w) neighborhood N centered at (x, y) x , I y . Calculate the gradient product sum in the neighborhood N and construct the autocorrelation matrix at the pixel (x, y). Output the autocorrelation matrix graph M xx , M xy , M yy .

[0039] For a small local area of ​​the image, we use a local window to move around to observe the grayscale changes in the window area. If the grayscale changes significantly during the window movement, it is considered that a potential feature corner point is encountered in the window. Assume that the coordinates of each pixel in the current window are (x i ,y i ), where (x i ,y i)∈W, W is the size of the window. When the sliding window is offset by (u,v) pixels, the weighted sum of squared differences (SSD) is defined in a small local area of ​​the image as: Among them, w(x i ,y i ) is the weight of each pixel in the window, which is usually set to a Gaussian kernel. Then, the above formula is expanded by Taylor expansion to obtain: Among them, I x , I y are the gradients of the corresponding images in the x and y directions respectively. The matrix in the middle is called the autocorrelation coefficient A, that is The corner point is determined by the eigenvalue of the autocorrelation coefficient A. If the eigenvalues ​​are large, it means that the point is a corner point; if the eigenvalues ​​are large and small, it means an edge; if the eigenvalues ​​are small, it means that the point is a flat area. The above method can realize the extraction of Harris corner points, which is recorded as H i .

[0040] For each input image I, extract the ORB feature point set F orb , SURF feature point set F surf and Harris corner set F harris For each feature point p, extract its geometric information such as position coordinates (x, y), scale s, main direction θ, and the corresponding feature descriptor d. Convert the geometric information and descriptors of each feature point into a unified data structure, such as a feature tuple or vector of (x, y, s, θ, d). For each image I, the three feature point sets F orb 、F surf 、F harris , perform matching fusion of spatial neighborhoods. For each ORB feature point p orb , search for SURF feature points p that meet the conditions in its neighborhood surf and Harris corner point p harris , construct triples (p orb ,p surf ,p harris ). Perform weighted averaging on each triple to obtain the fused feature point coordinates (x', y') and descriptor d'. All fused feature points constitute the cascade feature point set F of the image. I , each cascade feature point contains the fused position, scale, direction, and descriptor information. I Screening and optimization are performed, such as using non-maximum suppression, feature point response sorting and other methods to obtain a streamlined cascade feature point set F I '.

[0041] like Figure 2 As shown, for the image I to be matched 1and I 2 , extract their multi-source feature points respectively and perform cascade fusion (as described above). Get image I 1 and I 2 The fusion feature point cloud P 1_fused and P 2_fused , each fused feature point contains its two-dimensional coordinates (x, y) and high-dimensional descriptor d. 1_fused and P 2_fused Input the Transformer encoder and treat each feature point as a token. At each layer of the encoder, perform self-attention encoding on the feature point sequence. Map the coordinates (x, y) of the feature point and the descriptor d to the query vector q, key vector k and value vector v. Calculate the attention weight matrix between the feature points, calculate the encoding vector e=Av of the feature point, and incorporate the information aggregated by the self-attention into each feature point. Between the layers of the encoder, introduce feedforward networks and residual connections to improve the nonlinear expression ability of the feature point encoding. At the output layer of the encoder, we get the image I 1 and I 2 The encoded feature point cloud E 1 and E 2 , each encoded feature point contains its original coordinates (x, y) and the updated high-dimensional feature vector e.

[0042] Based on Transformer feature point matching, calculate E 1 and E 2 The cosine similarity between all feature point pairs in E is obtained to obtain the similarity matrix S. S is input into the Transformer decoder, and each feature point pair is regarded as a token. At each layer of the decoder, the feature point pair sequence is cross-attention decoded. 1 The encoding vector e of each feature point in 1 Mapped to query vector q, E 2 The encoding vector e of each feature point in 2 Mapped into a key vector k and a value vector v. Calculate e 1 To e 2 The attention weight matrix is ​​calculated as 1 The decoding vector d1 = Av, e 2 The matching information is aggregated to e 1 In. 2 Perform the same operation to obtain its decoding vector d 2 At the output layer of the decoder, the decoded vector d of each feature point is 1 and d 2 splicing, and then sending it to the fully connected layer and the softmax layer to predict the matching confidence p of the feature point pair. According to the matching confidence p, the nearest neighbor, mutually different nearest neighbor and other criteria are used to obtain E 1 and E2 The initial matching relationship M init .

[0043] Feature point refinement based on depth information, based on the initial matching M init , using the epipolar geometry principle of binocular vision, calculate the depth information z of each matching feature point. According to the depth information z, the encoded feature point cloud E 1 and E 2 For E 1 Each feature point (x 1 ,y 1 ,e 1 ), according to its depth z 1 , for E 2 Each feature point (x 2 ,y 2 ,e 2 ), according to its depth z 2 . Get the scaled encoded feature point cloud E 1_refine and E 2_refine Repeat the E 1_refine and E 2_refine Input the Transformer decoder to obtain the refined matching relationship M refine Repeat the iteration to update the matching relationship and feature point encoding until the preset convergence condition (such as the number of iterations or the matching accuracy threshold) is reached. Output the final matching relationship M_final and get P 1_fused and P 2_fused The successfully matched feature point pairs (p 1 ,p 2 ).

[0044] For each pair of images in the dataset (I i ,I j ), and repeat to get their fused feature point cloud (P i ,P j ) between the matching relationship M ij . Using the matching relationship M ij , all matching feature point pairs (p i ,p j ) are unified into the global coordinate system to form a matching point cloud. The matching point cloud is post-processed by filtering and smoothing to remove wrong matches and outliers to obtain the final dense matching point cloud.

[0045] This application unifies the matching points into the same three-dimensional coordinate system and constructs a three-dimensional feature point cloud based on the matched feature point pairs through a series of pose solving and optimization steps. Solve the homography matrix and decompose the homography matrix to obtain the camera internal parameters to ensure the accuracy of the camera model. Then establish the relative pose equation between the left and right cameras, and solve the rotation matrix R and translation vector t by the least squares method to obtain the relative pose relationship between the left and right cameras. Then use the triangulation method to solve the three-dimensional coordinates of the feature points, and optimize the camera pose by the back-projection algorithm to reduce the reprojection error. Finally, use the BA algorithm for global optimization to minimize the reprojection error on all images and three-dimensional points to obtain a three-dimensional feature point cloud with higher accuracy and better consistency.

[0046] like Figure 3 As shown in the figure, after obtaining the feature matching point cloud between multi-view images, the feature information between images is used to select two initial images, and the 2D solution method from image feature points to corresponding image feature points is used to solve the camera pose. The parameter solution between two adjacent images is used as an example to illustrate. The camera intrinsic parameters use the 8-point algorithm to solve the homography matrix mapped by the matching point pairs of the two images, and decompose the essential matrix to solve the camera intrinsic parameters. The external parameter solution process is as follows. Figure 3 As shown, assuming that the two images in the above figure are acquired by the left and right cameras respectively, their corresponding external parameters are R l , T l and R r , T r , R l , T l Respectively represent the relative position of the left camera and the world coordinate system, R r , T r Respectively represent the relative position of the right camera and the world coordinate system. For any point P1 in space, assume that its non-homogeneous coordinates in the world coordinate system, left camera coordinate system and right camera coordinate system are x w 、x l 、x r , then Eliminate x w ,get Therefore, the geometric relationship R, T between the two cameras can be expressed as follows: We can find R l , T l and R r , T r , and finally the geometric relationship R and T of the two cameras are obtained.

[0047] Then, the camera's internal and external parameters and the same-name points in the same spatial position are used to triangulate the 3D coordinates of the feature points in the 3D scene. Since the 3D coordinates of the initialized spatial points have been solved, the back-projection algorithm is used to optimize the pose parameters. Repeat the above process until all images are optimized. The BA (Bundle Adjustment) algorithm is used to optimize again to avoid cascade errors, and accurate reconstruction of 3D feature points can be achieved.

[0048] To perform 3D reconstruction modeling, we first need to prepare the hardware and software environment: operating system: Windows 11, memory: 32 GB, processor: Intel (R) Xeon (R) Gold 6136 CPU @ 3.00 GHz 2.99 GHz (2 processors), GPU: 1 NVIDIA GeForce RTX 3090 Ti graphics card with 24 GB video memory capacity, deep learning framework: Pytorch, Python version: 3.11.

[0049] A high-resolution camera is used to rotate around the dry-type transformer for one week to collect multi-view RGB images. The image resolution is set to 2048×2400 pixels to ensure that there is enough overlap between adjacent images for subsequent feature matching. The multi-source feature point fusion method proposed in this paper is used to extract dense feature points on the surface of the dry-type transformer. The feature points include grayscale change information, surface texture information and corner point information, which fully reflect the surface structural characteristics of the transformer. Any two adjacent sequence images are selected and accurately matched using the extracted dense feature points. Through feature point matching, the camera pose is estimated, laying the foundation for subsequent three-dimensional reconstruction. A deep learning model for image feature point matching is constructed, and the number of training rounds (epoch) is set to 500, the initial learning rate is set to 0.01, and the batch size (batchsize) is set to 64. The AdaGrad adaptive gradient descent algorithm is used to optimize the model parameters. Using the matched dense feature points and the estimated camera pose, the three-dimensional point coordinates are calculated through the triangulation principle, and the solved three-dimensional points are spliced ​​into a complete point cloud model. The point cloud is post-processed such as denoising and filtering to improve the model quality. Point cloud sharpening and rendering technology are used to fill the holes on the model surface to obtain a detailed three-dimensional digital twin model.

[0050] In order to evaluate the performance of the proposed algorithm, a group of transformer images were selected for comparative experiments. The images were processed using traditional feature point extraction algorithms such as ORB, SURF, and Harris to obtain the corresponding 3D reconstruction results. The same group of images were processed using the multi-source feature point fusion algorithm proposed in this paper to obtain the 3D reconstruction results. Figure 4 , Figure 5The results show that the proposed algorithm can obtain a denser and more accurate point cloud by fusing multi-source feature points, and reconstruct a more realistic 3D model.

[0051] In order to evaluate the computational efficiency of the proposed algorithm, it is compared with common digital twin modeling equipment (such as radar point cloud scanning and laser scanning). The same transformer was scanned 10 times, and the average 3D reconstruction time of each method was recorded. The proposed algorithm only needs to take photos or record videos around the transformer, and data acquisition is simple and fast. By extracting dense point clouds as the basis for 3D reconstruction and processing them with optimized algorithm models, the proposed algorithm has high computational efficiency. The comparison results (as shown in Table 1) show that the processing time of the proposed algorithm is the shortest, and the proportion of holes in the reconstructed 3D model is comparable to that of the radar point cloud device. In contrast, although the radar point cloud scanning has the smallest proportion of holes, it needs to be matched with known device parameters for point cloud registration, and the processing time is relatively long. The laser scanning device is affected by factors such as scanning distance and occlusion, and the operation is complicated, and it is easy to produce more holes, and the 3D reconstruction effect is relatively poor.

[0052] Table 1 3D reconstruction results of different methods

[0053] algorithm Void ratio % Processing time (minutes) This algorithm 16.3 5 minutes 49 seconds Radar point cloud equipment 12.7 19 minutes 27 seconds Laser scanning equipment 43.4 83 minutes 12 seconds

[0054] In order to cope with the three-dimensional structural reconstruction of dry-type transformers under complex operating environments and improve the intelligent operation and maintenance efficiency and equipment reliability of transformers, this paper studies a method for three-dimensional digital structural reconstruction of dry-type transformers based on multi-feature point fusion. The dense point cloud extraction of multi-feature points and adaptive precise model matching are used to realize the flexible three-dimensional modeling method of dry-type transformers. The comparative experimental analysis shows that compared with other common methods, the algorithm proposed in this paper takes the least time and is the most convenient to operate during the reconstruction process. The proportion of voids in the surface model in the three-dimensional structural model is the smallest, and the three-dimensional reconstruction effect is the best. Next, we will continue to study the optimization of the three-dimensional structural model of the transformer and the integration of information data of on-site monitoring sensor equipment into the three-dimensional digital model. In the three-dimensional model, the status changes of on-site transformers and other equipment are monitored in real time, the maintenance strategy of the transformer is optimized, and the transformation of transformers and other equipment from passive maintenance to active prevention is realized, thereby improving the reliability and stability of power equipment.

[0055] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the present invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A method for reconstructing the three-dimensional structure of a dry-type transformer, characterized in that: include: Acquire images of dry-type transformers; Extract the ORB features, SURF features and Harris features of the image respectively; Cascade fusion of ORB features, SURF features and Harris features to obtain the fused feature point cloud of the image; Construct a Transformer-based matching model, use the matching model to match the fused feature point cloud, and obtain matching feature point pairs; According to the matched feature point pairs, the matched feature point pairs are converted into a three-dimensional coordinate system by solving the camera pose parameters to construct a three-dimensional feature point cloud; According to the three-dimensional feature point cloud, the three-dimensional digital twin model of the dry-type transformer is obtained through three-dimensional point cloud reconstruction.

2. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 1, characterized in that: Extract the ORB features of the image, including: Perform FAST corner point detection on the image to obtain a candidate corner point set A1; According to the candidate corner point set A1, an image pyramid is constructed, and the corner point set A2 is obtained by performing FAST corner point detection on the image pyramid; For each corner point in the diagonal point set A2, ORB p , with corner point ORB p Take the neighborhood image block as the center and establish the matrix M; According to the matrix M, calculate the corner point ORB p The main direction θ; For each corner point p in the corner point set A2, at the corner point ORB p Select N point pairs in the neighborhood of , compare the grayscale value of each point pair, and generate the corner point ORB according to the comparison results. p BRIEF descriptor; ORB p The main direction θ and BRIEF descriptor as the corresponding corner point ORB p ORB feature output.

3. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 2, characterized in that: According to the matrix M, calculate the corner point ORB p The main direction θ is given by the following formula: Where x and y represent the pixel coordinates of the pixel point, and I(x,y) represents the grayscale value of the pixel at this point; p represents x to the power of p, x q represents x to the power of q, m p,q represents the p+q moment of the image, m 01 and m 10 Represents the first-order moment of the image along the x-axis and y-axis respectively.

4. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 2, characterized in that: Extract SURF features of the image, including: Construct the Gaussian scale space of the image, and obtain the candidate feature point set B1 by detecting extreme points in the Gaussian scale space; For each feature point in the candidate feature point set B1, SURF p , by comparing with the feature point SURF p Interpolate the neighboring pixel information to obtain the feature point set B2; For each feature point in the feature point set B2, SURF p , with feature point SURF p As the center, at the feature point SURF p Calculate the wavelet responses in multiple directions in the neighborhood of and construct a direction histogram; obtain the feature point set B3 according to the direction histogram; The direction with the largest wavelet response value in the direction histogram is taken as the feature point SURF p The main direction of For each feature point in the feature point set B3, SURF p , according to the feature point SURF p The main direction of the feature point is divided into multiple sub-areas; Calculate the feature vector of each sub-region, and concatenate the feature vectors of multiple sub-regions to obtain the SURF feature descriptor; The main direction and SURF feature descriptor of the feature point are output as the SURF features of the corresponding feature point.

5. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 4, characterized in that: Extract Harris features of the image, including: Set the size of the local window of the image, For each pixel (x, y) in the local window, calculate the pixel gradient I in the horizontal and vertical directions x (x,y) and I y (x,y); According to the pixel gradient I x (x,y) and I y (x, y), calculate the autocorrelation matrix A of pixel (x, y); Calculate the eigenvalues ​​of the autocorrelation matrix A; obtain the corner point set C1 according to the eigenvalues; Calculate the Harris of each corner point in the corner point set C1 p Harris response function value of; Perform non-maximum suppression on the Harris response function value and extract the local maximum point as the Harris corner point; The Harris corner points are output as detection results.

6. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 5, characterized in that: According to the pixel gradient I x (x,y) and I y (x, y), calculate the autocorrelation matrix A of the pixel (x, y), the specific steps include: Among them, w(x i ,y i ) is the weight of each pixel in the window; I x and I y are the gradients of the corresponding image in the x and y directions respectively; i Indicates the horizontal coordinate of the corresponding point pixel, y i Indicates the vertical coordinate of the pixel at that point.

7. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 5, characterized in that: Get matching feature point pairs, including: Get the images I1 and I2 to be matched, and extract the corresponding fusion feature point clouds P respectively 1_fused and P 2_fused ; The fused feature point cloud P 1_fused and P 2_fused Input the Transformer-based matching model, encode the feature points through the self-attention mechanism, and obtain the encoded feature points E1 and E2; Calculate the similarity matrix between feature points E1 and E2; According to the similarity matrix, the initial matching relationship M between the two sets of feature points is obtained by decoding through the Transformer-based matching model. init ; According to the initial matching relationship M init Calculate the depth information of the feature point, use the depth information to adjust the size of feature points E1 and E2, and get feature point E 1_refine and E 2_refine ; Repeat the above matching steps until the preset convergence condition is reached to obtain the final matching relationship M final ; According to the final matching relationship M final , for the fused feature point cloud P 1_fused and P 2_fused Perform matching to obtain a set of matching feature point pairs; Repeat the above image matching steps to obtain all matching feature point pairs.

8. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 7, characterized in that: According to the matched feature point pairs, the camera pose parameters are calculated, including: Get images I1 and I2, and solve the homography matrix based on the matching feature point pairs between images I1 and I2; By decomposing the homography matrix, the camera intrinsic parameters are obtained; According to the coordinate transformation relationship of the matching feature point pairs, the relative pose equation between the left and right cameras is established. The relative pose equation contains the rotation matrix R and the translation vector t. Solve the relative pose equation by the least squares method to obtain the rotation matrix R and translation vector t between the left and right cameras; Using the obtained camera intrinsic parameters, rotation matrix R and translation vector t, as well as the pixel coordinates of the matching feature point pairs, the coordinates of the feature points in three-dimensional space are solved by triangulation method; According to the coordinates of the feature points in three-dimensional space, the rotation matrix R and translation vector t are optimized using the back-projection algorithm; The BA algorithm is used to perform global optimization on the optimized rotation matrix R and translation vector t to obtain a three-dimensional feature point cloud.

9. The method for reconstructing the three-dimensional structure of a dry-type transformer according to claim 8, characterized in that: According to the coordinate transformation relationship of the matching feature point pairs, the relative pose equation between the left and right cameras is established, and the expression is as follows: Among them, R l Represents the 3D point x in the left camera space w To the left camera coordinate x l The rotation matrix, T l Represents the 3D point x in the left camera space w To the left camera coordinate x l The translation matrix, R r Represents the 3D point x in the right camera space w To the right camera coordinate x r The rotation matrix, T r Represents the 3D point x in the right camera space w To the right camera coordinate x r The translation matrix of .

10. A three-dimensional structure reconstruction system for a dry-type transformer, characterized in that: include: An acquisition module, for acquiring an image of a dry-type transformer; Feature module, extracts ORB features, SURF features and Harris features of the image respectively; The extracted ORB features, SURF features and Harris features are cascaded and fused to obtain the fusion features of the image; The matching module builds a matching model based on Transformer and uses the matching model to match the fused features to obtain matching feature point pairs; The modeling module converts the matched feature point pairs into a three-dimensional coordinate system and constructs a three-dimensional feature point cloud by solving the camera pose parameters according to the matched feature point pairs. And according to the three-dimensional feature point cloud, the three-dimensional model of the dry-type transformer is obtained through three-dimensional point cloud reconstruction.

Images