Grid equipment three-dimensional reconstruction method based on Gaussian splashing

Abstract

The invention relates to a three-dimensional reconstruction method for power grid equipment based on Gaussian splashing. The method comprises the following steps: acquiring laser radar point cloud, videos, pictures and camera internal parameters through data acquisition; performing frame extraction or view field cutting processing on the data, extracting picture feature points and descriptors, and performing feature matching by using a SuperGlue algorithm; and inputting the obtained information into a COLMAP database, and performing sparse reconstruction through a GLOMAP algorithm to obtain initial point cloud and pose estimation of the power equipment. And supplementing the initial point cloud with the laser radar point cloud to form a final point cloud of the power equipment. And combining the final point cloud, the picture and the pose information into a Gaussian splashing data set, and partitioning the Gaussian splashing data set according to a preset interval. And training the sub-models by using a Gaussian splash algorithm, trimming according to transparency and volume parameters, merging into a Gaussian splash model, and finally constructing a three-dimensional model of the power grid equipment.

Description

Technical Field

[0001] The invention relates to a three-dimensional reconstruction method of power grid equipment based on Gaussian splashing, and belongs to the technical field of digital modeling of power grid equipment. Background Art

[0002] With the continuous advancement of science and technology, the operation and management of power grid equipment are becoming more intelligent and refined. In this context, 3D reconstruction technology, with its intuitive and comprehensive visual presentation, is gradually becoming an important means of monitoring, fault diagnosis and asset management of power grid equipment. In recent years, the 3D reconstruction method based on Gaussian splashing has shown great application potential in the field of 3D reconstruction of power grid equipment due to its high efficiency and high quality.

[0003] Gaussian splatting technology originated in the field of computer graphics. Its core idea is to regard each data point or voxel in a three-dimensional scene as a Gaussian kernel, and to generate a continuous surface or volume representation by superimposing these Gaussian kernels in space. This technology can convert discrete point cloud data into a continuous three-dimensional model to achieve high-quality scene reconstruction and rendering. In the three-dimensional reconstruction of power grid equipment, Gaussian splatting technology can effectively process complex equipment structures and details, generate realistic three-dimensional models, and provide intuitive and comprehensive equipment information for operation and maintenance personnel.

[0004] The SuperPoint algorithm is an image feature extraction method based on deep learning. It has shown high efficiency and robustness in a variety of visual tasks such as image matching and target tracking. The SuperGlue algorithm is an image feature matching algorithm based on deep learning. It aims to learn the matching process from pre-existing local features to solve challenging image matching tasks. The NetVLAD algorithm is a feature encoding method widely used in the fields of image processing and computer vision. It is mainly used to aggregate local feature descriptors into global representations to support tasks such as image recognition and retrieval. CloudCompare is a powerful open source 3D point cloud and mesh processing software that is widely used in many fields such as geographic information systems (GIS), computer graphics, surveying and mapping, archaeology, architecture, and engineering.

[0005] In the current practice of promoting the 3D digitalization of power grid equipment, the mainstream method still relies on manually creating 3D models based on detailed design documents and ledger information. This method is particularly challenging when faced with the complex network structure of the power grid system, the wide distribution of equipment, and the unique operating characteristics of each device.

[0006] Prior art, such as the Chinese patent with patent number "CN117523081A", discloses a method and system for 3D reconstruction of underwater structures based on point cloud data. The method includes the following steps: In the data acquisition stage, the point cloud data of the target scene or object is acquired from multiple angles and distances through underwater sonar, depth camera or structured light equipment, and saved in a 3D coordinate format. Data preprocessing includes denoising, filtering, downsampling and registration to improve data quality and consistency and reduce redundancy and error. Data segmentation divides the point cloud into different sub-regions or instances according to its spatial distribution and semantic information. Feature extraction uses a deep neural network to extract global and local features from the point cloud to describe the shape, structure and texture. In the data reconstruction stage, combined with the underwater structure design, a generative adversarial network or variational autoencoder is used to generate a 3D mesh or voxel model, or an implicit function is used to generate a continuous surface model, and conditional batch normalization technology is applied to generate texture information. Finally, data post-processing optimizes and repairs the generated 3D model to improve quality and integrity.

[0007] The problem with the above-mentioned prior art is that segmentation and feature extraction based on the spatial distribution and semantic information of point cloud data may be difficult for complex scenes or objects, and more advanced algorithms and technologies are needed to achieve accurate segmentation and effective feature extraction. The generative adversarial network or variational autoencoder method used to generate a 3D model may not be as good as the method of using lidar point cloud data to supplement the model in some cases. Summary of the invention

[0008] In order to solve the above problems existing in the prior art, the present invention proposes a three-dimensional reconstruction method of power grid equipment based on Gaussian splashing.

[0009] The technical solution of the present invention is as follows:

[0010] The present invention provides a three-dimensional reconstruction method of power grid equipment based on Gaussian splashing, comprising the following steps:

[0011] Collect data of power grid equipment scenes, including laser radar point cloud data, video data, image data, and camera internal parameter information of the acquisition device; perform frame extraction processing on the video data and field angle cutting processing on the image data to obtain corresponding images after data processing;

[0012] Use the SuperPoint algorithm to extract the feature points and descriptors of the image, and use the SuperGlue algorithm to obtain feature matching between the image and random images based on the feature points and descriptors of the image;

[0013] The image, its feature points, descriptors, and feature matching information are input into the COLMAP database, and the image, its feature points, descriptors, feature matching information, and the camera intrinsic parameter information of the acquisition device are sparsely reconstructed based on the GLOMAP algorithm of the COLMAP database to obtain the initial point cloud of the power equipment and the pose estimation information of the equipment during image acquisition;

[0014] Use the LiDAR point cloud data to supplement the initial point cloud of the power equipment and obtain the final point cloud of the power equipment;

[0015] The final point cloud, image and pose estimation information of the power equipment during image acquisition are combined into a Gaussian splash data set; the Gaussian splash data set is divided into several blocks according to the preset interval, and the Gaussian splash algorithm is used to train the blocks to construct a Gaussian splash sub-model;

[0016] The Gaussian splash sub-models are pruned according to the transparency and volume parameters, and the pruned Gaussian splash sub-models are merged into the Gaussian splash model;

[0017] The three-dimensional model of power grid equipment is represented according to the Gaussian splash model.

[0018] As a preferred embodiment, the video data is collected by a drone;

[0019] The image data is collected by a ground panoramic camera with a 360° viewing angle;

[0020] The laser radar point cloud data is collected by a laser radar device.

[0021] As a preferred implementation, for the video data collected by the drone, frame extraction processing is performed on the video data, specifically:

[0022] Remove 2 frames from the video data every 30 frames;

[0023] Generate a picture for each frame of video data after frame extraction;

[0024] For the image data collected by the ground panoramic camera, the 360° image data is cut in the range of 80 to 120 according to the field of view to obtain several cut images;

[0025] The mask of the cut image is extracted by using the Grounding DINO target detection algorithm and the SAM image segmentation algorithm.

[0026] As a preferred implementation, the specific method of extracting feature points and descriptors using the SuperPoint algorithm is:

[0027] Input the image into the shared encoder to obtain the feature map of the image;

[0028] Input the feature map into the feature point detection decoder and the descriptor decoder respectively to obtain the feature points and descriptors of the image data;

[0029] The mask is used to remove feature points and descriptors obtained from the image data collected by the ground panoramic camera, which are irrelevant to the power grid equipment.

[0030] As a preferred implementation, the specific method for feature matching using the SuperGlue algorithm is as follows:

[0031] The NetVLAD algorithm is used to extract the global feature information of the image, the similarity between the images is calculated based on the global feature information, and the N images that are most similar to the target image are selected as matching pairs;

[0032] The feature points and descriptors of the image and its matching pair are input into the attention graph neural network, the feature point positions and descriptors of the image and its matching pair are processed through mapping transformation, and the similarity is calculated through self-attention and cross-attention mechanisms, and the corresponding matching descriptors are obtained according to the similarity;

[0033] The feature point score matrix between the image and its matching pair is generated according to the optimal matching layer. The feature points with scores lower than the set threshold are filtered out through the score matrix to obtain the final matching information.

[0034] As a preferred embodiment, the method of using laser radar point cloud data to supplement the initial point cloud of power equipment is as follows:

[0035] Use CloudCompare software to filter outliers on the initial point cloud of power equipment and perform thinning on the LiDAR point cloud data;

[0036] Perform point cloud registration on the laser radar point cloud data after thinning and the point cloud after outlier filtering;

[0037] Use the registered point cloud as the final point cloud of the power equipment.

[0038] As a preferred implementation, the steps of constructing the Gaussian splash model are:

[0039] Construct a coordinate system based on the Gaussian splash data set, and obtain the maximum value x1, y1 and the minimum value x2, y2 of the point cloud in the X-axis and Y-axis directions of the Gaussian splash data set;

[0040] Divide the Gaussian splash data set according to the preset interval a to obtain the block set S:

[0041]

[0042] Among them, si represents the i-th block, block s i The coordinate ranges expressed in the X-axis and Y-axis directions are represents the horizontal coordinate of the origin, represents the ordinate of the origin, and n represents the total number of blocks;

[0043] According to the block coordinate range, point cloud, image and pose estimation information of the device during image acquisition are allocated to each block;

[0044] Use Gaussian splash algorithm to train the blocks to obtain Gaussian splash sub-model;

[0045] Traverse the pixel points of the divided image, and determine whether each point of the point cloud of the Gaussian splash sub-model intersects with the pixel point. If so, calculate the product of the transparency and volume in the Gaussian distribution parameters of the intersection point, and the product of the transparency and volume represents the importance score of the point to the pixel point; if not, do not process;

[0046] Add the importance scores to obtain a global importance score of the pixels in the image;

[0047] Set a global importance score threshold, and remove the Gaussian distribution parameters of the corresponding points in the point cloud of the Gaussian splash sub-model whose global importance score is less than the global importance score threshold;

[0048] The Gaussian splatter model is constructed by combining Gaussian splatter sub-models.

[0049] As a preferred implementation, the steps of using the Gaussian splash algorithm to train the blocks are:

[0050] Use the Gaussian splash algorithm to construct Gaussian distribution parameters for each point of the divided point cloud to obtain an initial Gaussian splash sub-model;

[0051] Inputting the posture information of the device when the block images are collected into the initial Gaussian splash sub-model to generate a rasterized rendering image under the posture information;

[0052] The difference between the divided image and the rasterized rendered image is calculated. When the difference exceeds a preset value, the Gaussian distribution parameters of the initial Gaussian splash sub-model are updated using the gradient descent method to obtain the Gaussian splash sub-model.

[0053] On the other hand, the present invention further provides an electronic device having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing as described in any embodiment of the present invention is implemented.

[0054] On the other hand, the present invention also provides a computer-readable medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional reconstruction method of power grid equipment based on Gaussian splashing as described in any embodiment of the present invention.

[0055] The present invention has the following beneficial effects:

[0056] The present invention can accurately capture key frames in dynamic data through frame extraction processing, effectively reduce data redundancy, and retain important information. Field of view cutting adjusts the data perspective according to actual needs, making data processing more flexible and improving the efficiency and accuracy of subsequent processing. This step ensures the refined processing of data and lays a solid foundation for subsequent three-dimensional modeling. The SuperPoint algorithm is used to extract feature points and descriptors of the image, which is efficient and robust. Subsequently, the SuperGlue algorithm is used to achieve accurate matching of feature points. The algorithm improves matching accuracy and reduces mismatching through deep learning and attention mechanism, and provides reliable feature point pairs for sparse reconstruction. The GLOMAP algorithm combined with the COLMAP database is used for sparse reconstruction, which can efficiently and accurately restore the initial point cloud of the power equipment and the pose estimation information of the equipment from the feature points. This step not only improves the accuracy of the three-dimensional model, but also provides strong support for subsequent point cloud supplementation and model optimization. The initial point cloud is supplemented by laser radar point cloud data, which can further enrich the point cloud data and improve the integrity and accuracy of the three-dimensional model. The high precision and wide coverage characteristics of laser radar make the final point cloud of the power equipment more real and detailed. The point cloud, image and pose estimation information are combined into a Gaussian splash dataset, and the sub-models are trained through block processing and Gaussian splash algorithm, and finally merged into a Gaussian splash model. This process makes full use of the diversity of the dataset and improves the efficiency and effect of model training. At the same time, the Gaussian splash model can better represent the three-dimensional form of power grid equipment and enhance the realism and usability of the model. The Gaussian splash sub-model is pruned according to the transparency and volume parameters to remove unnecessary details and noise, making the model more concise and clear. The pruned sub-models are merged into the final Gaussian splash model, which further improves the accuracy and practicality of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flow chart of the implementation steps of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0060] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0061] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0062] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0063] Embodiment 1:

[0064] See also Figure 1 The present invention provides a three-dimensional reconstruction method of power grid equipment based on Gaussian splashing, comprising the following steps:

[0065] Collect data of power grid equipment scenes, including laser radar point cloud data, video data, image data, and camera internal parameter information of the acquisition device; perform frame extraction processing on the video data and field angle cutting processing on the image data to obtain corresponding images after data processing;

[0066] Use the SuperPoint algorithm to extract the feature points and descriptors of the image, and use the SuperGlue algorithm to obtain feature matching between the image and random images based on the feature points and descriptors of the image;

[0067] The image, its feature points, descriptors, and feature matching information are input into the COLMAP database, and the image, its feature points, descriptors, feature matching information, and the camera intrinsic parameter information of the acquisition device are sparsely reconstructed based on the GLOMAP algorithm of the COLMAP database to obtain the initial point cloud of the power equipment and the pose estimation information of the equipment during image acquisition;

[0068] The GLOMAP algorithm of the COLMAP database is an innovative method in the field of Structure-from-Motion (SfM), which aims to improve the efficiency and accuracy of 3D reconstruction.

[0069] Use the LiDAR point cloud data to supplement the initial point cloud of the power equipment and obtain the final point cloud of the power equipment;

[0070] The final point cloud, image and pose estimation information of the power equipment during image acquisition are combined into a Gaussian splash data set; the Gaussian splash data set is divided into several blocks according to the preset interval, and the Gaussian splash algorithm is used to train the blocks to construct a Gaussian splash sub-model;

[0071] The Gaussian splash sub-models are pruned according to the transparency and volume parameters, and the pruned Gaussian splash sub-models are merged into the Gaussian splash model;

[0072] The three-dimensional model of power grid equipment is represented according to the Gaussian splash model.

[0073] As a preferred embodiment, the video data is collected by a drone;

[0074] The image data is collected by a ground panoramic camera with a 360° viewing angle;

[0075] The laser radar point cloud data is collected by a laser radar device.

[0076] As a preferred implementation, for the video data collected by the drone, frame extraction processing is performed on the video data, specifically:

[0077] Remove 2 frames from the video data every 30 frames;

[0078] Generate a picture for each frame of video data after frame extraction;

[0079] For the image data collected by the ground panoramic camera, the 360° image data is cut in the range of 80 to 120 according to the field of view to obtain several cut images;

[0080] The mask of the cut image is extracted by the Grounding DINO target detection algorithm and the SAM image segmentation algorithm. The specific method is as follows:

[0081] Input the image into the target detection algorithm Grounding DINO, use prompt words such as sky as text input, set the bounding box threshold to 0.25 to filter out some boxes with low confidence, set the text threshold to 0.25 to select words with high similarity as labels, and get the bounding box of the sky area in the image;

[0082] The obtained sky area bounding box is input as the prompt word and input into the image segmentation algorithm SAM (SegmentAnything Model) to obtain a mask image of the sky. At the same time, the pixel values ​​of the power grid equipment part in the mask image are set to white, and the pixel values ​​of the sky part are set to black.

[0083] Through the preliminary detection of Grounding DINO, the image segmentation algorithm can receive a more accurate target area as input, reducing the interference of background noise and other non-target objects. This pre-processing makes the segmentation algorithm more adaptable and robust when facing diverse and complex scenes, and can more accurately segment the boundaries of the target object.

[0084] Compared with the disadvantage that traditional segmentation models only work well in their pre-trained categories, the GroundingDINO algorithm combines the language pre-training model to obtain better detection results without pre-trained category labels.

[0085] The mask image obtained using this method can extract the sky part more accurately, which plays an important role in the subsequent feature point filtering and the generation of sparse reconstruction point cloud.

[0086] At the same time, using this method, you only need to set the category to be detected when inputting to get a relatively good mask result, which reduces manual intervention.

[0087] As a preferred implementation, the specific method of extracting feature points and descriptors using the SuperPoint algorithm is:

[0088] Input the image into the shared encoder to obtain the feature map of the image;

[0089] Input the feature map into the feature point detection decoder and the descriptor decoder respectively to obtain the feature points and descriptors of the image data;

[0090] The mask is used to remove feature points and descriptors obtained from the image data collected by the ground panoramic camera, which are irrelevant to the power grid equipment.

[0091] The specific steps are:

[0092] S1. The image data first passes through the shared encoder to obtain a preliminary feature map. The structure of the shared encoder consists of a combination of multiple convolution layers with a convolution kernel of 3, a maximum pooling layer with a pooling kernel of 2, and a nonlinear activation function layer. The image size after the shared encoder is gradually reduced to 1 / 8 of the original size, while the number of channels of the feature map is increased. The final output feature map has a dimension of (128, W / 8, H / 8), where 128 represents the number of channels of the feature map, and W / 8 and H / 8 are the width and height of the feature map, respectively.

[0093] S2. Input the feature map obtained in step S1 into the feature point detection decoder part. The feature map is first dimensionally transformed into a dimension (65, W / 8, H / 8), the feature map is normalized, and then the feature map is transformed into the original size, the dimension is (1, W, H) feature points.

[0094] S3, input the feature map obtained in step S1 into the descriptor decoder part. Among them, it first passes through a convolution layer to transform the dimension to (256, W / 8, H / 8), then transforms it to (256, H, W) through bicubic interpolation, and finally normalizes the descriptor of each pixel value.

[0095] S4: Based on the features obtained from the data collected on the ground, use a mask to remove the feature points of the parts that are not related to the power grid equipment in step S2.

[0096] Compared with the traditional SIFT algorithm, the feature points extracted by the SuperPoint algorithm can more effectively extract feature points with strong robustness from complex and changeable power scenes, thereby improving the accuracy and reliability of feature matching.

[0097] As a preferred implementation, the specific method for feature matching using the SuperGlue algorithm is as follows:

[0098] The NetVLAD algorithm is used to extract the global feature information of the image, the similarity between the images is calculated based on the global feature information, and the N images that are most similar to the target image are selected as matching pairs;

[0099] The feature points and descriptors of the image and its matching pair are input into the attention graph neural network, the feature point positions and descriptors of the image and its matching pair are processed through mapping transformation, and the similarity is calculated through self-attention and cross-attention mechanisms, and the corresponding matching descriptors are obtained according to the similarity;

[0100] The feature point score matrix between the image and its matching pair is generated according to the optimal matching layer. The feature points with scores lower than the set threshold are filtered out through the score matrix to obtain the final matching information.

[0101] By using the SuperGlue algorithm, compared with traditional image matching methods, this method uses a neural network model, which can consider more semantic and contextual information in the matching process, and combines the optimal transmission algorithm to effectively eliminate mismatching points and improve feature matching accuracy.

[0102] As a preferred embodiment, the method of using laser radar point cloud data to supplement the initial point cloud of power equipment is as follows:

[0103] Use CloudCompare software to filter outliers on the initial point cloud of power equipment and perform thinning on the LiDAR point cloud data;

[0104] Perform point cloud registration on the laser radar point cloud data after thinning and the point cloud after outlier filtering;

[0105] Use the registered point cloud as the final point cloud of the power equipment.

[0106] Traditionally, only sparsely reconstructed point clouds are used as the initialization of the Gaussian splash model. The Gaussian splash model obtained after training has holes or blurs in the power equipment part after rendering. This is because the sparsely reconstructed point cloud is too sparse. Using this method, the thinned LiDAR point cloud is used as the initialization of the Gaussian splash model, which can improve the effect of model rendering without increasing the resources required for Gaussian splash algorithm training.

[0107] As a preferred implementation, the steps of constructing the Gaussian splash model are:

[0108] Construct a coordinate system based on the Gaussian splash data set, and obtain the maximum value x1, y1 and the minimum value x2, y2 of the point cloud in the X-axis and Y-axis directions of the Gaussian splash data set;

[0109] Divide the Gaussian splash data set according to the preset interval a to obtain the block set S:

[0110]

[0111] Among them, s i represents the i-th block, block s i The coordinate ranges expressed in the X-axis and Y-axis directions are represents the horizontal coordinate of the origin, represents the ordinate of the origin, and n represents the total number of blocks;

[0112] According to the block coordinate range, point cloud, image and pose estimation information of the device during image acquisition are allocated to each block;

[0113] Use Gaussian splash algorithm to train the blocks to obtain Gaussian splash sub-model;

[0114] Traverse the pixel points of the divided image, and determine whether each point of the point cloud of the Gaussian splash sub-model intersects with the pixel point. If so, calculate the product of the transparency and volume in the Gaussian distribution parameters of the intersection point, and the product of the transparency and volume represents the importance score of the point to the pixel point; if not, do not process;

[0115] Add the importance scores to obtain a global importance score of the pixels in the image;

[0116] Set a global importance score threshold, and remove the Gaussian distribution parameters of the corresponding points in the point cloud of the Gaussian splash sub-model whose global importance score is less than the global importance score threshold;

[0117] The Gaussian splatter model is constructed by combining Gaussian splatter sub-models.

[0118] Pruning the Gaussian splash sub-model can reduce the disadvantages of using the thinned lidar point cloud as the initialization of the Gaussian splash model, which leads to an increase in the file size of the sub-model and an increase in the point density, resulting in a worse rendering effect.

[0119] As a preferred implementation, the steps of using the Gaussian splash algorithm to train the blocks are:

[0120] Use the Gaussian splash algorithm to construct Gaussian distribution parameters for each point of the divided point cloud to obtain an initial Gaussian splash sub-model;

[0121] Inputting the posture information of the device when the block images are collected into the initial Gaussian splash sub-model to generate a rasterized rendering image under the posture information;

[0122] The difference between the divided image and the rasterized rendered image is calculated. When the difference exceeds a preset value, the Gaussian distribution parameters of the initial Gaussian splash sub-model are updated using the gradient descent method to obtain the Gaussian splash sub-model.

[0123] The adopted block training strategy effectively solves the problem of limited computing resources. By dividing the large-scale, high-resolution data set into multiple small blocks for parallel processing, it not only improves the training efficiency, but also ensures that each data block can be fully trained and optimized, thus ensuring the overall training effect.

[0124] Embodiment 2:

[0125] This embodiment provides an electronic device having a computer program stored thereon. When the computer program is executed by a processor, the method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing as described in any embodiment of the present invention is implemented.

[0126] Embodiment three:

[0127] This embodiment provides a computer-readable medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional reconstruction method of power grid equipment based on Gaussian splashing as described in any embodiment of the present invention.

[0128] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0129] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0130] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0131] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0132] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A three-dimensional reconstruction method for power grid equipment based on Gaussian splashing, characterized in that: The following steps are involved: Collect data of power grid equipment scenes, including laser radar point cloud data, video data, image data, and camera internal parameter information of the acquisition device; perform frame extraction processing on the video data and field angle cutting processing on the image data to obtain corresponding images after data processing; Use the SuperPoint algorithm to extract the feature points and descriptors of the image, and use the SuperGlue algorithm to obtain feature matching between the image and random images based on the feature points and descriptors of the image; The image, its feature points, descriptors, and feature matching information are input into the COLMAP database, and the image, its feature points, descriptors, feature matching information, and the camera intrinsic parameter information of the acquisition device are sparsely reconstructed based on the GLOMAP algorithm of the COLMAP database to obtain the initial point cloud of the power equipment and the pose estimation information of the equipment during image acquisition; Use the LiDAR point cloud data to supplement the initial point cloud of the power equipment and obtain the final point cloud of the power equipment; The final point cloud, image and pose estimation information of the power equipment during image acquisition are combined into a Gaussian splash data set; the Gaussian splash data set is divided into several blocks according to the preset interval, and the Gaussian splash algorithm is used to train the blocks to construct a Gaussian splash sub-model; The Gaussian splash sub-models are pruned according to the transparency and volume parameters, and the pruned Gaussian splash sub-models are merged into the Gaussian splash model; The three-dimensional model of power grid equipment is represented according to the Gaussian splash model.

2. The method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing according to claim 1, characterized in that: The video data is collected by a drone; The image data is collected by a ground panoramic camera with a 360° viewing angle; The laser radar point cloud data is collected by a laser radar device.

3. The method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing according to claim 1, characterized in that: For the video data collected by the drone, the video data is processed by frame extraction, specifically: Remove 2 frames from the video data every 30 frames; Generate a picture for each frame of video data after frame extraction; For the image data collected by the ground panoramic camera, the 360° image data is cut in the range of 80 to 120 according to the field of view to obtain several cut images; The mask of the cut image is extracted by using the Grounding DINO target detection algorithm and the SAM image segmentation algorithm.

4. The method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing according to claim 3, characterized in that: The specific method of extracting feature points and descriptors using the SuperPoint algorithm is: Input the image into the shared encoder to obtain the feature map of the image; The feature map is input into the feature point detection decoder and the descriptor decoder respectively to obtain the feature points and descriptors of the image data; The mask is used to remove feature points and descriptors obtained from the image data collected by the ground panoramic camera, which are irrelevant to the power grid equipment.

5. The method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing according to claim 1, characterized in that: The specific method of using the SuperGlue algorithm for feature matching is: The NetVLAD algorithm is used to extract the global feature information of the image, the similarity between the images is calculated based on the global feature information, and the N images that are most similar to the target image are selected as matching pairs; The feature points and descriptors of the image and its matching pair are input into the attention graph neural network, the feature point positions and descriptors of the image and its matching pair are processed through mapping transformation, and the similarity is calculated through self-attention and cross-attention mechanisms, and the corresponding matching descriptors are obtained according to the similarity; The feature point score matrix between the image and its matching pair is generated according to the optimal matching layer. The feature points with scores lower than the set threshold are filtered out through the score matrix to obtain the final matching information.

6. The method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing according to claim 1, characterized in that: The method of using LiDAR point cloud data to supplement the initial point cloud of power equipment is as follows: Use CloudCompare software to filter outliers on the initial point cloud of power equipment and perform thinning on the LiDAR point cloud data; Perform point cloud registration on the laser radar point cloud data after thinning and the point cloud after outlier filtering. Use the registered point cloud as the final point cloud of the power equipment.

7. The method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing according to claim 1, characterized in that: The steps of constructing the Gaussian splash model are: Construct a coordinate system based on the Gaussian splash data set, and obtain the maximum value x1, y1 and the minimum value x2, y2 of the point cloud in the X-axis and Y-axis directions of the Gaussian splash data set; Divide the Gaussian splash data set according to the preset interval a to obtain the block set S: Among them, s i represents the i-th block, block s i The coordinate ranges expressed in the X-axis and Y-axis directions are represents the horizontal coordinate of the origin, represents the ordinate of the origin, and n represents the total number of blocks; According to the block coordinate range, point cloud, image and pose estimation information of the device during image acquisition are allocated to each block; Use Gaussian splash algorithm to train the blocks to obtain Gaussian splash sub-model; Traverse the pixel points of the divided image, and determine whether each point of the point cloud of the Gaussian splash sub-model intersects with the pixel point. If so, calculate the product of the transparency and volume in the Gaussian distribution parameters of the intersection point, and the product of the transparency and volume represents the importance score of the point to the pixel point; if not, do not process; Add the importance scores to obtain a global importance score of the pixels in the image; Set a global importance score threshold, and remove the Gaussian distribution parameters of the corresponding points in the point cloud of the Gaussian splash sub-model whose global importance score is less than the global importance score threshold; The Gaussian splatter model is constructed by combining Gaussian splatter sub-models.

8. The method for three-dimensional reconstruction of power grid equipment based on Gaussian splashing according to claim 7, characterized in that: The steps to train the blocks using the Gaussian splash algorithm are: Use the Gaussian splash algorithm to construct Gaussian distribution parameters for each point of the divided point cloud to obtain an initial Gaussian splash sub-model; Inputting the posture information of the device when the block images are collected into the initial Gaussian splash sub-model to generate a rasterized rendering image under the posture information; The difference between the divided image and the rasterized rendered image is calculated. When the difference exceeds a preset value, the Gaussian distribution parameters of the initial Gaussian splash sub-model are updated using the gradient descent method to obtain the Gaussian splash sub-model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the three-dimensional reconstruction method of power grid equipment based on Gaussian splashing as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the three-dimensional reconstruction method of power grid equipment based on Gaussian splashing as described in any one of claims 1 to 8 is implemented.

Images