Power grid equipment and facility real scene reconstruction method and system based on AI vision three-dimensional reconstruction

Abstract

The application provides a power grid equipment and facility real scene reconstruction method and system based on AI vision three-dimensional reconstruction, the method comprises the following steps: obtaining initial scene image data of a power grid from multiple data sources, preprocessing, and generating effective scene image data with global coordinates; self-adaptive block is carried out on the effective scene image data, and a plurality of image blocks are generated; an explicit-implicit coupled reconstruction model is used, and sparse point clouds corresponding to the image blocks are used as input to carry out parallel three-dimensional reconstruction and generate three-dimensional sub-models; for the overlapping area of any adjacent three-dimensional sub-models, the boundary fusion is carried out according to the corresponding feature point clouds to generate a globally integrated power grid scene three-dimensional model. The method and system realize seamless splicing of a kilometer-level scene, ensure high efficiency and low cost of large scene three-dimensional real scene modeling, and effectively improve the splicing precision and accuracy.

Description

Technical Field

[0001] This invention relates to the field of visual 3D reconstruction technology for power systems, and more specifically, to a method and system for real-world reconstruction of power grid equipment and facilities based on AI visual 3D reconstruction. Background Technology

[0002] The most mature existing technology for large-scale power grid visual 3D reconstruction is oblique photogrammetry-based real-scene reconstruction and physicalization. This technology uses drones or flying platforms equipped with multi-angle oblique cameras to acquire high-overlap images of the target area from both vertical and oblique angles. Based on computer vision and photogrammetry principles (such as multi-view stereo matching), high-density point clouds are generated through dense matching, and textured triangular mesh models (such as OSGB or OBJ formats) are further constructed. This technology is suitable for large-scale geographical scene reconstruction (such as power grid corridors and terrain features), and is characterized by high efficiency and low cost. The main drawbacks of this technology include: First, its accuracy is highly dependent on image quality and ambient lighting, resulting in poor matching effects in areas with weak texture (such as solid-color walls) or severely occluded areas, easily leading to hollow or distorted models; second, oblique photogrammetry struggles to acquire geometric information of occluded areas (such as under equipment or dense vegetation), resulting in insufficient model integrity; finally, data processing is susceptible to interference from image distortion, shadows, and other factors, potentially leading to overlapping, distortion, or inconsistencies in texture mapping, affecting the accuracy of visualization and analysis. Summary of the Invention

[0003] To address the technical problem that existing 3D reconstruction methods are efficient and low-cost in large power grid scenarios, but also have low accuracy and precision, this invention provides a method and system for real-scene reconstruction of power grid equipment and facilities based on AI vision 3D reconstruction.

[0004] According to one aspect of the present invention, the present invention provides a method for real-scene reconstruction of power grid equipment and facilities based on AI vision 3D reconstruction, comprising:

[0005] Initial scene image data of the power grid is obtained from multiple data sources;

[0006] The initial scene image data is subjected to GPS coordinate correction and point cloud preprocessing to generate effective scene image data with global coordinates. The equipment and facilities in the effective scene image data are divided into corresponding categories, including rigid and non-rigid categories.

[0007] Based on the custom device density entropy, the effective scene image data is adaptively divided into blocks to generate several image blocks. The equipment and facilities contained in each image block are divided into rigid and non-rigid parts according to their corresponding categories.

[0008] An explicit-implicit coupling reconstruction model is adopted, which takes the sparse point cloud corresponding to the image block as input to perform parallel 3D reconstruction and outputs a 3D sub-model with local coordinates and the corresponding feature point cloud for each image block.

[0009] For any overlapping region of adjacent 3D sub-models, based on their corresponding feature point clouds, the boundary is fused by rigid alignment of rigid parts and deformation constraints of non-rigid parts to determine the first coordinate transformation result from a 3D sub-model to its adjacent 3D sub-model.

[0010] Based on the first coordinate transformation result and the second coordinate transformation result, a globally integrated three-dimensional model of the power grid scene is generated. The second coordinate transformation result is the global coordinate of the three-dimensional sub-model generated based on the correspondence between the local coordinates and the global coordinates of each three-dimensional sub-model.

[0011] According to another aspect of the present invention, the present invention provides a real-scene reconstruction system for power grid equipment and facilities based on AI visual 3D reconstruction, the system comprising:

[0012] The data acquisition module is used to acquire initial scene image data of the power grid from multiple data sources;

[0013] An initial processing module is used to perform GPS coordinate correction and point cloud preprocessing on the initial scene image data to generate effective scene image data with global coordinates. The equipment and facilities in the effective scene image data are divided into corresponding categories, including rigid and non-rigid categories.

[0014] The block partitioning module is used to adaptively partition the effective scene image data based on a custom device density entropy to generate several image blocks. The equipment and facilities contained in each image block are divided into rigid and non-rigid parts according to their corresponding categories.

[0015] The parallel reconstruction module is used to perform parallel 3D reconstruction using an explicit-implicit coupled reconstruction model, taking the sparse point cloud corresponding to the image block as input, and outputting a 3D sub-model with local coordinates and the corresponding feature point cloud for each image block.

[0016] The boundary fusion module is used to perform boundary fusion on the overlapping areas of any two adjacent 3D sub-models based on their corresponding feature point clouds, through rigid alignment of rigid parts and deformation constraints of non-rigid parts, to determine the first coordinate transformation result from one 3D sub-model to its adjacent 3D sub-models.

[0017] The result output module is used to generate a globally integrated three-dimensional model of the power grid scene based on the first coordinate transformation result and the second coordinate transformation result. The second coordinate transformation result is the global coordinate of the three-dimensional sub-model generated based on the correspondence between the local coordinates and the global coordinates of each three-dimensional sub-model.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing a computer program that, when executed by a processor, implements the methods described in any of the above aspects of the present invention.

[0019] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method described in any of the preceding aspects of the present invention.

[0020] The present invention discloses a method and system for real-scene reconstruction of power grid equipment and facilities based on AI vision 3D reconstruction. The method includes: acquiring initial scene image data of the power grid from multiple data sources; performing GPS coordinate correction and point cloud preprocessing on the initial scene image data to generate effective scene image data with global coordinates; adaptively dividing the effective scene image data into several image blocks based on a custom equipment density entropy; using an explicit-implicit coupling reconstruction model, performing parallel 3D reconstruction with the sparse point cloud corresponding to the image block as input, and outputting a 3D sub-model with local coordinates and the corresponding feature point cloud for each image block; performing boundary fusion on the overlapping areas of any adjacent 3D sub-models based on their corresponding feature point clouds to determine a first coordinate transformation result from one 3D sub-model to its adjacent 3D sub-models; and generating a globally integrated 3D model of the power grid scene based on the first coordinate transformation result and the second coordinate transformation result, wherein the second coordinate transformation result is the global coordinate of the 3D sub-model generated based on the correspondence between the local coordinates and global coordinates of each 3D sub-model. The method and system are based on an adaptive block-segmentation strategy for power grid equipment density, a multi-node parallel reconstruction mechanism based on an explicit-implicit coupling reconstruction model, and a boundary fusion algorithm combining rigid alignment and non-rigid deformation. This enables seamless stitching of kilometer-level scenes, effectively improving the accuracy and precision of stitching while ensuring high efficiency and low cost in large-scale 3D real-scene modeling. Attached Figure Description

[0021] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0022] Figure 1 This is a flowchart of a method for real-scene reconstruction of power grid equipment and facilities based on AI visual 3D reconstruction according to a preferred embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of a power grid equipment and facility real-scene reconstruction system based on AI vision 3D reconstruction according to a preferred embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the present invention. Detailed Implementation

[0025] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0026] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0027] Exemplary methods

[0028] Figure 1 This is a flowchart illustrating a preferred embodiment of a method for real-world reconstruction of power grid equipment and facilities based on AI-based visual 3D reconstruction according to the present invention. Figure 1 As shown, the method for real-scene reconstruction of power grid equipment and facilities based on AI visual 3D reconstruction in this preferred embodiment starts from step 101.

[0029] In step 101, initial scene image data of the power grid is obtained from multiple data sources.

[0030] In step 102, the initial scene image data is subjected to GPS coordinate correction and point cloud preprocessing to generate effective scene image data with global coordinates. The equipment and facilities in the effective scene image data are divided into corresponding categories, including rigid and non-rigid categories.

[0031] In this preferred embodiment, the initial scene image is a kilometer-scale two-dimensional power grid scene photo or two-dimensional video frame taken from multiple perspectives. GPS coordinate correction is performed on these two-dimensional images, and the shooting position obtained by GPS for each image is associated with the image itself, so that the two-dimensional image carries a global coordinate attribute, that is, the real-world shooting position corresponding to the image.

[0032] In step 103, based on the custom device density entropy, the effective scene image data is adaptively divided into blocks to generate several image blocks. The devices and facilities contained in each image block are divided into rigid and non-rigid parts according to their corresponding categories.

[0033] Preferably, based on a custom device density entropy, the effective image data is adaptively divided into blocks to generate several image blocks, including:

[0034] Step 201: Using the towers in the effective image data as initial anchor points, construct several initial blocks with custom length * width.

[0035] Step 202: For the i-th block set, calculate the device density entropy of each block based on the custom device density entropy, where, when i=1, the elements in the block set are the initial blocks, and the formula for calculating the device density entropy H is:

[0036]

[0037] Among them, P k The percentage of the k-th type of device within a unit volume of a block-mapped 3D space; M is the total number of device types in the effective image data.

[0038] Step 203: For all blocks in the i-th block set whose device density entropy satisfies the adaptive partitioning criterion, perform adaptive partitioning according to the adaptive partitioning rules to generate 2 sub-blocks;

[0039] Step 204: After adaptive block partitioning is completed, an updated set of the i-th block set is generated, wherein the updated set includes sub-blocks generated after adaptive block partitioning of the blocks in the i-th block set and blocks that have not undergone adaptive block partitioning.

[0040] Step 205: When the device density entropy of any two blocks in the updated set satisfies the block partitioning convergence rule, stop the block partitioning; otherwise, let i = i + 1, and let the i-th block set be the updated set of the (i-1)-th block set, and return to step 203.

[0041] In this preferred embodiment, the operation using the power pole as the initial anchor point is completed using a two-dimensional image recognition model in the prior art. For example, a target detection network directly detects various types of equipment and facilities in the two-dimensional image, as well as their corresponding positions (pixel coordinates), and divides the image into blocks based on the detected power pole. The reason for dividing the image into blocks using the power pole as the initial anchor point is that the power pole is a prominent landmark in the power scene, which is easy to identify and does not constitute a restriction on using other objects as anchor points. In addition, since the detected equipment and facilities in the scene have different physical characteristics, they are classified. For rigid and fixed-shape structures such as power poles and insulators, their deformation is negligible, so they are classified as rigid bodies. However, flexible structures such as conductors may bend and deform under the influence of gravity, wind, etc., so they are classified as non-rigid bodies.

[0042] Preferably, for all blocks in the i-th block set whose device density entropy satisfies the adaptive partitioning criterion, adaptive partitioning is performed according to the adaptive partitioning rules to generate two sub-blocks, wherein the expression of the adaptive partitioning criterion is:

[0043]

[0044] In the formula, T H Customize the device entropy density threshold;

[0045] The adaptive block division rules include:

[0046] When performing adaptive segmentation, splitting is performed along a custom main direction of the device within it;

[0047] The width d of the overlapping area of ​​the two generated sub-blocks satisfies d=max(d0,k0*width of the block to be divided), where d0 is the minimum width of the custom overlapping block and k0 is the custom width coefficient, which takes a value less than 1.

[0048] Preferably, block partitioning stops when the device density entropy of any two blocks in the updated set satisfies the block partitioning convergence rule, wherein the block partitioning convergence rule includes:

[0049]

[0050] In the formula, H i and H j H represents the device density entropy values ​​for the i-th and j-th sub-blocks. avg The arithmetic mean of the device density entropy values ​​of all sub-blocks. It is a positive number less than 100.

[0051] In this preferred embodiment, multiple initial blocks are constructed for the overall power grid scene. Each initial block uses a "tower" as an anchor point. Within the entire area to be reconstructed (e.g., a kilometer-long transmission line), multiple grids of 100m x 100m size are divided to cover the entire scene. For example, if the scene is 2km long and 1km wide, the initial number of grids is approximately 20 x 10 = 200. Block division based on equipment density entropy is chosen because power grid equipment is a three-dimensional entity (e.g., towers have height, cables have spatial orientation), and two-dimensional area alone cannot reflect its distribution density in real space. For example, the same tower may appear with different pixel areas in two-dimensional images from different angles, but its volume is fixed in three-dimensional space. The spatial distribution (length, height) of non-rigid equipment (e.g., cables) has a greater impact on the accuracy of 3D reconstruction, and its density needs to be reflected through volume proportion. Therefore, calculating equipment density entropy using "unit volume proportion" more accurately reflects the distribution complexity of equipment in the 3D scene, providing a reasonable basis for subsequent adaptive block division to ensure a balanced 3D reconstruction difficulty for each block. For the initial block, although it appears as a pixel region of "length × width" in a 2D image, because the image has been corrected to global spatial coordinates via GPS coordinates (i.e., the shooting position and pose of each image are known), each pixel in the 2D image can be mapped to a real-world 3D space through camera intrinsic and extrinsic parameters (focal length, distortion coefficients, shooting pose, etc.), forming a corresponding 3D spatial range (containing length, width, and height information). Therefore, the "custom length × width initial block" (2D pixel region) actually corresponds to a 3D cube / cuboid in the real world (containing volume attributes), and its volume can be calculated using the mapped 3D coordinates (length × width × height). k "P represents the proportion of the k-th type of equipment within a unit volume," where "unit volume" refers to the three-dimensional spatial volume mapped by the aforementioned two-dimensional block. The calculation logic is as follows: First, identify the pixel regions of the k-th type of equipment (such as poles, cables, etc.) contained within the block using the two-dimensional image; convert these pixel regions into point sets / regions in three-dimensional space through coordinate mapping, and calculate their proportion in the corresponding three-dimensional volume of the block (e.g., the three-dimensional point set volume of the pole ÷ the total three-dimensional volume of the block), thus obtaining P. k The custom device master direction refers to the extension direction of the main devices (such as wires) in the scene (for example, if wires are mostly erected along the north-south direction, then the master direction is north-south). During splitting, the original block will be divided into two along this direction, and the size of the sub-block depends on the master direction. Using the device master direction for splitting makes the resulting blocks more closely match the device distribution, facilitating subsequent fine-grained reconstruction. During adaptive tiling, overlapping areas are also manually set to preserve sufficient boundary features, such as wires and insulators crossing blocks, thereby providing more matching point pairs for subsequent boundary fusion.

[0052] In step 104, an explicit-implicit coupling reconstruction model is adopted, and parallel 3D reconstruction is performed with the sparse point cloud corresponding to the image block as input, and the output is a 3D sub-model with local coordinates and the corresponding feature point cloud of each image block.

[0053] Preferably, an explicit-implicit coupled reconstruction model is used, which performs parallel 3D reconstruction with the sparse point cloud corresponding to the image block as input, and outputs a 3D sub-model with local coordinates and the corresponding feature point cloud for each image block, including:

[0054] The image blocks are distributed to worker nodes using a master node in a master-slave distributed computing framework.

[0055] Each worker node independently runs an explicit-implicit coupled reconstruction model, using the sparse point cloud corresponding to the image patch assigned to that worker node as input for 3D reconstruction, and outputting a 3D sub-model with local coordinates and the corresponding feature point cloud for each image patch. The explicit-implicit coupled reconstruction model includes a 3D Gaussian model, a SlimMLP network, and a model convergence verification unit, specifically:

[0056] In the 3D Gaussian model, a differentiable renderer is used to calculate the current 3D Gaussian set to obtain the rendered image. Here, the current 3D Gaussian set refers to the initial 3D Gaussian set generated by K-means clustering of the sparse point cloud corresponding to the image block in the first iteration, and the 3D Gaussian set obtained in the previous iteration as fed back by the iterative verification unit in subsequent iterations. Gauss in Defined as:

[0057]

[0058] In the formula, With the center coordinates, The covariance matrix characterizing scale and rotation, Based on the color, Opacity;

[0059] The image luminance loss is calculated based on the rendered image and the corresponding real image of the image block, wherein the formula for calculating the image luminance loss is:

[0060]

[0061] In the formula, For image photometric loss, and They are real images and rendered images, respectively. Let p be the set of all pixels in the real image and the rendered image, where p is any one of the pixels. This indicates taking the square of the L2 norm;

[0062] The EM algorithm is used to dynamically split / merge the Gaussians in the current 3D Gaussian set under the condition of minimizing the image photometric loss, so as to generate an optimized current 3D Gaussian set.

[0063] For the optimized current 3D Gaussian set, the SlimMLP network samples according to a normal distribution within the 3σ neighborhood of each Gaussian to obtain several sampling points;

[0064] For any sampling point x, its local relative coordinates With Gaussian central features As input, the output is the calculation result based on all sampled points. ,in, for The square root of the eigenvalue, the calculation result The expression is:

[0065]

[0066] In the formula, For color residue, For geometric displacement residuals, This is the covariance correction factor;

[0067] The iterative verification unit performs geometric fusion and color fusion on each Gaussian in the optimized current 3D Gaussian set based on the calculation results, including:

[0068] For each Gaussian, perform geometric fusion, and generate a corrected covariance based on the geometric displacement residuals and covariance correction coefficients in the calculation results. The calculation formula is as follows:

[0069] ;

[0070] Perform color fusion on each Gaussian and generate a corrected color residual based on the color residual in the calculation results. The calculation formula is as follows:

[0071]

[0072]

[0073] In the formula, For dynamic weights, j represents other Gaussians in the neighborhood related to the color fusion of the i-th Gaussian, and J is the number of related Gaussians set.

[0074] Based on the joint loss function, and according to the calculation results, the joint loss value is calculated from the corrected color residual and the minimum image luminance loss, wherein the expression of the joint loss function is:

[0075]

[0076]

[0077]

[0078] In the formula, For the joint loss value, To determine the minimum image photometric loss when optimizing the current 3D Gaussian set, and These are the edge sharpening weights and residual regularization weights, respectively. and These are the edge sharpening loss and the residual regularization term, respectively. This is the set of sampling points where the grayscale / color of the current 3D Gaussian set changes drastically after optimization. Represents an L1 normal form function;

[0079] Using the minimum joint loss value as a constraint, the network parameters are... and Update;

[0080] Iterative optimization is performed based on the condition that the joint loss value converges to the convergence threshold. When the joint loss value converges to the convergence threshold, the three-dimensional sub-model of the image block with local coordinates and the corresponding feature point cloud are output. Otherwise, the current 3D Gaussian set for the next iteration is generated using the geometric displacement residual generated in this iteration, the corrected covariance, and the corrected color residual.

[0081] In this preferred embodiment, the divided blocks are two-dimensional images, while the model reconstruction uses three-dimensional point clouds as input. Therefore, for any image block, a motion recovery structure can be used to generate a corresponding sparse point cloud with local coordinates. The explicit-implicit coupled reconstruction model uses 3D Gaussian as explicit anchor points and introduces a lightweight MLP network in its local neighborhood for detail enhancement. This retains the efficient rendering characteristics of 3D Gaussian splashing while capturing 0.1mm-level geometric details and material textures through an implicit network. In addition, the model also designs a dynamic weight allocation mechanism, which adaptively adjusts the optimization ratio of the explicit / implicit parts according to the scene complexity, solving the problems of "overfitting explicit anchor points" or "implicit prediction drift" in traditional hybrid methods. Thus, a high-precision three-dimensional model is generated through joint optimization and rendering fusion strategies. Furthermore, a master-slave distributed computing architecture is adopted to divide a large power grid scene into multiple blocks, and each block is reconstructed in parallel using an explicit-implicit coupled reconstruction model. This improves reconstruction accuracy while ensuring high reconstruction efficiency.

[0082] In step 105, for any overlapping region of adjacent 3D sub-models, based on their corresponding feature point clouds, boundary fusion is performed by rigid alignment of rigid parts and deformation constraints of non-rigid parts to determine the first coordinate transformation result from a 3D sub-model to its adjacent 3D sub-model.

[0083] Preferably, for any overlapping region of adjacent 3D sub-models, based on its corresponding feature point cloud, boundary fusion is performed by rigid alignment of rigid parts and deformation constraints of non-rigid parts to determine the first coordinate transformation value from block to block, including:

[0084] For the rigid portions in the overlapping regions of any two adjacent 3D sub-models a and b, rigid alignment is performed based on their corresponding feature point clouds, where:

[0085] Extract the first FPFH feature descriptor from the feature point cloud of the rigid portion;

[0086] Based on the first FPFH feature descriptor, feature matching is performed to obtain the first set of point pairs;

[0087] For the point pairs in the first set of point pairs, the second set of point pairs is determined by the Random Sampling Consensus (RANSAC) algorithm.

[0088] The corresponding point pair determined when the rigid body transformation value of the corresponding point pair in the second point pair set is the minimum is taken as the rigid alignment matching point pair;

[0089] For the non-rigid portion of the overlapping region of any adjacent 3D sub-models a and b, deformation constraints are applied based on their corresponding feature point clouds, where:

[0090] Extract the second FPFH feature descriptor from the feature point cloud of the non-rigid portion;

[0091] Feature matching is performed based on the second FPFH feature descriptor to obtain a third set of point pairs, wherein the total number of point pairs in the third set is N, and the point pair elements are represented as ( ;

[0092] Based on the elastic deformation model, determine the third point as any point corresponding to the three-dimensional sub-model a in the set. After non-rigid deformation, a fourth set of point pairs is generated, where the elements of the fourth set of point pairs are represented as ( The expression for the elastic deformation model is:

[0093]

[0094] In the formula, It is the i-th point after non-rigid deformation, and K is the total number of preset control points in the non-rigid part of the overlapping region corresponding to the 3D sub-model a. These are radial basis functions used to calculate the radial basis function values ​​between point x and the j-th control point. Let the displacement be the j-th control point.

[0095] Using the minimum value of the deformation constraint function as a condition, non-rigid deformation matching point pairs are determined according to the fourth set of point pairs, where the expression of the deformation constraint function is:

[0096]

[0097] In the formula, The deformation regularization weights are determined based on empirical tuning.

[0098] Based on the spatial continuity of the block division, the first coordinate transformation result of the block transformation corresponding to the three-dimensional sub-models a and b is determined according to the rigid alignment matching point pairs and the non-rigid deformation matching point pairs.

[0099] In this preferred embodiment, considering that a block may simultaneously contain both rigid body parts corresponding to rigid objects and non-rigid body parts corresponding to non-rigid objects, different strategies are used to process these two parts during the boundary fusion of adjacent sub-models, and then the two are integrated to complete the overall boundary fusion. Specifically, after performing point-to-point matching on the rigid body and non-rigid body parts of the overlapping area of ​​adjacent sub-models, all matching point pairs of the overlapping area are obtained. Then, combined with the non-overlapping parts of two adjacent 3D sub-models a and b, the coordinate transformation result of 3D sub-model b with global coordinate transformation can be determined. And the coordinate transformation results of the 3D sub-model a with global coordinate transformation. Thus, the first coordinate transformation result of the block transformation corresponding to the three-dimensional sub-models a and b is obtained. This strategy of "separate processing + collaborative integration" respects the physical properties of different types of equipment (rigid invariance, non-rigid flexibility) and ensures the overall spatial consistency of overlapping areas, thereby improving the accuracy of scene stitching.

[0100] In step 106, a globally integrated three-dimensional model of the power grid scene is generated based on the first coordinate transformation result and the second coordinate transformation result. The second coordinate transformation result is the global coordinate of the three-dimensional sub-model generated based on the correspondence between the local coordinates and the global coordinates of each three-dimensional sub-model.

[0101] Preferably, based on a custom global graph optimization function, a globally integrated 3D model of the power grid scenario is generated according to the first coordinate transformation result and the second coordinate transformation result, wherein the expression of the custom global graph optimization function is:

[0102] ,

[0103] In the formula, edges is the set of adjacent blocks. This is the result of the first coordinate transformation. and This is the result of the second coordinate transformation. Denotes the F-norm of a matrix. This indicates taking the square of the F-norm;

[0104] By solving the problem with the constraint of minimizing the value of the custom global graph optimization function, a globally integrated 3D model of the power grid scenario can be obtained.

[0105] The preferred embodiment of the power grid equipment and facility real-scene reconstruction method based on AI vision 3D reconstruction describes the following: For the original image data of the acquired large-scale power grid scene, the image is divided into blocks based on an adaptive block-segmentation strategy according to the density of power grid equipment. Then, a distributed computing framework is used to generate a 3D sub-model and corresponding feature point cloud for each image block based on a multi-node parallel reconstruction mechanism of an explicit-implicit coupling reconstruction model. Then, combined with a boundary fusion algorithm of rigid alignment and non-rigid deformation, boundary fusion is performed based on the feature point cloud of the overlapping area of ​​adjacent 3D sub-models to achieve seamless stitching of kilometer-level scenes. While ensuring high efficiency and low cost of large-scale 3D real-scene modeling, the stitching accuracy is effectively improved.

[0106] Exemplary System

[0107] Figure 2 This is a schematic diagram of a power grid equipment and facility real-scene reconstruction system based on AI vision 3D reconstruction according to a preferred embodiment of the present invention. Figure 2 As shown, the AI-based visual 3D reconstruction system 200 for real-scene reconstruction of power grid equipment and facilities according to this preferred embodiment includes:

[0108] The data acquisition module 201 is used to acquire initial scene image data of the power grid from multiple data sources;

[0109] The initial processing module 202 is used to perform GPS coordinate correction and point cloud preprocessing on the initial scene image data to generate effective scene image data with global coordinates. The equipment and facilities in the effective scene image data are divided into corresponding categories, including rigid and non-rigid categories.

[0110] The block division module 203 is used to adaptively divide the effective scene image data into blocks based on a custom device density entropy, generating a number of image blocks. The equipment and facilities contained in each image block are divided into rigid and non-rigid parts according to their corresponding categories.

[0111] Parallel reconstruction module 204 is used to perform parallel 3D reconstruction using an explicit-implicit coupled reconstruction model, taking the sparse point cloud corresponding to the image block as input, and outputting a 3D sub-model with local coordinates and the corresponding feature point cloud for each image block.

[0112] The boundary fusion module 205 is used to perform boundary fusion on the overlapping areas of any adjacent 3D sub-models based on their corresponding feature point clouds, by rigid alignment of rigid parts and deformation constraints of non-rigid parts, to determine the first coordinate transformation result from a 3D sub-model to its adjacent 3D sub-models.

[0113] The result output module 206 is used to generate a globally integrated three-dimensional model of the power grid scene based on the first coordinate transformation result and the second coordinate transformation result, wherein the second coordinate transformation result is the global coordinate of the three-dimensional sub-model generated based on the correspondence between the local coordinates and the global coordinates of each three-dimensional sub-model.

[0114] Preferably, the block partitioning module 203 adaptively partitions the effective image data based on a custom device density entropy to generate several image blocks, including:

[0115] Step 1201: Using the towers in the effective image data as initial anchor points, construct several initial blocks with custom length * width.

[0116] Step 1202: For the i-th block set, calculate the device density entropy of each block based on the custom device density entropy, where when i=1, the elements in the block set are the initial blocks, and the formula for calculating the device density entropy H is:

[0117]

[0118] Among them, P kThe percentage of the k-th type of device within a unit volume of a block-mapped 3D space; M is the total number of device types in the effective image data.

[0119] Step 1203: For all blocks in the i-th block set whose device density entropy satisfies the adaptive partitioning criterion, perform adaptive partitioning according to the adaptive partitioning rules to generate 2 sub-blocks;

[0120] Step 1204: After adaptive block partitioning is completed, an updated set of the i-th block set is generated, wherein the updated set includes sub-blocks generated after adaptive block partitioning of the blocks in the i-th block set and blocks that have not undergone adaptive block partitioning.

[0121] Step 1205: When the device density entropy of any two blocks in the updated set satisfies the block partitioning convergence rule, stop the block partitioning; otherwise, let i = i + 1, and let the i-th block set be the updated set of the (i-1)-th block set, and return to step 1203.

[0122] Preferably, the block partitioning module 203 performs adaptive partitioning on all blocks in the i-th block set whose device density entropy satisfies the adaptive partitioning criterion, generating two sub-blocks according to the adaptive partitioning rules. The expression for the adaptive partitioning criterion is:

[0123]

[0124] In the formula, T H Customize the device entropy density threshold;

[0125] The adaptive block division rules include:

[0126] When performing adaptive segmentation, splitting is performed along a custom main direction of the device within it;

[0127] The width d of the overlapping area of ​​the two generated sub-blocks satisfies d=max(d0,k0*width of the block to be divided), where d0 is the minimum width of the custom overlapping block and k0 is the custom width coefficient, which takes a value less than 1.

[0128] Preferably, the block partitioning module 203 stops partitioning when the device density entropy of any two blocks in the updated set satisfies the block partitioning convergence rule, wherein the block partitioning convergence rule includes:

[0129]

[0130] In the formula, H i and H j H represents the device density entropy values ​​for the i-th and j-th sub-blocks. avgThe arithmetic mean of the device density entropy values ​​of all sub-blocks. It is a positive number less than 100.

[0131] Preferably, the parallel reconstruction module 204 employs an explicit-implicit coupled reconstruction model, using the sparse point cloud corresponding to the image block as input for parallel 3D reconstruction, and outputs a 3D sub-model with local coordinates for each image block and the corresponding feature point cloud, including:

[0132] The image blocks are distributed to worker nodes using a master node in a master-slave distributed computing framework.

[0133] Each worker node independently runs an explicit-implicit coupled reconstruction model, using the sparse point cloud corresponding to the image patch assigned to that worker node as input for 3D reconstruction, and outputting a 3D sub-model with local coordinates and the corresponding feature point cloud for each image patch. The explicit-implicit coupled reconstruction model includes a 3D Gaussian model, a SlimMLP network, and a model convergence verification unit, specifically:

[0134] In the 3D Gaussian model, a differentiable renderer is used to calculate the current 3D Gaussian set to obtain the rendered image. Here, the current 3D Gaussian set refers to the initial 3D Gaussian set generated by K-means clustering of the sparse point cloud corresponding to the image block in the first iteration, and the 3D Gaussian set obtained in the previous iteration as fed back by the iterative verification unit in subsequent iterations. Gauss in Defined as:

[0135]

[0136] In the formula, With the center coordinates, The covariance matrix characterizing scale and rotation, Based on the color, Opacity;

[0137] The image luminance loss is calculated based on the rendered image and the corresponding real image of the image block, wherein the formula for calculating the image luminance loss is:

[0138]

[0139] In the formula, For image photometric loss, and They are real images and rendered images, respectively. Let p be the set of all pixels in the real image and the rendered image, where p is any one of the pixels. This indicates taking the square of the L2 norm;

[0140] The EM algorithm is used to dynamically split / merge the Gaussians in the current 3D Gaussian set under the condition of minimizing the image photometric loss, so as to generate an optimized current 3D Gaussian set.

[0141] For the optimized current 3D Gaussian set, the SlimMLP network samples according to a normal distribution within the 3σ neighborhood of each Gaussian to obtain several sampling points;

[0142] For any sampling point x, its local relative coordinates With Gaussian central features As input, the output is the calculation result based on all sampled points. ,in, for The square root of the eigenvalue, the calculation result The expression is:

[0143]

[0144] In the formula, For color residue, For geometric displacement residuals, This is the covariance correction factor;

[0145] The iterative verification unit performs geometric fusion and color fusion on each Gaussian in the optimized current 3D Gaussian set based on the calculation results, including:

[0146] For each Gaussian, perform geometric fusion, and generate a corrected covariance based on the geometric displacement residuals and covariance correction coefficients in the calculation results. The calculation formula is as follows:

[0147] ;

[0148] Perform color fusion on each Gaussian and generate a corrected color residual based on the color residual in the calculation results. The calculation formula is as follows:

[0149]

[0150]

[0151] In the formula, For dynamic weights, j represents other Gaussians in the neighborhood related to the color fusion of the i-th Gaussian, and J is the number of related Gaussians set.

[0152] Based on the joint loss function, and according to the calculation results, the joint loss value is calculated from the corrected color residual and the minimum image luminance loss, wherein the expression of the joint loss function is:

[0153]

[0154]

[0155]

[0156] In the formula, For the joint loss value, To determine the minimum image photometric loss when optimizing the current 3D Gaussian set, and These are the edge sharpening weights and residual regularization weights, respectively. and These are the edge sharpening loss and the residual regularization term, respectively. This is the set of sampling points where the grayscale / color of the current 3D Gaussian set changes drastically after optimization. Represents an L1 normal form function;

[0157] Using the minimum joint loss value as a constraint, the network parameters are... and Update;

[0158] Iterative optimization is performed based on the condition that the joint loss value converges to the convergence threshold. When the joint loss value converges to the convergence threshold, the three-dimensional sub-model of the image block with local coordinates and the corresponding feature point cloud are output. Otherwise, the current 3D Gaussian set for the next iteration is generated using the geometric displacement residual generated in this iteration, the corrected covariance, and the corrected color residual.

[0159] Preferably, the boundary fusion module 205 determines the first coordinate transformation value from block to block for the overlapping regions of any adjacent 3D sub-models, based on their corresponding feature point clouds, through rigid alignment of rigid parts and deformation constraints of non-rigid parts, including:

[0160] For the rigid portions in the overlapping regions of any two adjacent 3D sub-models a and b, rigid alignment is performed based on their corresponding feature point clouds, where:

[0161] Extract the first FPFH feature descriptor from the feature point cloud of the rigid portion;

[0162] Based on the first FPFH feature descriptor, feature matching is performed to obtain the first set of point pairs;

[0163] For the point pairs in the first set of point pairs, the second set of point pairs is determined by the Random Sampling Consensus (RANSAC) algorithm.

[0164] The corresponding point pair determined when the rigid body transformation value of the corresponding point pair in the second point pair set is the minimum is taken as the rigid alignment matching point pair;

[0165] For the non-rigid portion of the overlapping region of any adjacent 3D sub-models a and b, deformation constraints are applied based on their corresponding feature point clouds, where:

[0166] Extract the second FPFH feature descriptor from the feature point cloud of the non-rigid portion;

[0167] Feature matching is performed based on the second FPFH feature descriptor to obtain a third set of point pairs, wherein the total number of point pairs in the third set is N, and the point pair elements are represented as ( ;

[0168] Based on the elastic deformation model, determine the third point as any point corresponding to the three-dimensional sub-model a in the set. After non-rigid deformation, a fourth set of point pairs is generated, where the elements of the fourth set of point pairs are represented as ( The expression for the elastic deformation model is:

[0169]

[0170] In the formula, It is the i-th point after non-rigid deformation, and K is the total number of preset control points in the non-rigid part of the overlapping region corresponding to the 3D sub-model a. These are radial basis functions used to calculate the radial basis function values ​​between point x and the j-th control point. Let the displacement be the j-th control point.

[0171] Using the minimum value of the deformation constraint function as a condition, non-rigid deformation matching point pairs are determined according to the fourth set of point pairs, where the expression of the deformation constraint function is:

[0172]

[0173] In the formula, The deformation regularization weights are determined based on empirical tuning.

[0174] Based on the spatial continuity of the block division, the first coordinate transformation result of the block transformation corresponding to the three-dimensional sub-models a and b is determined according to the rigid alignment matching point pairs and the non-rigid deformation matching point pairs.

[0175] Preferably, the result output module 206 generates a globally integrated 3D model of the power grid scene based on a custom global graph optimization function, according to the first coordinate transformation result and the second coordinate transformation result. The expression of the custom global graph optimization function is:

[0176] ,

[0177] In the formula, edges is the set of adjacent blocks. This is the result of the first coordinate transformation. and This is the result of the second coordinate transformation. Denotes the F-norm of a matrix. This indicates taking the square of the F-norm;

[0178] By solving the problem with the constraint of minimizing the value of the custom global graph optimization function, a globally integrated 3D model of the power grid scenario can be obtained.

[0179] The AI-based visual 3D reconstruction system for real-scene reconstruction of power grid equipment and facilities described in this preferred embodiment has the same steps for performing 3D reconstruction of large-scale power grid images as the AI-based visual 3D reconstruction method, and achieves the same technical effects, so it will not be repeated here.

[0180] Exemplary electronic devices

[0181] Figure 3 This is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the present invention. Figure 3 As shown, the electronic device includes one or more processors 301 and memory 302.

[0182] The processor 301 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0183] The memory 302 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 301 may execute the program instructions to implement the AI-based visual 3D reconstruction method for real-world reconstruction of power grid equipment and facilities, and / or other desired functions, as described in the various embodiments disclosed above. In one example, the electronic device may also include an input device 303 and an output device 304, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0184] In addition, the input device 303 may also include, for example, a keyboard, a mouse, etc.

[0185] The output device 304 can output various information to the outside. The output device 304 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0186] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0187] Exemplary computer program products and computer-readable storage media

[0188] In addition to the methods and devices described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the AI-based visual 3D reconstruction method for real-world reconstruction of power grid equipment and facilities according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0189] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0190] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the AI-based visual 3D reconstruction method for real-world reconstruction of power grid equipment and facilities according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0191] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0192] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0193] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0194] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0195] The apparatus and methods of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0196] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps are decomposable and / or recombinable. Such decomposition and / or recombination should be considered equivalent to the present disclosure. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0197] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for real-scene reconstruction of power grid equipment and facilities based on AI visual 3D reconstruction, characterized in that, The method includes: Initial scene image data of the power grid is obtained from multiple data sources; The initial scene image data is subjected to GPS coordinate correction and point cloud preprocessing to generate effective scene image data with global coordinates. The equipment and facilities in the effective scene image data are divided into corresponding categories, including rigid and non-rigid categories. Based on a custom device density entropy, the effective scene image data is adaptively divided into blocks to generate several image blocks, including: Step 1201: Using the towers in the effective scene image data as initial anchor points, construct several initial blocks with custom length * width. Step 1202: For the i-th block set, calculate the device density entropy of each block based on the custom device density entropy. Each block contains equipment and facilities categorized into rigid and non-rigid parts according to their corresponding categories. When i=1, the elements in the block set are the initial blocks. The formula for calculating the device density entropy H is: Among them, P k The percentage of the k-th type of device within a unit volume of a block-mapped 3D space; M is the total number of device types in the effective scene image data. Step 1203: For all blocks in the i-th block set whose device density entropy satisfies the adaptive partitioning criterion, perform adaptive partitioning according to the adaptive partitioning rules to generate 2 sub-blocks; Step 1204: After adaptive block partitioning is completed, an updated set of the i-th block set is generated, wherein the updated set includes sub-blocks generated after adaptive block partitioning of the blocks in the i-th block set and blocks that have not undergone adaptive block partitioning. Step 1205: When the device density entropy of any two blocks in the updated set satisfies the block partitioning convergence rule, stop the block partitioning; otherwise, let i = i + 1, and let the i-th block set be the updated set of the (i-1)-th block set, and return to step 1203. An explicit-implicit coupling reconstruction model is adopted, which takes the sparse point cloud corresponding to the image block as input to perform parallel 3D reconstruction and outputs a 3D sub-model with local coordinates and the corresponding feature point cloud for each image block. For any overlapping region of adjacent 3D sub-models, based on their corresponding feature point clouds, the boundary is fused by rigid alignment of rigid parts and deformation constraints of non-rigid parts to determine the first coordinate transformation result from a 3D sub-model to its adjacent 3D sub-model. Based on the first coordinate transformation result and the second coordinate transformation result, a globally integrated three-dimensional model of the power grid scene is generated. The second coordinate transformation result is the global coordinate of the three-dimensional sub-model generated based on the correspondence between the local coordinates and the global coordinates of each three-dimensional sub-model.

2. The method according to claim 1, characterized in that, For all blocks in the i-th block set whose device density entropy satisfies the adaptive partitioning criterion, adaptive partitioning is performed according to the adaptive partitioning rules to generate two sub-blocks. The expression for the adaptive partitioning criterion is: In the formula, T H Customize the device entropy density threshold; The adaptive block division rules include: When performing adaptive segmentation, splitting is performed along a custom main direction of the device within it; The width d of the overlapping area of ​​the two generated sub-blocks satisfies d=max(d0,k0*width of the block to be divided), where d0 is the minimum width of the custom overlapping block and k0 is the custom width coefficient, which takes a value less than 1.

3. The method according to claim 1, characterized in that, When the device density entropy of any two blocks in the updated set satisfies the block partitioning convergence rule, the block partitioning is stopped, wherein the block partitioning convergence rule includes: In the formula, H i and H j H represents the device density entropy values ​​for the i-th and j-th sub-blocks. avg The arithmetic mean of the device density entropy values ​​of all sub-blocks. It is a positive number less than 100.

4. The method according to claim 1, characterized in that, An explicit-implicit coupling reconstruction model is adopted, using the sparse point cloud corresponding to the image block as input for parallel 3D reconstruction. The output includes a 3D sub-model with local coordinates for each image block and its corresponding feature point cloud, comprising: The image blocks are distributed to worker nodes using a master node in a master-slave distributed computing framework. Each worker node independently runs an explicit-implicit coupled reconstruction model, using the sparse point cloud corresponding to the image patch assigned to that worker node as input for 3D reconstruction, and outputting a 3D sub-model with local coordinates and the corresponding feature point cloud for each image patch. The explicit-implicit coupled reconstruction model includes a 3D Gaussian model, a SlimMLP network, and a model convergence verification unit, specifically: In the 3D Gaussian model, a differentiable renderer is used to calculate the current 3D Gaussian set to obtain the rendered image. Here, the current 3D Gaussian set refers to the initial 3D Gaussian set generated by K-means clustering of the sparse point cloud corresponding to the image block in the first iteration, and the 3D Gaussian set obtained in the previous iteration as fed back by the iterative verification unit in subsequent iterations. Gauss in Defined as: In the formula, With the center coordinates, The covariance matrix characterizing scale and rotation, Based on the color, Opacity; The image luminance loss is calculated based on the rendered image and the corresponding real image of the image block, wherein the formula for calculating the image luminance loss is: In the formula, For image photometric loss, and They are real images and rendered images, respectively. Let p be the set of all pixels in the real image and the rendered image, where p is any one of the pixels. This indicates taking the square of the L2 norm; The EM algorithm is used to dynamically split / merge the Gaussians in the current 3D Gaussian set under the condition of minimizing the image photometric loss, so as to generate an optimized current 3D Gaussian set. For the optimized current 3D Gaussian set, the SlimMLP network samples according to a normal distribution within the 3σ neighborhood of each Gaussian to obtain several sampling points; For any sampling point x, its local relative coordinates With Gaussian central features As input, the output is the calculation result based on all sampled points. ,in, for The square root of the eigenvalue, the calculation result The expression is: In the formula, For color residue, For geometric displacement residuals, This is the covariance correction factor; The iterative verification unit performs geometric fusion and color fusion on each Gaussian in the optimized current 3D Gaussian set based on the calculation results, including: For each Gaussian, perform geometric fusion, and generate a corrected covariance based on the geometric displacement residuals and covariance correction coefficients in the calculation results. The calculation formula is as follows: ； Perform color fusion on each Gaussian and generate a corrected color residual based on the color residual in the calculation results. The calculation formula is as follows: In the formula, For dynamic weights, j represents other Gaussians in the neighborhood related to the color fusion of the i-th Gaussian, and J is the number of related Gaussians set. Based on the joint loss function, and according to the calculation results, the joint loss value is calculated from the corrected color residual and the minimum image luminance loss, wherein the expression of the joint loss function is: In the formula, For the joint loss value, To determine the minimum image photometric loss when optimizing the current 3D Gaussian set, and These are the edge sharpening weights and residual regularization weights, respectively. and These are the edge sharpening loss and the residual regularization term, respectively. This is the set of sampling points where the grayscale / color of the current 3D Gaussian set changes drastically after optimization. Represents an L1 normal form function; Using the minimum joint loss value as a constraint, the network parameters are... and Update; Iterative optimization is performed based on the condition that the joint loss value converges to the convergence threshold. When the joint loss value converges to the convergence threshold, the three-dimensional sub-model of the image block with local coordinates and the corresponding feature point cloud are output. Otherwise, the current 3D Gaussian set for the next iteration is generated using the geometric displacement residual generated in this iteration, the corrected covariance, and the corrected color residual.

5. The method according to claim 4, characterized in that, For any overlapping region of adjacent 3D sub-models, based on their corresponding feature point clouds, boundary fusion is performed by rigid alignment of rigid parts and deformation constraints of non-rigid parts to determine the first coordinate transformation value from block to block, including: For the rigid portions in the overlapping regions of any two adjacent 3D sub-models a and b, rigid alignment is performed based on their corresponding feature point clouds, where: Extract the first FPFH feature descriptor from the feature point cloud of the rigid portion; Based on the first FPFH feature descriptor, feature matching is performed to obtain the first set of point pairs; For the point pairs in the first set of point pairs, the second set of point pairs is determined by the Random Sampling Consensus (RANSAC) algorithm. The corresponding point pair determined when the rigid body transformation value of the corresponding point pair in the second point pair set is the minimum is taken as the rigid alignment matching point pair; For the non-rigid portion of the overlapping region of any adjacent 3D sub-models a and b, deformation constraints are applied based on their corresponding feature point clouds, where: Extract the second FPFH feature descriptor from the feature point cloud of the non-rigid portion; Feature matching is performed based on the second FPFH feature descriptor to obtain a third set of point pairs, wherein the total number of point pairs in the third set is N, and the point pair elements are represented as ( ; Based on the elastic deformation model, determine the third point as any point corresponding to the three-dimensional sub-model a in the set. After non-rigid deformation, a fourth set of point pairs is generated, where the elements of the fourth set of point pairs are represented as ( The expression for the elastic deformation model is: In the formula, It is the i-th point after non-rigid deformation, and K is the total number of preset control points in the non-rigid part of the overlapping region corresponding to the 3D sub-model a. These are radial basis functions used to calculate the radial basis function values ​​between point x and the j-th control point. The displacement of the j-th control point; Using the minimum value of the deformation constraint function as a condition, non-rigid deformation matching point pairs are determined according to the fourth set of point pairs, where the expression of the deformation constraint function is: In the formula, The deformation regularization weights are determined based on empirical tuning. Based on the spatial continuity of the block division, the first coordinate transformation result of the block transformation corresponding to the three-dimensional sub-models a and b is determined according to the rigid alignment matching point pairs and the non-rigid deformation matching point pairs.

6. The method according to claim 5, characterized in that, Based on a custom global graph optimization function, a globally integrated 3D model of the power grid scene is generated according to the results of the first and second coordinate transformations. The expression of the custom global graph optimization function is as follows: ， In the formula, edges is the set of adjacent blocks. This is the result of the first coordinate transformation. and This is the result of the second coordinate transformation. Denotes the F-norm of a matrix. This indicates taking the square of the F-norm; By solving the problem with the constraint of minimizing the value of the custom global graph optimization function, a globally integrated 3D model of the power grid scenario can be obtained.

7. A real-scene reconstruction system for power grid equipment and facilities based on AI visual 3D reconstruction, characterized in that, The system includes: The data acquisition module is used to acquire initial scene image data of the power grid from multiple data sources; An initial processing module is used to perform GPS coordinate correction and point cloud preprocessing on the initial scene image data to generate effective scene image data with global coordinates. The equipment and facilities in the effective scene image data are divided into corresponding categories, including rigid and non-rigid categories. The block partitioning module is used to adaptively partition the effective scene image data into blocks based on a custom device density entropy, generating several image blocks, including: Step 1201: Using the towers in the effective scene image data as initial anchor points, construct several initial blocks with custom length * width. Step 1202: For the i-th block set, calculate the device density entropy of each block based on the custom device density entropy. Each block contains equipment and facilities categorized into rigid and non-rigid parts according to their corresponding categories. When i=1, the elements in the block set are the initial blocks. The formula for calculating the device density entropy H is: Among them, P k The percentage of the k-th type of device within a unit volume of a block-mapped 3D space; M is the total number of device types in the effective scene image data. Step 1203: For all blocks in the i-th block set whose device density entropy satisfies the adaptive partitioning criterion, perform adaptive partitioning according to the adaptive partitioning rules to generate 2 sub-blocks; Step 1204: After adaptive block partitioning is completed, an updated set of the i-th block set is generated, wherein the updated set includes sub-blocks generated after adaptive block partitioning of the blocks in the i-th block set and blocks that have not undergone adaptive block partitioning. Step 1205: When the device density entropy of any two blocks in the updated set satisfies the block partitioning convergence rule, stop the block partitioning; otherwise, let i = i + 1, and let the i-th block set be the updated set of the (i-1)-th block set, and return to step 1203. The parallel reconstruction module is used to perform parallel 3D reconstruction using an explicit-implicit coupled reconstruction model, taking the sparse point cloud corresponding to the image block as input, and outputting a 3D sub-model with local coordinates and the corresponding feature point cloud for each image block. The boundary fusion module is used to perform boundary fusion on the overlapping areas of any two adjacent 3D sub-models based on their corresponding feature point clouds, through rigid alignment of rigid parts and deformation constraints of non-rigid parts, to determine the first coordinate transformation result from one 3D sub-model to its adjacent 3D sub-models. The result output module is used to generate a globally integrated three-dimensional model of the power grid scene based on the first coordinate transformation result and the second coordinate transformation result. The second coordinate transformation result is the global coordinate of the three-dimensional sub-model generated based on the correspondence between the local coordinates and the global coordinates of each three-dimensional sub-model.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-6.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the steps of the method according to any one of claims 1-6.

Images