A Method for Surface Modeling of Porcelain-Shelled Substations Based on UAVs and Lightweight 3D Gaussian Modeling

Abstract

This invention discloses a method for surface modeling of porcelain-insulated substation equipment based on UAVs and lightweight 3D Gaussian compression, comprising: determining the UAV inspection route; taking multi-view photos of the porcelain-insulated substation equipment with the UAV to obtain an equipment image set; performing noise reduction processing and camera pose estimation on the equipment image set; and obtaining surface depth information of the equipment image set based on the SDF function; generating a dense point cloud based on the equipment images and corresponding camera pose information; constructing a 3D Gaussian model based on the dense point cloud; and obtaining the rendered image; compressing the 3D Gaussian model using a lightweight Gaussian compression method; establishing a neural implicit surface reconstruction model; constructing a composite loss function; and training and optimizing the reconstruction model based on the composite loss function. This invention introduces lightweight Gaussian compression technology into the implicit surface reconstruction method, achieving efficient compression and fast rendering of the scene, significantly reducing storage requirements, and improving the modeling accuracy of the equipment surface and the ability to capture complex surface details.

Description

Technical Field

[0001] This field relates to the area of ​​3D reconstruction, and more particularly to a method for modeling the surface of ceramic-insulated substations based on UAVs and lightweight 3D Gaussian. Background Technology

[0002] In modern power systems, substation equipment is a critical facility, and the efficiency of its management and monitoring directly affects the safety and stability of power supply. Porcelain-insulated substation equipment is widely used in power systems due to its superior insulation performance. However, during long-term operation, the surface of porcelain-insulated equipment is susceptible to environmental pollution, crack formation, and material degradation. If these potential problems are not detected and addressed in a timely manner, they will seriously threaten the stable operation of the power system. In particular, surface defects of porcelain-insulated equipment, such as cracks and dirt accumulation, not only reduce its insulation performance but may also directly lead to equipment failure, posing a significant risk to power supply. Therefore, exploring an efficient and reliable detection method to achieve early warning of surface defects in porcelain-insulated equipment has become an urgent technical challenge.

[0003] Currently, drones are increasingly used in substation inspections. However, accurate environmental models are crucial for drone inspections to ensure obstacle avoidance and equipment safety. Traditional modeling methods, such as oblique photogrammetry and viewpoint-based composite rendering, often fall short of ideal modeling accuracy when dealing with the complex and ever-changing scenes of substations. This not only limits the efficiency and safety of drone inspections but also affects the accurate detection of surface defects in ceramic bushing equipment. Existing 3D modeling methods, such as neural radiation fields and 3D Gaussian sputtering, still have many limitations when applied to substation scenarios. Neural radiation fields may not capture sufficient detail when handling complex scenes and are computationally and rendering-intensive; while 3D Gaussian sputtering requires significant storage space to preserve motion-recovery structure points, posing a considerable challenge for resource-constrained drone inspection systems. Summary of the Invention

[0004] To address the problems in the prior art, the present invention aims to provide a method for surface modeling of ceramic bushing substations based on UAVs and lightweight 3D Gaussian arrays, thereby solving the problems of insufficient modeling accuracy, high computational cost, and large storage space requirements in the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] This invention provides a method for surface modeling of porcelain bushing substations based on unmanned aerial vehicles (UAVs) and lightweight 3D Gaussian methods, including:

[0007] S1: Determine the drone inspection route, and the drone will execute the inspection route to take pictures of the porcelain bushing substation from multiple perspectives to obtain video data.

[0008] S2: Construct a device image set based on video data, perform noise reduction processing and camera pose estimation on the device image set, and obtain the surface depth information of the device image set based on the SDF function;

[0009] S3: Generate a dense point cloud based on the device images and their corresponding camera pose information in the denoised device image set, construct a 3D Gaussian based on the dense point cloud, and obtain the rendered device image.

[0010] S4: Compress the 3D Gaussian using the lightweight Gaussian method to prune redundant Gaussians;

[0011] S5: Establish a neural implicit surface reconstruction model and perform regularization processing on the compressed 3D Gaussian surface.

[0012] S6: Construct a composite loss function, and train and optimize the reconstruction model based on the composite loss function;

[0013] The composite loss function includes the SDF gradient constraint function, the point cloud constraint function, and the color loss function.

[0014] Optionally, step S1 includes the following specific steps:

[0015] Based on the images of the surrounding environment of the porcelain bushing substation collected by the UAV, the environmental images are analyzed, and the route of the UAV inspection task is planned according to the analysis results.

[0016] The drone executes the inspection route and uses its onboard camera to capture video data of the porcelain-insulated substation equipment from multiple angles.

[0017] Optionally, step S2 includes the following specific steps:

[0018] S201: Split the continuous video data into single frames to obtain a continuous static device image set including multiple frames;

[0019] S202: Obtain the device image of the current frame and compare it with the device image of the previous frame to locate the differences between the two frames, identify and optimize noise points in the device image;

[0020] S203: Periodically calculate the noise level of the device image in the current frame. If the noise level is detected to exceed the preset threshold, reduce the influence of the Gaussian point that generates noise by backpropagating its corresponding negative gradient value to reduce the noise.

[0021] S204: Based on the structure-of-motion method, estimate the camera pose corresponding to the device image in the device image set to obtain the camera pose information corresponding to the device image;

[0022] S205: Calculate the device image set based on the SDF function to obtain the surface depth information of the device image.

[0023] Optionally, step S3 includes the following specific steps:

[0024] S301: Generate dense point cloud based on the denoised device image and the corresponding camera pose information;

[0025] S302: Based on each point in the dense point cloud, construct and initialize a 3D Gaussian, and adjust the 3D Gaussian to adapt to different viewpoints through rotation and scaling transformations;

[0026] The attribute representations of each 3D Gaussian include center position, covariance, color, and opacity;

[0027] S303: View transformation is performed using sputtering technology to calculate a new covariance matrix, so as to project the three-dimensional Gaussian onto the two-dimensional image plane and obtain the two-dimensional Gaussian distribution;

[0028] S304: For each pixel in a 2D image, use the Gaussian rendering formula to calculate the color and opacity of all 2D Gaussians within that pixel to obtain the final color of that pixel, and output the rendered 2D image.

[0029] Optionally, the three-dimensional Gaussian in S302 has the following functional expression:

[0030]

[0031]

[0032] In the formula, m Let be the coordinates of any point in three-dimensional space; n The center of the three-dimensional Gaussian; Σ The covariance matrix is ​​used to control the size, shape, and orientation of the three-dimensional Gaussian. R It is a rotation matrix; S It is a diagonal matrix; T The table represents the transpose symbol;

[0033] The new covariance matrix in S303 has the following functional expression:

[0034]

[0035] In the formula, For the new covariance matrix; JLet be the Jacobian matrix of the projection transformation; W This is the viewpoint transformation matrix;

[0036] The Gaussian rendering formula in S304 is expressed as follows:

[0037]

[0038] In the formula, C The final pixel color; For the first j A two-dimensional Gaussian color; For the first j A two-dimensional Gaussian transparency; N The number of two-dimensional Gaussians that affect the color of this pixel. The cumulative transparency factor represents the number of transparency factors from the first to the second. j -1 is the cumulative product of the transparency of a 2D Gaussian, used to adjust the contribution of each 2D Gaussian to the final pixel color.

[0039] Optionally, step S4 includes the following specific steps:

[0040] S401: Calculate the global saliency score of each 3D Gaussian to determine their contribution to scene reconstruction;

[0041] S402: Calculate the importance score of each three-dimensional Gaussian to the noise level, and determine whether each three-dimensional Gaussian should be compressed to prune redundant Gaussians based on the importance score and the global significance score.

[0042] S403: In the process of 3D Gaussian compression, vector quantization is introduced to process the spherical harmonic function coefficients of 3D Gaussian in order to further compress the model size and retain key visual information;

[0043] S404: By information distillation, the coefficients of spherical harmonic functions in an uncompressed 3D Gaussian are reduced from higher to lower orders to reduce the complexity of the model.

[0044] Optionally, step S402 includes the following specific steps:

[0045] Construct a convolutional neural network and analyze the contribution of each Gaussian point to the noise level based on the convolutional neural network to obtain the importance score of each three-dimensional Gaussian.

[0046] All three-dimensional Gaussians are sorted according to the calculated global significance score, and the three-dimensional Gaussians with high global significance scores are retained.

[0047] For a 3D Gaussian with a low global significance score, determine whether to compress the 3D Gaussian based on its importance score. If the importance score of the 3D Gaussian is high, it is not compressed, and the 3D Gaussian is optimized to make up for the details lost due to compression. If the importance score of the 3D Gaussian is low, it is compressed.

[0048] Optionally, step S5 includes the following specific steps:

[0049] S501: Based on multilayer perceptron and SDF function, construct a neural implicit surface reconstruction model and predict the normal vector of the reconstructed model surface;

[0050] S502: Scale regularize the 3D Gaussian by minimizing the scaling factor to compress the 3D Gaussian into a plane, and adjust the direction of the scaling factor to align the normal vector of the plane Gaussian with the normal vector predicted by the neural implicit surface reconstruction model, thereby improving the accuracy of surface reconstruction.

[0051] Optionally, step S6 includes the following specific steps:

[0052] S601: Divide the images in the device image set into a training set and a test set;

[0053] S602: Based on the device images before and after rendering, construct a color loss function to measure the difference between the device images before and after rendering;

[0054] Based on the Eikonal Loss function, the gradient of the SDF function is constrained, and an SDF gradient constraint function is constructed to forcibly constrain the gradient norm of the SDF function, thereby improving the surface reconstruction quality.

[0055] A point cloud constraint function is constructed based on the dense point cloud constraint SDF function to constrain the surface depth information of the device image to approach zero;

[0056] S603: Construct a composite loss function based on the color loss function, SDF gradient constraint function, and point cloud constraint function, and assign initial weights to the different functions in the composite loss function;

[0057] S604: Train the neural implicit surface reconstruction model based on the training set, calculate the error value through the composite loss function and backpropagate to update the parameters of the neural implicit surface reconstruction model, and continuously iterate the training.

[0058] S605: When the number of training iterations of the neural implicit surface reconstruction model reaches the preset number of tests, the neural implicit surface reconstruction model is tested for performance based on the test set; if the model performance is qualified, the training of the neural implicit surface reconstruction model is completed; if the model performance is unqualified, the weights of different functions in the composite loss function are adjusted, and the process returns to step S604.

[0059] Optionally, the expression for the SDF gradient constraint function in S602 is:

[0060]

[0061] In the formula, Here is the SDF gradient constraint function; S It is the total number of sample points; It is at point The gradient of the surface depth function; It is the norm of the gradient;

[0062] The expression for the composite loss function in S603 is as follows:

[0063]

[0064] In the formula, It is a composite loss function; Here is the color loss function; Point cloud constraint functions; The weights are for the color loss function; These are the weights of the SDF gradient constraint function; The weights are the constraint functions for the point cloud.

[0065] The beneficial effects of the embodiments provided by the present invention include:

[0066] This invention introduces a lightweight Gaussian compression technique into a three-dimensional Gaussian sputtering-guided implicit surface reconstruction method, achieving efficient compression and fast rendering of the scene, significantly reducing storage requirements, and improving the modeling accuracy of porcelain bushing substation surfaces and the ability to capture complex surface details.

[0067] This invention introduces an attention mechanism into lightweight Gaussian compression technology. It uses a deep learning convolutional neural network to analyze the importance score of each Gaussian point to the noise level and compares it with the global saliency score of Gaussian to determine whether to compress the Gaussian point. This reduces noise interference in areas where the compressed image has a large impact and optimizes the Gaussian distribution parameters, thereby improving the modeling accuracy of complex geometric shapes and surface features.

[0068] In this invention, the neural implicit surface reconstruction model is trained using color loss function, SDF gradient constraint function and point cloud constraint function as constraints to ensure that the reconstruction model focuses on true surface optimization during training, thereby improving the precision of complex surface reconstruction and the realism of rendered images. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 A flowchart illustrating a method for modeling the surface of porcelain-insulated substations based on a drone and a lightweight 3D Gaussian according to an embodiment of this specification is shown.

[0071] Figure 2 A structural block diagram of a surface modeling system for porcelain bushing substations based on UAVs and lightweight 3D Gaussian is shown in an embodiment of this specification. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0073] However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of embodiments of this disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this disclosure.

[0074] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The term "comprising" as used herein indicates the presence of features, steps, or operations, but does not exclude the presence or addition of one or more other features. It should be noted that all terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be understood that the terms used herein should be interpreted in a manner consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0075] Example 1

[0076] like Figure 1As shown, this embodiment provides a method for surface modeling of porcelain bushing substations based on UAVs and lightweight 3D Gaussian arrays, including:

[0077] S1: Determine the drone inspection route, and the drone will execute the inspection route to take pictures of the porcelain bushing substation from multiple perspectives to obtain video data.

[0078] For example, S1 includes the following specific steps:

[0079] Based on the images of the surrounding environment of the porcelain bushing substation collected by the UAV, the environmental images are analyzed, and the route of the UAV inspection task is planned according to the analysis results.

[0080] When using drones to collect images of the surrounding environment of porcelain-insulated substations, it is necessary to determine the scanning range and scanning path in advance and conduct a comprehensive inspection of the environment surrounding the porcelain-insulated substations.

[0081] The drone executes the inspection route and uses its onboard camera to capture video data of the porcelain-insulated substation equipment from multiple angles.

[0082] S2: Construct a device image set based on video data, perform noise reduction processing and camera pose estimation on the device image set, and obtain the surface depth information of the device image set based on the SDF function;

[0083] For example, S2 includes the following specific steps:

[0084] S201: Split the continuous video data into single frames to obtain a continuous static device image set including multiple frames;

[0085] S202: Obtain the device image of the current frame and compare it with the device image of the previous frame to locate the differences between the two frames, identify and optimize noise points in the device image;

[0086] S203: Periodically calculate the noise level of the device image in the current frame. If the noise level is detected to exceed the preset threshold, reduce the influence of the Gaussian point that generates noise by backpropagating its corresponding negative gradient value to reduce the noise.

[0087] S204: Based on the structure-of-motion method, estimate the camera pose corresponding to the device image in the device image set to obtain the camera pose information corresponding to the device image;

[0088] S205: Calculate the device image set based on the SDF function to obtain the surface depth information of the device image.

[0089] In this embodiment, the surface depth information of the porcelain bushing substation in the equipment image is obtained through the SDF (Symbol Distance Field) function, so as to recover the three-dimensional geometric structure of the porcelain bushing substation from the equipment image, which is beneficial to creating an accurate three-dimensional model of the porcelain bushing substation.

[0090] S3: Generate a dense point cloud based on the device images and their corresponding camera pose information in the denoised device image set; construct a 3D Gaussian cloud based on the dense point cloud; and obtain the rendered device image.

[0091] For example, S3 includes the following specific steps:

[0092] S301: Generate dense point cloud based on the device image and corresponding camera pose information after noise reduction;

[0093] In this embodiment, the correspondence between adjacent images is first found through a feature matching algorithm, and the sparse point cloud and camera trajectory of the scene are reconstructed using the structure of motion reconstruction algorithm. The structure of motion reconstruction algorithm will generate a sparse 3D point cloud based on the feature matching results. Then, a dense reconstruction is performed on the basis of the sparse 3D point cloud to generate a dense point cloud.

[0094] In some embodiments, the surface depth information of the device image in step S205 is involved in the generation process of the dense point cloud described above.

[0095] S302: Based on each point in the dense point cloud, construct and initialize a 3D Gaussian, and adjust the 3D Gaussian to adapt to different viewpoints through rotation and scaling transformations;

[0096] The attributes of each 3D Gaussian include center position, covariance, color, and opacity.

[0097] In some embodiments, the three-dimensional Gaussian in S302 has the following function expression:

[0098]

[0099]

[0100] In the formula, m Let be the coordinates of any point in three-dimensional space; n The center of the three-dimensional Gaussian; Σ The covariance matrix is ​​used to control the size, shape, and orientation of the three-dimensional Gaussian. R It is a rotation matrix; S It is a diagonal matrix; T The table represents the transpose symbol;

[0101] S303: View transformation is performed using sputtering technology to calculate a new covariance matrix, so as to project the three-dimensional Gaussian onto the two-dimensional image plane and obtain the two-dimensional Gaussian distribution;

[0102] In some embodiments, the new covariance matrix in S303 has the following functional expression:

[0103]

[0104] In the formula, For the new covariance matrix; J Let be the Jacobian matrix of the projection transformation; W This is the viewpoint transformation matrix;

[0105] S304: For each pixel in a 2D image, use the Gaussian rendering formula to calculate the color and opacity of all 2D Gaussians within that pixel to obtain the final color of that pixel, and output the rendered 2D image.

[0106] In some embodiments, the Gaussian rendering formula in S304 is expressed as follows:

[0107]

[0108] In the formula, C The final pixel color; For the first j A two-dimensional Gaussian color; For the first j A two-dimensional Gaussian transparency; N The number of two-dimensional Gaussians that affect the color of this pixel. The cumulative transparency factor represents the cumulative transparency product of the first to the (j-1)th two-dimensional Gaussian, used to adjust the contribution of each two-dimensional Gaussian to the final pixel color.

[0109] S4: Compress the 3D Gaussian using the lightweight Gaussian method to prune redundant Gaussians;

[0110] For example, S4 includes the following specific steps:

[0111] S401: Calculate the global saliency score of each 3D Gaussian to determine their contribution to scene reconstruction;

[0112] In some embodiments, the functional expression for the global saliency score in S401 is:

[0113]

[0114] In the formula, The global significance score; M Number of device images; H andK These are the height and width of the device image, respectively; p For pixels, q is the Gaussian exponent; t is an indicator function used to determine whether the Gaussian curve intersects with the ray, thus changing the fraction value. Let the opacity be Gaussian. The adaptive function measures the dimension of the Gaussian volume, and the global saliency score reflects the degree to which the Gaussian contributes to the final rendered image.

[0115] S402: Calculate the importance score of each three-dimensional Gaussian to the noise level, and determine whether each three-dimensional Gaussian should be compressed to prune redundant Gaussians based on the importance score and the global significance score.

[0116] In some embodiments, S403 includes the following specific steps:

[0117] Construct a convolutional neural network and analyze the contribution of each Gaussian point to the noise level based on the convolutional neural network to obtain the importance score of each three-dimensional Gaussian.

[0118] All three-dimensional Gaussians are sorted according to the calculated global significance score, and the three-dimensional Gaussians with high global significance scores are retained.

[0119] In this embodiment, the global significance score of the three-dimensional Gaussian is determined based on whether it is higher than a preset score standard. If it is, the global significance score of the three-dimensional Gaussian is considered to be high, and the three-dimensional Gaussian is retained. If not, the global significance score of the three-dimensional Gaussian is considered to be low, and the process proceeds to the next step.

[0120] For a 3D Gaussian with a low global significance score, determine whether to compress the 3D Gaussian based on its importance score. If the importance score of the 3D Gaussian is high, it is not compressed, and the 3D Gaussian is optimized to make up for the details lost due to compression. If the importance score of the 3D Gaussian is low, it is compressed.

[0121] In this embodiment, the three-dimensional Gaussian model with low global saliency score and high importance score needs to undergo joint attribute adjustment, including adjusting their mean, variance and weight, to adapt to and compensate for the Gaussian loss and surface depth information loss caused by pruning, and to ensure the stability and accuracy of the overall model.

[0122] S403: In the process of 3D Gaussian compression, vector quantization is introduced to process the spherical harmonic function coefficients of 3D Gaussian in order to further compress the model size and retain key visual information;

[0123] S404: By information distillation, the coefficients of spherical harmonic functions in an uncompressed 3D Gaussian are reduced from higher to lower orders to reduce the complexity of the model.

[0124] In this embodiment, the vast majority of 3D Gaussian data in the uncompressed 3D Gaussian model consists of higher-order spherical harmonic coefficients, leading to increased storage overhead and computational complexity. To achieve a balance between disk storage efficiency and scene reflection quality, information distillation is used to transfer higher-order spherical harmonic coefficients to lower-order spherical harmonics, moving information to a more compact representation while maintaining minimal changes to the scene's appearance and reducing model complexity. Specifically, information distillation involves extracting knowledge from a teacher model using full-order spherical harmonic coefficients and transferring it to a student model with truncated features and lower-order distilled, compressed spherical harmonic coefficients. Through this information distillation process, the visual quality of the scene can be maintained while reducing model size.

[0125] Specifically, the lightweight Gaussian method proposed in this embodiment achieves significant reduction in data redundancy in 3D Gaussian representation while maintaining visual quality through three key steps: Gaussian trimming and restoration, distillation and compression of spherical harmonic function coefficients, and Gaussian attribute vector quantization. This improves rendering and storage efficiency. It provides an effective solution for novel perspective synthesis, particularly in handling large-scale scenes such as substations, where it can significantly enhance performance.

[0126] S5: Establish a neural implicit surface reconstruction model and perform regularization processing on the compressed 3D Gaussian surface.

[0127] For example, S5 includes the following specific steps:

[0128] S501: Based on multilayer perceptron and SDF function, construct a neural implicit surface reconstruction model and predict the normal vector of the reconstructed model surface;

[0129] In this embodiment, the neural implicit surface reconstruction model uses a multilayer perceptron (MLP) as its core architecture and utilizes the SDF function to implicitly represent three-dimensional geometric information.

[0130] S502: Scale regularize the 3D Gaussian by minimizing the scaling factor to compress the 3D Gaussian into a plane, and adjust the direction of the scaling factor to align the normal vector of the plane Gaussian with the normal vector predicted by the neural implicit surface reconstruction model, thereby improving the accuracy of surface reconstruction.

[0131] S6: Construct a composite loss function, and train and optimize the reconstruction model based on the composite loss function;

[0132] For example, S6 includes the following specific steps:

[0133] S601: Divide the images in the device image set into a training set and a test set;

[0134] S602: Based on the device images before and after rendering, construct a color loss function to measure the difference between the device images before and after rendering;

[0135] In some embodiments, the expression for the color loss function in S602 is:

[0136]

[0137] In the formula, Here is the color loss function; C The original image color; The colors of the rendered image;

[0138] Based on the Eikonal Loss function, the gradient of the SDF function is constrained, and an SDF gradient constraint function is constructed to forcibly constrain the gradient norm of the SDF function, thereby improving the surface reconstruction quality.

[0139] In some embodiments, the expression for the SDF gradient constraint function in S602 is:

[0140]

[0141] In the formula, Here is the SDF gradient constraint function; S It is the total number of sample points; It is at point The gradient of the surface depth function; It is the norm of the gradient;

[0142] A point cloud constraint function is constructed based on the dense point cloud constraint SDF function to constrain the surface depth information of the device image to approach zero;

[0143] In some embodiments, the expression for the point cloud constraint function in S602 is:

[0144]

[0145] In the formula, Point cloud constraint functions; At point The predicted SDF value at point represents the point Distance to the nearest surface; It is a point in a dense point cloud;

[0146] Among them, when When , it indicates that the point is outside the surface; when When negative, it indicates that the point is inside the surface; when When the value is 0, it means that the point is exactly on the surface. In this embodiment, the SDF value at the point cloud position is close to zero through the point cloud constraint function, that is, these points should be located on the surface, thereby performing geometric constraints.

[0147] S603: Construct a composite loss function based on the color loss function, SDF gradient constraint function, and point cloud constraint function, and assign initial weights to the different functions in the composite loss function;

[0148] In some embodiments, the expression for the composite loss function in S603 is:

[0149]

[0150] In the formula, It is a composite loss function; Weights for the color loss function; These are the weights of the SDF gradient constraint function; The weights are the constraint functions for the point cloud.

[0151] S604: Train the neural implicit surface reconstruction model based on the training set, calculate the error value through the composite loss function and backpropagate to update the parameters of the neural implicit surface reconstruction model, and continuously iterate the training.

[0152] In this embodiment, the neural implicit surface reconstruction model selects ADAM as the training optimizer, and the adaptive learning rate is selectively set according to the requirements.

[0153] S605: When the number of training iterations of the neural implicit surface reconstruction model reaches the preset number of tests, the neural implicit surface reconstruction model is tested for performance based on the test set; if the model performance is qualified, the training of the neural implicit surface reconstruction model is completed; if the model performance is unqualified, the weights of different functions in the composite loss function are adjusted, and the process returns to step S604.

[0154] Example 2

[0155] like Figure 2 As shown, this embodiment provides a surface modeling system 200 for porcelain bushing substations based on UAVs and lightweight 3D Gaussian, comprising:

[0156] The image acquisition module 201 is used to determine the inspection route of the UAV. The UAV performs the inspection route and takes pictures of the porcelain bushing substation from multiple perspectives to obtain video data.

[0157] The image processing module 202 is used to construct a device image set based on video data, perform noise reduction processing and camera pose estimation on the device image set, and obtain the surface depth information of the device image set based on the SDF function.

[0158] Gaussian generation module 203 is used to generate a dense point cloud based on the denoised device image and the corresponding camera pose information, construct a three-dimensional Gaussian based on the dense point cloud, and obtain the rendered device image.

[0159] Gaussian compression module 204 is used to compress three-dimensional Gaussians using a lightweight Gaussian method to prune redundant Gaussians;

[0160] Model building module 205 is used to build a neural implicit surface reconstruction model and to perform regularization processing on the compressed 3D Gaussian.

[0161] The model training module 206 is used to construct a composite loss function, and to train and optimize the reconstruction model based on the composite loss function.

[0162] Example 3

[0163] This embodiment provides an electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor;

[0164] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory and may also include non-transitory...

[0165] The memory includes, for example, at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0166] The non-transient software program and instructions required to implement the surface modeling method for porcelain-insulated substations based on UAVs and lightweight 3D Gaussian vectors in the above embodiments are stored in memory. When executed by a processor, the surface modeling method for porcelain-insulated substations based on UAVs and lightweight 3D Gaussian vectors in the above embodiments is executed. For example, the method described above is executed. Figure 1 The method steps S1 to S6 are described in the text.

[0167] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Example 4

[0169] This embodiment provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute a method for modeling the surface of porcelain bushing substations based on UAVs and lightweight 3D Gaussian as described in Embodiment 1.

[0170] It should be noted that the computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0171] In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this embodiment, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0172] The computer-readable storage medium described above can be used to write computer programs for executing this embodiment in one or more programming languages ​​or combinations thereof. These programming languages ​​include object-oriented programming languages ​​such as Python, Java, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] In summary, this embodiment introduces lightweight Gaussian compression technology into the implicit surface reconstruction method guided by 3D Gaussian sputtering, achieving efficient compression and fast rendering of the scene, effectively pruning redundant Gaussians, reducing storage space requirements and computing resource consumption, and improving the modeling accuracy of porcelain bushing substation surfaces and the ability to capture complex surface details.

[0174] This embodiment utilizes drone inspection technology, which uses drones to photograph porcelain bushing substations from multiple perspectives. This allows for more detailed monitoring of the equipment, comprehensively capturing surface features and promptly identifying potential problems such as cracks and dirt accumulation. This enables early warning and handling, reducing the risk of equipment failure.

[0175] This embodiment introduces an attention mechanism into the lightweight Gaussian compression technique. It uses a deep learning convolutional neural network to analyze the importance score of each Gaussian point to the noise level and compares it with the global saliency score of Gaussian to determine whether to compress the Gaussian point. This reduces noise interference in areas where the compressed image has a large impact and optimizes the Gaussian distribution parameters, thereby improving the modeling accuracy of complex geometric shapes and surface features.

[0176] In this embodiment, the neural implicit surface reconstruction model is trained using color loss function, SDF gradient constraint function and point cloud constraint function as constraints to ensure that the reconstruction model focuses on true surface optimization during training, thereby improving the precision of complex surface reconstruction and the realism of rendered images.

[0177] The above description is merely a preferred embodiment of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the present invention is defined by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention.

Claims

1. A method for surface modeling of porcelain bushing substations based on UAVs and lightweight 3D Gaussian vectoring, characterized in that, include: S1: Determine the drone inspection route, and the drone will execute the inspection route to take pictures of the porcelain bushing substation from multiple perspectives to obtain video data. S2: Construct a device image set based on video data, perform noise reduction processing and camera pose estimation on the device image set, and obtain the surface depth information of the device image set based on the SDF function; S3: Generate a dense point cloud based on the device images and their corresponding camera pose information in the denoised device image set, construct a 3D Gaussian based on the dense point cloud, and obtain the rendered device image. S4: Compress the 3D Gaussian using the lightweight Gaussian method to prune redundant Gaussians; S5: Establish a neural implicit surface reconstruction model and perform regularization processing on the compressed 3D Gaussian surface. S6: Construct a composite loss function, and train and optimize the reconstruction model based on the composite loss function; The composite loss function includes the SDF gradient constraint function, point cloud constraint function, and color loss function; S4 includes the following specific steps: S401: Calculate the global saliency score of each 3D Gaussian to determine their contribution to scene reconstruction; S402: Calculate the importance score of each three-dimensional Gaussian to the noise level, and determine whether each three-dimensional Gaussian should be compressed to prune redundant Gaussians based on the importance score and the global significance score. S403: In the process of 3D Gaussian compression, vector quantization is introduced to process the spherical harmonic function coefficients of 3D Gaussian in order to further compress the model size and retain key visual information; S404: By information distillation, the coefficients of spherical harmonic functions in an uncompressed 3D Gaussian are reduced from higher order to lower order to reduce the complexity of the model. S402 includes the following specific steps: Construct a convolutional neural network and analyze the contribution of each Gaussian point to the noise level based on the convolutional neural network to obtain the importance score of each three-dimensional Gaussian. All three-dimensional Gaussians are sorted according to the calculated global significance score, and the three-dimensional Gaussians with high global significance scores are retained. For a 3D Gaussian with a low global significance score, determine whether to compress the 3D Gaussian based on its importance score. If the importance score of the 3D Gaussian is high, it is not compressed, and the 3D Gaussian is optimized to make up for the details lost due to compression. If the importance score of the 3D Gaussian is low, it is compressed.

2. The method according to claim 1, characterized in that, S1 includes the following specific steps: Based on the images of the surrounding environment of the porcelain bushing substation collected by the UAV, the environmental images are analyzed, and the route of the UAV inspection task is planned according to the analysis results. The drone executes the inspection route and uses its onboard camera to capture video data of the porcelain-insulated substation equipment from multiple angles.

3. The method according to claim 1, characterized in that, S2 includes the following specific steps: S201: Split the continuous video data into single frames to obtain a continuous static device image set including multiple frames; S202: Obtain the device image of the current frame and compare it with the device image of the previous frame to locate the differences between the two frames, identify and optimize noise points in the device image; S203: Periodically calculate the noise level of the device image in the current frame. If the noise level is detected to exceed the preset threshold, reduce the influence of the Gaussian point that generates noise by backpropagating its corresponding negative gradient value to reduce the noise. S204: Based on the structure-of-motion method, estimate the camera pose corresponding to the device image in the device image set to obtain the camera pose information corresponding to the device image; S205: Calculate the device image set based on the SDF function to obtain the surface depth information of the device image.

4. The method according to claim 1, characterized in that, S3 includes the following specific steps: S301: Generate dense point cloud based on the denoised device image and the corresponding camera pose information; S302: Based on each point in the dense point cloud, construct and initialize a 3D Gaussian, and adjust the 3D Gaussian to adapt to different viewpoints through rotation and scaling transformations; The attribute representations of each 3D Gaussian include center position, covariance, color, and opacity; S303: View transformation is performed using sputtering technology to calculate a new covariance matrix, so as to project the three-dimensional Gaussian onto the two-dimensional image plane and obtain the two-dimensional Gaussian distribution; S304: For each pixel in a 2D image, use the Gaussian rendering formula to calculate the color and opacity of all 2D Gaussians within that pixel to obtain the final color of that pixel, and output the rendered 2D image.

5. The method according to claim 4, characterized in that, The three-dimensional Gaussian in S302 has the following function expression: In the formula, m is the coordinate of any point in three-dimensional space; n is the center of the three-dimensional Gaussian; Σ is the covariance matrix, used to control the size, shape and direction of the three-dimensional Gaussian; R is the rotation matrix; S is the diagonal matrix; T represents the transpose symbol. The new covariance matrix in S303 has the following functional expression: In the formula, Let J be the new covariance matrix; J be the Jacobian matrix of the projection transformation; and W be the view transformation matrix. The Gaussian rendering formula in S304 is expressed as follows: In the formula, C is the final pixel color; For the first j A two-dimensional Gaussian color; For the first j A two-dimensional Gaussian transparency; N The number of two-dimensional Gaussians that affect the color of this pixel; The cumulative transparency factor represents the value from the 1st to the 2nd. j- A product of the transparency of a 2D Gaussian is used to adjust the contribution of each 2D Gaussian to the final pixel color.

6. The method according to claim 1, characterized in that, S5 includes the following specific steps: S501: Based on multilayer perceptron and SDF function, construct a neural implicit surface reconstruction model and predict the normal vector of the reconstructed model surface; S502: Scale regularize the 3D Gaussian by minimizing the scaling factor to compress the 3D Gaussian into a plane, and adjust the direction of the scaling factor to align the normal vector of the plane Gaussian with the normal vector predicted by the neural implicit surface reconstruction model, thereby improving the accuracy of surface reconstruction.

7. The method according to claim 1, characterized in that, S6 includes the following specific steps: S601: Divide the images in the device image set into a training set and a test set; S602: Based on the device images before and after rendering, construct a color loss function to measure the difference between the device images before and after rendering; Based on the Eikonal Loss function, the gradient of the SDF function is constrained, and an SDF gradient constraint function is constructed to forcibly constrain the gradient norm of the SDF function, thereby improving the surface reconstruction quality. A point cloud constraint function is constructed based on the dense point cloud constraint SDF function to constrain the surface depth information of the device image to approach zero; S603: Construct a composite loss function based on the color loss function, SDF gradient constraint function, and point cloud constraint function, and assign initial weights to the different functions in the composite loss function. S604: Train the neural implicit surface reconstruction model based on the training set, calculate the error value through the composite loss function and backpropagate to update the parameters of the neural implicit surface reconstruction model, and continuously iterate the training. S605: When the number of training iterations of the neural implicit surface reconstruction model reaches the preset number of tests, the neural implicit surface reconstruction model is tested for performance based on the test set; if the model performance is qualified, the training of the neural implicit surface reconstruction model is completed; if the model performance is unqualified, the weights of different functions in the composite loss function are adjusted, and the process returns to step S604.

8. The method according to claim 7, characterized in that, The expression for the SDF gradient constraint function in S602 is as follows: In the formula, Here is the SDF gradient constraint function; S It is the total number of sample points; It is at point The gradient of the surface depth function; It is the norm of the gradient; The expression for the composite loss function in S603 is as follows: In the formula, It is a composite loss function; Here is the color loss function; Point cloud constraint functions; The weights are for the color loss function; These are the weights of the SDF gradient constraint function; The weights are the constraint functions for the point cloud.

Images