Surface modeling method for porcelain bushing power transformation equipment based on unmanned aerial vehicle and light three-dimensional Gaussian

Abstract

The invention discloses a porcelain bushing power transformation equipment surface modeling method based on an unmanned aerial vehicle and light three-dimensional Gaussian, and the method comprises the steps: determining an unmanned aerial vehicle inspection task route, shooting porcelain bushing power transformation equipment through the unmanned aerial vehicle in a multi-view manner, obtaining an equipment image set, carrying out the noise reduction processing of the equipment image set, and carrying out the camera pose estimation. Obtaining surface depth information of the equipment image set based on an SDF function; dense point clouds are generated based on the equipment images and the corresponding camera pose information, three-dimensional Gaussian is constructed according to the dense point clouds, and rendered images are obtained; compressing the three-dimensional Gaussian through a light Gaussian method; establishing a neural implicit surface reconstruction model; constructing a composite loss function, and training and optimizing the reconstruction model based on the composite loss function; according to the method, the lightweight Gaussian compression technology is introduced into the implicit surface reconstruction method, efficient compression and rapid rendering of the scene are achieved, the storage requirement is remarkably reduced, and the modeling precision of the equipment surface and the capturing capacity of complex surface details are improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction, and in particular to a surface modeling method of porcelain bushing transformer equipment based on unmanned aerial vehicle and light three-dimensional Gaussian. Background Art

[0002] In modern power systems, substation equipment is a key facility, and the efficiency of its management and monitoring is directly related to the safety and stability of power supply. Porcelain bushing substation equipment is widely used in power systems due to its excellent insulation performance. However, during long-term operation, the surface of porcelain bushing equipment is easily affected by factors such as environmental pollution, crack generation and material degradation. If these potential problems are not discovered and handled in time, they will seriously threaten the stable operation of the power system. In particular, defects on the surface of porcelain bushing equipment, such as cracks and dirt accumulation, will not only reduce its insulation performance, but may also directly lead to equipment failure, bringing huge risks to power supply. Therefore, exploring an efficient and reliable detection method to achieve early warning of surface defects of porcelain bushing equipment has become a technical problem that needs to be solved urgently.

[0003] At present, drone equipment is gradually becoming popular in substation inspections. However, drone equipment requires an accurate environmental model during the inspection process to ensure obstacle avoidance and equipment safety. Traditional modeling methods, such as oblique photography and perspective synthesis rendering-based technology, often find it difficult to achieve ideal modeling accuracy when dealing with complex and changeable scenes in substations. This not only limits the efficiency and safety of drone equipment inspections, but also affects the accurate detection of surface defects in porcelain sleeve equipment. Existing three-dimensional modeling methods, such as neural radiation fields and three-dimensional Gaussian sputtering, still have many limitations when applied to substation scenarios. Neural radiation fields may not be able to capture enough details when dealing with complex scenes, and the calculation and rendering costs are high; while three-dimensional Gaussian sputtering requires a large amount of storage space to save motion recovery structure points, which is a considerable challenge for resource-limited drone equipment inspection systems. Summary of the invention

[0004] In view of the above-mentioned problems in the prior art, the purpose of the present invention is to provide a surface modeling method of porcelain sleeve substation equipment based on drone and lightweight three-dimensional Gaussian, so as to solve the problems of insufficient modeling accuracy, high calculation cost and large storage space requirement in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: The embodiment of the present invention provides a surface modeling method of a porcelain bushing transformer equipment based on an unmanned aerial vehicle and a light three-dimensional Gaussian, comprising: S1: Determine the inspection mission route of the drone. The drone executes the inspection mission route and shoots the porcelain bushing substation equipment from multiple perspectives to obtain video data. S2: constructing a device image set based on the video data, performing noise reduction processing and camera pose estimation on the device image set, and obtaining surface depth information of the device image set based on the SDF function; S3: Generate a dense point cloud based on the device image and its corresponding camera pose information in the denoised device image set, construct a three-dimensional Gaussian based on the dense point cloud, and obtain a rendered device image; S4: compressing the 3D Gaussian by the light Gaussian method to prune redundant Gaussians; S5: Establish a neural implicit surface reconstruction model and regularize the compressed 3D Gaussian. S6: constructing a composite loss function, and training and optimizing the reconstruction model based on the composite loss function; The composite loss function includes an SDF gradient constraint function, a point cloud constraint function and a color loss function.

[0006] Optionally, S1 includes the following specific steps: The drone collects the surrounding environment images of the porcelain bushing substation equipment, analyzes the environmental images, and plans the drone inspection task route according to the analysis results; The drone executes the inspection mission route and uses the camera carried by the drone to shoot the porcelain bushing substation equipment from multiple angles to obtain video data.

[0007] Optionally, S2 includes the following specific steps: S201: Splitting the continuous video data into single frames to obtain a continuous static device image set including multiple frames; S202: Acquire the current frame device image, compare it with the previous frame device image, locate the difference between the two frames, and identify and optimize the noise points in the device image; S203: regularly calculating the noise level of the current frame device image, and if it is detected that the noise level exceeds a preset threshold, for the Gaussian point generating noise, reducing the influence of the point by back-propagating its corresponding negative gradient value, so as to reduce the noise; S204: estimating the camera pose corresponding to the device image in the device image set based on the structure-from-motion method to obtain the camera pose information corresponding to the device image; S205: Calculate a device image set based on an SDF function to obtain surface depth information of the device image.

[0008] Optionally, S3 includes the following specific steps: S301: Generate a dense point cloud based on the denoised device image and the corresponding camera pose information; S302: construct and initialize a three-dimensional Gaussian based on each point in the dense point cloud, and adjust the three-dimensional Gaussian to adapt to different viewing angles through rotation and scaling transformations; The attribute representation of each three-dimensional Gaussian includes center position, covariance, color, and opacity; S303: performing view transformation by using a sputtering technique, calculating a new covariance matrix, projecting the three-dimensional Gaussian on a two-dimensional image plane, and obtaining a two-dimensional Gaussian distribution; S304: For a pixel point of the two-dimensional image, use a Gaussian rendering formula to calculate the color and opacity of all two-dimensional Gaussians in the pixel point to obtain a final color of the pixel point, and output a rendered two-dimensional image.

[0009] Optionally, the three-dimensional Gaussian in S302 has a function expression as follows:

[0010]

[0011] In the formula, m is the coordinate of any point in three-dimensional space; n is the center of the three-dimensional Gaussian; Σ is the covariance matrix, which is used to control the size, shape and direction of the three-dimensional Gaussian; R is the rotation matrix; S is a diagonal matrix; T The table is the transposed symbol; The new covariance matrix in S303 has the following functional expression:

[0012] In the formula, is the new covariance matrix; J is the Jacobian matrix of the projection transformation; W is the perspective transformation matrix; The Gaussian rendering formula in S304 is expressed as:

[0013] In the formula, C is the final pixel color; For the j The color of a two-dimensional Gaussian; For the j The transparency of a two-dimensional Gaussian; N is the number of two-dimensional Gaussians that affect the color of the pixel. Multiply the transparency factor, indicating the number of j -1 The transparency accumulation product of the two 2D Gaussians is used to adjust the contribution of each 2D Gaussian to the final pixel color.

[0014] Optionally, S4 includes the following specific steps: S401: Calculate the global saliency score of each three-dimensional Gaussian to determine their contribution to scene reconstruction; S402: Calculate the importance score of each three-dimensional Gaussian to the noise level, and determine whether each three-dimensional Gaussian needs to be compressed according to the importance score and the global significance score, so as to prune redundant Gaussians; S403: In the three-dimensional Gaussian compression process, vector quantization is introduced to process the spherical harmonic function coefficients of the three-dimensional Gaussian to further compress the model size and retain key visual information; S404: Through information distillation, the coefficients of the spherical harmonics in the uncompressed three-dimensional Gaussian are reduced from high order to low order to reduce the complexity of the model.

[0015] Optionally, the S402 includes the following specific steps: Construct a convolutional neural network, and analyze the contribution of each Gaussian point to the noise level according to the convolutional neural network to obtain the importance score of each three-dimensional Gaussian; Sort all three-dimensional Gaussians according to the calculated global significance scores, and retain the three-dimensional Gaussians with high global significance scores; For three-dimensional Gaussians with low global significance scores, whether to compress the three-dimensional Gaussians is determined based on the importance score of each three-dimensional Gaussian; if the importance score of the three-dimensional Gaussian is high, the three-dimensional Gaussian is not compressed, and the three-dimensional Gaussian is optimized to compensate for the details lost due to the compression of the three-dimensional Gaussian; if the importance score of the three-dimensional Gaussian is low, the three-dimensional Gaussian is compressed.

[0016] Optionally, S5 includes the following specific steps: S501: Based on the multi-layer perceptron and SDF function, a neural implicit surface reconstruction model is constructed, and the normal vector of the reconstructed model surface is predicted; S502: Scale regularization is performed on the three-dimensional Gaussian by minimizing the scaling factor so that the three-dimensional Gaussian is compressed into a plane, and the direction of the scaling factor is adjusted so that the plane Gaussian normal vector is aligned with the normal vector predicted by the neural implicit surface reconstruction model to improve the accuracy of surface reconstruction.

[0017] Optionally, S6 includes the following specific steps: S601: Divide the images in the device image set into a training set and a test set; S602: constructing a color loss function based on the device images before and after rendering to measure the difference between the device images before and after rendering; Based on the Eikonal Loss function, the gradient of the SDF function is constrained, and the SDF gradient constraint function is constructed to constrain the gradient norm of the SDF function and improve the surface reconstruction quality. Based on the dense point cloud constraint SDF function, a point cloud constraint function is constructed to constrain the surface depth information of the device image to approach zero; S603: constructing a composite loss function based on the color loss function, the SDF gradient constraint function and the point cloud constraint function, and assigning initial weights to different functions in the composite loss function; S604: training the neural implicit surface reconstruction model based on the training set, calculating the error value through the composite loss function and back-propagating it to update the parameters of the neural implicit surface reconstruction model, and continuously iterating the training; S605: When the number of training iterations of the neural implicit surface reconstruction model reaches the preset number of test iterations, the performance of the neural implicit surface reconstruction model is tested based on the test set; if the model performance is qualified, the training of the neural implicit surface reconstruction model is completed; if the model performance is unqualified, the weights of different functions in the composite loss function are adjusted, and then the process returns to step S604.

[0018] Optionally, the expression of the SDF gradient constraint function in S602 is:

[0019] In the formula, is the SDF gradient constraint function; S is the total number of sample points; It's on point The gradient of the surface depth function at ; is the norm of the gradient; The expression of the composite loss function in S603 is:

[0020] In the formula, is the composite loss function; is the color loss function; is the point cloud constraint function; is the weight of the color loss function; is the weight of the SDF gradient constraint function; is the point cloud constraint function weight.

[0021] The beneficial effects brought by the embodiments provided by the present invention include: The present invention introduces a lightweight Gaussian compression technology into an implicit surface reconstruction method guided by three-dimensional Gaussian sputtering, thereby achieving efficient compression and fast rendering of the scene, significantly reducing storage requirements, and improving the modeling accuracy of the surface of porcelain bushing substation equipment and the ability to capture complex surface details.

[0022] The present invention introduces an attention mechanism into the lightweight Gaussian compression technology, uses a deep learning convolutional neural network to analyze the importance score of each Gaussian point to the noise level, and compares it with the global significance score of the Gaussian to determine whether to compress the Gaussian point, reduces noise interference in areas with a large impact on the compressed image, optimizes the Gaussian distribution parameters, and thus improves the modeling accuracy of complex geometric shapes and surface features.

[0023] When training the neural implicit surface reconstruction model in the present invention, the color loss function, SDF gradient constraint function and point cloud constraint function are used as constraints to ensure that the reconstruction model focuses on true surface optimization during the training process, thereby improving the fineness of complex surface reconstruction and the realism of rendered images. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 A flow chart of a surface modeling method of a porcelain bushing transformer equipment based on a drone and a light three-dimensional Gaussian according to an embodiment of this specification is shown; Figure 2 A structural block diagram of a surface modeling system for porcelain bushing transformer substation equipment based on a drone and a lightweight three-dimensional Gaussian according to an embodiment of this specification is shown. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0027] However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, the description of the known technology is omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The term "comprising" used herein indicates the existence of features, steps, operations, but does not exclude the existence or addition of one or more other features. It should be noted that all terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used here should be interpreted as having meanings consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0029] Example 1 like Figure 1 As shown, this embodiment provides a surface modeling method of a porcelain bushing transformer equipment based on a drone and a light three-dimensional Gaussian, including: S1: Determine the inspection mission route of the drone. The drone executes the inspection mission route and shoots the porcelain bushing substation equipment from multiple perspectives to obtain video data. Exemplarily, S1 includes the following specific steps: The drone collects the surrounding environment images of the porcelain bushing substation equipment, analyzes the environmental images, and plans the drone inspection task route according to the analysis results; When the drone collects images of the surrounding environment of the porcelain sleeve substation equipment, it is necessary to determine the scanning range and scanning path in advance and conduct a comprehensive inspection of the surrounding environment of the porcelain sleeve equipment; The drone executes the inspection mission route and uses the camera carried by the drone to shoot the porcelain bushing substation equipment from multiple angles to obtain video data.

[0030] S2: constructing a device image set based on the video data, performing noise reduction processing and camera pose estimation on the device image set, and obtaining surface depth information of the device image set based on the SDF function; Exemplarily, S2 includes the following specific steps: S201: Splitting the continuous video data into single frames to obtain a continuous static device image set including multiple frames; S202: Acquire the current frame device image, compare it with the previous frame device image, locate the difference between the two frames, and identify and optimize the noise points in the device image; S203: regularly calculating the noise level of the current frame device image, and if it is detected that the noise level exceeds a preset threshold, for the Gaussian point generating noise, reducing the influence of the point by back-propagating its corresponding negative gradient value, so as to reduce the noise; S204: estimating the camera pose corresponding to the device image in the device image set based on the structure-from-motion method to obtain the camera pose information corresponding to the device image; S205: Calculate a device image set based on an SDF function to obtain surface depth information of the device image.

[0031] Among them, this embodiment obtains the surface depth information of the porcelain sleeve substation equipment in the equipment image through the SDF (signed distance field) function to restore the three-dimensional geometric structure of the porcelain sleeve substation equipment from the equipment image, which is conducive to creating an accurate three-dimensional model of the porcelain sleeve substation equipment.

[0032] S3: Generate a dense point cloud based on the device image in the denoised device image set and its corresponding camera pose information, construct a three-dimensional Gaussian based on the dense point cloud, and obtain a rendered device image; Exemplarily, S3 includes the following specific steps: S301: generating a dense point cloud based on the device images after noise reduction and the corresponding camera pose information of the device images; In this embodiment, the correspondence between adjacent images is first found through a feature matching algorithm, and a motion recovery structure algorithm is used to reconstruct a sparse point cloud and a camera trajectory of the scene. The motion recovery structure algorithm generates a sparse three-dimensional point cloud based on the feature matching result; then, dense reconstruction is performed on the basis of the sparse three-dimensional point cloud to generate a dense point cloud; In some embodiments, the surface depth information of the device image in step S205 participates in the generation process of the above-mentioned dense point cloud.

[0033] S302: construct and initialize a three-dimensional Gaussian based on each point in the dense point cloud, and adjust the three-dimensional Gaussian to adapt to different viewing angles through rotation and scaling transformations; Among them, the attribute representation of each three-dimensional Gaussian includes center position, covariance, color, and opacity; In some embodiments, the three-dimensional Gaussian in S302 has a function expression as follows:

[0034]

[0035] In the formula, m is the coordinate of any point in three-dimensional space; n is the center of the three-dimensional Gaussian; Σ is the covariance matrix, which is used to control the size, shape and direction of the three-dimensional Gaussian; R is the rotation matrix; S is a diagonal matrix; T The table is the transposed symbol; S303: performing view transformation by using a sputtering technique, calculating a new covariance matrix, projecting the three-dimensional Gaussian on a two-dimensional image plane, and obtaining a two-dimensional Gaussian distribution; In some embodiments, the new covariance matrix in S303 has a functional expression as follows:

[0036] In the formula, is the new covariance matrix; J is the Jacobian matrix of the projection transformation; W is the perspective transformation matrix; S304: For a pixel point of the two-dimensional image, use a Gaussian rendering formula to calculate the color and opacity of all two-dimensional Gaussians in the pixel point to obtain a final color of the pixel point, and output a rendered two-dimensional image.

[0037] In some embodiments, the Gaussian rendering formula in S304 is expressed as:

[0038] In the formula, C is the final pixel color; For the j The color of a two-dimensional Gaussian; For the j The transparency of a two-dimensional Gaussian; N is the number of two-dimensional Gaussians that affect the color of the pixel. The multiplicative transparency factor represents the cumulative transparency product from the 1st to the j-1th 2D Gaussian, which is used to adjust the contribution of each 2D Gaussian to the final pixel color.

[0039] S4: compressing the 3D Gaussian by the light Gaussian method to prune redundant Gaussians; Exemplarily, S4 includes the following specific steps: S401: Calculate the global saliency score of each three-dimensional Gaussian to determine their contribution to scene reconstruction; In some embodiments, the function expression of the global significance score in S401 is:

[0040] In the formula, is the global significance score; M is the number of device images; H and K The height and width of the device image respectively; p is a pixel, q is the Gaussian index; t is an indicative function used to determine whether the Gaussian intersects with the ray, thereby changing the score. is the opacity of the Gaussian, is a fitness function used to measure the dimension of the Gaussian volume, and the global saliency score reflects the contribution of the Gaussian to the final rendered image.

[0041] S402: Calculate the importance score of each three-dimensional Gaussian to the noise level, and determine whether each three-dimensional Gaussian needs to be compressed according to the importance score and the global significance score, so as to prune redundant Gaussians; In some embodiments, S403 includes the following specific steps: Construct a convolutional neural network, and analyze the contribution of each Gaussian point to the noise level according to the convolutional neural network to obtain the importance score of each three-dimensional Gaussian; Sort all three-dimensional Gaussians according to the calculated global significance scores, and retain the three-dimensional Gaussians with high global significance scores; In this embodiment, the global significance score level of the three-dimensional Gaussian is determined according to whether the global significance score of the three-dimensional Gaussian is higher than a preset score standard; if so, the global significance score of the three-dimensional Gaussian is considered to be high, and the three-dimensional Gaussian is retained; if not, the global significance score of the three-dimensional Gaussian is considered to be low, and the next step is entered; For three-dimensional Gaussians with low global significance scores, whether to compress the three-dimensional Gaussians is determined based on the importance score of each three-dimensional Gaussian; if the importance score of the three-dimensional Gaussian is high, the three-dimensional Gaussian is not compressed, and the three-dimensional Gaussian is optimized to compensate for the details lost due to the compression of the three-dimensional Gaussian; if the importance score of the three-dimensional Gaussian is low, the three-dimensional Gaussian is compressed.

[0042] In this embodiment, three-dimensional Gaussians with low global significance scores and high importance scores need to undergo joint attribute adjustment, including adjusting their means, variances, and weights, to adapt to and compensate for the Gaussian loss and surface depth information loss caused by pruning, thereby ensuring the stability and accuracy of the overall model.

[0043] S403: In the three-dimensional Gaussian compression process, vector quantization is introduced to process the spherical harmonic function coefficients of the three-dimensional Gaussian to further compress the model size and retain key visual information; S404: Through information distillation, the coefficients of the spherical harmonics in the uncompressed three-dimensional Gaussian are reduced from high order to low order to reduce the complexity of the model.

[0044] In this embodiment, in the uncompressed three-dimensional Gaussian, most of the three-dimensional Gaussian data is composed of high-order spherical harmonic coefficients, which leads to an increase in storage overhead and computational complexity. In order to strike a balance between disk storage efficiency and scene reflection quality, the high-order spherical harmonic coefficients are transferred to low-order spherical harmonics through information distillation, and the information is transferred to a more compact representation, while keeping the scene appearance unchanged, thereby reducing the complexity of the model. Information distillation is to extract knowledge from a teacher model that uses full-order spherical harmonic coefficients and transfer it to a student model with truncated low-order distilled compressed spherical harmonic coefficients; through the information distillation process, the visual quality of the scene can be maintained while reducing the model size.

[0045] Specifically, the lightweight Gaussian method proposed in this embodiment achieves significant reduction of data redundancy of three-dimensional Gaussian representation while maintaining visual effects through three key steps: Gaussian pruning and restoration, distillation and compression of spherical harmonic function coefficients, and Gaussian attribute vector quantization, thereby improving rendering efficiency and storage efficiency. It provides an effective solution for new perspective synthesis, especially when dealing with large-scale scenes such as substations, which can significantly improve performance.

[0046] S5: Establish a neural implicit surface reconstruction model and regularize the compressed 3D Gaussian. Exemplarily, S5 includes the following specific steps: S501: Based on the multi-layer perceptron and SDF function, a neural implicit surface reconstruction model is constructed, and the normal vector of the reconstructed model surface is predicted; In this embodiment, the neural implicit surface reconstruction model uses a multi-layer perceptron (MLP) as the core architecture and uses the SDF function to implicitly represent the three-dimensional geometric information; S502: Scale regularization is performed on the three-dimensional Gaussian by minimizing the scaling factor so that the three-dimensional Gaussian is compressed into a plane, and the direction of the scaling factor is adjusted so that the plane Gaussian normal vector is aligned with the normal vector predicted by the neural implicit surface reconstruction model to improve the accuracy of surface reconstruction.

[0047] S6: constructing a composite loss function, and training and optimizing the reconstruction model based on the composite loss function; Exemplarily, S6 includes the following specific steps: S601: Divide the images in the device image set into a training set and a test set; S602: constructing a color loss function based on the device images before and after rendering to measure the difference between the device images before and after rendering; In some embodiments, the expression of the color loss function in S602 is:

[0048] In the formula, is the color loss function; C is the original image color; is the color of the rendered image; Based on the Eikonal Loss function, the gradient of the SDF function is constrained, and the SDF gradient constraint function is constructed to constrain the gradient norm of the SDF function and improve the surface reconstruction quality. In some embodiments, the expression of the SDF gradient constraint function in S602 is:

[0049] In the formula, is the SDF gradient constraint function; S is the total number of sample points; It's on point The gradient of the surface depth function at ; is the norm of the gradient; Based on the dense point cloud constraint SDF function, a point cloud constraint function is constructed to constrain the surface depth information of the device image to approach zero; In some embodiments, the expression of the point cloud constraint function in S602 is:

[0050] In the formula, is the point cloud constraint function; At the point The predicted SDF value at , indicating that the point distance to the nearest surface; is a point in a dense point cloud; Among them, when When is, it means the point is outside the surface; when When it is negative, it means the point is inside the surface; when When it is 0, it means that the point is exactly on the surface; this embodiment uses the point cloud constraint function to ensure that the value of SDF at the point cloud position is close to zero, that is, these points should be located on the surface, thereby performing geometric constraints.

[0051] S603: constructing a composite loss function based on the color loss function, the SDF gradient constraint function and the point cloud constraint function, and assigning initial weights to different functions in the composite loss function; In some embodiments, the expression of the composite loss function in S603 is:

[0052] In the formula, is the composite loss function; is the weight of the color loss function; is the weight of the SDF gradient constraint function; is the point cloud constraint function weight.

[0053] S604: training the neural implicit surface reconstruction model based on the training set, calculating the error value through the composite loss function and back-propagating it to update the parameters of the neural implicit surface reconstruction model, and continuously iterating the training; In this embodiment, the neural implicit surface reconstruction model selects ADAM as the training optimizer, and the adaptive learning rate is selectively set according to demand.

[0054] S605: When the number of training iterations of the neural implicit surface reconstruction model reaches the preset number of test iterations, the performance of the neural implicit surface reconstruction model is tested based on the test set; if the model performance is qualified, the training of the neural implicit surface reconstruction model is completed; if the model performance is unqualified, the weights of different functions in the composite loss function are adjusted, and then the process returns to step S604.

[0055] Example 2 like Figure 2 As shown, this embodiment provides a surface modeling system 200 of a porcelain bushing transformer equipment based on a drone and a lightweight three-dimensional Gaussian, comprising: Image acquisition module 201, used to determine the inspection task route of the drone, the drone executes the inspection task route, and shoots the porcelain bushing transformer equipment from multiple perspectives to obtain video data; An image processing module 202 is used to construct a device image set based on the video data, perform noise reduction and camera pose estimation on the device image set, and obtain surface depth information of the device image set based on an SDF function; A Gaussian generation module 203 is used to generate a dense point cloud based on the device image after noise reduction and the corresponding camera pose information, construct a three-dimensional Gaussian according to the dense point cloud, and obtain a rendered device image; A Gaussian compression module 204, for compressing the three-dimensional Gaussian by a light Gaussian method to prune redundant Gaussians; A model building module 205 is used to build a neural implicit surface reconstruction model and perform regularization processing on the compressed three-dimensional Gaussian; The model training module 206 is used to construct a composite loss function, and train and optimize the reconstruction model based on the composite loss function.

[0056] Example 3 This embodiment provides an electronic device, comprising at least one control processor and a memory for communicating with the at least one control processor; As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transitory Memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory optionally includes a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0057] The non-transient software program and instructions required to implement the above-mentioned embodiment of the surface modeling method of the porcelain sleeve substation equipment based on the drone and the light three-dimensional Gaussian are stored in the memory. When executed by the processor, the above-mentioned embodiment of the surface modeling method of the porcelain sleeve substation equipment based on the drone and the light three-dimensional Gaussian is executed, for example, the above-mentioned Figure 1 The method comprises steps S1 to S6.

[0058] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0059] Example 4 This embodiment provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a surface modeling method for porcelain bushing substation equipment based on an unmanned aerial vehicle and a lightweight three-dimensional Gaussian as described in Example 1.

[0060] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0061] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In this embodiment, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device. The computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0062] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Python, Java, C++, and also includes a conventional procedural programming language, such as C language or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0063] In summary, this embodiment introduces lightweight Gaussian compression technology into the implicit surface reconstruction method guided by three-dimensional Gaussian sputtering, thereby achieving efficient compression and fast rendering of the scene, effectively pruning redundant Gaussians, reducing the demand for storage space and the consumption of computing resources, and improving the modeling accuracy of the surface of the porcelain bushing substation equipment and the ability to capture complex surface details.

[0064] This embodiment utilizes drone inspection technology to photograph porcelain sleeve substation equipment from multiple perspectives through drones, so as to conduct more detailed monitoring of porcelain sleeve substation equipment, comprehensively capture the characteristics of the equipment surface, and promptly discover potential problems of the equipment, such as cracks and dirt accumulation, so as to provide early warning and processing, and reduce the risk of equipment failure.

[0065] This embodiment introduces an attention mechanism into the lightweight Gaussian compression technology, and uses a deep learning convolutional neural network to analyze the importance score of each Gaussian point to the noise level, and compares it with the global significance score of the Gaussian to determine whether to compress the Gaussian point, reduce noise interference in areas with a large impact on the compressed image, and optimize the Gaussian distribution parameters, thereby improving the modeling accuracy of complex geometric shapes and surface features.

[0066] In this embodiment, when training the neural implicit surface reconstruction model, the color loss function, SDF gradient constraint function and point cloud constraint function are used as constraints to ensure that the reconstruction model focuses on true surface optimization during the training process, thereby improving the fineness of complex surface reconstruction and the realism of rendered images.

[0067] The above description is only a preferred embodiment of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention.

Claims

1. A surface modeling method for porcelain bushing transformer equipment based on drone and light three-dimensional Gaussian, characterized in that: include: S1: Determine the inspection mission route of the drone. The drone executes the inspection mission route and shoots the porcelain bushing substation equipment from multiple perspectives to obtain video data. S2: constructing a device image set based on the video data, performing noise reduction processing and camera pose estimation on the device image set, and obtaining surface depth information of the device image set based on the SDF function; S3: Generate a dense point cloud based on the device image and its corresponding camera pose information in the denoised device image set, construct a three-dimensional Gaussian based on the dense point cloud, and obtain a rendered device image; S4: compressing the 3D Gaussian by the light Gaussian method to prune redundant Gaussians; S5: Establish a neural implicit surface reconstruction model and regularize the compressed 3D Gaussian. S6: constructing a composite loss function, and training and optimizing the reconstruction model based on the composite loss function; The composite loss function includes an SDF gradient constraint function, a point cloud constraint function and a color loss function.

2. The method according to claim 1, characterized in that The S1 comprises the following specific steps: The drone collects the surrounding environment images of the porcelain bushing substation equipment, analyzes the environmental images, and plans the drone inspection task route according to the analysis results; The drone executes the inspection mission route and uses the camera carried by the drone to shoot the porcelain bushing substation equipment from multiple angles to obtain video data.

3. The method according to claim 1, characterized in that The S2 comprises the following specific steps: S201: Splitting the continuous video data into single frames to obtain a continuous static device image set including multiple frames; S202: Acquire the current frame device image, compare it with the previous frame device image, locate the difference between the two frames, and identify and optimize the noise points in the device image; S203: regularly calculating the noise level of the current frame device image, and if it is detected that the noise level exceeds a preset threshold, for the Gaussian point generating noise, reducing the influence of the point by back-propagating its corresponding negative gradient value, so as to reduce the noise; S204: estimating the camera pose corresponding to the device image in the device image set based on the structure-from-motion method to obtain the camera pose information corresponding to the device image; S205: Calculate a device image set based on an SDF function to obtain surface depth information of the device image.

4. The method according to claim 1, characterized in that: The S3 includes the following specific steps: S301: Generate a dense point cloud based on the denoised device image and the corresponding camera pose information; S302: construct and initialize a three-dimensional Gaussian based on each point in the dense point cloud, and adjust the three-dimensional Gaussian to adapt to different viewing angles through rotation and scaling transformations; The attribute representation of each three-dimensional Gaussian includes center position, covariance, color, and opacity; S303: performing view transformation by using a sputtering technique, calculating a new covariance matrix, projecting the three-dimensional Gaussian on the two-dimensional image plane, and obtaining a two-dimensional Gaussian distribution; S304: For a pixel point of the two-dimensional image, use a Gaussian rendering formula to calculate the color and opacity of all two-dimensional Gaussians in the pixel point to obtain a final color of the pixel point, and output a rendered two-dimensional image.

5. The method according to claim 4, characterized in that The three-dimensional Gaussian in S302 has the following function expression: ； ； In the formula, m is the coordinate of any point in three-dimensional space; n is the center of the three-dimensional Gaussian; Σ is the covariance matrix, which is used to control the size, shape and direction of the three-dimensional Gaussian; R is the rotation matrix; S is a diagonal matrix; T The table is the transposed symbol; The new covariance matrix in S303 has a functional expression as follows: ； In the formula, is the new covariance matrix; J is the Jacobian matrix of the projection transformation; W is the perspective transformation matrix; The Gaussian rendering formula in S304 is expressed as: ； In the formula, C is the final pixel color; For the j The color of a two-dimensional Gaussian; For the j The transparency of a two-dimensional Gaussian; N is the number of two-dimensional Gaussians that affect the color of the pixel. Multiply the transparency factor, indicating the number of j -1 The transparency accumulation product of the two 2D Gaussians is used to adjust the contribution of each 2D Gaussian to the final pixel color.

6. The method according to claim 1, characterized in that The S4 comprises the following specific steps: S401: Calculate the global saliency score of each three-dimensional Gaussian to determine their contribution to scene reconstruction; S402: Calculate the importance score of each three-dimensional Gaussian to the noise level, and determine whether each three-dimensional Gaussian needs to be compressed according to the importance score and the global significance score, so as to prune redundant Gaussians; S403: In the three-dimensional Gaussian compression process, vector quantization is introduced to process the spherical harmonic function coefficients of the three-dimensional Gaussian to further compress the model size and retain key visual information; S404: Through information distillation, the coefficients of the spherical harmonics in the uncompressed three-dimensional Gaussian are reduced from high order to low order to reduce the complexity of the model.

7. The method according to claim 6, characterized in that The S402 includes the following specific steps: Construct a convolutional neural network, and analyze the contribution of each Gaussian point to the noise level according to the convolutional neural network to obtain the importance score of each three-dimensional Gaussian; Sort all three-dimensional Gaussians according to the calculated global significance scores, and retain the three-dimensional Gaussians with high global significance scores; For three-dimensional Gaussians with low global significance scores, whether to compress the three-dimensional Gaussians is determined based on the importance score of each three-dimensional Gaussian; if the importance score of the three-dimensional Gaussian is high, the three-dimensional Gaussian is not compressed, and the three-dimensional Gaussian is optimized to compensate for the details lost due to the compression of the three-dimensional Gaussian; if the importance score of the three-dimensional Gaussian is low, the three-dimensional Gaussian is compressed.

8. The method according to claim 1, characterized in that: The S5 comprises the following specific steps: S501: Based on the multi-layer perceptron and SDF function, a neural implicit surface reconstruction model is constructed, and the normal vector of the reconstructed model surface is predicted; S502: Scale regularization is performed on the three-dimensional Gaussian by minimizing the scaling factor so that the three-dimensional Gaussian is compressed into a plane, and the direction of the scaling factor is adjusted so that the normal vector of the plane Gaussian is aligned with the normal vector predicted by the neural implicit surface reconstruction model to improve the accuracy of surface reconstruction.

9. The method according to claim 1, characterized in that: The S6 comprises the following specific steps: S601: Divide the images in the device image set into a training set and a test set; S602: constructing a color loss function based on the device images before and after rendering to measure the difference between the device images before and after rendering; Based on the Eikonal Loss function, the gradient of the SDF function is constrained, and the SDF gradient constraint function is constructed to constrain the gradient norm of the SDF function and improve the surface reconstruction quality. Based on the dense point cloud constraint SDF function, a point cloud constraint function is constructed to constrain the surface depth information of the device image to approach zero; S603: constructing a composite loss function based on the color loss function, the SDF gradient constraint function and the point cloud constraint function, and assigning initial weights to different functions in the composite loss function; S604: training the neural implicit surface reconstruction model based on the training set, calculating the error value through the composite loss function and back-propagating it to update the parameters of the neural implicit surface reconstruction model, and continuously iterating the training; S605: When the number of training iterations of the neural implicit surface reconstruction model reaches the preset number of test iterations, the performance of the neural implicit surface reconstruction model is tested based on the test set; if the model performance is qualified, the training of the neural implicit surface reconstruction model is completed; if the model performance is unqualified, the weights of different functions in the composite loss function are adjusted, and then the process returns to step S604.

10. The method according to claim 9, characterized in that The expression of the SDF gradient constraint function in S602 is: ； In the formula, is the SDF gradient constraint function; S is the total number of sample points; It's on point The gradient of the surface depth function at ; is the norm of the gradient; The expression of the composite loss function in S603 is: ； In the formula, is the composite loss function; is the color loss function; is the point cloud constraint function; is the weight of the color loss function; is the weight of the SDF gradient constraint function; is the point cloud constraint function weight.

Images