A three-dimensional reconstruction method and system for a power inspection scene

Abstract

The application relates to the technical field of three-dimensional reconstruction, in particular to a three-dimensional reconstruction method and system for a power inspection scene, which comprises the following steps: acquiring camera poses and an initialized sparse point cloud of the power inspection scene, establishing a three-dimensional Gaussian representation containing three-dimensional Gaussian ellipsoids based on the camera poses and the initialized sparse point cloud; calculating eigenvalues of a covariance matrix of the three-dimensional Gaussian ellipsoid for a slender structure in the power inspection scene, and estimating local surface normals according to the distribution of neighborhood three-dimensional Gaussian ellipsoids; constructing a linear consistency constraint loss for constraining the eigenvalue proportion and spatial orientation of the three-dimensional Gaussian ellipsoid to fit the slender structure; combining an image reconstruction photometric loss and the linear consistency constraint loss to construct a three-dimensional reconstruction loss function, optimizing parameters of the three-dimensional Gaussian ellipsoid based on the three-dimensional reconstruction loss function, and outputting a three-dimensional reconstruction model of the power inspection scene after iterative convergence, so that the integrity and accuracy of three-dimensional reconstruction are improved.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, specifically a three-dimensional reconstruction method and system for power line inspection scenarios. Background Technology

[0002] Power line inspection operations typically rely on drones and handheld devices to acquire multi-view images for 3D scene reconstruction. 3D Gaussian Splashing (3DGS), with its advantages of high-precision rendering and lightweight modeling, is gradually being applied to the digital modeling of power equipment. This technology uses a 3D Gaussian representation as the scene carrier, representing spatial equipment through a massive number of 3D Gaussian ellipsoids, and relying on iterative optimization of ellipsoid parameters through image reconstruction photometric loss to achieve reconstruction. However, traditional reconstruction schemes based on 3D Gaussian splashing rely solely on a single photometric loss to constrain the 3D Gaussian ellipsoid. Power line inspection scenarios contain numerous slender columnar structures such as conductors, insulating bushings, long insulator strings, and tower bracing. The galvanized reflective surfaces of these components have monotonous textures, resulting in insufficient photometric loss constraint effectiveness. Consequently, the 3D Gaussian ellipsoid is prone to problems such as scale expansion, positional drift, and orientation confusion, causing breakage in the slender structure reconstruction model and reducing the integrity and accuracy of the 3D model. Summary of the Invention

[0003] This application provides a three-dimensional reconstruction method and system for power line inspection scenarios, which solves the problem of fracture in the reconstruction of slender structures in power line inspection scenarios and improves the integrity and accuracy of three-dimensional reconstruction.

[0004] To address the aforementioned technical problems, this application adopts the following technical solution: a 3D reconstruction method for power line inspection scenarios, comprising: acquiring the camera pose and initializing a sparse point cloud of the power line inspection scenario; establishing a 3D Gaussian representation containing a 3D Gaussian ellipsoid based on the camera pose and the initializing sparse point cloud; in the 3D Gaussian representation, for slender structures within the power line inspection scenario, calculating the eigenvalues ​​of the covariance matrix of the 3D Gaussian ellipsoid, and estimating local surface normals based on the distribution of the neighborhood 3D Gaussian ellipsoids; constructing a linear consistency constraint loss based on the eigenvalues ​​and the local surface normals, wherein the linear consistency constraint loss is used to constrain the eigenvalue ratio and spatial orientation of the 3D Gaussian ellipsoid to fit the slender structure; constructing a 3D reconstruction loss function by combining the image reconstruction photometric loss and the linear consistency constraint loss; optimizing the parameters of the 3D Gaussian ellipsoid based on the 3D reconstruction loss function; and outputting a 3D reconstruction model of the power line inspection scenario after iterative convergence.

[0005] Optionally, the step of constructing a linear consistency constraint loss based on the eigenvalues ​​and the local surface normal includes: using the local surface normal as a constraint reference, obtaining three eigenvalues ​​corresponding to the covariance matrix of the three-dimensional Gaussian ellipsoid; The longest axis of the three-dimensional Gaussian ellipsoid is constrained to be perpendicular to the local surface normal, and the eigenvalue of the longest axis of the three-dimensional Gaussian ellipsoid is constrained to be greater than the sum of the eigenvalues ​​of the remaining two axes; the linear consistency constraint loss value is calculated based on the ratio of the eigenvalue of the longest axis to the sum of the eigenvalues ​​of the remaining two axes.

[0006] Optionally, the step of constructing a three-dimensional reconstruction loss function by combining the image reconstruction photometric loss and the linear consistency constraint loss includes: in the three-dimensional Gaussian representation, constructing a plane consistency constraint loss for the planar structure in the power inspection scenario, wherein the plane consistency constraint loss is used to constrain the three-dimensional Gaussian ellipsoid to be flattened; and constructing a three-dimensional reconstruction loss function by combining the image reconstruction photometric loss, the linear consistency constraint loss, and the plane consistency constraint loss.

[0007] Optionally, the step of obtaining the camera pose and initializing the sparse point cloud of the power inspection scene includes: obtaining a multi-view image of the power inspection scene, and performing feature extraction, feature matching, camera pose estimation and sparse point cloud recovery on the multi-view image to obtain the camera pose and initializing the sparse point cloud of the power inspection scene.

[0008] Optionally, the step of acquiring multi-view images of the power inspection scene includes: acquiring a video stream and an RGB image sequence of the power inspection scene, extracting frames from the video stream at a fixed frame rate, and sorting the RGB image sequence to obtain multi-view images of the power inspection scene.

[0009] Optionally, after the step of outputting the 3D reconstruction model of the power inspection scene after the iterative convergence, the method further includes: performing semantic segmentation on the multi-view image to generate a 2D semantic mask; and mapping the 2D semantic mask onto the 3D Gaussian ellipsoid in the 3D reconstruction model based on the camera pose to obtain a 3D reconstruction model with semantic information.

[0010] Optionally, the step of semantically segmenting the multi-view image to generate a two-dimensional semantic mask includes: constructing power scene prompt words containing the names of power equipment; and using an open vocabulary segmentation model to perform vocabulary detection and semantic segmentation on the multi-view image based on the power scene prompt words to generate a two-dimensional semantic mask.

[0011] Optionally, the power scene prompts include main equipment prompts, key component prompts, and background interference prompts; the main equipment prompts are used to describe the main equipment in the power scene, the key component prompts are used to describe local components on the power equipment, and the background interference prompts are used to describe the sky, ground, walls, cabinets, buildings, or other non-core equipment areas.

[0012] Optionally, the parameters of the three-dimensional Gaussian ellipsoid include spatial position, covariance matrix, opacity, and color parameters.

[0013] This application also provides a 3D reconstruction system for power line inspection scenarios, including: The acquisition module is configured to acquire the camera pose and initial sparse point cloud of the power inspection scene, and establish a three-dimensional Gaussian representation containing a three-dimensional Gaussian ellipsoid based on the camera pose and the initial sparse point cloud. The calculation module is configured to calculate the eigenvalues ​​of the covariance matrix of the three-dimensional Gaussian ellipsoid for the slender structure in the power inspection scenario in the three-dimensional Gaussian representation, and estimate the local surface normal based on the distribution of the neighborhood three-dimensional Gaussian ellipsoid. The constraint loss module is configured to construct a linear consistency constraint loss based on the eigenvalues ​​and the local surface normals. The linear consistency constraint loss is used to constrain the eigenvalue ratios and spatial orientation of the three-dimensional Gaussian ellipsoid to fit the slender structure. The 3D reconstruction module is configured to construct a 3D reconstruction loss function by combining the image reconstruction photometric loss and the linear consistency constraint loss, optimize the parameters of the 3D Gaussian ellipsoid based on the 3D reconstruction loss function, and output a 3D reconstruction model of the power inspection scene after iterative convergence.

[0014] This application also provides a terminal device, including a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method described above.

[0015] This application also provides a storage medium storing a computer program, which is invoked and executed by a computer to implement the method described above.

[0016] The beneficial effects of this application are as follows: This application targets slender structures in power inspection scenarios. It extracts the eigenvalues ​​of the three-dimensional Gaussian ellipsoid covariance matrix and estimates the local surface normals by combining the distribution of the neighborhood three-dimensional Gaussian ellipsoid. The surface orientation of the component is derived from the geometric distribution of the Gaussian ellipsoid itself, eliminating the need for an external depth prediction network and achieving the acquisition of surface geometric information, thus adapting to power inspection modeling scenarios. By combining the eigenvalues ​​of the covariance matrix and the local surface normals to construct a linear consistency constraint loss, it constrains the proportion of the three-axis eigenvalues ​​of the three-dimensional Gaussian ellipsoid, limiting excessive ellipsoid expansion. Furthermore, the local surface normals constrain the spatial orientation of the three-dimensional Gaussian ellipsoid, forcing the minor axis of the Gaussian ellipsoid to align with the surface normal of the component, and the major axis to extend tangentially along the slender component. This results in the Gaussian ellipsoid appearing as a continuous, slender, thin sheet that fits onto slender cylindrical equipment such as pipelines and bushings, eliminating the problem of fracture in the reconstruction of slender structures and improving the integrity of the three-dimensional reconstruction of power scenarios. By fusing the photometric loss from image reconstruction with the linear consistency constraint loss to form a joint 3D reconstruction loss function, texture rendering constraints and geometric constraints of slender structures are completed simultaneously. When optimizing the 3D Gaussian ellipsoid parameters, the texture and color of the reconstructed model are consistent with the real inspection image, while the geometric constraints maintain the continuity and integrity of slender components, thus achieving the geometric integrity and rendering accuracy of the 3D reconstruction model. Attached Figure Description

[0017] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0018] Figure 1 This is a flowchart of a three-dimensional reconstruction method for power line inspection scenarios according to this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely one preferred embodiment of this application and are only used to explain this application. They do not limit the scope of protection of this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] As one implementation method, such as Figure 1 As shown, this embodiment provides a 3D reconstruction method for power line inspection scenarios to solve the problem of fractured reconstruction of slender structures in power line inspection scenarios, including: S1. Obtain the camera pose and initial sparse point cloud of the power inspection scene, and establish a 3D Gaussian representation containing 3D Gaussian ellipsoids based on the camera pose and the initial sparse point cloud. The 3D Gaussian representation refers to a spatial representation carrier of the power inspection scene, generated based on the camera pose and the initial sparse point cloud, and composed of several 3D Gaussian ellipsoids. It is used to store optimizable parameters such as the position, covariance, opacity, and color of all 3D Gaussian ellipsoids in the scene, and is the underlying data carrier of the scene in 3D Gaussian splashing (3DGS) technology.

[0021] Specifically, the process involves acquiring video streams and RGB image sequences of the power line inspection scene, extracting frames from the video stream at a fixed frame rate, and sorting the RGB image sequences to obtain multi-view images of the power line inspection scene. Feature extraction, feature matching, camera pose estimation, and sparse point cloud reconstruction are then performed on the multi-view images to obtain the camera pose and initial sparse point cloud of the power line inspection scene. Furthermore, Structure for Motion Restoration (SfM) technology can be used for feature extraction and feature matching of the multi-view images. The Bundle Adjustment algorithm can be used to calculate the camera pose (position and rotation angle) corresponding to each image and reconstruct the initial sparse point cloud of the scene.

[0022] 3D Gaussian Splash (3DGS) uses thousands of 3D ellipsoids (Gaussian ellipsoids) to fill space. Each ellipsoid's parameters include spatial position, covariance matrix, opacity, and color parameters. Spatial position refers to the coordinates of the ellipsoid's 3D center point in the coordinate system. The covariance matrix controls the ellipsoid's size, three-axis stretching ratio, and spatial orientation. Opacity, ranging from 0 to 1, controls the ellipsoid's light transmission; multiple ellipsoids are stacked to achieve a smooth surface. Color parameters store viewpoint-dependent RGB colors, ensuring realistic rendering of device reflections and textures. In power line inspection scenarios, the problem of fractured reconstructions of slender structures arises because these ellipsoids are not correctly arranged or their shapes are out of control on slender structures (such as power lines), resulting in poor rendering effects.

[0023] S2. In the 3D Gaussian representation, for slender structures in power line inspection scenarios, the eigenvalues ​​of the covariance matrix of the 3D Gaussian ellipsoid are calculated, and the local surface normal is estimated based on the distribution of neighborhood 3D Gaussian ellipsoids. Each Gaussian ellipsoid corresponds to a 3×3 covariance matrix. Eigenvalue decomposition of the matrix yields three eigenvalues ​​and corresponding eigenvectors, representing the lengths of the ellipsoid's three axes. These three eigenvalues ​​represent the extent of the ellipsoid's extension along the three principal axes. For slender structures in power equipment (such as conductors), the corresponding Gaussian ellipsoid will be very flat or thin. In this case, the eigenvector corresponding to the smallest eigenvalue naturally points to the normal direction of the local surface represented by that ellipsoid.

[0024] A local surface normal is a vector in 3D geometry representing the orientation of a point on an object's surface, perpendicular to the tangent plane of that point. In power line inspection scenarios, it's used to accurately describe the orientation of surfaces such as transformer housings, bushings, and conductors. Normals calculated from a single Gaussian ellipsoid may exhibit jumps due to noise or optimization instability. To obtain a continuous and smooth normal field that closely matches the actual surface of the equipment, it's necessary to take a group of spatially adjacent Gaussian ellipsoids, combine their spatial coordinates with the initial normal, and perform a weighted average or surface fitting to obtain a local surface normal with high stability and accuracy.

[0025] S3. Construct a linear consistency constraint loss based on the eigenvalues ​​and the local surface normals. The linear consistency constraint loss is used to constrain the eigenvalue ratios and spatial orientation of the three-dimensional Gaussian ellipsoid to fit the slender structure, including: Using the local surface normal as the constraint benchmark, the three eigenvalues ​​corresponding to the covariance matrix of the 3D Gaussian ellipsoid are obtained. The direction of the longest axis of the 3D Gaussian ellipsoid is constrained to be perpendicular to the local surface normal, and the eigenvalue of the longest axis is constrained to be greater than the sum of the eigenvalues ​​of the remaining two axes. For a cylindrical cable, its surface normal is radially outward (perpendicular to the cable's central axis), and the cable's extension direction (longest axis) is along the central axis. The longest axis must be perpendicular to the surface normal. The linear consistency constraint loss value is calculated based on the ratio of the longest axis eigenvalue to the sum of the eigenvalues ​​of the remaining two axes.

[0026] Furthermore, a scale-constraint sub-loss is constructed, based on the three eigenvalues ​​of the covariance matrix of the 3D Gaussian ellipsoid. The scale-constraint sub-loss is zero when the eigenvalue of the longest axis of the ellipsoid is significantly greater than the sum of the eigenvalues ​​of the other two axes. A normal orientation constraint sub-loss is also constructed, based on the dot product of the eigenvectors corresponding to the local surface normal and the longest axis of the ellipsoid. This constraint is used to ensure that the major axis of the ellipsoid is perpendicular to the local surface normal. The scale-constraint sub-loss and the normal orientation constraint sub-loss are then fused to obtain the linear consistency constraint loss.

[0027] Using the local surface normal calculated in step S2 as a priori, a linear consistency constraint loss is constructed. By limiting the ratio between the three eigenvalues, the shape of the Gaussian ellipsoid is controlled, thus solving the technical problems of Gaussian drift and segmental fracture in slender structures. In 3DGS (3D Gaussian Geometry) technology, the shape of each Gaussian ellipsoid is determined by the covariance matrix, which can be decomposed into three eigenvalues. , representing the lengths of the three axes of the ellipsoid, respectively, are used for adaptive shape regularization based on local geometric features. First, the local surface normal at the current location of the Gaussian ellipsoid is calculated; if it is on a slender structure, the proportion of eigenvalues ​​is restricted.

[0028] Linearity consistency constraint loss requires the longest axis of the ellipsoid to be within the range of linear consistency constraints. It must be larger than the other two minor axes. and The sum is set as: In the formula, Represents the linear consistency constraint loss. This represents the scale constraint loss value. This represents the loss value due to normal constraint. The loss function represents the normal constraint loss weight, which is used to balance the effects of scale constraints and orientation constraints. This loss function simultaneously regulates the ratio of the three axes of the ellipsoid and the spatial orientation, constraining the Gaussian ellipsoid from both geometric and spatial orientation dimensions, thus avoiding Gaussian drift and model segmentation problems caused by slender structures. In the formula, This represents the scale constraint loss value. When this loss value approaches 0, it indicates that the ellipsoid has been stretched into a linear shape, fitting the power line and avoiding breakage. When this loss value is positive, it indicates that the ellipsoid is somewhat circular. Represents a minimal constant. Longest axis It is the maximum value among the three eigenvalues ​​of the covariance matrix, rather than a fixed eigenvalue representing a specific coordinate. In the formula, This represents the loss value due to normal constraint. Indicates the longest axis The corresponding eigenvectors represent the direction of the major axis of the ellipsoid. This represents the estimated local surface normal unit vector.

[0029] S4. A 3D reconstruction loss function is constructed by combining image reconstruction photometric loss and linear consistency constraint loss. The parameters of the 3D Gaussian ellipsoid are optimized based on this loss function, and the 3D reconstruction model of the power inspection scene is output after iterative convergence. The process includes: using the current Gaussian ellipsoid parameters, simulating the camera's perspective, generating a predicted image, and calculating which Gaussian ellipsoids contribute to each pixel in the image. The predicted image is compared with the real image to calculate the photometric loss; simultaneously, the shape of the Gaussian ellipsoid on the slender structure is examined to calculate the linear consistency loss. The two are added together to obtain the total error (3D reconstruction loss function). Working backward along the rendering pipeline, the color parameters of the ellipsoid are adjusted, or the ellipsoid is rotated to a direction perpendicular to the normal, calculating the direction and magnitude (i.e., gradient) that each parameter should be adjusted. Based on the calculated gradient, the parameters of the Gaussian ellipsoid are updated slightly using an optimizer (such as Adam). As iterations proceed, the value of the 3D reconstruction loss function will decrease until it no longer decreases significantly, or the preset number of iterations is reached, meaning the model can no longer be significantly optimized. At this point, each Gaussian ellipsoid has found its optimal position and shape. The power lines have become continuous, smooth lines, and the surface texture of the equipment is clear and geometrically accurate. Exporting these finalized Gaussian ellipsoid parameters constitutes the 3D reconstruction model of the power inspection scenario. The 3D reconstruction loss function can be as follows: In the formula, Indicates the loss in 3D reconstruction. Indicates the luminance loss during image reconstruction. Represents the linear consistency constraint loss. The weight represents the loss due to linear consistency constraints.

[0030] Furthermore, in the 3D Gaussian representation, a planar consistency constraint loss is constructed for planar structures within the power inspection scenario. This loss constrains the 3D Gaussian ellipsoid to have a flattened distribution. The linear consistency loss forces the ellipsoid to elongate along one axis, adapting to slender cylindrical components such as conductors, insulators, and bushings. The planar consistency loss forces the ellipsoid to compress along one axis and extend along two axes, adapting to planar components such as substation cabinets, walls, and metal plates. A 3D reconstruction loss function is constructed by combining the image reconstruction photometric loss, the linear consistency constraint loss, and the planar consistency constraint loss. The 3D reconstruction loss function can be as follows: In the formula, Represents the loss due to plane consistency constraints. The weight represents the loss due to the plane consistency constraint. In the formula, Represents a minimal constant. Represents the three eigenvalues ​​of the covariance matrix, with the shortest axis being... It is the minimum value among the three eigenvalues ​​of the covariance matrix, rather than a fixed eigenvalue representing a certain coordinate.

[0031] S5. Perform semantic segmentation on the multi-view image to generate a two-dimensional semantic mask. Based on the camera pose, map the two-dimensional semantic mask onto the three-dimensional Gaussian ellipsoid in the three-dimensional reconstruction model to obtain a three-dimensional reconstruction model with semantic information. Specifically, construct power scene prompt words containing the names of power equipment; use an open vocabulary segmentation model to perform vocabulary detection and semantic segmentation on the multi-view image based on the power scene prompt words to generate a two-dimensional semantic mask.

[0032] The process includes: using the camera pose (i.e., the position and orientation of the camera) obtained in the initial stage of 3D reconstruction, for the 3D Gaussian ellipsoid in the 3D model, performing a planar projection of the 3D Gaussian ellipsoid onto the 2D image based on the camera pose, filtering out the 3D Gaussian ellipsoids that are not occluded and are visible from the current image viewpoint, for the visible Gaussian ellipsoids, obtaining the image pixel corresponding to the projection center, reading the semantic label of the pixel in the 2D semantic mask, filtering the labels of occluded and completely invisible images, and using only the labels of the valid observation images to participate in the voting.

[0033] A 3D Gaussian ellipsoid may be captured in multiple images. The semantic labels of the same 3D Gaussian ellipsoid are collected from all valid visible images. A majority voting strategy is used to select the semantic label with the highest frequency as the final semantic category of the 3D Gaussian ellipsoid. If the number of votes for each category is the same, labels for key components or main equipment categories can be prioritized, while background interference labels are weakened. Ultimately, each 3D Gaussian ellipsoid is assigned a semantic label. The entire 3D model is no longer simply geometric and color information, but becomes a semantic 3D model with an identity ID that can be understood and queried by computers.

[0034] Power scene prompts include main equipment prompts, key component prompts, and background interference prompts. Main equipment prompts describe the main equipment in the power scene, including transformers, circuit breakers, disconnect switches, current transformers, and voltage transformers. Key component prompts describe specific parts of the power equipment, including insulators, bushings, conductors, heat sinks, and vibration dampers. Background interference prompts distinguish non-core power equipment areas in the scene and reduce background pixel interference on component segmentation; these include the sky, ground, grass, trees, and walls.

[0035] This embodiment addresses the issue of fractured reconstructions of slender structures such as bushings and pipelines in power inspection scenarios. For slender structures within these scenarios, it extracts the eigenvalues ​​of the 3D Gaussian ellipsoid covariance matrix and estimates the local surface normals by combining this with the distribution of the neighborhood 3D Gaussian ellipsoid. The surface orientation of the component is derived from the Gaussian ellipsoid's own geometric distribution, eliminating the need for an external depth prediction network and enabling the acquisition of surface geometric information, thus adapting to power inspection modeling scenarios. A linear consistency constraint loss is constructed by combining the eigenvalues ​​of the covariance matrix and the local surface normals. This constraint constrains the proportions of the three axes of the 3D Gaussian ellipsoid, limiting excessive ellipsoid expansion. Furthermore, the local surface normals constrain the spatial orientation of the 3D Gaussian ellipsoid, forcing the minor axis of the Gaussian ellipsoid to align with the surface normal of the component, and the major axis to extend tangentially along the slender component. This results in the Gaussian ellipsoid appearing as a continuous, slender, thin sheet that fits the slender cylindrical equipment such as pipelines and bushings, eliminating the fracture problem in the reconstruction of slender structures and improving the integrity of the 3D reconstruction of power scenarios. By fusing the photometric loss from image reconstruction with the linear consistency constraint loss to form a joint 3D reconstruction loss function, texture rendering constraints and geometric constraints of slender structures are completed simultaneously. When optimizing the 3D Gaussian ellipsoid parameters, the texture and color of the reconstructed model are consistent with the real inspection image, while the geometric constraints maintain the continuity and integrity of slender components, thus achieving the geometric integrity and rendering accuracy of the 3D reconstruction model.

[0036] As one implementation, this embodiment provides a 3D reconstruction system for power line inspection scenarios, comprising: an acquisition module configured to acquire camera pose and initial sparse point cloud of the power line inspection scenario, and establish a 3D Gaussian representation containing a 3D Gaussian ellipsoid based on the camera pose and the initial sparse point cloud; a calculation module configured to calculate the eigenvalues ​​of the covariance matrix of the 3D Gaussian ellipsoid for slender structures within the power line inspection scenario in the 3D Gaussian representation, and estimate local surface normals based on the distribution of neighborhood 3D Gaussian ellipsoids; a constraint loss module configured to construct a linear consistency constraint loss based on the eigenvalues ​​and the local surface normals, the linear consistency constraint loss being used to constrain the eigenvalue ratio and spatial orientation of the 3D Gaussian ellipsoid to fit the slender structure; and a 3D reconstruction module configured to construct a 3D reconstruction loss function by combining image reconstruction photometric loss and the linear consistency constraint loss, optimize the parameters of the 3D Gaussian ellipsoid based on the 3D reconstruction loss function, and output a 3D reconstruction model of the power line inspection scenario after iterative convergence.

[0037] As one implementation, this embodiment provides a terminal device, including a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method described above.

[0038] As one implementation method, this embodiment provides a storage medium storing a computer program, which is called and executed by a computer to implement the method described above.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solutions of this application, as long as they do not depart from the spirit and scope of the technical solutions of this application, should be covered within the scope of the claims of this application.

Claims

1. A three-dimensional reconstruction method for power line inspection scenarios, characterized in that, include: The camera pose and initial sparse point cloud of the power inspection scene are obtained, and a three-dimensional Gaussian representation containing a three-dimensional Gaussian ellipsoid is established based on the camera pose and the initial sparse point cloud. In the three-dimensional Gaussian representation, for the slender structure in the power inspection scenario, the eigenvalues ​​of the covariance matrix of the three-dimensional Gaussian ellipsoid are calculated, and the local surface normal is estimated based on the distribution of the neighborhood three-dimensional Gaussian ellipsoid. A linear consistency constraint loss is constructed based on the eigenvalues ​​and the local surface normals. The linear consistency constraint loss is used to constrain the eigenvalue ratio and spatial orientation of the three-dimensional Gaussian ellipsoid to fit the slender structure. A three-dimensional reconstruction loss function is constructed by combining the image reconstruction photometric loss and the linear consistency constraint loss. The parameters of the three-dimensional Gaussian ellipsoid are optimized based on the three-dimensional reconstruction loss function. After iterative convergence, a three-dimensional reconstruction model of the power inspection scene is output.

2. The three-dimensional reconstruction method for power line inspection scenarios according to claim 1, characterized in that, The step of constructing a linear consistency constraint loss based on the eigenvalues ​​and the local surface normals includes: Using the local surface normal as a constraint reference, the three eigenvalues ​​corresponding to the covariance matrix of the three-dimensional Gaussian ellipsoid are obtained; The longest axis of the three-dimensional Gaussian ellipsoid is constrained to be perpendicular to the local surface normal, and the eigenvalue of the longest axis of the three-dimensional Gaussian ellipsoid is constrained to be greater than the sum of the eigenvalues ​​of the remaining two axes. The linear consistency constraint loss value is calculated based on the ratio of the longest axis eigenvalue to the sum of the remaining two axis eigenvalues.

3. The three-dimensional reconstruction method for power line inspection scenarios according to claim 1, characterized in that, The step of constructing a 3D reconstruction loss function by combining the image reconstruction photometric loss and the linear consistency constraint loss includes: In the 3D Gaussian representation, a plane consistency constraint loss is constructed for the planar structure in the power inspection scenario. The plane consistency constraint loss is used to constrain the 3D Gaussian ellipsoid to have a flattened distribution. A 3D reconstruction loss function is constructed by combining the image reconstruction photometric loss, the linear consistency constraint loss, and the planar consistency constraint loss.

4. The three-dimensional reconstruction method for power line inspection scenarios according to claim 1, characterized in that, The steps of acquiring the camera pose and initializing the sparse point cloud for the power line inspection scene include: A multi-view image of a power inspection scene is acquired, and feature extraction, feature matching, camera pose estimation, and sparse point cloud recovery are performed on the multi-view image to obtain the camera pose and initialized sparse point cloud of the power inspection scene.

5. A three-dimensional reconstruction method for power line inspection scenarios according to claim 4, characterized in that, The step of acquiring multi-view images of a power inspection scene includes: A video stream and RGB image sequence of a power inspection scene are acquired. The video stream is then frame-segmented at a fixed frame rate, and the RGB image sequence is sorted to obtain a multi-view image of the power inspection scene.

6. A three-dimensional reconstruction method for power line inspection scenarios according to claim 4, characterized in that, After the step of outputting the 3D reconstruction model of the power inspection scene after iterative convergence, the following steps are also included: Perform semantic segmentation on the multi-view image to generate a two-dimensional semantic mask; Based on the camera pose, the two-dimensional semantic mask is mapped onto the three-dimensional Gaussian ellipsoid in the three-dimensional reconstruction model to obtain a three-dimensional reconstruction model with semantic information.

7. A three-dimensional reconstruction method for power line inspection scenarios according to claim 6, characterized in that, The step of semantic segmentation of the multi-view image and generation of a two-dimensional semantic mask includes: Construct power scenario prompts that include the names of power equipment; An open vocabulary segmentation model is used to perform vocabulary detection and semantic segmentation on the multi-view image based on the power scene prompts, in order to generate a two-dimensional semantic mask.

8. A three-dimensional reconstruction method for power line inspection scenarios according to claim 7, characterized in that, The power scene prompts include main equipment prompts, key component prompts, and background interference prompts. The main equipment prompts describe the main equipment in the power scene, the key component prompts describe local components on the power equipment, and the background interference prompts describe the sky, ground, walls, cabinets, buildings, or other non-core equipment areas.

9. A three-dimensional reconstruction method for power line inspection scenarios according to claim 1, characterized in that, The parameters of the three-dimensional Gaussian ellipsoid include spatial position, covariance matrix, opacity, and color parameters.

10. A three-dimensional reconstruction system for power line inspection scenarios, characterized in that, include: The acquisition module is configured to acquire the camera pose and initial sparse point cloud of the power inspection scene, and establish a three-dimensional Gaussian representation containing a three-dimensional Gaussian ellipsoid based on the camera pose and the initial sparse point cloud. The calculation module is configured to calculate the eigenvalues ​​of the covariance matrix of the three-dimensional Gaussian ellipsoid for the slender structure in the power inspection scenario in the three-dimensional Gaussian representation, and estimate the local surface normal based on the distribution of the neighborhood three-dimensional Gaussian ellipsoid. The constraint loss module is configured to construct a linear consistency constraint loss based on the eigenvalues ​​and the local surface normals. The linear consistency constraint loss is used to constrain the eigenvalue ratios and spatial orientation of the three-dimensional Gaussian ellipsoid to fit the slender structure. The 3D reconstruction module is configured to construct a 3D reconstruction loss function by combining the image reconstruction photometric loss and the linear consistency constraint loss, optimize the parameters of the 3D Gaussian ellipsoid based on the 3D reconstruction loss function, and output a 3D reconstruction model of the power inspection scene after iterative convergence.

11. A terminal device, characterized in that, The method includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method as described in any one of claims 1 to 9.

12. A storage medium, characterized in that, The storage medium stores a computer program, which is invoked and executed by a computer to implement the method as described in any one of claims 1 to 9.

Images