Gaussian cell semantic injection method based on multi-view consistency and credibility gating

Abstract

The application discloses a Gaussian primitive semantic injection method based on multi-view consistency and credibility gating, and belongs to the technical field of three-dimensional vision and scene perception. The method comprises the following steps: constructing a sparse point cloud and camera parameters by a motion structure recovery algorithm and initializing a Gaussian primitive; extracting a power distribution tower center main rod and a vertical main shaft component level structure topology; projecting the Gaussian primitive to a training image to obtain a two-dimensional semantic probability distribution, calculating a semantic field space consistency component, a multi-view coverage component and a structure topology consistency component, and obtaining a semantic confidence factor by weighted fusion; according to the Gaussian primitive, the Gaussian primitive is divided into high, medium and low credibility primitives, and semantic voting, state maintenance or semantic gradient return blocking and RGB rendering contribution reservation are activated respectively; and the spatial topology structure is perfected and the missed structure is completed, and iteration is carried out until convergence. The application can eliminate the false annotation and semantic adhesion problems caused by the tree shelter and low texture area in the distribution network inspection.

Description

Technical Field

[0001] This invention relates to the field of 3D vision and scene perception technology, specifically to multi-view based... Figure 1 A Gaussian semantic injection method with consistency and credibility gating. Background Technology

[0002] In the field of power line inspection, 3D Gaussian Splatting (3DGS) uses a set of anisotropic Gaussian primitives to explicitly parametrically model the scene, offering significant advantages in rendering quality and real-time performance. It is gradually being introduced into 3D reconstruction and semantic understanding tasks in power line scenarios. Existing solutions typically use 2D semantic perception models to obtain semantic information from training views, and then pass semantic labels to Gaussian primitives through multi-view projection voting. However, during the 3DGS optimization process, floating Gaussians and semi-transparent artifacts inevitably occur in spatially sparse or occluded regions. These primitives do not correspond to real physical surfaces, yet they participate in semantic voting indiscriminately under the unconditional fusion paradigm, injecting incorrect labels into the 3D semantic field.

[0003] In distribution network inspection scenarios, the aforementioned problems are exacerbated by two unique factors: First, dense vegetation along distribution network lines causes trees and leaves to obscure electrical equipment from multiple angles. The obscured areas lack effective multi-view geometric constraints, leading to depth estimation errors and the generation of numerous floating primitives at the obstruction boundaries. Second, low-texture areas such as concrete poles, crossarms, and the sky background provide weak photometric gradient signals, resulting in insufficient depth optimization and further generating semi-transparent artifacts with positional drift. These floating primitives, derived from depth ambiguity rather than physical surfaces, participate in semantic voting indiscriminately under the unconditional fusion paradigm, causing systematic pollution of the 3D semantic field. Existing technologies lack quantitative evaluation methods for the geometric credibility of Gaussian primitives, making it impossible to distinguish between surface primitives with sufficient depth constraints and floating artifacts caused by occlusion or low texture at the decision-making level. Therefore, a primitive quality assessment and semantic injection method for complex distribution network scenarios is urgently needed. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-view based Figure 1 The Gaussian primitive semantic injection method with consistency and credibility gating obtains a two-dimensional semantic probability distribution by projecting Gaussian primitives, calculates the semantic field spatial consistency component, multi-view coverage component, and structural topology consistency component, and fuses them to obtain a semantic confidence factor. Based on this, Gaussian primitives are divided into high, medium, and low confidence primitives and differential semantic injection is performed. The semantic confidence is enhanced by utilizing the spatial topology of distribution towers to eliminate the problems of floating artifact mislabeling and semantic adhesion caused by occlusion and low texture in distribution network inspection.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a multi-view based Figure 1 The Gaussian meta-semantic injection method with consistency and credibility gating includes the following steps: Collect multiple training images of power distribution towers; Sparse point cloud and camera parameters are constructed using multiple training images through a motion structure recovery algorithm. Multiple Gaussian elements are initialized with each cloud point in the sparse point cloud as the center. Each Gaussian element includes position parameters and semantic embedding vectors. The center of the power distribution tower is determined based on the camera parameters, and the main pole and vertical main axis of the power distribution tower are extracted to form a component-level structural topology. The position parameters of each Gaussian primitive are projected onto the image plane of each training image, and the visibility of each Gaussian primitive in each training image is determined. The training image that meets the preset visibility condition is the visible training image of that Gaussian primitive. The training image is input into the two-dimensional semantic perception model to obtain the two-dimensional semantic probability distribution of each Gaussian primitive at the projection position of the visible training image. For each Gaussian element, the semantic field spatial consistency component is calculated based on the two-dimensional semantic probability distribution. The ratio of the number of visible training images of the Gaussian element to the total number of training images is calculated to obtain the multi-view coverage component. The spatial topology of the power distribution tower is constructed based on the component-level structural topology and the structural topology consistency component is calculated. The semantic field spatial consistency component, the multi-view coverage component, and the structural topology consistency component are weighted and fused to obtain the semantic confidence factor. Based on the semantic confidence factor, and by comparing it with the preset high confidence threshold and low confidence threshold, each Gaussian primitive is divided into high confidence primitive, medium confidence primitive, and low confidence primitive. The semantic embedding vector of the high confidence primitive is activated to participate in multi-view semantic voting and injection. The semantic embedding vector of the medium confidence primitive remains unchanged. The low confidence primitive is given an empty semantic label and its semantic gradient backpropagation is blocked, while its RGB rendering contribution remains unchanged. The extracted central main pole and structural components of the power distribution tower are used to improve the spatial topology of the power distribution tower. The spatial topology nodes in the spatial topology of the power distribution tower are used to enhance the semantic confidence of the Gaussian elements. The process is iterated until the semantic confidence factors of each Gaussian element converge.

[0006] Furthermore, based on the camera parameters, the center of the power distribution tower is determined, and the central main pole and the vertical main axis of the power distribution tower are extracted to form a component-level structural topology; the specific steps are as follows: The surrounding center of the power distribution tower is determined based on the camera parameters, and the surrounding center is used as the initial position of the center of the power distribution tower. Principal component analysis and point cloud fitting are performed on the sparse point cloud within a defined neighborhood of the initial position of the distribution tower center to extract the central main pole and the vertical main axis of the distribution tower. The component-level structural topology is formed by taking the center position of the main pole of the power distribution tower as the center of the power distribution tower and the extension direction of the vertical main axis of the power distribution tower.

[0007] Furthermore, the preset visibility condition is as follows: after the Gaussian primitive is projected onto the image plane of the training image, the projection point of the Gaussian primitive falls within the effective area of ​​the image plane, and the Gaussian primitive is not completely occluded by other Gaussian primitives in the depth sorting along the viewing direction.

[0008] Furthermore, the semantic field spatial consistency component is calculated as follows: ; Where k is the Gaussian element index, d is the semantic category index, and D is the total number of semantic categories. Let k be the semantic field space consistency component of the k-th Gaussian element. For the set of visible training images of the k-th Gaussian element, Let v be the number of visible training images in the visible training image set, and v be the index of the visible training image in the visible training image set. Let be the probability that the k-th Gaussian element belongs to the d-th semantic category, as output by the two-dimensional semantic perception model at the projection position of the v-th visible training image.

[0009] Furthermore, the structural topology consistency component is used to characterize the reliability of the actual location of the Gaussian element in the structural component declared by the spatial topology of the distribution tower; the structural topology consistency component is calculated as follows: ; in, For the k-th Gaussian element, the structural topological consistency component is... The hierarchical matching degree of the kth Gaussian element is determined based on the degree of matching between the height of the kth Gaussian element and the height range of the corresponding structural component's hierarchy. The normalized attachment distance value of the kth Gaussian element is determined based on the ratio of the vertical distance from the kth Gaussian element to the central main pole of the power distribution tower to the preset maximum attachment distance. The morphological matching degree of the kth Gaussian element is determined based on the angle between the direction of the first principal component of the local neighborhood where the kth Gaussian element is located and the extension direction of the corresponding structural component. The multi-view depth consistency of the k-th Gaussian element is determined based on the standard deviation of the projection depth of the k-th Gaussian element in each visible training image. These are the weighting coefficients.

[0010] Furthermore, the semantic confidence factor is calculated as follows: ; in, For the first The semantic confidence factor of Gaussian elements, For the first Semantic field space consistency components of Gaussian elements. For the first The multi-view coverage component of the Gaussian element. For the first The structural topological consistency component of each Gaussian element. The balance coefficients for the semantic field space consistency components. The balance coefficient for the multi-view coverage components. Let be the balance coefficient of the structural topological consistency component, and .

[0011] Furthermore, the balance coefficient of the structural topology consistency component Adaptively increasing with the number of iterations: The structural self-confidence component is calculated based on the degree to which the spatial topology of the power distribution tower is verified by the Gaussian elements. This structural self-confidence component is determined by at least one of the following: the interior point ratio fitted to the central main pole of the tower; the spatial coverage density of Gaussian elements at the structural component locations declared by the spatial topology of the power distribution tower; and the consistency ratio between the Gaussian element semantic labels and the spatial topology of the power distribution tower. As the structural self-confidence component increases, while maintaining... Under the premise of monotonically increasing the balance coefficient of the topological consistency component of the structure And correspondingly reduce the balance coefficient of the semantic field spatial consistency component. Balance coefficient with multi-view coverage components .

[0012] Further, based on the semantic confidence factor compared with preset high confidence thresholds and low confidence thresholds, each Gaussian primitive is divided into high confidence primitives, medium confidence primitives, and low confidence primitives; the specific steps are as follows: When the semantic confidence factor is greater than or equal to the high confidence threshold, the Gaussian primitive is classified as a high confidence primitive; when the semantic confidence factor is less than the low confidence threshold, the Gaussian primitive is classified as a low confidence primitive; when the semantic confidence factor is greater than or equal to the low confidence threshold and less than the high confidence threshold, the Gaussian primitive is classified as a medium confidence primitive.

[0013] Furthermore, a preheating iteration count is set. The classification of high-confidence primitives, medium-confidence primitives, and low-confidence primitives is temporarily suspended until the preheating iteration count is reached. Once classification begins, the semantic confidence factors of all high-confidence primitives are recalculated and classification is re-executed every preset update iteration count. If a medium-confidence primitive's semantic confidence factor reaches or exceeds the high-confidence threshold in subsequent re-evaluations, it is upgraded to a high-confidence primitive, and its semantic embedding vector is activated, allowing it to resume participation in multi-view semantic voting and injection.

[0014] Furthermore, the spatial topology of the power distribution tower is composed of the position nodes of the central main pole, crossarms, insulators, conductors, and surge arresters; improving the spatial topology of the power distribution tower includes completing the unidentified structural components: When the two-dimensional semantic perception model only identifies the structural components on one side of the power distribution tower but not the symmetrical structural components on the other side, the position of the identified structural components on one side of the power distribution tower is mirrored about the axis of symmetry, taking the central main pole of the power distribution tower as the axis of symmetry, to obtain the spatial position of the symmetrical structural components on the other side that were not identified. Within a preset distance range of the spatial location, search for Gaussian primitives. For Gaussian primitives that have not yet been assigned a semantic label or are in the medium confidence primitive category, if the hierarchical matching degree of the Gaussian primitive is greater than a preset hierarchical threshold, and the attachment distance normalized value of the Gaussian primitive is greater than a preset attachment threshold, and the morphological matching degree of the Gaussian primitive is greater than a preset morphological threshold, then assign a semantic label to the unidentified structural component and set the value of the semantic confidence factor of the Gaussian primitive to be higher than the high confidence threshold, so that the Gaussian primitive is classified as a high confidence primitive. If the hierarchical matching degree of the Gaussian primitive is not greater than the preset hierarchical threshold, or the normalized value of the attachment distance of the Gaussian primitive is not greater than the preset attachment threshold, or the morphological matching degree of the Gaussian primitive is not greater than the preset morphological threshold, then the Gaussian primitive is treated as a low-confidence primitive.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention constructs a three-dimensional confidence evaluation system by using semantic field spatial consistency components, multi-view coverage components, and structural topology consistency components. It comprehensively evaluates the confidence of Gaussian primitives from three dimensions: semantic stability, geometric constraint sufficiency, and structural prior matching degree, thereby achieving the distinction between real surface primitives and floating artifact primitives.

[0016] This invention employs a three-state gating classification and differentiated semantic injection strategy. It activates semantic voting for high-confidence primitives, maintains the state of medium-confidence primitives for re-evaluation, and blocks semantic gradient backpropagation for low-confidence primitives while retaining their RGB rendering contribution. It replaces unconditional fusion with selective injection, thereby eliminating mislabeling and semantic adhesion caused by floating artifacts at the decision-making level.

[0017] This invention introduces a structural topology consistency component, using the analytical geometric prior of power distribution towers as a source of structural evidence independent of the two-dimensional semantic perception model. Even if the two-dimensional semantic model has systematic false detections or missed detections, as long as the position of the Gaussian unit does not conform to the spatial topology of the power distribution tower, it will be blocked or downweighted, thus breaking through the bottleneck of semantic quality being limited by the upper limit of the two-dimensional model.

[0018] This invention utilizes the spatial topology of power distribution towers to complete the detection of missing structural components. Based on symmetry and hierarchical constraints, it extrapolates the spatial location of unidentified structural components, backfills semantic labels on neighboring Gaussian units and improves confidence, effectively salvaging the missed detections of the two-dimensional semantic perception model and improving the completeness of semantic injection of structural components.

[0019] In this invention, the semantic field spatial consistency component and the multi-view coverage component share the same set of visible training images and are calculated together during the semantic projection process. This eliminates the need to maintain additional gradient statistics or learnable parameters, does not increase computational overhead, and is applicable to any semantic understanding framework based on 3D Gaussian splashing. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention.

[0021] Figure 2 This is a semantic rendering diagram of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific examples. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0023] In this embodiment, based on the representation of a three-dimensional Gaussian splash scene (3DGS), the application scenario is the three-dimensional semantic perception and reconstruction of core equipment components such as poles, crossarms, insulators, conductors, and surge arresters in power distribution network inspection.

[0024] like Figure 1 As shown, the present invention provides a multi-view based Figure 1 The Gaussian meta-semantic injection method with consistency and credibility gating includes the following steps: S1: Collect multiple training images of power distribution towers.

[0025] In this embodiment, the inspection drone collects a set of real visible light images from multiple perspectives around the power distribution tower, which serve as multiple training images for the 3D Gaussian splash scene. The number of training images ranges from 10 to 500, with the specific number determined based on the scene size and the drone's flight path planning to ensure sufficient coverage of the power distribution tower. Optionally, the visible light image set is collected in a circular pattern, with the camera's optical center approximately circling the target power distribution tower.

[0026] S2: Construct sparse point cloud and camera parameters using multiple training images through a motion structure recovery algorithm, and initialize multiple Gaussian primitives with each cloud point in the sparse point cloud as the center.

[0027] A sparse point cloud and camera parameters (camera pose and intrinsic parameters) are constructed from the multiple training images using the Structure from Motion (SfM) algorithm. Multiple Gaussian primitives (GRPs) are initialized centered on each point in the sparse point cloud. Each GRP includes positional parameters and a semantic embedding vector. Each GRP also contains parameters such as covariance matrix, opacity, and spherical harmonic coefficients, where the dimension of the semantic embedding vector is equal to the total number of semantic categories, D.

[0028] S3: Determine the center of the power distribution tower based on the camera parameters and extract the central main pole and vertical main axis of the power distribution tower to form a component-level structural topology.

[0029] The surrounding center of the power distribution tower is determined based on the camera parameters. Specifically, the average of the optical center coordinates of each visible camera is used to obtain the surrounding center, which is then used as the initial position of the power distribution tower center. Within a defined neighborhood of the initial position of the power distribution tower center, principal component analysis (PCA) and random sampling fitting (such as RANSAC fitting) are performed on the sparse point cloud to obtain slender main rod chains with high local density extending in a near-vertical direction as candidate main rods. The center position of the candidate main rods is used as the center of the power distribution tower, and the principal component direction of the candidate main rods is used as the extension direction of the vertical main axis of the power distribution tower. The position of the power distribution tower center and the extension direction of the vertical main axis of the power distribution tower constitute the component-level structural topology, which includes the position of the power distribution tower center and the extension direction of the vertical main axis of the power distribution tower.

[0030] S4: Project the position parameters of each Gaussian primitive onto the image plane of each training image, determine the visibility of each Gaussian primitive in each training image, and designate the training image that meets the preset visibility condition as the visible training image of that Gaussian primitive; input the training image into the two-dimensional semantic perception model to obtain the two-dimensional semantic probability distribution of each Gaussian primitive at the projection position of the visible training image.

[0031] The position parameters of each Gaussian primitive are projected onto the image plane of each training image according to the camera parameters, and the visibility of each Gaussian primitive in each training image is determined. The preset visibility condition is: after the Gaussian primitive is projected onto the image plane of the training image, the projection point of the Gaussian primitive falls within the effective area of ​​the image plane (i.e., the pixel coordinates are within the width and height range of the image), and the Gaussian primitive is not completely occluded by other Gaussian primitives in the depth sorting along the viewing direction. The viewing direction is the direction of light rays from the optical center of the camera, passing through the pixels of the training image, and pointing to three-dimensional space. Training images that meet the preset visibility condition are the visible training images of that Gaussian primitive.

[0032] The specific method for occlusion determination is as follows: For a certain pixel position in the training image, all visible Gaussian elements at that pixel position along the viewing direction are sorted in ascending order of depth value, and the opacity of each Gaussian element is accumulated in turn. If the accumulated opacity reaches or exceeds a preset threshold (e.g., 0.95), then all Gaussian elements after the Gaussian element corresponding to the accumulated opacity are determined to be invisible.

[0033] The two-dimensional semantic probability distribution of each Gaussian unit at its projection position in each visible training image is extracted. This two-dimensional semantic probability distribution is obtained by a two-dimensional semantic perception model outputting the probability of each category or the semantic prior at the region level for each training image. The two-dimensional semantic perception model includes, but is not limited to, models based on 2D object detection algorithms (such as the YOLO series, which outputs the 2D rectangular bounding box of the object as a coarse-grained semantic region distribution), segmentation models based on deep convolutional neural networks, semantic segmentation models based on Transformers (such as the Segment Anything Model), or multimodal visual language models (such as CLIP). When using an object detection model, the internal region of the output 2D rectangular bounding box can be assigned the semantic prior probability of the corresponding category, thereby constructing a coarse-grained two-dimensional semantic probability distribution. Any model capable of directly or indirectly outputting pixel-level or region-level semantic category probability distributions is applicable to this invention.

[0034] S5: For each Gaussian element, calculate the semantic field spatial consistency component based on the two-dimensional semantic probability distribution, calculate the multi-view coverage component by the ratio of the number of visible training images of the Gaussian element to the total number of training images, construct the distribution tower spatial topology based on the component-level structural topology and calculate the structural topology consistency component, and weightedly fuse the semantic field spatial consistency component, the multi-view coverage component and the structural topology consistency component to obtain the semantic confidence factor.

[0035] For each Gaussian element, the following three components are computed: Semantic field spatial consistency component: Project the Gaussian element onto all its visible training images, calculate the average probability of each semantic category across all visible training images, and take the maximum value among the average probabilities of all semantic categories. Where k is the Gaussian element index, d is the semantic category index, and D is the total number of semantic categories. Let k be the semantic field space consistency component of the k-th Gaussian element. For the set of visible training images of the k-th Gaussian element, Let v be the number of visible training images in the visible training image set, and v be the index of the visible training image in the visible training image set. Let be the probability that the k-th Gaussian element belongs to the d-th semantic category, as output by the two-dimensional semantic perception model at the projection position of the v-th visible training image.

[0036] Multi-view coverage component: The ratio of the number of visible training images to the total number of training images for this Gaussian element.

[0037] Structural topology consistency component: used to characterize the reliability of the actual location of the structural component as declared by the spatial topology of the distribution tower. The structural topology consistency component is calculated as follows: ; in, For the k-th Gaussian element, the structural topological consistency component is... The hierarchical matching degree of the kth Gaussian element is determined based on the degree of matching between the height of the kth Gaussian element and the height range of the corresponding structural component's hierarchy. For example, the value is 1 when the height of the Gaussian element falls within the height range of the corresponding structural component's hierarchy, and decreases according to a Gaussian function as the deviation distance increases when it deviates from the height range.

[0038] The normalized attachment distance value of the kth Gaussian element is determined based on the ratio of the vertical distance from the kth Gaussian element to the central main pole of the power distribution tower to the preset maximum attachment distance. ;in, Vertical distance For the preset maximum attachment distance (e.g., 2 meters), when hour, . The shape matching degree of the k-th Gaussian element is determined based on the angle between the first principal component direction of the local neighborhood where the k-th Gaussian element is located and the extension direction of the corresponding structural component. The first principal component direction is determined as follows: taking the k-th Gaussian element as the center, a spherical neighborhood with radius r (e.g., r=0.3) is taken. Principal component analysis is performed on the positions of all Gaussian elements within the spherical neighborhood to obtain three principal component directions and their corresponding variances. The principal component direction with the largest variance is taken as the first principal component direction of the local neighborhood. The cosine value of the angle between the first principal component direction and the extension direction of the corresponding structural component is used as the shape matching degree. The multi-view depth consistency of the k-th Gaussian element is determined based on the standard deviation of the projected depth of the k-th Gaussian element in each visible training image, for example... ,in, is the standard deviation of the projected depth values ​​in each visible training image; This is the weighting coefficient. In this embodiment... , , , .

[0039] The semantic confidence factor is obtained by weighted fusion of the above three components: ;in, For the first The semantic confidence factor of Gaussian elements, For the first Semantic field space consistency components of Gaussian elements. For the first The multi-view coverage component of the Gaussian element. For the first The structural topological consistency component of each Gaussian element. The balance coefficients for the semantic field space consistency components. The balance coefficient for the multi-view coverage components. Let be the balance coefficient of the structural topological consistency component, and In this embodiment , , .

[0040] Furthermore, the balance coefficient of the structural topology consistency component Adaptively increases with the number of iterations: The structural self-confidence component is calculated based on the degree to which the spatial topology of the power distribution tower is verified by the Gaussian elements, and the structural self-confidence component is determined by at least one of the following three items: (1) The in-point ratio of the fitting of the main pole at the center of the tower, that is, the ratio of the number of in-points to the total number of point clouds participating in the fitting when the Random Sampling Consensus Algorithm (RANSAC) is used, and the value ranges from 0 to 1.

[0041] (2) The spatial coverage density of the Gaussian elements at the location of the structural component declared by the spatial topology of the power distribution tower is defined as the range centered on the declared location of the structural component and with the typical size of the structural component as the radius (for example, if the typical size of the insulator is 0.3 meters, then the spherical area centered on the declared location of the insulator and with a radius of 0.3 meters is the location of the insulator). The actual number of Gaussian elements in this range is counted, and the ratio of the number of Gaussian elements to the expected number of elements is calculated as the spatial coverage density, with a value range of 0 to 1.

[0042] (3) The consistency ratio between the Gaussian semantic tag and the spatial topology of the power distribution tower, that is, for the Gaussian element that falls at the location of the structural component declared by the spatial topology of the power distribution tower, the semantic category corresponding to the maximum probability after the semantic embedding vector of the Gaussian element is processed by the normalized exponential function is compared with the structural component category declared by the structural topology at the location of the Gaussian element; the consistency ratio is the number of Gaussian elements whose semantic category is consistent with the structural component category divided by the total number of Gaussian elements participating in the evaluation that fall at the location of the structural component, and the value range is from 0 to 1.

[0043] The location of the structural component refers to the area within which the location of the structural component as declared by the spatial topology of the power distribution tower is centered and the typical size of the structural component is radii.

[0044] If one of the above three items is selected, the structural self-confidence component is directly equal to that item; if multiple of the above three items are selected, the structural self-confidence component is the weighted average of the selected items, and the sum of the weight coefficients is 1.

[0045] After obtaining the structural self-confidence component, as the structural self-confidence component increases, while maintaining... Under the premise of monotonically increasing the balance coefficient of the topological consistency component of the structure And correspondingly reduce the balance coefficient of the semantic field spatial consistency component. Balance coefficient with multi-view coverage components When the structural self-confidence component is low, the intervention of structural evidence is weakened, and the semantic field spatial consistency component and multi-view coverage component become dominant. As the iteration progresses, the weight of structural evidence gradually increases, and the constraint of structural evidence on the semantic confidence factor gradually strengthens. The above adjustments are performed according to preset rules and do not introduce learnable parameters.

[0046] S6: Based on the semantic confidence factor, compare it with the preset high confidence threshold and low confidence threshold, and divide each Gaussian primitive into high confidence primitive, medium confidence primitive and low confidence primitive; activate the semantic embedding vector of the high confidence primitive to participate in multi-view semantic voting and injection; keep the semantic embedding vector of the medium confidence primitive unchanged; assign an empty semantic label to the low confidence primitive and block its semantic gradient backpropagation, while keeping its RGB rendering contribution unchanged.

[0047] Specifically, when the semantic confidence factor is greater than or equal to the high confidence threshold, the Gaussian primitive is classified as a high-confidence primitive; when the semantic confidence factor is less than the low confidence threshold, the Gaussian primitive is classified as a low-confidence primitive; and when the semantic confidence factor is greater than or equal to the low confidence threshold and less than the high confidence threshold, the Gaussian primitive is classified as a medium-confidence primitive. In this embodiment, the high confidence threshold is set to 0.6, and the low confidence threshold is set to 0.3.

[0048] Specifically, for high-confidence primitives, the two-dimensional semantic information of each visible training image is fused and updated to the semantic embedding vector of that primitive. This can be done using an exponential moving average method. .in, Let be the semantic embedding vector of the k-th Gaussian element before the t-th semantic injection, and its dimension is equal to the total number of semantic categories D; For the first During the next injection, the Gaussian primitive is in the currently visible training image. The two-dimensional semantic probability distribution vector retrieved at the projection position is... A dimensional vector whose elements sum to 1; The momentum coefficient has a value greater than 0 and less than 1. In this embodiment, it is taken as... =0.9.

[0049] For low-confidence primitives, the semantic embedding vector of the low-confidence primitive is set to zero, and the gradient stopping flag of the low-confidence primitive is set to block the gradient backpropagation of the semantic embedding vector, so that the low-confidence primitive does not participate in the semantic backpropagation update; however, the low-confidence primitive can still be retained in RGB rendering to maintain the integrity of scene appearance reconstruction and achieve decoupling between semantic field and appearance field.

[0050] For primitives with medium confidence, maintain their current semantic state, that is, keep their current semantic embedding vector unchanged, and do not participate in this round of multi-view semantic voting.

[0051] A preheating iteration count is set. Before this count is reached, the classification of high-confidence, medium-confidence, and low-confidence primitives is temporarily suspended. Classification only begins after the preheating iteration count is reached. In this embodiment, the preheating iteration count is set to 3000, which can be adjusted between 1000 and 8000 depending on the scenario scale. The preheating mechanism is based on the fact that in the early stages, the geometric parameters of Gaussian primitives have not yet converged, and inaccurate position projection will cause the two-dimensional semantic probability query position to shift and the semantic confidence factor to be distorted. If gating is performed in the early stages, a large number of primitives whose geometry has not yet converged but could be optimized will be misjudged as low-confidence and blocked, resulting in irreversible loss of semantic information. After classification begins, the semantic confidence factor of all Gaussian primitives is recalculated and classification is re-executed every preset update iteration count. In this embodiment, the update iteration count is set to 500, which can be adjusted between 200 and 1000. If a medium-confidence primitive reaches or exceeds the high-confidence threshold in a subsequent re-evaluation, it is upgraded to a high-confidence primitive and its semantic embedding vector is activated, allowing it to resume participation in multi-view semantic voting and injection. This reversible upgrade mechanism represents a targeted improvement over traditional low-quality, i.e., permanent, one-way gating.

[0052] S7: Improve the spatial topology of the power distribution tower by using the extracted central main pole and structural components of the power distribution tower, and use the spatial topology nodes in the spatial topology of the power distribution tower to strengthen the semantic confidence of the Gaussian elements, iterating until the semantic confidence factors of each Gaussian element converge.

[0053] The extracted central main pole and structural components of the power distribution tower are used to refine the spatial topology of the power distribution tower. The spatial topology nodes within this topology are then used to enhance the semantic confidence of the Gaussian elements. As the iteration progresses, the structural components classified as high-confidence primitives are continuously enhanced and extracted, thus updating the spatial topology of the power distribution tower. The spatial point-line structure and its structural node values ​​become increasingly precise. The updated spatial topology nodes are used to iterate the next round of semantic confidence calculation and enhancement, forming a "structure-semantics-gating" closed loop. This iteration continues until the semantic confidence factors of each Gaussian element converge.

[0054] Furthermore, the spatial topology of the power distribution tower is composed of the location nodes of the central main pole, crossarms, insulators, conductors, and surge arresters. Improving the spatial topology of the power distribution tower includes completing the missing structural components: When the two-dimensional semantic perception model only identifies structural components on one side of the distribution tower and fails to identify symmetrical structural components on the other side, the position of the identified structural components on one side is mirrored about the axis of symmetry, using the central main pole of the distribution tower as the axis of symmetry, to obtain the spatial position of the unidentified symmetrical structural components on the other side. For example, when the two-dimensional semantic perception model only identifies the insulator on the right crossarm, the position of the right insulator is mirrored about the central main pole, using the central main pole of the distribution tower as the axis of symmetry, to obtain the spatial position of the left insulator.

[0055] Within a preset distance range (e.g., 0.3 meters) of the spatial location, Gaussian primitives are searched. For Gaussian primitives found that are not currently assigned a semantic label or are in the medium confidence level, if the hierarchical matching degree of the Gaussian primitive is greater than a preset hierarchical threshold (e.g., 0.6), the normalized attachment distance value of the Gaussian primitive is greater than a preset attachment threshold (e.g., 0.5), and the morphological matching degree of the Gaussian primitive is greater than a preset morphological threshold (e.g., 0.7), then a semantic label corresponding to the unidentified structural component is assigned, and the semantic confidence factor of the Gaussian primitive is set to a value higher than the high confidence threshold (e.g., set to 0.8), thus classifying the Gaussian primitive as a high confidence primitive. If the hierarchical matching degree of the Gaussian primitive is not greater than the preset hierarchical threshold, or the normalized attachment distance value of the Gaussian primitive is not greater than the preset attachment threshold, or the morphological matching degree of the Gaussian primitive is not greater than the preset morphological threshold, then the Gaussian primitive is treated as a low confidence primitive.

[0056] Thus, the spatial topology of the power distribution tower plays a dual role in semantic recognition, simultaneously verifying identified candidates and suppressing false detections, as well as filling in and recovering from missed detections in unidentified areas.

[0057] After the iterative process is completed, the semantic rendering stage begins. In this stage, when rendering semantics by weighted accumulation of Gaussian primitives along the viewing direction using opacity, the semantic contribution of each Gaussian primitive is as follows: if the Gaussian primitive is classified as a low-confidence primitive, its semantic contribution is zero; otherwise, its semantic contribution is the normalized exponential function value of the Gaussian primitive's semantic embedding vector, thus ensuring that semantics are presented only within geometrically reliable regions. Finally, the category corresponding to the maximum value of the accumulated pixel-by-pixel semantic vector is taken as the semantic category label for that pixel, and the semantic segmentation result for that viewpoint is output. Figure 2 As shown, the left image is the original RGB image, the middle image is the two-dimensional semantic image from the corresponding viewpoint, and the right image is the semantic image rendered from the same viewpoint after reconstruction. It can be seen that the method proposed in this invention has high reconstruction quality.

[0058] The power distribution towers are power distribution line towers, the training images are inspection images collected by drones from multiple perspectives around the power distribution towers, and the semantic categories include poles, crossarms, insulators, conductors, and surge arresters.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A Gaussian meta-semantic injection method based on multi-view consistency and credibility gating, characterized in that, Includes the following steps: Collect multiple training images of power distribution towers; Sparse point cloud and camera parameters are constructed using multiple training images through a motion structure recovery algorithm. Multiple Gaussian elements are initialized with each cloud point in the sparse point cloud as the center. Each Gaussian element includes position parameters and semantic embedding vectors. The center of the power distribution tower is determined based on the camera parameters, and the main pole and vertical main axis of the power distribution tower are extracted to form a component-level structural topology. The position parameters of each Gaussian primitive are projected onto the image plane of each training image, and the visibility of each Gaussian primitive in each training image is determined. The training image that meets the preset visibility condition is the visible training image of that Gaussian primitive. The training image is input into the two-dimensional semantic perception model to obtain the two-dimensional semantic probability distribution of each Gaussian primitive at the projection position of the visible training image. For each Gaussian element, the semantic field spatial consistency component is calculated based on the two-dimensional semantic probability distribution. The ratio of the number of visible training images of the Gaussian element to the total number of training images is calculated to obtain the multi-view coverage component. The spatial topology of the power distribution tower is constructed based on the component-level structural topology and the structural topology consistency component is calculated. The semantic field spatial consistency component, the multi-view coverage component, and the structural topology consistency component are weighted and fused to obtain the semantic confidence factor. Based on the semantic confidence factor, and by comparing it with the preset high confidence threshold and low confidence threshold, each Gaussian primitive is divided into high confidence primitive, medium confidence primitive and low confidence primitive; the semantic embedding vector of the high confidence primitive is activated to participate in multi-view semantic voting and injection; the semantic embedding vector of the medium confidence primitive remains unchanged. The low-confidence primitive is given an empty semantic label and its semantic gradient backpropagation is blocked, while its RGB rendering contribution remains unchanged. The extracted central main pole and structural components of the power distribution tower are used to improve the spatial topology of the power distribution tower. The spatial topology nodes in the spatial topology of the power distribution tower are used to enhance the semantic confidence of the Gaussian elements. The process is iterated until the semantic confidence factors of each Gaussian element converge.

2. The method according to claim 1, characterized in that, The center of the power distribution tower is determined based on the camera parameters, and the central main pole and vertical main axis of the power distribution tower are extracted to form a component-level structural topology; the specific steps are as follows: The surrounding center of the power distribution tower is determined based on the camera parameters, and the surrounding center is used as the initial position of the center of the power distribution tower. Principal component analysis and point cloud fitting are performed on the sparse point cloud within a defined neighborhood of the initial position of the distribution tower center to extract the central main pole and the vertical main axis of the distribution tower. The component-level structural topology is formed by taking the center position of the main pole of the power distribution tower as the center of the power distribution tower and the extension direction of the vertical main axis of the power distribution tower.

3. The method according to claim 2, characterized in that, The preset visibility condition is as follows: after the Gaussian primitive is projected onto the image plane of the training image, the projection point of the Gaussian primitive falls within the effective area of ​​the image plane, and the Gaussian primitive is not completely occluded by other Gaussian primitives in the depth sorting along the line of sight.

4. The method according to claim 3, characterized in that, The semantic field spatial consistency component is calculated as follows: ； Where k is the Gaussian element index, d is the semantic category index, and D is the total number of semantic categories. Let k be the semantic field space consistency component of the k-th Gaussian element. For the set of visible training images of the k-th Gaussian element, Let v be the number of visible training images in the visible training image set, and v be the index of the visible training image in the visible training image set. Let be the probability that the k-th Gaussian element belongs to the d-th semantic category, as output by the two-dimensional semantic perception model at the projection position of the v-th visible training image.

5. The method according to claim 4, characterized in that, The structural topology consistency component is used to characterize the reliability of the actual location of the Gaussian element in the structural component declared by the spatial topology of the distribution tower; the structural topology consistency component is calculated as follows: ； in, For the k-th Gaussian element, the structural topological consistency component is... The hierarchical matching degree of the kth Gaussian element is determined based on the degree of matching between the height of the kth Gaussian element and the height range of the corresponding structural component's hierarchy. The normalized attachment distance value of the kth Gaussian element is determined based on the ratio of the vertical distance from the kth Gaussian element to the central main pole of the power distribution tower to the preset maximum attachment distance. The morphological matching degree of the kth Gaussian element is determined based on the angle between the direction of the first principal component of the local neighborhood where the kth Gaussian element is located and the extension direction of the corresponding structural component. The multi-view depth consistency of the k-th Gaussian element is determined based on the standard deviation of the projection depth of the k-th Gaussian element in each visible training image. These are the weighting coefficients.

6. The method according to claim 5, characterized in that, The semantic confidence factor is calculated as follows: ； in, For the first The semantic confidence factor of Gaussian elements, For the first Semantic field space consistency components of Gaussian elements. For the first The multi-view coverage component of the Gaussian element. For the first The structural topological consistency component of each Gaussian element. The balance coefficients for the semantic field space consistency components. The balance coefficient for the multi-view coverage components. Let be the balance coefficient of the structural topological consistency component, and .

7. The method according to claim 6, characterized in that, The balance coefficient of the structural topology consistency component Adaptively increasing with the number of iterations: The structural self-confidence component is calculated based on the degree to which the spatial topology of the power distribution tower is verified by the Gaussian elements. This structural self-confidence component is determined by at least one of the following: the interior point ratio fitted to the central main pole of the tower; the spatial coverage density of Gaussian elements at the structural component locations declared by the spatial topology of the power distribution tower; and the consistency ratio between the Gaussian element semantic labels and the spatial topology of the power distribution tower. As the structural self-confidence component increases, while maintaining... Under the premise of monotonically increasing the balance coefficient of the topological consistency component of the structure And correspondingly reduce the balance coefficient of the semantic field spatial consistency component. Balance coefficient with multi-view coverage components .

8. The method according to claim 7, characterized in that, Based on the semantic confidence factor, and by comparing it with preset high confidence thresholds and low confidence thresholds, each Gaussian primitive is divided into high confidence primitives, medium confidence primitives, and low confidence primitives; the specific steps are as follows: When the semantic confidence factor is greater than or equal to the high confidence threshold, the Gaussian primitive is classified as a high confidence primitive; when the semantic confidence factor is less than the low confidence threshold, the Gaussian primitive is classified as a low confidence primitive; when the semantic confidence factor is greater than or equal to the low confidence threshold and less than the high confidence threshold, the Gaussian primitive is classified as a medium confidence primitive.

9. The method according to claim 8, characterized in that, A preheating iteration count is set. The classification of high-confidence, medium-confidence, and low-confidence primitives is temporarily suspended until the preheating iteration count is reached. Once classification begins, the semantic confidence factors of all high-confidence primitives are recalculated and classification is re-executed every preset update iteration count. If a medium-confidence primitive's semantic confidence factor reaches or exceeds the high-confidence threshold in subsequent re-evaluations, it is upgraded to a high-confidence primitive, and its semantic embedding vector is activated, allowing it to resume participation in multi-view semantic voting and injection.

10. The method according to claim 9, characterized in that, The spatial topology of the power distribution tower consists of the location nodes of the central main pole, crossarms, insulators, conductors, and surge arresters; improving the spatial topology of the power distribution tower includes completing the unidentified structural components: When the two-dimensional semantic perception model only identifies the structural components on one side of the power distribution tower but not the symmetrical structural components on the other side, the position of the identified structural components on one side of the power distribution tower is mirrored about the axis of symmetry, taking the central main pole of the power distribution tower as the axis of symmetry, to obtain the spatial position of the symmetrical structural components on the other side that were not identified. Within a preset distance range of the spatial location, search for Gaussian primitives. For Gaussian primitives that have not yet been assigned a semantic label or are in the medium confidence primitive category, if the hierarchical matching degree of the Gaussian primitive is greater than a preset hierarchical threshold, and the attachment distance normalized value of the Gaussian primitive is greater than a preset attachment threshold, and the morphological matching degree of the Gaussian primitive is greater than a preset morphological threshold, then assign a semantic label to the unidentified structural component and set the value of the semantic confidence factor of the Gaussian primitive to be higher than the high confidence threshold, so that the Gaussian primitive is classified as a high confidence primitive. If the hierarchical matching degree of the Gaussian primitive is not greater than the preset hierarchical threshold, or the normalized value of the attachment distance of the Gaussian primitive is not greater than the preset attachment threshold, or the morphological matching degree of the Gaussian primitive is not greater than the preset morphological threshold, then the Gaussian primitive is treated as a low-confidence primitive.

Images