Three-dimensional semantic object generation method and system based on 3DGS new perspective evidence fusion

Abstract

The application discloses a three-dimensional semantic object generation method and system based on 3DGS new perspective evidence fusion, and belongs to the field of three-dimensional reconstruction and semantic map. The method comprises the following steps: acquiring a 3DGS map, an original collection perspective, a camera pose and semantic candidate information; determining a new rendering perspective according to object visibility, imaging quality, evidence uncertainty reduction and perspective correlation; extracting a two-dimensional evidence unit, combining the camera pose and Gaussian distribution information to determine a three-dimensional evidence anchor point, a spatial range and anchor uncertainty; constructing an evidence correlation graph, performing multi-perspective evidence aggregation and inhibiting the influence of repeated evidence, and generating a structured semantic object with three-dimensional position, geometric range, evidence chain and confidence.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion. Background Technology

[0002] 3D Gaussian Splatting (3DGS) can utilize a large number of 3D Gaussian primitives to represent the color, transparency, and spatial distribution of a scene, and supports real-time or near real-time new perspective rendering. It has been applied in scenarios such as data center inspection, warehouse management, exhibition hall navigation, and digital twins. Compared with traditional triangular mesh models, 3DGS maps have the advantages of high construction efficiency, better appearance fidelity, and flexible perspective generation.

[0003] However, existing 3DGS maps are typically used primarily for scene visualization, and their Gaussian primitives do not naturally correspond to semantic objects such as queryable equipment, shelves, meters, or exhibits. Current practices usually involve performing object detection, segmentation, or text recognition separately on the original or rendered image, and then directly overlaying the results. Since the same physical object can be repeatedly identified in multiple original viewpoints and continuously generated new viewpoints, it is difficult to directly determine whether multiple 2D candidate results belong to the same 3D object. Especially when adjacent new viewpoints are rendered from the same 3DGS representation, their recognition results are highly correlated. If only 2D detection confidence scores are accumulated or 2D non-maximum suppression is used, it can easily lead to duplicate object generation, artificially high object confidence scores, or incorrect merging of different objects.

[0004] Furthermore, 3DGS is not a mesh model with stable surface boundaries, making it difficult to reliably convert 2D detection boxes, masks, or OCR text regions into 3D positions and object extents. Some device text, nameplates, or features are only visible from specific viewpoints, and single-view recognition results are easily affected by occlusion, reflections, and imaging distance. Therefore, existing technologies struggle to reliably fuse 2D semantic evidence from multiple viewpoints, especially relevant new viewpoints, into 3D semantic objects with stable positions, well-defined extents, and high confidence levels. Summary of the Invention

[0005] The specific technical problem to be solved by this invention is: how to determine the three-dimensional correspondence between the original viewpoint and the new rendering viewpoint two-dimensional evidence generated for the same physical object in a 3DGS map and suppress the repeated accumulation of related viewpoint evidence, so as to generate a three-dimensional semantic object with reliable location, range and confidence.

[0006] To address the aforementioned technical problems, the first aspect of this invention discloses a method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion, comprising the following steps: Acquire a 3DGS map of the target scene, the original acquisition viewpoint associated with the 3DGS map, camera pose information, and semantic candidate information; For the three-dimensional semantic object to be generated, a new rendering perspective is determined to supplement the object evidence based on the object visibility under the candidate observation view, the imaging quality of the object region, the amount of uncertainty reduction of existing evidence, and the correlation of evidence between viewpoints. Rendering or image acquisition is performed on the original acquisition perspective and the new rendering perspective to extract two-dimensional evidence units containing object category, two-dimensional position, text information, recognition confidence, perspective identifier and evidence source; Based on the camera pose information, the two-dimensional position, and the Gaussian distribution information corresponding to the two-dimensional evidence unit in the 3DGS map, the three-dimensional evidence anchor point, spatial range, and anchoring uncertainty of the two-dimensional evidence unit are determined. Construct an evidence association diagram between two-dimensional evidence units. Based on the spatial consistency, line-of-sight consistency, object category consistency, text consistency, and evidence relevance of the three-dimensional evidence anchor points, aggregate the two-dimensional evidence units into evidence clusters and suppress the non-independence of related evidence. Based on the effective viewpoint coverage, anchoring uncertainty, category consistency, text consistency, non-independence suppression results, and occlusion risk of each evidence cluster, a three-dimensional semantic object is generated, and the object identifier, object category, three-dimensional position, geometric range, evidence chain, and object confidence of the three-dimensional semantic object are output.

[0007] Furthermore, the determination of a new rendering perspective to supplement object evidence based on object visibility under candidate observation viewpoints, object region imaging quality, reduction in uncertainty of existing evidence, and correlation of evidence between viewpoints includes: Generate multiple candidate viewing angles based on the object’s initial 3D position, geometric extent, or text region to be confirmed. The 3DGS map is used to estimate the visible area, occlusion ratio, viewing direction, imaging distance, and pixel percentage of the target area of ​​the object under each candidate viewing angle. The recognition quality score corresponding to each candidate viewing angle is determined based on the visible area, the occlusion ratio, the viewing direction, the imaging distance, and the pixel ratio of the target area. Based on the category uncertainty, location uncertainty, text uncertainty, or asset matching uncertainty of the three-dimensional semantic object to be generated, determine the amount of uncertainty reduction corresponding to each candidate observation perspective; The correlation of evidence between the viewpoints is determined based on the camera position distance, the angle of observation direction, the overlap of the visible Gaussian region, or the similarity of the rendered images between the candidate observation viewpoint and the used viewpoint. A new rendering perspective is determined based on the recognition quality score, the amount of uncertainty reduction, and the correlation of evidence between perspectives.

[0008] Furthermore, determining the three-dimensional evidence anchor point, spatial range, and anchoring uncertainty of the two-dimensional evidence unit based on the camera pose information, the two-dimensional position, and the Gaussian distribution information corresponding to the two-dimensional evidence unit in the 3DGS map includes: Generate a corresponding projection ray based on the camera pose information and the two-dimensional position; Along the projection ray, obtain the density value, depth value, transparency value, and color consistency information corresponding to the Gaussian distribution; Based on the density value, the depth value, the transparency value, and the color consistency information, the spatial support range of the two-dimensional evidence unit on the projection ray is determined; The three-dimensional evidence anchor points are determined based on the overlapping areas or line-of-sight intersection areas of spatial support intervals from multiple perspectives. The spatial range is determined based on the back projection results of the object frame, object mask, or text region corresponding to the two-dimensional evidence unit under multiple viewpoints; The anchoring uncertainty is determined based on the dispersion of the spatial support interval, the line-of-sight intersection residual, and the number of effective viewing angles involved in the anchoring.

[0009] Furthermore, the construction of the evidence association diagram between the two-dimensional evidence units includes: Each two-dimensional evidence unit is treated as a graph node; When the distance between the three-dimensional evidence anchor points of two two-dimensional evidence units is less than a preset distance threshold, and the similarity of the corresponding object categories is greater than a preset category threshold, candidate association edges are established between the corresponding graph nodes. The association weights of candidate association edges are determined based on the line-of-sight intersection residuals, text similarity, spatial range overlap, visible Gaussian region overlap, and interview evidence correlation corresponding to the candidate association edges. Two-dimensional evidence units rendered by adjacent camera positions, similar viewing directions, or the same Gaussian region are marked as relevant evidence nodes; Based on the association weights between relevant evidence nodes, the maximum number of effective contributions of relevant evidence nodes is determined to limit the cumulative contribution of the same relevant evidence group to the confidence of the object.

[0010] Furthermore, the step of aggregating the two-dimensional evidence units into evidence clusters and suppressing the non-independence of related evidence includes: Based on the association weights in the evidence association graph, two-dimensional evidence units with association weights greater than a preset aggregation threshold are grouped into the same evidence cluster; For related evidence nodes in the same evidence cluster, the corresponding identification confidence is attenuated according to the correlation between evidence from different perspectives. The object confidence of the evidence cluster is calculated based on the attenuated recognition confidence, effective viewpoint number, anchoring uncertainty, category consistency, text consistency, and occlusion risk. If there are object category conflicts, text conflicts, spatial range discontinuities, or anchoring uncertainties greater than a preset threshold within the same evidence cluster, the evidence cluster shall be split or a pending mark shall be added. When the spatial range corresponding to adjacent evidence clusters meets the merging conditions and their object categories, text information, or asset candidates meet the association conditions, the adjacent evidence clusters are merged.

[0011] Furthermore, it also includes: Obtain asset ledgers, asset type dictionaries, region dictionaries, asset location descriptions, and historical audit records; The asset candidate set is determined based on the object category, text information, 3D location, region, and spatial relationship of the 3D semantic object; Calculate the category matching score, text matching score, region matching score, location description matching score, and historical confirmation score between the three-dimensional semantic object and each asset candidate, respectively. When the same asset candidate is matched by multiple 3D semantic objects, the 3D semantic objects and the asset candidates are globally bound and assigned according to the asset uniqueness constraint, the spatial relationship between objects and the matching score of each asset candidate; When the highest asset matching score is less than the preset binding threshold, or when the difference between the highest asset matching score and the second highest asset matching score is less than the preset difference threshold, an asset pending mark and a corresponding supplementary evidence request are generated.

[0012] Furthermore, it also includes: Candidate observation poses are generated based on the 3D position, geometric extent, principal orientation, effective viewpoint set, and task type of the 3D semantic object; The 3DGS map is used to estimate the visible area of ​​objects, the degree of frontal gaze, the risk of occlusion, the text readability, and the recognition stability of each candidate observation pose. The availability of the candidate observation poses is filtered based on the sensor's working distance, the reachability of personnel or robots, and the task type. Available candidate observation poses are ranked according to the visible area of ​​the object, the degree of frontal gaze, the risk of occlusion, the text readability, the recognition stability, and the accessibility. Output the master observation pose and at least one alternative observation pose, and record the reason why the unavailable candidate observation pose is unavailable.

[0013] Furthermore, it also includes: after the 3DGS map is updated, obtaining the Gaussian region, spatial range, and effective view set corresponding to the confirmed 3D semantic objects in the 3DGS map before and after the update; Compare the changes in Gaussian region, spatial range shift, object visibility, and observation pose availability before and after the update; When the degree of change in the Gaussian region exceeds the preset change threshold, the degree of spatial range offset exceeds the preset offset threshold, the updated map cannot support the original 3D evidence anchor points, or the original observation pose is unavailable, the corresponding 3D semantic object is determined as the object to be reviewed. Based on the reason for the change of the object to be reviewed, the object to be reviewed is divided into the relocation queue, rebinding queue, regenerated observation pose queue, supplementary evidence queue or deletion confirmation queue. When the degree of change in the Gaussian region and the degree of spatial range offset before and after the update do not exceed the corresponding thresholds, and the asset binding relationship and observation pose remain valid, the object identifier, asset binding relationship and semantic version information of the corresponding 3D semantic object are inherited.

[0014] Secondly, the present invention also provides a three-dimensional semantic object generation system based on 3DGS novel perspective evidence fusion, applicable to any of the three-dimensional semantic object generation methods based on 3DGS novel perspective evidence fusion, comprising: The acquisition module is used to acquire a 3DGS map of the target scene, the original acquisition viewpoint associated with the 3DGS map, camera pose information, and semantic candidate information. The viewpoint determination module is used to determine a new rendering viewpoint to supplement object evidence for the three-dimensional semantic object to be generated, based on the object visibility under candidate observation viewpoints, the imaging quality of the object region, the amount of uncertainty reduction of existing evidence, and the correlation of evidence between viewpoints. The evidence extraction module is used to render or acquire images from the original acquisition perspective and the new rendering perspective, and extract two-dimensional evidence units containing object category, two-dimensional position, text information, recognition confidence, perspective identifier and evidence source; An anchoring module is used to determine the three-dimensional evidence anchor point, spatial range, and anchoring uncertainty of the two-dimensional evidence unit based on the camera pose information, the two-dimensional position, and the Gaussian distribution information corresponding to the two-dimensional evidence unit in the 3DGS map. The aggregation module is used to construct an evidence association diagram between two-dimensional evidence units. Based on the spatial consistency, line-of-sight consistency, object category consistency, text consistency, and evidence relevance of the three-dimensional evidence anchor points, the two-dimensional evidence units are aggregated into evidence clusters, and non-independence of related evidence is suppressed. The generation module is used to generate a three-dimensional semantic object based on the effective view coverage, anchoring uncertainty, category consistency, text consistency, non-independence suppression results and occlusion risk of each evidence cluster, and output the object identifier, object category, three-dimensional position, geometric range, evidence chain and object confidence of the three-dimensional semantic object.

[0015] Thirdly, the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, cause the processor to perform any of the three-dimensional semantic object generation methods based on 3DGS novel perspective evidence fusion as described in the present invention.

[0016] The beneficial technical effects of the present invention are at least as follows: Compared to existing methods that only perform two-dimensional recognition and overlay display on the original image or a single rendered image, this invention determines a new rendering perspective based on object visibility, image quality, reduction in evidence uncertainty, and viewpoint relevance. This allows for the supplementation of effective observational evidence in situations where the original acquisition perspective contains occlusion, reflection, unsuitable distance, blurred text, or partially invisible targets. Furthermore, by restricting the repeated selection of perspectives with similar heights or strong correlations, it avoids excessive redundant rendering and repeated recognition, thereby improving the effectiveness of new perspective generation and semantic evidence extraction.

[0017] This invention transforms two-dimensional evidence units, such as category, 2D location, text, confidence level, and source identifier, from the original acquisition perspective and the new rendering perspective into 3D evidence anchor points, spatial ranges, and anchoring uncertainties by combining camera pose and 3DGS Gaussian distribution information. Because it does not rely on stable surfaces, mesh patches, or object segmentation boundaries in traditional mesh models, it is applicable to non-mesh 3DGS high-fidelity maps, enabling 3D positioning and range representation of objects such as equipment, meters, shelves, and exhibits, thus solving the problem that 2D recognition results are difficult to directly form stable 3D objects.

[0018] This invention further constructs an evidence association graph, integrating spatial consistency, line-of-sight consistency, category consistency, textual consistency, and evidence relevance to associate, aggregate, and cluster two-dimensional evidence from different perspectives. This allows for more reliable identification of whether candidate results from multiple original and newly rendered perspectives correspond to the same physical object, reducing the likelihood of duplicate creation of the same object, incorrect merging of adjacent objects, and incorrect association of different categories or textual clues.

[0019] Furthermore, this invention suppresses the repetitive influence of evidence from adjacent viewpoints, similar observation directions, or evidence based on the same Gaussian region, preventing the misinterpretation of homologous or highly correlated identification results as multiple independent pieces of evidence and the simple accumulation of confidence scores. By combining effective viewpoint coverage, anchoring uncertainty, text consistency, and occlusion risk, a three-dimensional semantic object is generated, giving the generated result three-dimensional location, geometric range, evidence chain, and confidence score. Therefore, the output not only supports scene visualization but also provides a more queryable, traceable, and reliable structured data foundation for asset association, manual review, and semantic map applications. Attached Figure Description

[0020] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion, as disclosed in one embodiment of the present invention. Figure 2 This is a schematic diagram illustrating a novel rendering perspective determination according to one embodiment of the present invention; Figure 3 This is a schematic diagram of three-dimensional anchoring of two-dimensional evidence units, evidence association, and evidence cluster aggregation disclosed in one embodiment of the present invention; Figure 4 This is a flowchart of an abnormal event identification, early warning classification, and impact deduction method disclosed in one embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this application. For those skilled in the art, any modifications, substitutions, combinations, or equivalent improvements made to the following embodiments without departing from the concept of this application should fall within the scope of protection of this application.

[0023] In this application, "3DGS map" refers to a 3D scene representation constructed using 3D Gaussian Splatting (3DGS). The 3DGS map may include multiple Gaussian primitives, each of which may possess one or more of the following parameters: 3D center position, scale parameter, covariance parameter, rotation parameter, opacity parameter, color parameter, appearance feature parameter, and rendering contribution parameter. The 3DGS map may have a unified map coordinate system and may be associated with original acquired images, camera pose, point clouds, trajectories, reference planes, region boundaries, or building plan data.

[0024] In this application, "original acquisition viewpoint" refers to the observation viewpoint corresponding to the image frame actually acquired by a camera, mobile terminal, robot, drone, inspection equipment, or other image acquisition device during the 3DGS map construction, training, or update process. "New rendering viewpoint" refers to the scene observation viewpoint generated based on the 3DGS map, virtual camera parameters, and virtual camera pose rendering. The new rendering viewpoint can be the same as the original acquisition viewpoint, or it can be located between, near, or at a position not directly traversed by the original acquisition trajectory.

[0025] In this application, "two-dimensional evidence unit" refers to two-dimensional semantic information obtained from the original acquisition viewpoint or the new rendering viewpoint, used to prove or describe the object to be identified. A two-dimensional evidence unit may include an object bounding box, instance mask, keypoint region, text region, QR code region, category label, OCR text, recognition confidence score, image coordinates, viewpoint identifier, evidence source identifier, image quality score, occlusion status, and other relevant information. In this application, "three-dimensional evidence anchor point" refers to the three-dimensional position representation of the object determined based on the two-dimensional evidence unit, camera pose, and 3DGS map. It can be a three-dimensional point, a three-dimensional point set, a probability center, a spatial support region center, a local Gaussian region center, or other spatial representations that can be used for three-dimensional localization.

[0026] This application is applicable to various scenarios, including data center inspection, warehousing and logistics, exhibition hall guidance, industrial site management, park facility management, digital twin scenario construction, equipment ledger verification, and robot inspection data production, but is not limited to these scenarios. For example, in a data center scenario, three-dimensional semantic objects such as server racks, servers, meters, indicator lights, access control equipment, fire protection facilities, and nameplates can be generated; in a warehousing and logistics scenario, three-dimensional semantic objects such as shelves, storage areas, storage locations, cargo boxes, aisles, and pick-up / drop-off points can be generated; and in an exhibition hall scenario, three-dimensional semantic objects such as exhibits, booths, service points, signs, and aisles can be generated.

[0027] like Figure 1 As shown, Figure 1This is a flowchart illustrating a method for generating 3D semantic objects based on 3DGS novel perspective evidence fusion, provided as an embodiment of this application. The method can be executed by a server, edge computing device, graphics workstation, inspection platform, digital twin platform, or other electronic device with computing capabilities. The method includes the following steps.

[0028] In step S101, a 3DGS map of the target scene, the original acquisition viewpoint associated with the 3DGS map, camera pose information, and semantic candidate information are acquired.

[0029] The 3DGS map can be generated from multiple raw image frames, video frames, or other image data. The raw acquisition viewpoint can include image frame identifiers, camera intrinsic parameters, camera extrinsic parameters, timestamps, acquisition device identifiers, image sharpness, exposure information, and acquisition trajectory information. Camera pose information can at least include the camera's position and orientation in the map coordinate system, specifically represented using rotation matrices, translation vectors, pose transformation matrices, Euler angles, quaternions, or other equivalent forms.

[0030] Semantic candidate information can be generated by object detection models, instance segmentation models, OCR recognition models, text detection models, QR code recognition models, manual annotation modules, rule recognition modules, or other semantic analysis modules. The semantic candidate information may include the object category of the object to be recognized, the two-dimensional object bounding box, the object mask, the text region, the OCR text, the recognition confidence score, and the candidate source information.

[0031] In one implementation, the system can first perform object detection and text recognition on the image from the original acquisition perspective to obtain initial two-dimensional semantic candidates. For example, in a data center scenario, it can identify server racks, meters, nameplates, indicator lights, access control equipment, etc.; in a warehouse scenario, it can identify shelves, location labels, boxes, aisles, and warehouse area signs. Objects in the initial two-dimensional semantic candidates that have object occlusion, incomplete text, uncertain categories, unstable spatial locations, conflicting candidates, or unclear asset matching can be marked as objects to be supplemented as evidence.

[0032] In step S102, for the three-dimensional semantic object to be generated, a new rendering perspective is determined to supplement the object evidence based on the object visibility under the candidate observation viewpoint, the imaging quality of the object region, the amount of uncertainty reduction of existing evidence, and the correlation of evidence between viewpoints.

[0033] like Figure 2 As shown, Figure 2This diagram illustrates a novel rendering perspective determination method provided in an embodiment of this application. The system can generate multiple candidate viewing perspectives based on the object's initial 3D position, initial geometric range, suspected Gaussian region, text region to be confirmed, projection area of ​​existing 2D evidence, or region to which the object belongs. Candidate viewing perspectives can be located in front of, to the side front of, diagonally above, or in other observable directions. They can also be discretely or continuously sampled along the original camera trajectory, areas accessible to personnel, areas traversable by robots, or reference planes.

[0034] For example, for meters or nameplates installed on cabinet doors, multiple candidate viewing angles can be generated near the normal direction of the target surface, and the camera position can be adjusted within a preset distance range; for warehouse rack labels, multiple candidate viewing angles located in the aisle can be generated according to the direction of the rack aisle; for exhibit labels, candidate viewing angles that are close to the plane of the label can be generated first.

[0035] The system uses 3DGS maps to render or estimate the visibility of each candidate viewing angle, and obtains information such as the visible area, occlusion ratio, pixel ratio of the target area, viewing direction, imaging distance, image sharpness estimate, text readability estimate, and illumination stability estimate of the object to be identified under each candidate viewing angle.

[0036] In one alternative implementation, the recognition quality score of the candidate viewing angle can be represented by the following formula: , in, Indicates the first The quality score for identifying each candidate observation perspective; Indicates the visible area or visible proportion of an object; Indicates the occlusion ratio; Indicates the percentage of pixels in the target area; Indicates imaging distance adaptability; The ratings indicate image sharpness, viewing orientation, text readability, or lighting stability. to This indicates the corresponding weight.

[0037] The imaging distance adaptation can be determined based on the object category, camera focal length, sensor resolution, and task requirements. For objects containing fine-grained text or scale information, such as meters, nameplates, location labels, and equipment numbers, candidate viewing angles that allow the text or scale area to reach a preset pixel size can be prioritized. For large-scale objects such as cabinets, shelves, and booths, the weight of the overall visible area and geometric outline integrity of the object can be appropriately increased.

[0038] The reduction in uncertainty of existing evidence can reflect the degree of improvement in object category, three-dimensional location, text content, object range, or asset matching results after a candidate observation perspective is added. The uncertainty may include one or more of the following: category uncertainty, location uncertainty, textual uncertainty, geometric range uncertainty, and asset matching uncertainty.

[0039] For example, when an object has been identified as a "meter" but it is impossible to determine whether it is a pressure gauge or an ammeter, a candidate observation angle that can display the dial scale, unit, or model information can be selected first; when the OCR text corresponding to the object has missing characters, garbled characters, or multiple recognition results, a candidate observation angle with a higher degree of frontal visibility and less text area occlusion can be selected; when the three-dimensional positions obtained by back projection under different angles are relatively dispersed, a candidate observation angle that can increase the angle with the existing observation ray can be selected, thereby improving the accuracy of three-dimensional intersection positioning.

[0040] In one alternative implementation, the reduction in uncertainty corresponding to the candidate observation viewpoint can be expressed as: , in, This indicates the uncertainty of an object based on existing evidence. This indicates that the prediction incorporates candidate observation perspectives. Uncertainty of the object after corresponding evidence The larger the value, the greater the supplementary value of the candidate observation perspective.

[0041] Furthermore, the system determines the inter-view evidence correlation between candidate observation viewpoints and used viewpoints. This inter-view evidence correlation can be determined based on camera position distance, observation direction angle, object projection area overlap, visible Gaussian region overlap, rendered image similarity, common source identifiers, or other features. When the camera position distance between two viewpoints is small, the observation direction angle is small, the visible Gaussian region overlap is high, or the rendered image similarity is high, it indicates that the object evidence generated by the two viewpoints has a high correlation.

[0042] In one implementation, the first The combined selection score of the candidate observation perspectives can be expressed as: , in, This represents the overall selection score for the candidate observation perspectives; Indicates the quality score of the identification; This indicates the amount of uncertainty reduction; This indicates the relevance of evidence between candidate observation perspectives and already used perspectives; , and This indicates the corresponding weight. The system can select one or more candidate viewpoints with the highest comprehensive score as the new rendering viewpoint.

[0043] Optionally, the system can also set a minimum identification quality threshold, a minimum uncertainty reduction threshold, and a maximum relevance threshold. A candidate observation view is only identified as a new rendering view if it meets the following conditions: the identification quality score is greater than the minimum identification quality threshold, the uncertainty reduction is greater than the minimum reduction threshold, and the inter-view evidence relevance is less than the maximum relevance threshold. If none of the candidate observation views meet the requirements, the system can output a supplementary acquisition request, a manual verification marker, or a low-confidence object marker.

[0044] In step S103, the original acquisition perspective and the new rendering perspective are rendered or image acquired, and two-dimensional evidence units containing object category, two-dimensional location, text information, recognition confidence, perspective identifier and evidence source are extracted.

[0045] For the original acquisition viewpoint, the system can directly read the original image frame associated with the corresponding camera pose; for the new rendering viewpoint, the system generates a rendered image based on the 3DGS map and the corresponding virtual camera pose. Subsequently, the system performs object detection, instance segmentation, text detection, OCR recognition, feature recognition, or other semantic analysis processing on the original image frame and the rendered image.

[0046] Each two-dimensional unit of evidence can be represented as: , in, Represents the object category or category probability distribution; Represents a two-dimensional object box; Represents an object mask, text region, or key point region; This indicates OCR text, device name, serial number, model, storage location code, or other textual clues; Indicates the confidence level of identification; Indicates the viewpoint identifier; This indicates the source of evidence and is used to distinguish between original acquired images, 3DGS rendered images, manually annotated results, or other sources. This indicates the quality information of two-dimensional evidence.

[0047] In one implementation, when an object frame contains a clearly defined text area, the object category, object frame, text area, and OCR text can be merged into a composite two-dimensional evidence unit. For cases where only text is recognized but no specific object category is identified, a text-based two-dimensional evidence unit can also be generated for subsequent association with object candidates based on spatial location, text similarity, and other object evidence. For evidence units with low recognition confidence, insufficient image clarity, or excessive occlusion, the system can retain them as weak evidence instead of directly participating in object confidence accumulation as independent high-confidence evidence.

[0048] In step S104, the three-dimensional evidence anchor point, spatial range, and anchoring uncertainty of the two-dimensional evidence unit are determined based on the camera pose information, two-dimensional position, and Gaussian distribution information corresponding to the two-dimensional evidence unit in the 3DGS map.

[0049] like Figure 3 As shown, Figure 3 This diagram illustrates the three-dimensional anchoring, evidence association, and evidence cluster aggregation of two-dimensional evidence units provided in this application embodiment. For any two-dimensional evidence unit, the system generates a corresponding projection ray based on camera intrinsic parameters, camera extrinsic parameters, and the two-dimensional position. The two-dimensional position can be the center point of the object frame, multiple pixels sampled from the object mask, the center point of the text region, the location of a key point, or a combination of the above positions.

[0050] For object masks, object bounding boxes, or text regions, the system can select multiple pixels within the corresponding region and generate multiple projection rays to obtain multiple spatial support regions of the object. For object edge regions, text regions, and high-texture regions, the sampling density can be increased to improve the accuracy of object spatial extent estimation.

[0051] Along the projection ray, the system obtains the corresponding Gaussian distribution information from the 3DGS map. This Gaussian distribution information may include the center position of the Gaussian primitive, density value, depth value, opacity value, color contribution, scale parameter, covariance parameter, rendering weight, and visibility weight. Based on the density contribution, cumulative opacity contribution, depth peak, color consistency, and local spatial continuity of the Gaussian primitives along the ray direction, the system determines the spatial support interval of the two-dimensional evidence unit on the projection ray.

[0052] The spatial support region corresponding to a single two-dimensional evidence unit can be represented as: , in, This indicates the position of the camera's optical center in the map coordinate system. This represents the unit direction vector obtained by backprojection of a two-dimensional position. This represents the deep support interval determined based on Gaussian distribution information.

[0053] When the Gaussian density corresponding to the ray is concentrated, the depth distribution is unimodal, the color consistency is high, and there is no occlusion, a narrower spatial support interval can be determined; when there is foreground occlusion, multimodal depth, transparent areas, background interference, inconsistent colors, or sparse local Gaussian distribution, a wider spatial support interval can be determined, and the anchoring uncertainty can be increased.

[0054] For two-dimensional evidence units from multiple perspectives that may correspond to the same physical object, the system can determine three-dimensional evidence anchor points based on the overlapping areas of spatial support intervals, the shortest distance between projected rays, line-of-sight intersection residuals, the overlap of local Gaussian regions, or the fusion results of three-dimensional probability distributions. The three-dimensional evidence anchor point can be the geometric center, weighted center, maximum a posteriori position, local Gaussian distribution peak position, or the position that minimizes the line-of-sight residuals of the overlapping areas of multiple spatial support intervals.

[0055] For object bounding boxes, object masks, or text regions, the system can back-project their corresponding multiple pixel regions onto a 3DGS map to obtain the spatial extent of the object. This spatial extent can be represented using a 3D bounding box, oriented bounding box, convex hull, ellipsoid, point set, voxel set, two-dimensional extent and height extent on a local reference plane, or a combination of the above.

[0056] For example, for three-dimensional objects such as cabinets, shelves, and boxes, an oriented three-dimensional bounding box can be used to represent their spatial range; for near-planar objects such as nameplates, labels, access control panels, and display boards, a local plane can be fitted based on associated Gaussian elements, and a two-dimensional boundary range can be generated on the local plane; for local small-scale objects such as meters and indicator lights, a three-dimensional center point and a preset radius, a local ellipsoid, or a compact three-dimensional bounding box can be used to represent them.

[0057] Anchoring uncertainty can be determined by one or more of the following: spatial support interval dispersion, residuals of multiple line-of-sight intersections, number of effective viewpoints, camera baseline length, object occlusion ratio, multimodality of depth distribution, Gaussian unit spatial stability, and image recognition quality. Optionally, anchoring uncertainty can be expressed as a scalar, covariance matrix, confidence ellipsoid parameter, confidence level, or a combination thereof.

[0058] In one implementation, the anchoring uncertainty can be expressed as: , in, Indicates the anchoring uncertainty; This indicates the degree of dispersion in spatial support intervals or depth estimation; Indicates the line-of-sight residual; Indicates the number of valid viewpoints participating in anchoring; Indicates the risk of obstruction; to This indicates the weight. The lower the anchoring uncertainty, the more stable the three-dimensional position of the object.

[0059] For two-dimensional evidence units where a reliable camera pose cannot be obtained, an effective spatial support interval cannot be formed, the corresponding Gaussian distribution region cannot be determined, or the anchoring uncertainty exceeds a preset threshold, the system can mark them as unreliable anchored evidence. Unreliable anchored evidence can be used to assist in text judgment, category judgment, or manual review, but it does not directly and independently generate three-dimensional semantic objects.

[0060] In step S105, an evidence association diagram between two-dimensional evidence units is constructed. Based on the spatial consistency, line-of-sight consistency, object category consistency, text consistency, and evidence relevance of the three-dimensional evidence anchor points, the two-dimensional evidence units are aggregated into evidence clusters, and non-independence suppression is applied to related evidence.

[0061] Specifically, the system treats each two-dimensional evidence unit as a graph node in the evidence association graph. For two two-dimensional evidence units, when the distance between their corresponding three-dimensional evidence anchor points is less than a preset distance threshold and the object category similarity is greater than a preset category threshold, a candidate association edge is established between the corresponding graph nodes.

[0062] Furthermore, the system determines the association weight of candidate association edges based on the line-of-sight intersection residual, text similarity, spatial overlap, visible Gaussian region overlap, category similarity, and inter-viewpoint evidence relevance. A higher association weight indicates a higher probability that the two two-dimensional evidence units belong to the same physical object.

[0063] In one implementation, the association weight between two two-dimensional evidence units can be expressed as: , in, Representing a two-dimensional evidence unit and The correlation weight between them; This indicates the consistency of the three-dimensional evidence anchor point space; Indicates a consistent line of sight; Indicates category consistency; Indicates text similarity; Indicates the degree of overlap in spatial extent; Indicates the relevance of evidence between perspectives; to Indicates the weight.

[0064] Text similarity can be determined through string edit distance, keyword overlap rate, character-level similarity, semantic vector similarity, device number matching degree, or dictionary matching results. For OCR text with missing characters, typos, glare interference, or partial occlusion, fault-tolerant matching can be performed by combining object category, spatial location, and candidate fields in the asset ledger.

[0065] The system can group two-dimensional evidence units with association weights greater than a preset aggregation threshold into the same evidence cluster based on the association weights in the evidence association graph. Evidence clusters can be generated using connected component partitioning, graph community discovery, hierarchical clustering, density clustering, minimum cut partitioning, or other equivalent aggregation methods.

[0066] Since two-dimensional evidence units from adjacent camera positions, similar viewing directions, the same Gaussian region, or similar rendering conditions are not independent of each other, directly accumulating their identification confidence scores can easily lead to artificially high object confidence scores. Therefore, this application suppresses the non-independence of relevant evidence.

[0067] Specifically, the system can mark two-dimensional evidence units that are adjacent to each other, have similar viewing directions, have an overlap of visible Gaussian regions greater than a preset threshold, or have a similarity of rendered images greater than a preset threshold as relevant evidence nodes. For multiple two-dimensional evidence units in the same relevant evidence group, the recognition confidence can be attenuated according to the evidence correlation between viewpoints, or the maximum effective contribution of the relevant evidence group to the object's confidence can be limited.

[0068] For example, for the first The decayed confidence of each relevant evidence node can be expressed as: , in, This represents the initial confidence level of the identification. This indicates the relevance between the two-dimensional evidence unit and the related evidence already included. This indicates the confidence level of identification after attenuation. When the relevance is high, the confidence level after attenuation is low; when the relevance is low, more independent evidence contribution is retained.

[0069] In another implementation, the system can limit the maximum number of valid contributions from the same relevant set of evidence to a preset number, such as one, two, or a number corresponding to the coverage of the viewpoint baseline. This way, even if the same object is repeatedly identified in multiple consecutive near-new rendering views, it will not be mistakenly considered as a large number of independent valid pieces of evidence.

[0070] When there are conflicts in object categories, text, spatial discontinuities, excessive line-of-sight recurrence residuals, or anchoring uncertainties exceeding a preset threshold within the same evidence cluster, the system can split the evidence cluster or add a pending label to it. When the spatial range between adjacent evidence clusters meets the merging conditions, and their object categories, text information, local Gaussian regions, or asset candidates are related, the system can merge the adjacent evidence clusters.

[0071] In step S106, based on the effective view coverage, anchoring uncertainty, category consistency, text consistency, non-independence suppression results and occlusion risk of each evidence cluster, a three-dimensional semantic object is generated, and the object identifier, object category, three-dimensional position, geometric range, evidence chain and object confidence are output.

[0072] Effective viewpoint coverage refers to the number of observation viewpoints, the distribution range of observation directions, or the degree of viewpoint baseline coverage that can be considered to have effective independent supplementary value after relevant evidence suppression is completed. Category consistency can be determined based on the category distribution of each two-dimensional evidence unit within the same evidence cluster, category voting results, or category probability entropy. Text consistency can be determined based on OCR text similarity, device number consistency, keyword consistency, asset name matching degree, or dictionary matching results. Occlusion risk can be determined based on the occlusion ratio of objects under each viewpoint, visible area, local Gaussian distribution occlusion relationship, and image quality information.

[0073] In one implementation, object confidence can be expressed as: , in, Indicates the confidence level of an object; Indicates effective vantage point coverage; Indicates the anchoring uncertainty; Indicates category consistency; Indicates textual consistency; Indicates the effective evidence contribution after the suppression of non-independence; Indicates the risk of obstruction; to Indicates the weight.

[0074] The generated 3D semantic object can include object identifier, object category, 3D location, geometric range, local Gaussian region identifier, effective viewpoint set, evidence chain, object confidence, anchoring uncertainty, text information, object status, and review status. The evidence chain records the original image frame identifiers, new rendering viewpoint identifiers, 2D evidence unit identifiers, recognition results, association weights, related evidence suppression results, and manual review records involved in object generation, thus enabling traceability of the object generation process.

[0075] Based on the generation of three-dimensional semantic objects in Example 1, the system can also perform asset binding candidate generation, observation pose candidate generation, spatial relationship generation, and review queue diversion.

[0076] For asset binding, the system retrieves the asset ledger, asset type dictionary, region dictionary, asset location description, and historical audit records. The asset ledger may include information such as asset number, asset name, asset type, region, installation location, equipment model, associated text fields, maintenance status, and historical confirmation records.

[0077] The system determines a candidate set of assets based on the object category, text information, 3D location, region, and spatial relationships of a 3D semantic object. For example, for a 3D semantic object identified as a server rack, candidate assets can be selected from the server rack ledgers within the same server room area; for a 3D semantic object identified as a meter, candidate assets can be selected based on its server rack, adjacent equipment, text number, and installation height.

[0078] The system calculates category matching scores, text matching scores, region matching scores, location description matching scores, and historical confirmation scores between the 3D semantic objects and each asset candidate. Category matching scores reflect the consistency between the object category and the asset type; text matching scores reflect the consistency between OCR text, equipment name, number, or model and ledger fields; region matching scores reflect the consistency between the spatial region of the object and the region of the asset; location description matching scores reflect the consistency between the object's 3D location, adjacency relationships, vertical relationships, horizontal relationships, or installation location description and ledger records; and historical confirmation scores reflect the history of manual review, existing binding relationships, or long-term stability.

[0079] When the same asset candidate is matched by multiple 3D semantic objects, the system can globally bind and assign the 3D semantic objects and asset candidates based on asset uniqueness constraints, spatial relationships between objects, and matching scores of each asset candidate. If the highest asset matching score is lower than a preset binding threshold, or if the difference between the highest and second-highest asset matching scores is less than a preset difference threshold, the system can generate an asset pending flag and output a request for supplementary evidence or an asset confirmation review task.

[0080] For observation pose generation, the system generates candidate observation poses based on the 3D position, geometric range, principal direction, effective viewpoint set, and task type of the 3D semantic object. Task types can include inspection and recognition, text verification, meter reading, equipment appearance inspection, asset inventory, guided tours, or manual review, etc.

[0081] The system estimates the visible area, frontal gaze, occlusion risk, text readability, recognition stability, and background interference level of objects under each candidate observation pose using 3DGS maps. The system can also filter the availability of candidate observation poses by combining factors such as sensor working distance, camera field of view, robot reachability, personnel standing area, passage boundaries, restricted areas, and safety distances.

[0082] For available candidate observation poses, the system sorts them according to the visible area of ​​the object, degree of frontal gaze, occlusion risk, text readability, recognition stability, and reachability, and outputs the main observation pose and at least one alternative observation pose. For unavailable candidate observation poses, the system can record the reasons for their unavailability, such as severe occlusion, insufficient distance, excessive distance, field of view not meeting requirements, unreachable channel, located in a restricted area, or inability to obtain stable recognition results.

[0083] The system can also generate spatial relationship candidates based on the object's 3D location, spatial extent, region boundaries, reference plane, and geometric relationships. These can include relationships such as belonging to a region, inclusion, being included, adjacency, vertical, horizontal, front-back, channel adjacency, and sorting relationships on the reference plane. Spatial relationships can serve as an auxiliary basis for asset matching, observation pose filtering, object retrieval, and manual review.

[0084] The system assigns objects to different review queues based on object confidence, asset matching results, spatial conflicts, observation pose availability, and the need for supplementary evidence. Objects with consistent multi-view evidence, clear asset matching, and available observation poses can enter the fast confirmation queue; objects with similar scores among multiple asset candidates can enter the asset confirmation queue; objects with conflicting scopes, spatial relationships, or abnormal overlap can enter the spatial correction queue; objects with few effective viewpoints, weak text, unstable anchoring, or high risk of occlusion can enter the supplementary review queue; and objects with insufficient evidence or duplicates of already confirmed objects can enter the deletion confirmation queue.

[0085] After manual review, the system can solidify the semantic version and output a list of semantic objects, asset binding relationships, observation poses, spatial relationships, evidence sources, review records, data model files, verification files, and result reports. This transforms the 3DGS high-fidelity map from a visual map into a semantic map foundation that is queryable, location-based, asset-binding, capable of generating observation poses, and continuously reviewed and maintained.

[0086] After the 3DGS map is retrained, partially re-photographed, incrementally updated, or the map coordinates are corrected, the system can perform semantic verification and judgment on the confirmed 3D semantic objects.

[0087] Specifically, the system acquires the Gaussian region, spatial extent, and effective view set corresponding to the confirmed 3D semantic objects in the 3DGS map before and after the update, and compares the degree of change of Gaussian region, spatial extent offset, object visibility, and observation pose availability before and after the update.

[0088] The degree of change in the Gaussian region can be determined based on changes in the number, density, spatial position, color, opacity, local rendering results, or local visibility of Gaussian pixels around the object's anchor point. The degree of spatial offset can be determined based on the positional offset between the object's anchor point, bounding box, object reference plane, or the center of the local Gaussian region before and after the update. The degree of change in object visibility can be determined based on changes in the object's visible area, occlusion ratio, and text readability from the same or similar viewing angles before and after the update.

[0089] When the degree of change in the Gaussian region exceeds the preset change threshold, the degree of spatial range offset exceeds the preset offset threshold, the updated 3DGS map cannot support the original 3D evidence anchor points, or the original observation pose is unavailable, the system determines the corresponding 3D semantic object as an object to be reviewed.

[0090] The system can categorize objects to be reviewed into relocation queues, rebinding queues, regenerated observation pose queues, supplementary evidence queues, or deletion confirmation queues based on the reasons for the changes. For example, if the anchor point region of an object undergoes significant spatial changes but the object category and text remain stable, it can be placed in the relocation queue; if the object's position remains stable but the nameplate text, asset number, or ledger association changes, it can be placed in the rebinding queue; if the object itself remains valid but the original recommended observation point becomes invalid due to occlusion or changes in accessibility, it can be placed in the regenerated observation pose queue; and if the corresponding region of the object no longer has valid Gaussian support or the new perspective cannot reproduce the original object evidence, it can be placed in the deletion confirmation queue.

[0091] When the degree of change in the Gaussian region and the degree of spatial range offset before and after the update do not exceed the corresponding thresholds, and the asset binding relationship and observation pose remain valid, the system can inherit the object identifier, asset binding relationship, observation pose and semantic version information of the corresponding 3D semantic object, thereby avoiding repeated review of all objects due to the overall update of the 3DGS map.

[0092] like Figure 4 As shown, Figure 4This is a schematic diagram of the structure of a 3D semantic object generation system based on 3DGS novel perspective evidence fusion, provided in an embodiment of this application. The system includes an acquisition module 401, a perspective determination module 402, an evidence extraction module 403, an anchoring module 404, an aggregation module 405, and a generation module 406; optionally, it may also include an asset binding module 407, an observation pose module 408, a semantic verification module 409, an approval queue module 410, a storage module 411, and an interaction module 412.

[0093] The acquisition module 401 is used to acquire the 3DGS map of the target scene, the original acquisition viewpoint, camera pose information, semantic candidate information, and optional point cloud, trajectory, reference plane, area boundary, asset ledger and historical audit records.

[0094] The viewpoint determination module 402 is used to generate candidate observation viewpoints for the three-dimensional semantic object to be generated, calculate the object visibility, the imaging quality of the object region, the amount of uncertainty reduction of existing evidence and the correlation of evidence between viewpoints, and determine a new rendering viewpoint to supplement the object evidence.

[0095] The evidence extraction module 403 is used to acquire or render images from the original acquisition perspective and the new rendering perspective, and extract two-dimensional evidence units consisting of object category, two-dimensional location, text information, recognition confidence, perspective identifier and evidence source.

[0096] The anchoring module 404 is used to determine the projection ray, spatial support interval, three-dimensional evidence anchor point, spatial range and anchoring uncertainty of the two-dimensional evidence unit based on the camera pose information, two-dimensional position and corresponding Gaussian distribution information.

[0097] The aggregation module 405 is used to construct an evidence association graph between two-dimensional evidence units, calculate spatial consistency, line-of-sight consistency, object category consistency, text consistency and evidence relevance, and complete evidence cluster aggregation, conflict splitting, adjacent evidence cluster merging and related evidence non-independence suppression.

[0098] The generation module 406 is used to generate a three-dimensional semantic object based on the effective viewpoint coverage, anchoring uncertainty, category consistency, text consistency, non-independence suppression results and occlusion risk, and output the object identifier, object category, three-dimensional position, geometric range, evidence chain and object confidence.

[0099] The asset binding module 407 is used to generate a set of candidate assets, calculate matching scores, and perform global binding assignment based on object category, text clues, three-dimensional location, region, spatial relationship, asset ledger, and historical audit records.

[0100] The observation pose module 408 is used to generate, filter, and sort candidate observation poses based on the object's 3D position, geometric range, principal direction, effective viewpoint set, 3DGS rendering results, reachability, and task type.

[0101] The semantic verification module 409 is used to compare the object-related Gaussian regions, spatial extent, object visibility, and observation pose availability before and after the 3DGS map is updated, and to determine the objects to be verified and their verification types.

[0102] The review queue module 410 is used to sort objects into review queues based on object confidence, asset matching results, spatial conflicts, observation pose availability, and map update impact.

[0103] Storage module 411 is used to store 3DGS maps, original acquisition viewpoints, camera poses, two-dimensional evidence units, three-dimensional semantic objects, asset binding relationships, observation poses, review records, and semantic version information. Interaction module 412 is used to provide users with functions such as object viewing, evidence chain viewing, asset binding confirmation, spatial range correction, observation pose adjustment, review comments input, and result export.

[0104] The system can be deployed on a single computing device or in a distributed system consisting of cloud servers, edge servers, robot terminals, graphics workstations, and user terminals. Each module can be implemented through software, hardware, firmware, or a combination thereof. The functions of each module can also be executed by the same processor or by multiple processors working collaboratively.

[0105] This application also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, the computer-executable instructions cause the processor to perform the 3D semantic object generation method based on 3DGS novel perspective evidence fusion in any of the above embodiments. The computer-readable storage medium may include, but is not limited to, a disk, optical disk, read-only memory, random access memory, flash memory, solid-state drive, memory card, or other media capable of storing program instructions.

[0106] Through the above implementation methods, this application utilizes the new perspective rendering capabilities of 3DGS maps to supplement object identification evidence and converts multi-view two-dimensional candidates into evidence units with three-dimensional anchor points, spatial range, and uncertainty. Furthermore, through evidence association graphs and related evidence non-independence suppression mechanisms, it avoids the repeated accumulation of adjacent or highly related new perspective evidence, thereby reducing the risks of duplicate object generation, erroneous merging, and inflated confidence levels. The generated three-dimensional semantic objects can also be further associated with asset ledgers, observation poses, spatial relationships, and review queues, making them suitable for semantic delivery and continuous updating and maintenance of objects under non-mesh high-fidelity maps.

Claims

1. A method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion, characterized in that, include: Acquire a 3DGS map of the target scene, the original acquisition viewpoint associated with the 3DGS map, camera pose information, and semantic candidate information; For the three-dimensional semantic object to be generated, a new rendering perspective is determined to supplement the object evidence based on the object visibility under the candidate observation view, the imaging quality of the object region, the amount of uncertainty reduction of existing evidence, and the correlation of evidence between viewpoints. Rendering or image acquisition is performed on the original acquisition perspective and the new rendering perspective to extract two-dimensional evidence units containing object category, two-dimensional position, text information, recognition confidence, perspective identifier and evidence source; Based on the camera pose information, the two-dimensional position, and the Gaussian distribution information corresponding to the two-dimensional evidence unit in the 3DGS map, the three-dimensional evidence anchor point, spatial range, and anchoring uncertainty of the two-dimensional evidence unit are determined. Construct an evidence association diagram between two-dimensional evidence units. Based on the spatial consistency, line-of-sight consistency, object category consistency, text consistency, and evidence relevance of the three-dimensional evidence anchor points, aggregate the two-dimensional evidence units into evidence clusters and suppress the non-independence of related evidence. Based on the effective viewpoint coverage, anchoring uncertainty, category consistency, text consistency, non-independence suppression results, and occlusion risk of each evidence cluster, a three-dimensional semantic object is generated, and the object identifier, object category, three-dimensional position, geometric range, evidence chain, and object confidence of the three-dimensional semantic object are output.

2. The method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion according to claim 1, characterized in that, The process of determining new rendering perspectives to supplement object evidence based on object visibility from candidate viewing perspectives, object region imaging quality, reduction in uncertainty of existing evidence, and correlation of evidence between perspectives includes: Based on the object’s initial 3D position, geometric extent, or text region to be confirmed, generate multiple candidate viewing angles; The 3DGS map is used to estimate the visible area, occlusion ratio, viewing direction, imaging distance, and pixel percentage of the target area of ​​the object under each candidate viewing angle. The recognition quality score corresponding to each candidate viewing angle is determined based on the visible area, the occlusion ratio, the viewing direction, the imaging distance, and the pixel ratio of the target area. Based on the category uncertainty, location uncertainty, text uncertainty, or asset matching uncertainty of the three-dimensional semantic object to be generated, determine the amount of uncertainty reduction corresponding to each candidate observation perspective; The correlation of evidence between the viewpoints is determined based on the camera position distance, the angle of observation direction, the overlap of the visible Gaussian region, or the similarity of the rendered images between the candidate viewpoint and the used viewpoint. A new rendering perspective is determined based on the recognition quality score, the amount of uncertainty reduction, and the correlation of evidence between perspectives.

3. The method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion according to claim 1, characterized in that, The step of determining the three-dimensional evidence anchor point, spatial range, and anchoring uncertainty of the two-dimensional evidence unit based on the camera pose information, the two-dimensional position, and the Gaussian distribution information corresponding to the two-dimensional evidence unit in the 3DGS map includes: Generate a corresponding projection ray based on the camera pose information and the two-dimensional position; Along the projection ray, obtain the density value, depth value, transparency value, and color consistency information corresponding to the Gaussian distribution; Based on the density value, the depth value, the transparency value, and the color consistency information, the spatial support range of the two-dimensional evidence unit on the projection ray is determined; The three-dimensional evidence anchor points are determined based on the overlapping areas or line-of-sight intersection areas of spatial support intervals from multiple perspectives. The spatial range is determined based on the back projection results of the object frame, object mask, or text region corresponding to the two-dimensional evidence unit under multiple viewpoints; The anchoring uncertainty is determined based on the dispersion of the spatial support interval, the line-of-sight intersection residual, and the number of effective viewing angles involved in the anchoring.

4. The method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion according to claim 1, characterized in that, The construction of the evidence association diagram between two-dimensional evidence units includes: Each two-dimensional evidence unit is treated as a graph node; When the distance between the three-dimensional evidence anchor points of two two-dimensional evidence units is less than a preset distance threshold, and the similarity of the corresponding object categories is greater than a preset category threshold, candidate association edges are established between the corresponding graph nodes. The association weights of candidate association edges are determined based on the line-of-sight intersection residuals, text similarity, spatial range overlap, visible Gaussian region overlap, and interview evidence correlation corresponding to the candidate association edges. Two-dimensional evidence units rendered by adjacent camera positions, similar viewing directions, or the same Gaussian region are marked as relevant evidence nodes; Based on the association weights between relevant evidence nodes, the maximum number of effective contributions of relevant evidence nodes is determined to limit the cumulative contribution of the same relevant evidence group to the confidence of the object.

5. The method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion according to claim 1, characterized in that, The step of aggregating the two-dimensional evidence units into evidence clusters and suppressing the non-independence of related evidence includes: Based on the association weights in the evidence association graph, two-dimensional evidence units with association weights greater than a preset aggregation threshold are grouped into the same evidence cluster; For related evidence nodes in the same evidence cluster, the corresponding identification confidence is attenuated according to the correlation between evidence from different perspectives. The object confidence of the evidence cluster is calculated based on the attenuated recognition confidence, effective viewpoint number, anchoring uncertainty, category consistency, text consistency, and occlusion risk. If there are object category conflicts, text conflicts, spatial range discontinuities, or anchoring uncertainties greater than a preset threshold within the same evidence cluster, the evidence cluster shall be split or a pending mark shall be added. When the spatial range corresponding to adjacent evidence clusters meets the merging conditions and their object categories, text information, or asset candidates meet the association conditions, the adjacent evidence clusters are merged.

6. The method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion according to claim 1, characterized in that, Also includes: Obtain asset ledgers, asset type dictionaries, region dictionaries, asset location descriptions, and historical audit records; The asset candidate set is determined based on the object category, text information, 3D location, region, and spatial relationship of the 3D semantic object; Calculate the category matching score, text matching score, region matching score, location description matching score, and historical confirmation score between the three-dimensional semantic object and each asset candidate, respectively. When the same asset candidate is matched by multiple 3D semantic objects, the 3D semantic objects and the asset candidates are globally bound and assigned according to the asset uniqueness constraint, the spatial relationship between objects and the matching score of each asset candidate; When the highest asset matching score is less than the preset binding threshold, or when the difference between the highest asset matching score and the second highest asset matching score is less than the preset difference threshold, an asset pending mark and a corresponding supplementary evidence request are generated.

7. The method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion according to claim 1, characterized in that, Also includes: Candidate observation poses are generated based on the 3D position, geometric extent, principal orientation, effective viewpoint set, and task type of the 3D semantic object. The 3DGS map is used to estimate the visible area of ​​objects, the degree of frontal view, the risk of occlusion, the text readability, and the recognition stability of each candidate observation pose. The availability of the candidate observation poses is filtered based on the sensor's working distance, the reachability of personnel or robots, and the task type. Available candidate observation poses are ranked according to the visible area of ​​the object, the degree of frontal gaze, the risk of occlusion, the text readability, the recognition stability, and the accessibility. Output the master observation pose and at least one alternative observation pose, and record the reason why the unavailable candidate observation pose is unavailable.

8. The method for generating three-dimensional semantic objects based on 3DGS novel perspective evidence fusion according to claim 1, characterized in that, Also includes: After the 3DGS map is updated, obtain the Gaussian region, spatial extent, and effective view set corresponding to the confirmed 3D semantic objects in the 3DGS map before and after the update. Compare the changes in Gaussian region, spatial range shift, object visibility, and observation pose availability before and after the update; When the degree of change in the Gaussian region exceeds the preset change threshold, the degree of spatial range offset exceeds the preset offset threshold, the updated map cannot support the original 3D evidence anchor points, or the original observation pose is unavailable, the corresponding 3D semantic object is determined as the object to be reviewed. Based on the reason for the change of the object to be reviewed, the object to be reviewed is divided into the relocation queue, rebinding queue, regenerated observation pose queue, supplementary evidence queue, or deletion confirmation queue. When the degree of change in the Gaussian region and the degree of spatial range offset before and after the update do not exceed the corresponding thresholds, and the asset binding relationship and observation pose remain valid, the object identifier, asset binding relationship and semantic version information of the corresponding 3D semantic object are inherited.

9. A three-dimensional semantic object generation system based on 3DGS novel perspective evidence fusion, applied to the three-dimensional semantic object generation method based on 3DGS novel perspective evidence fusion as described in any one of claims 1-8, characterized in that, include: The acquisition module is used to acquire a 3DGS map of the target scene, the original acquisition viewpoint associated with the 3DGS map, camera pose information, and semantic candidate information. The viewpoint determination module is used to determine a new rendering viewpoint to supplement object evidence for the three-dimensional semantic object to be generated, based on the object visibility under candidate observation viewpoints, the imaging quality of the object region, the amount of uncertainty reduction of existing evidence, and the correlation of evidence between viewpoints. The evidence extraction module is used to render or acquire images from the original acquisition perspective and the new rendering perspective, and extract two-dimensional evidence units containing object category, two-dimensional position, text information, recognition confidence, perspective identifier and evidence source; An anchoring module is used to determine the three-dimensional evidence anchor point, spatial range, and anchoring uncertainty of the two-dimensional evidence unit based on the camera pose information, the two-dimensional position, and the Gaussian distribution information corresponding to the two-dimensional evidence unit in the 3DGS map. The aggregation module is used to construct an evidence association diagram between two-dimensional evidence units. Based on the spatial consistency, line-of-sight consistency, object category consistency, text consistency, and evidence relevance of the three-dimensional evidence anchor points, the two-dimensional evidence units are aggregated into evidence clusters, and non-independence of related evidence is suppressed. The generation module is used to generate a three-dimensional semantic object based on the effective view coverage, anchoring uncertainty, category consistency, text consistency, non-independence suppression results and occlusion risk of each evidence cluster, and output the object identifier, object category, three-dimensional position, geometric range, evidence chain and object confidence of the three-dimensional semantic object.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, cause the processor to perform the three-dimensional semantic object generation method based on 3DGS novel perspective evidence fusion as described in any one of claims 1 to 8.

Images