A cognitive-driven optimal view selection method and system for large-scale urban scenes

Abstract

The application discloses a cognitive-driven optimal view selection method and system for large-scale urban scenes, comprising: acquiring an original real scene three-dimensional grid, extracting geometric primitives therefrom, and performing semantic segmentation and structure analysis with the geometric primitives as prior constraints to obtain a structured scene representation containing semantic attributes, structure information and topological relations, wherein the structured scene representation comprises a plurality of semantic objects; organizing the semantic objects in the structured scene representation into a three-dimensional semantic scene graph, and quantifying the cognitive core degree of each semantic object in the scene according to the three-dimensional semantic scene graph to obtain the cognitive heat of each semantic object; constructing an adaptive candidate view surface, reducing camera pose search to a direction optimization problem on the adaptive candidate view surface, and constructing a cooperative observation objective function based on the cognitive heat to solve a view angle on the adaptive candidate view surface that maximizes the cooperative observation objective function as an optimal view angle.

Description

Technical Field

[0001] This invention belongs to the fields of computer graphics, real-scene 3D scene understanding, digital twin city visualization, and intelligent viewpoint planning. In particular, it relates to a method for achieving structured understanding of large-scale unstructured urban scene 3D models obtained from multiple sources such as oblique photography and LiDAR reconstruction, through semantic reconstruction and cognitive representation, and then realizing cognitive-driven optimal viewpoint decision-making. Background Technology

[0002] Best View Selection (BVS) technology is a core supporting technology for scenarios such as 3D model browsing, scene thumbnail generation, digital twin city entrance view recommendation, drone inspection observation site planning, and immersive scene navigation. Existing mainstream technical approaches include geometrically driven methods based on viewpoint entropy, projected area, curvature saliency, visible area maximization, and symmetry, as well as aesthetically scoring learning methods based on deep learning.

[0003] The aforementioned existing technologies generally treat 3D models as a set of undifferentiated geometric patches, using geometric statistics or low-level visual features in a single frame image as the evaluation criteria for viewpoint quality. While this approach yields good results in scenarios involving single objects or regularized CAD models, it suffers from systemic technical flaws when directly applied to large-scale real-world urban scenes. The core issue lies in the fact that traditional BVS (Browser Visual Recognition) defines the problem as a signal processing problem that maximizes undifferentiated geometric signals. This fundamentally conflicts with the "understand first, then decide" logic of human visual cognition in urban scenarios. Specific technical flaws are as follows:

[0004] 1. Weak anti-interference ability and easily misled by low-value information: Real-world 3D original meshes generally have problems such as tree canopy occlusion, holes, non-manifold connections, geometric fragments, and unstable textures. Relying on scoring functions based on curvature, local normals, and pure projected area, they are easily misled by noisy areas such as vegetation and gravel with "high-frequency geometric features but low semantic value", and cannot focus on the core elements that are truly valuable for observation.

[0005] 2. Undifferentiated processing, unable to adapt to the heterogeneity of urban scenes: Urban scenes are composed of objects with completely different semantics and functions, such as buildings, roads, vegetation, and ground. The contribution mechanism and evaluation criteria of different objects to the "high-quality view" are fundamentally different. However, existing methods treat all surfaces equally, which cannot reflect the differentiated weights at the object level, nor can they incorporate human observation preferences that conform to cognitive habits, such as the main facade of buildings and the extension of roads.

[0006] 3. Inability to depict spatial narrative needs and lack of collaborative observation capabilities: Relying solely on the visible area statistics of a single object cannot express the spatial narrative needs in human cognition, such as "the main building and surrounding roads should be presented together" and "landmark buildings should be seen together with their environment." It also cannot quantify the collaborative observation quality of multiple semantic objects, and is prone to problems such as "a single building is clear but the overall relationship of the scene is missing."

[0007] 4. Low solution efficiency, poor robustness, and easy to get trapped in local optima: The occlusion relationship of large-scale urban models is affected by building clusters and terrain undulations. Existing methods directly perform blind exhaustive search or local search in 3D space, which not only has a large amount of computation, but is also very easy to get trapped in suboptimal solutions caused by terrain anti-occlusion and local peaks, and cannot stably obtain a high-quality global view in complex scenes.

[0008] In summary, existing technologies cannot meet the requirements for optimal viewpoint selection in large-scale real-world urban scenarios. There is an urgent need for a technical solution that can recover semantic objects and structural priors from the original unstructured grid, construct cognitive relationships between objects, and stably obtain the globally optimal viewpoint within the feasible observation domain. Summary of the Invention

[0009] To overcome the shortcomings of the existing technologies, this invention provides a cognitive-driven optimal perspective selection method and system for large-scale urban scenarios. Through three sequentially connected core stages—semantic reconstruction, cognitive representation, and perspective decision-making—the original unstructured urban grid is transformed into a set of semantic objects, a structured scene graph, and a cognitive heat distribution that can be directly utilized for perspective optimization. Finally, the optimal perspective that conforms to human cognition is obtained.

[0010] According to one aspect of the present invention, a cognitive-driven optimal perspective selection method for large-scale urban scenarios is provided, comprising: The original real-world 3D mesh is acquired, geometric primitives are extracted from it, and semantic segmentation and structural parsing are performed using the geometric primitives as prior constraints to obtain a structured scene representation containing semantic attributes, structural information and topological relationships. The structured scene representation includes multiple semantic objects. The semantic objects in the structured scene representation are organized into a three-dimensional semantic scene graph, and the cognitive core degree of each semantic object in the scene is quantified based on the three-dimensional semantic scene graph to obtain the cognitive popularity of each semantic object. An adaptive candidate view surface is constructed, and the camera pose search is reduced to a direction optimization problem on the adaptive candidate view surface. A collaborative observation objective function is constructed based on the cognitive heat. The view that maximizes the collaborative observation objective function is solved on the adaptive candidate view surface and is taken as the optimal view.

[0011] As a further technical solution, the steps to obtain a structured scene representation also include: Extract planar primitives and linear structural primitives from the original real-world 3D mesh; Based on the aforementioned planar primitives and linear structural primitives, semantic segmentation is performed on urban scenes to identify four types of semantic objects: buildings, roads, vegetation, and ground. The intrinsic structure of the identified building objects is analyzed, and vegetation objects are merged and continuous terrain surfaces are restored.

[0012] As a further technical solution, the steps to obtain cognitive popularity also include: Using the buildings, roads, and vegetation objects as nodes and the spatial semantic relationships between objects as edges, the three-dimensional semantic scene graph is constructed, wherein the edge weights integrate the physical proximity and semantic connectivity between objects. Based on the three-dimensional semantic scene graph, the cognitive heat of each semantic object is calculated from three dimensions: physical saliency, topological connectivity, and semantic contextual saliency. The core objects and core regions in the scene are then identified through the cognitive heat.

[0013] As a further technical solution, an adaptive candidate viewpoint surface is constructed, including: For any observation direction, based on the preset camera field of view constraint, the minimum observation distance of each semantic object in that observation direction is calculated. The minimum observation distance is used to ensure that the semantic object is completely imaged within the view frustum. The maximum value of the minimum observation distance of all semantic objects in the observation direction is taken as the observation distance corresponding to the observation direction. A continuous and closed adaptive candidate view surface is generated from all observation directions and their corresponding observation distances, and the camera optical center is constrained on the adaptive candidate view surface.

[0014] As a further technical solution, the cooperative observation objective function is a quadratic function, and its expression is: , in, From the current perspective, This is a popularity vector composed of the cognitive popularity of each semantic object. The semantic adjacency matrix is ​​generated from the three-dimensional semantic scene graph. It is a diagonal matrix composed of the individual observation quality of each semantic object from the current perspective; The diagonal term of the collaborative observation objective function is used to characterize the independent observation gains of each semantic object, while the off-diagonal term is used to characterize the collaborative observation gains between related semantic objects.

[0015] As a further technical solution, the individual observation quality is obtained by weighted visibility calculation, wherein for building objects, the visible area weight of its main facade is higher than the weight of other facades and roof; for road objects, the visible area weight of the road side closest to the observation direction is higher than the weight of other parts.

[0016] As a further technical solution, a two-stage optimization strategy is adopted to solve for the viewpoint that maximizes the cooperative observation objective function on the adaptive candidate viewpoint surface: In the first stage, semantic prior-guided hierarchical coarse sampling is carried out: based on the orientation of the main facade of the core building, a high-density sampling zone is delineated in the azimuth domain. Sampling is performed in the high-density sampling zone with a first angular step size, and sampling is performed in the area outside the high-density sampling zone with a second angular step size. The first angular step size is smaller than the second angular step size. The collaborative observation objective function value of each sampling viewpoint is calculated, and the multiple views with the highest scores are selected as the initial seed views. In the second stage, the initial seed view is subjected to Riemann-constrained local optimization on the adaptive candidate view surface. For each initial seed view, the view is iteratively updated along the Riemann gradient direction of the objective function in the tangent space of the surface until convergence is obtained to obtain the local optimal view. The view with the highest co-observation objective function value among all local optimal views is selected as the global optimal view.

[0017] According to one aspect of the present invention, a cognitive-driven optimal perspective selection system for large-scale urban scenarios is provided, comprising: The semantic reconstruction module is configured to: acquire the original real-world 3D mesh, extract geometric primitives from it, and perform semantic segmentation and structural parsing using the geometric primitives as prior constraints to obtain a structured scene representation containing semantic attributes, structural information and topological relationships. The structured scene representation includes multiple semantic objects. The cognitive representation module is configured to: organize the semantic objects in the structured scene representation into a three-dimensional semantic scene graph, and quantify the cognitive core degree of each semantic object in the scene based on the three-dimensional semantic scene graph to obtain the cognitive heat of each semantic object; The viewpoint decision module is configured to: construct an adaptive candidate viewpoint surface, reduce the camera pose search to a direction optimization problem on the adaptive candidate viewpoint surface, construct a collaborative observation objective function based on the cognitive heat, and solve for the viewpoint that maximizes the collaborative observation objective function on the adaptive candidate viewpoint surface, which is then taken as the optimal viewpoint.

[0018] According to one aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the aforementioned cognitive-driven optimal perspective selection method for large-scale urban scenarios.

[0019] According to one aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the cognitive-driven optimal perspective selection method for large-scale urban scenarios.

[0020] Compared with existing optimal perspective selection techniques based on geometric statistics, this invention has the following clear technical advantages and beneficial effects: 1. Significantly improved anti-interference capability: By adopting the technical approach of "semantic reconstruction first, then perspective decision", vegetation and noise areas with low semantic value are first removed from the original grid, which solves the problem of traditional methods being misled by high-frequency geometric noise. Perspective evaluation always focuses on the core elements with cognitive value.

[0021] 2. More aligned with human observation and cognitive habits: By analyzing the inherent structure of the building's main facade and symmetrical surfaces, the evaluation system explicitly incorporates human preference for "frontal observation and clear structure" perspectives; at the same time, it constructs a cognitive representation model based on urban imagery theory, ensuring that perspective decisions conform to the spatial cognitive patterns of human perception of urban scenes.

[0022] 3. Balancing Subject Expression and Spatial Narrative: By designing a collaborative observation objective function, the diagonal term ensures the observation quality of high-profile core objects, while the off-diagonal term strengthens the collaborative presentation of related objects. This avoids the shortcomings of traditional methods that "emphasize the local and neglect the overall," and achieves an observation effect of "clear core subject and complete scene relationships."

[0023] 4. Significantly improved robustness and efficiency: By using an adaptive candidate view surface, the high-dimensional search problem is reduced to a two-dimensional manifold optimization problem, significantly reducing the computational load. Through a two-stage optimization strategy, the coverage of the global search is guaranteed by semantic prior guidance to avoid getting trapped in local optima, while the local optimization of Riemann constraints achieves an accurate solution. Even in large-scale urban scenes with complex occlusion and undulating terrain, the optimal view can still be obtained stably and efficiently.

[0024] 5. Strong engineering applicability: All modules of this invention are designed with robust mechanisms to address common defects in real-world 3D models (holes, non-manifold structures, reconstruction errors), and are equipped with complete parameter design logic and scene adaptation schemes. They can be directly adapted to oblique photogrammetry urban scenes of different precision and scale, and can be directly implemented in various business scenarios such as digital twin visualization, drone inspection, and 3D scene interaction. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram of the overall process of the cognitive-driven optimal perspective selection method for large-scale urban scenarios provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the surface structure recognition and line structure recognition results provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the semantic segmentation results of an urban scene provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the building symmetry plane and main facade identification results provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the distribution of scene object recognition heat provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the adaptive candidate viewpoint surface generation result provided in an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the optimal viewing angle effect output by the method provided in the embodiments of the present invention. Detailed Implementation

[0027] This invention addresses the core shortcomings of existing technologies by proposing an integrated optimal perspective selection method that combines semantic reconstruction, cognitive representation, and perspective decision-making. The core objective is: 1. Breaking through the limitations of the traditional paradigm of "maximizing geometric signals", the optimal perspective selection in urban scenes is reconstructed into a perspective decision problem driven by semantic cognition, thus solving the problem of being misled by low semantic noise at its root. 2. Achieve semantic and structural parsing of unstructured real-world meshes, automatically identify core semantic objects such as buildings, roads, vegetation, and ground, and parse the inherent structures of buildings such as main facades and symmetrical surfaces that conform to human cognition, providing semantic and structural priors for perspective evaluation; 3. Construct a scene representation system that conforms to the laws of human urban spatial cognition, build a three-dimensional semantic scene map based on Lynch's urban imagery theory, quantify the cognitive core degree of each object, automatically identify the core area of ​​the scene, and solve the problem that existing technologies cannot distinguish the core observation objects; 4. Construct a perspective evaluation system that takes into account both the subject's expression and spatial narrative, using a single-object observation quality operator and a collaborative observation objective function to ensure both the clear presentation of the core object and the collaborative expression of related objects; 5. Achieve efficient and robust optimal perspective solution: By adaptive candidate perspective surface dimensionality reduction search space and combined with a two-stage optimization strategy guided by prior knowledge of the main facade, the global optimal perspective is stably and efficiently obtained under complex occlusion and undulating terrain conditions.

[0028] The overall technical approach of this invention is divided into three sequentially connected core stages: semantic reconstruction, cognitive representation, and perspective decision-making. The original unstructured urban grid is transformed into a set of semantic objects, a structured scene map, and a cognitive heat distribution that can be directly utilized for perspective optimization. Ultimately, the optimal perspective that conforms to human cognition is obtained. The core technical solutions for each stage are as follows:

[0029] Phase 1: Semantic Reconstruction Starting from the input original real-world 3D mesh, the system first completes topology cleaning and robust computational basis construction, extracting two types of core geometric primitives: surface structure and line structure. Then, using the geometric primitives as prior constraints, it completes semantic segmentation and objectification of four core urban scene elements: vegetation, ground, buildings, and roads. Further, it analyzes the inherent structures such as building symmetry planes and main facades, while simultaneously merging vegetation objects and restoring continuous terrain surfaces. Finally, it outputs a structured scene representation with semantic attributes, structural information, and topological relationships, enabling the machine to "understand what's in the scene."

[0030] Phase Two: Cognitive Representation The discrete semantic objects output from semantic reconstruction are organized into a three-dimensional semantic scene graph with semantic entities as nodes and spatial semantic associations as edges. Combining Lynch's urban imagery theory, the cognitive core degree of each semantic object in the scene is quantified from three orthogonal dimensions: physical salience, topological connectivity, and semantic contextual salience, and the comprehensive cognitive heat of each object is calculated. Based on the heat distribution, the core area of ​​the scene is automatically identified, enabling the machine to "understand what is worth seeing in the scene".

[0031] Phase Three: Perspective Decision-Making First, based on perspective projection field constraints, an adaptive candidate view surface is generated, reducing the 6-DOF high-dimensional camera pose search to a directional optimization problem on a 2D manifold. Then, combining semantic scene graphs, cognitive heat, and structural priors, an object-level individual observation quality operator is constructed, while introducing a terrain occlusion penalty term to improve robustness. Furthermore, a collaborative observation objective function integrating cognitive importance, individual observation quality, and semantic association is constructed, while characterizing the independent observation benefits of core objects and the collaborative observation benefits of related objects. Finally, a two-stage optimization strategy guided by the main facade prior—hierarchical coarse sampling and Riemann constraint local optimization on the candidate surface—is adopted to efficiently and stably solve for the globally optimal view.

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings, core algorithm logic, formula definitions, parameter design rules, and engineering implementation paths. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. The implementation methods of the present invention are used to illustrate the implementable paths of the present invention and are not intended to limit the scope of protection of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Furthermore, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0033] The method described in this embodiment of the invention takes as input an OBJ format real-world 3D mesh model obtained by oblique photogrammetry reconstruction and outputs as camera pose parameters (including camera position, viewing direction, and viewing distance) corresponding to the optimal viewing angle.

[0034] The parameters of the entire link of this invention are divided into three categories according to their functions. This embodiment clearly defines the design logic, value selection principle, typical scenario configuration benchmark and debugging method of the parameters, as shown in Table 1. Those skilled in the art can determine the appropriate parameters through limited experiments based on this embodiment and in combination with the scale, accuracy and scenario characteristics of the input data.

[0035] Table 1 Parameter Stratification and Value Selection Principles .

[0036] For the three most common types of input data in urban scenarios, standardized configuration benchmarks for core parameters are provided, which can be directly implemented and used, as shown in Table 2:

[0037] Table 2. Core Parameter Configuration Baseline for Typical Scenarios .

[0038] If the baseline configuration cannot fully adapt to special scenarios, you can debug according to the following priority process to avoid blindly adjusting parameters: Fix the underlying parameters: First, set all robust fine-tuning parameters to default empirical values ​​and do not adjust them throughout the process; Prioritize determining core parameters: Start with the essential core parameters, fix the other parameters, adjust them one by one, and verify the adaptability through intermediate results (semantic segmentation effect, main facade recognition accuracy) until the expected results are achieved; Adaptation parameters for different scenarios: Adjust the scene adaptation parameters according to the scene characteristics of the data (such as increasing the elevation attenuation coefficient when the terrain is undulating and decreasing the voxel size when the buildings are dense). Finally, optimize the details: If there are more refined requirements for the final effect, fine-tune the robustness parameters.

[0039] This invention proposes a scene-optimal perspective computation framework based on "semantic reconstruction-cognitive representation-perspective decision-making." Unlike traditional methods that perform undifferentiated scalar searches directly on the original mesh, this invention advocates for first "understanding" and "structuring" the scene before selecting a perspective. First, starting from an unstructured oblique photogrammetry mesh, vegetation, ground, buildings, and roads are identified from the bottom up, organizing them into a scene graph with topological relationships. The core heat of each semantic component is then quantified using Lynch imagery theory, enabling the machine to understand the scene. Next, perspective selection is modeled as a collaborative observation optimization problem on candidate surfaces. Individual observation quality operators and collaborative observation objective functions are used to uniformly characterize object importance, object observation quality, and their semantic relationships. A two-stage optimization guided by a priori analysis of the main facade and Riemann-constrained local optimization on candidate surfaces are employed to ultimately solve for the globally stable optimal perspective.

[0040] See Figure 1 The module composition and data flow chain of the three core stages of semantic reconstruction, cognitive representation, and perspective decision-making are shown in the present invention. The specific implementation of the method in the semantic reconstruction stage is as follows: To enable the algorithm to understand the scene before making BVS decisions, this stage proposes a semantic reconstruction method for urban scenes: First, a robust computational basis is constructed through preprocessing, and the unique planar and linear structures of urban scenes are extracted as the basis. Then, semantic segmentation of four core categories—vegetation, ground, roads, and buildings—is achieved using geometric primitives as hard constraints. At the same time, the intrinsic structures such as building symmetry planes and main facades are analyzed, and the unstructured original mesh is transformed into structured entities with semantic attributes, providing core support for subsequent scene cognitive representation and viewpoint optimization.

[0041] S1, Data Preprocessing and Computational Basis Construction.

[0042] To combat scanning noise, a robust computational basis is first established. Specifically: First, vertex merging and degenerate triangle removal are performed. This includes: the system reads the vertices, triangles, texture coordinates, normals, and texture map information of the original mesh; calculates the basic geometric features of each triangle, such as center coordinates, average normal, area, slope, and roughness, providing basic data for subsequent end-to-end modules; for each triangle, a unique and stable signature is formed based on its center coordinates, normal, and area. The signature quantization rule is: the center coordinates and normal are discretized with fixed precision at the scene scale, and combined with the area hash value to generate a signature that uniquely identifies the facet. This signature is reused across all modules, including face structure recognition, semantic segmentation, structure parsing, and viewpoint decision-making, enabling stable facet-level indexing and result write-back across modules.

[0043] Secondly, a hybrid topology-spatial index is constructed using KDTree, defining the neighborhood as the union of topological and spatial neighborhoods to address the interruption of region growth caused by scanning holes. The topological adjacency relationship is built based on shared edges of triangular faces and is used for scenarios such as region growth, building connectivity analysis, road skeleton extraction, and hole filling. The spatial nearest neighbor relationship is a spatial proximity index built based on KDTree or voxel raster, used to compensate for topological discontinuities caused by holes and non-manifold connections, ensuring stable region growth even on models with local defects.

[0044] S2, extraction of surface and line structure primitives.

[0045] Based on S1, two types of core geometric primitives are extracted as prior constraints for semantic segmentation: (1) Planar primitives: based on the consistency of surface normals and flatness clustering, they are used to lock the building facade, roof and ground areas; (2) Linear structural primitives: based on dihedral angle and PCA linearity detection, after RANSAC fitting and orthogonalization correction, they are used to define road boundaries and building outlines.

[0046] Based on the above primitives, a unified seed growth segmentation framework was constructed. For any semantic category... The segmentation process follows the following paradigm: (1) Seed selection: using strong geometric features to select a set of high-confidence facets. (2) Constrained growth: Starting from the seed, growth extends to the neighborhood only under the premise of satisfying the geometric constraints and connectivity specific to this category. (3) Semantic verification: Perform morphological closing operation and logical verification on the generated semantic clusters to remove isolated noise.

[0047] In this embodiment, the core objective of surface structure recognition is to extract planar primitives from the original mesh, providing high-confidence seeds for the recognition of ground, building roofs, and building facades. The specific implementation steps are as follows: Feature filtering: For each triangular facet, candidate facets that meet the planar features are filtered by comprehensively considering normal consistency, flatness, roughness, elevation percentile, and vegetation exclusion features. Multi-stage regional growth: Adopting a phased regional growth strategy of "strict first, then relaxed", different threshold systems are configured for three typical planes: ground, roof, and vertical facade. The core rule is: In the initial stage, strict thresholds are used to screen high-confidence seed patches to ensure seed purity: the initial range of the normal cosine threshold is 0.95~0.98, and the initial plane distance threshold is 0.3~0.6 times the base distance threshold; In subsequent stages, the thresholds are gradually relaxed to accommodate slight fluctuations and reconstruction errors in the real-world model: the normal cosine threshold is gradually relaxed to 0.90~0.95, and the planar distance threshold is gradually relaxed to 0.8~1.2 times; Rejection rules for region growth: When the normal similarity between the newly added patch and the seed plane is lower than the current stage threshold, the distance to the seed plane exceeds the allowable range, the vegetation / noise probability is too high, or the roughness exceeds the threshold, the patch is rejected from entering the current plane cluster. Post-processing and void filling: After the phased growth is completed, conservative void filling is performed. The void is filled into the corresponding plane cluster only when most of the adjacency relationships in the neighborhood of the face to be filled point to the same plane cluster and its normal is consistent with the average normal height of the plane cluster. Output results: The final output includes the plane normal, plane center, area, plane type (ground / roof / facade / other), the index of the corresponding triangle face, and the corresponding signature set of each plane primitive, which serve as the core seed for subsequent semantic segmentation.

[0048] In this embodiment, the core objective of line structure recognition is to extract linear structures such as building edge lines, roof ridge lines, and road boundary lines, providing boundary constraints and structural priors for semantic segmentation. This embodiment employs a three-channel parallel detection strategy, specifically implemented as follows: Three-channel feature point extraction: (1) Sharp edge detection channel: Sharp edges are detected based on the normal angle between adjacent triangular faces. The larger the normal angle, the more likely it is to correspond to structural boundaries such as building edges and roof ridges; (2) Normal change point detection channel: The normal change degree is calculated in the vertex neighborhood, and normal change points are extracted to supplement the structural boundaries missed by sharp edge detection; (3) PCA linear feature detection channel: Neighborhood PCA geometric features are calculated on the scene point cloud, and points with high linearity are extracted to capture linear structures such as road edges, slender components, and roof ridges; Core parameter configuration: (1) Normal mutation grouping: neighborhood size knn (K nearest neighbor) general range 15~30, preferred value 20; normal dot product threshold general range 0.80~0.90, preferred value 0.85; (2) PCA linear feature: neighborhood radius general range 1.0~2.0m, preferred value 1.5m; linearity threshold general range 0.70~0.80, preferred value 0.75; (3) line segment fitting and merging: RANSAC residual threshold general range 0.3~0.8m, preferred value 0.5m; global merging angle tolerance general range 5°~10°, preferred value 7°; Feature point merging and line segment fitting: After merging the feature points extracted from the three channels, voxel downsampling and normal re-estimation are performed. Local segments are formed through region growing, and then the RANSAC algorithm is used to fit the line segments. Global segment merging and optimization: Perform global merging on segments with similar directions, small endpoint spacing, and that can be considered as collinear extensions to solve the segment fragmentation problem; Output results: The final output includes the classified road skeleton line, roof ridge line, and building edge line, providing linear priors for subsequent road bridging, building completion, and vegetation false detection rejection. Figure 2 The demonstration showcases the recognition performance of geometric primitives such as planar primitives, building edge lines, roof ridge lines, and road skeleton lines extracted from the original mesh.

[0049] S3, Semantic Segmentation and Objectification.

[0050] Based on the aforementioned extracted geometric primitives, this embodiment completes semantic segmentation and objectification of four core categories: vegetation, ground, buildings, and roads. It also performs analysis of the intrinsic structure of buildings and restoration of continuous terrain surfaces. The specific implementation method is as follows:

[0051] S3.1, Vegetation Identification and Merging.

[0052] The core objective of vegetation recognition is to accurately extract vegetation areas such as tree canopies, while avoiding false detections of green building textures, lawns, and rough walls. The texture color of candidate region pixels must be within the green spectrum of the HSV color space and possess a certain degree of geometric roughness. The specific implementation steps are as follows: Multi-feature candidate selection: Fusing color and geometric features to select vegetation candidate patches: Color characteristics: Convert the model texture to the HSV color space, and select the facets whose hue falls in the green range and whose saturation and brightness match the characteristics of vegetation as color candidates; Geometric features: Based on the consistency of the normals of the three vertices of the triangle, a geometric scatter score is constructed, with the following formula: , in, Let be any pair of normals among the three vertices of the triangle. This is the dispersion gain coefficient, with a typical value range of 0.8 to 1.5. The more inconsistent and higher the normal, the more likely the surface will be classified as an irregular surface such as vegetation. Structural exclusion rule: To avoid false detection, a structural exclusion constraint is introduced: If a patch falls into the neighborhood of a large planar structure or is close to an extracted line structure, it will not be included in the vegetation seed set even if its color and geometric features meet the conditions, thus avoiding false detection of green building facades and structured walls from the source. Region growth and post-processing: Region growth is performed based on high-confidence vegetation seeds, expanding only to adjacent facets with similar colors, similar normal dispersion, and no structural exclusion; after growth is completed, adjacent canopy fragments are connected by morphological closing operations, and isolated noise is removed according to the minimum cluster size and bounding box shape. Vegetation object merging: To address the fragmentation problem of the initial vegetation segmentation results, the spatial gaps, center distances, and volume balance of vegetation clusters are analyzed through disjoint-set data structure. Vegetation clusters that belong to the same tree group but are excessively fragmented are merged. The granularity of vegetation objects is adjusted to a scale that is coordinated with structural objects such as buildings and roads, so as to avoid the bias caused by too many vegetation nodes in subsequent scene graph construction and heat calculation.

[0053] S3.2, Ground and Building Recognition.

[0054] Ground and building identification share the same set of triangular faces, employing a "ground-first, building-complementary" strategy. The specific implementation method is as follows: (1) Implementation method of ground identification The core challenge of ground identification is overcoming interference from undulating terrain and low-rise, flat-roofed buildings to accurately locate the ground area. The specific implementation steps are as follows: Robust estimation of baseline ground elevation: Construct a weighted histogram of elevation attenuation, and perform weighted statistics on the elevation distribution across the entire scene. The weighting function is as follows: , in, Count the original histogram at elevation h. The attenuation coefficient, typically ranging from 0.05 to 0.2, is negatively correlated with the degree of terrain undulation. This function robustly locks the benchmark ground elevation by reducing the statistical weight of high-altitude planes and avoiding interference from rooftops and flat-roofed buildings in ground elevation estimation. ; Restricted area growth: Using a surface that is close to the reference ground elevation, with a normal that is nearly vertically upward and a gentle elevation difference from the surrounding area as a seed, region growth is performed; a step blocking rule is introduced: if there is an elevation change that exceeds the local tolerance between the candidate expansion surface and the current ground area, and the normal no longer remains nearly vertically upward, then region growth is prohibited from crossing this boundary, which preserves the natural terrain slope and delineates a clear boundary at the junction of the building and the ground, preventing growth from penetrating into the building facade; Output results: Output the initial set of ground patches to provide basic data for subsequent continuous terrain surface restoration.

[0055] (2) Implementation of building identification The core objective of building recognition is to accurately extract building objects and remove non-building noise such as vehicles and streetlights. The specific implementation steps are as follows: Facade seed selection and growth: Using facade plane elements that are nearly perpendicular to the ground, have a large area, and have a significant height relative to the ground as seeds, connected facade patches are aggregated into building facade clusters based on topological adjacency relationships. Building area integrity: The roof triangular facets that are topologically connected to the facade and have an upward component in the normal direction are absorbed into the same building object; the absorption of the sloping roof surface is supplemented by the extracted ridge line; the suspended and truncated exterior walls are filled downwards; and the adjacent facade clusters that are spatially continuous and cut by holes are merged to form a complete building unit. Structure-aware filtering: In order to eliminate non-building noise, filtering is performed on the candidate clusters after clustering. Valid building clusters must simultaneously meet the following requirements: they are highly isolated from the ground grid in space and contain or are adjacent to the previously extracted line and surface structure primitives. This is to eliminate noise such as vehicles and street lights that have local volume but no building structure features. Output results: Output the object number, bounding box, centroid, height range, total area, triangular face index and signature set of each building, providing a basis for subsequent building structure analysis.

[0056] S3.3, Analysis of the intrinsic structure of the building.

[0057] This embodiment further analyzes the two core intrinsic structures of the building: the symmetry plane and the main facade, providing structural priors that conform to human cognition for subsequent perspective optimization. The specific implementation method is as follows: (1) Implementation method for identifying building symmetry planes To address the global symmetry failure problem in complex buildings such as main buildings with annexes, L-shaped structures, and terraced structures, this invention employs a strategy of "first decomposition, then calculation, and optimal selection." The specific implementation steps are as follows: Pre-disassembly of building components: Project the center point of the building triangle onto the horizontal plane and obtain the main direction of the building through PCA; establish a cross-sectional density histogram around the main direction and the sampling direction to identify the bottleneck structure of "high density on both sides and low frequency in the middle", and divide the building into components along the bottleneck plane to obtain several building sub-components; Symmetry error calculation: For the building as a whole and for each sub-component, calculate the reflection symmetry error of the candidate symmetry plane separately. The formula is: , Where P is the set of sampling points for the building / sub-component. For candidate symmetric planes, Let point P be about the plane The mirror point; Symmetry plane selection: If the weighted average symmetry error after sub-component decomposition is significantly better than the overall symmetry error, then the local symmetry plane of each sub-component is retained; otherwise, the overall symmetry plane of the building is retained, and the final output is the set of symmetry planes of the building, providing prior knowledge for the identification of the main facade.

[0058] (2) Implementation method for identifying the main facade of a building The main facade is the core preferred direction for humans to observe a building. This embodiment comprehensively determines the orientation of the main facade from three dimensions: "coverage capacity, frontal attributes, and openness." The specific implementation steps are as follows: Candidate orientation construction: The projection of the building wall normal onto the horizontal plane is statistically analyzed to generate an orientation histogram, and the peak orientation is extracted as the candidate for the main facade; at the same time, the normal of the symmetry plane and its orthogonal direction are added to the candidate set to ensure that the orientation corresponding to the symmetrical structure is included in the evaluation. Multi-dimensional scoring calculation: For each candidate direction d, the following three core indicators are calculated: Area capture rate The formula for measuring the coverage of a candidate orientation on the main facade of a building is: , in, Let be the area of ​​the triangular facet of the wall. For the wall normal, For angular tolerance, the general range is 15°~30°, with a preferred value of 20°. This is an indicator function; it takes the value 1 if the condition is met, and 0 otherwise. Forward To prevent misclassification of the building's rear or side facades as the main facade, the formula for determining whether captured walls are concentrated on the front of the building is as follows: , The stronger the forward orientation, the more likely that direction is to form the building's facade.

[0059] Openness : To measure the openness of the candidate orientation, the main facade usually corresponds to a more open urban interface, and the formula is: , in, Let be the set of adjacent buildings in the corridor ahead of candidate direction d. The degree of obstruction / oppression of this direction by adjacent buildings, with a value ranging from 0 to 1; the greater the openness, the more conducive this direction is to forming a complete and clear main facade interface.

[0060] Comprehensive Score and Orientation Determination: After performing Min-Max normalization on the three indicators, the comprehensive score is obtained by weighted summation. The formula is as follows: , in, These are weighting coefficients, with a general value range of 0.2 to 0.5, and the sum of the three is 1. The direction with the highest score is selected as the orientation of the main facade of the building. If the difference between the highest score and the second highest score is less than the preset tolerance, the difference in the area of ​​the opposite facade, the consistency of the orientation of the main facade of the adjacent buildings, and the building volume characteristics are further checked to ensure the reliability of the main facade identification. Output results: Output the orientation of the main facade of each building and the set of triangular faces corresponding to the main facade, providing core support for the weight design of subsequent perspective evaluation and prior guidance for optimization solution. Figure 3 The semantic segmentation and objectification results of four core categories—vegetation, buildings, ground, and roads—are displayed. Figure 4 It shows the results of symmetry plane decomposition, main facade orientation identification, and corresponding facade areas of complex buildings.

[0061] S3.4, Continuous terrain restoration and road recognition (1) Implementation method for restoring continuous terrain surface Continuous terrain surfaces are not only used for ground area completion and reassessment, but also provide core data for terrain occlusion penalties in subsequent viewpoint decisions. The specific implementation steps are as follows: Elevation grid construction: The initial candidate ground triangular faces and non-building, non-vegetation gentle slope patches are projected into a two-dimensional regular grid; within each grid cell, the low quantile elevation is extracted as the original surface estimate to avoid interference from local protrusions on the surface elevation; the grid size has a general range of 0.5m to 2.0m, which is positively correlated with the scene accuracy; Elevation field optimization: Neighborhood search and compensation are performed on void areas in the grid, quantile suppression is performed on local spikes, and stricter peak filtering is applied to areas around buildings; after multiple rounds of smoothing, a continuous terrain grid covering the entire scene is formed and saved as a terrain surface structure that can be interpolated and queried. Ground reassessment: Based on the restored continuous terrain surface, calculate the elevation residuals of the center and vertices of the original triangular face relative to the terrain surface. Combined with the normal constraint of the facet, reassess the ground attributes of some edge facets and fill in the areas missed in the initial ground segmentation.

[0062] (2) Implementation method of road recognition The core objective of road recognition is to extract continuous road objects to provide urban skeleton data for scene cognitive representation. The specific implementation steps are as follows: Candidate surface screening: In the set of remaining surfaces after removing buildings and vegetation, road candidate surfaces are screened based on the following: the normal is nearly horizontal, the elevation difference relative to the terrain is within a reasonable range, and the texture uniformity and brightness stability meet the characteristics of roads; core thresholds: the minimum value of the normal z component is generally within the range of 0.4~0.6, and the maximum height from the ground surface is generally within the range of 5.0~10.0m; Connected component clustering and filtering: Perform connected component clustering on candidate patches, and remove small fragments and non-road areas based on the length, width, aspect ratio, and average distance of the connected components to surrounding buildings / vegetation. Road topology bridging repair: To address the problem of road fragmentation, the skeleton lines and endpoint tangent vectors of each road connected block are extracted. If the skeleton endpoints of adjacent road blocks are approximately collinear, the spatial gap is small, and the intermediate path is not obstructed by buildings, then bridging repair is automatically performed to form continuous road objects. Output results: Output the road object's number, centroid, bounding box, skeleton line, and set of faces to which it belongs, providing core data for subsequent scene graph construction.

[0063] The specific implementation of the method described in this embodiment of the invention in the cognitive representation stage is as follows: Based on the results of semantic reconstruction, this stage constructs a three-dimensional semantic scene graph, quantifies the cognitive popularity of each object, and identifies the core areas of the scene. The specific implementation method is as follows: S4.1, Construction of 3D Semantic Scene Graph.

[0064] This invention constructs a three-dimensional semantic scene graph with semantic entities as nodes and spatial semantic associations as edges. This provides a structured platform for calculating cognitive popularity. The specific implementation steps are as follows: Node definition: Instantiate the semantically reconstructed building, road, and vegetation objects as graph node sets respectively. Each node Encapsulates core attributes such as object type, unique ID, centroid, bounding box volume, height range, triangle facet representative point set, and semantic tags; Spatial Indexing and Candidate Edge Filtering: To reduce the computational overhead of adjacency queries in large-scale scenarios, a voxel hash grid is used to establish a spatial index, dividing the 3D space into voxels of fixed size. Only when the voxels occupied by two objects overlap / are adjacent, and the centroid distance is less than the adjacency threshold, are they included as candidate connection pairs, and undirected edges are established. The adjacency threshold generally ranges from 20.0 to 50.0 m, and is positively correlated with the scene scale. Semantic weighted edge calculation: Traditional scene graphs typically establish binary edges based solely on physical distance, neglecting the functional differences of various semantic elements in spatial organization. According to Lynch's theory of urban imagery, roads are the core framework connecting urban spaces and guiding visual attention. Therefore, this embodiment constructs an adjacency matrix that integrates physical proximity and semantic connectivity. For a pair of entities connected by an edge, the edge weight is defined as: , in, Let σ be the centroid of the two nodes, and σ be the Gaussian decay scale, with a general range of 8.0~25.0m. Ω is the semantic connectivity coefficient. When either end of the edge is a road node, Ω is preferably set to 2.0, and in other scenarios, it is set to 1.0, so as to highlight the connecting and guiding role of roads in urban space.

[0065] Output results: Output the node set and semantic adjacency matrix Asem of the 3D semantic scene graph, providing a foundation for subsequent cognitive heat calculation.

[0066] S4.2, Cognitive Heat Calculation and Core Region Identification.

[0067] This invention, in conjunction with Lynch's urban imagery theory, quantifies the cognitive core level of each object from three orthogonal dimensions: physical salience, topological connectivity, and semantic contextual salience. The specific implementation steps are as follows: Physical salience calculation: Physical salience corresponds to the "unique physical features that can be quickly captured" in the urban image. It primarily measures the visual appeal of an object's size and height. The formula is: , in, They are nodes Normalized size and height Normalization adopts the full-scene Min-Max approach; a logarithmic function is introduced to compress the dynamic range and avoid extremely large objects from overwhelming small core structures; for vegetation objects, their size is truncated at the 95th percentile to avoid large areas of vegetation crowding out the normalization interval. Topological connectivity computation: Topological connectivity corresponds to the "structural position of an object in the urban spatial network" in urban imagery. It simulates the propagation and reinforcement of human visual attention in a scene using a random walk model. The specific implementation method is as follows: , , After the iteration converges, Min-Max normalization is performed on r to obtain the topological connectivity score for each node. Nodes located at structural hubs receive a sustained boost in influence during random walks, resulting in higher topology scores; u represents a globally uniform vector.

[0068] Semantic contextual saliency calculation: Contextual saliency measures the visual contrast of an object within its local neighborhood, corresponding to human visual attentional preference for objects that are "semantically unique and scale-prominent." The formula is as follows: , in, For nodes The first term of the formula is the semantic heterogeneity of the neighborhood, which measures the semantic difference between the object and its surrounding environment; the second term is the scale contrast, which measures the size difference between the object and its surrounding objects; the two are multiplied to obtain the saliency score of the local context; the greater the scale difference between the object and its surrounding objects, the stronger the visual enhancement effect of the surrounding environment on the object.

[0069] Comprehensive cognitive popularity calculation: The scores from the three dimensions are merged, and the distinction between high-scoring sections is enhanced using the Sigmoid function to obtain the comprehensive cognitive popularity of each node. The formula is as follows: , Core area identification: Dynamic thresholding is used to filter core objects. Preferably, the core object set is composed of nodes whose heat distribution is the mean plus 0.3 to 0.8 times the standard deviation. The core cognitive area corresponding to the scene. Figure 5 The results of comprehensive cognitive popularity calculation for each semantic object are presented, as well as the core objects and core regions identified based on popularity recognition.

[0070] The specific implementation of the method described in this embodiment of the invention during the perspective decision-making stage is as follows: This stage, based on prior semantic, structural, and cognitive knowledge, completes the solution for the optimal perspective. The specific implementation involves four core steps: candidate perspective surface construction, observation quality operator design, collaborative observation objective function construction, and two-stage optimization solution, as detailed below: S5.1, Adaptive candidate viewpoint surface construction.

[0071] To address the high-dimensional and ill-posed problem of pose search for a 6-DOF camera, this embodiment of the invention reduces the high-dimensional search to a direction optimization problem on a two-dimensional manifold by using an adaptive candidate viewpoint surface. The specific implementation steps are as follows: , To reduce the dimensionality of the high-dimensional search, the camera focus is anchored at the center of the scene. Viewpoint parameterization ,in It is the azimuth angle. Angle of elevation, This represents the observation distance from the camera to the center of the scene.

[0072] Minimum observation distance calculation at the direction level: for any observation direction Based on the field-of-view constraints of perspective projection, the minimum viewing distance of each semantic object in this direction is calculated using the following formula: , in The vertical field of view of the camera. This refers to the horizontal field of view of the camera; Defines the vertical height and horizontal width of object i in the current viewing coordinate system; This formula compensates for the depth component of an object that protrudes beyond the center of the scene along the viewing direction. Traditional methods often sample on a fixed bounding box, which can easily lead to edge objects being clipped by the camera's view frustum. This formula ensures that the object is fully imaged within the view frustum and is not clipped.

[0073] Adaptive surface generation: For each viewing direction, the maximum value of the minimum viewing distances of all objects is taken as the final viewing distance for that direction, using the following formula: , Thus, a continuous, closed, adaptive candidate view surface that closely follows the semantic distribution of the scene is generated in three-dimensional space. The camera optical center is strictly constrained on this surface, which not only avoids the object clipping problem caused by fixed bounding sphere sampling, but also reduces the high-dimensional search problem to an optimization problem on a two-dimensional spherical manifold, greatly reducing the difficulty of solving the problem. Figure 6 It demonstrates the difference between continuous closed candidate view surfaces generated based on scene semantic objects and field of view constraints and traditional fixed bounding sphere sampling.

[0074] S5.2, Design of Individual Observation Quality Operator.

[0075] This embodiment constructs an object-level single-entity observation quality operator, which measures the effective visibility of an object, incorporates differentiated weights based on structural priors, and introduces terrain occlusion penalties to improve robustness. The specific implementation method is as follows: (1) Weighted visibility calculation: For a specific viewpoint v, off-screen rendering is used to obtain the depth buffer and object-level visible pixels. The weighted visible area ratio of each object is calculated using the following formula: , in, Let i be the set of all surfaces of object i. Let A(p) be the set of visible surfaces of object i from viewpoint v, where A(p) is the actual area of ​​surface unit p, and ω(p) is the structural weight of surface unit p. The differentiation rule is: Building objects: The main facade has a significantly higher weight than parallel facades, side walls and roof. The preferred weight ratio is: main facade: other facades: roof = 10:2:1, which not only strengthens the core position of the main facade, but also guides the algorithm to achieve a balance between "clear main facade" and "three-dimensionality". Road objects: Construct a virtual space volume, and use the side of the road that is closest to the viewing direction as the main display surface, giving it higher weight and strengthening the sense of road extension and spatial connectivity. Vegetation objects: Approximately uniform weights are used to avoid vegetation interfering with the evaluation of core objects.

[0076] (2) Construction of terrain occlusion penalty term: To address the overall occlusion problem caused by terrain undulation in large-scale scenes, a low-frequency terrain penalty term is constructed to guide global optimization and prevent the algorithm from falling into the inferior region of terrain occlusion. The specific implementation method is as follows: Extract the surface elevation profile along the observation direction and calculate the terrain uplift along the line of sight path. Thus, the terrain penalty term is constructed: , in For terrain penalty weights, This represents the tolerance threshold. Here, terrain obstruction is quantified as a soft penalty, which, while suppressing the risk of terrain obstruction, allows reasonable passage through unobstructed low-lying areas such as canyons.

[0077] Individual observation quality operator integration: Combining weighted visibility with terrain penalty term to obtain the final individual observation quality of object i at viewpoint v:

[0078] This formula quantifies observation quality from both the terrain penalty of individual objects and the overall model, providing input for subsequent perspective decisions. The individual observation quality of all objects constitutes a diagonal observation quality matrix. , where N is the total number of semantic objects.

[0079] S5.3, Construction of the collaborative observation objective function.

[0080] The previous section modeled the observation quality for a single semantic object. However, in urban building scenarios, semantic objects are not singular; it is necessary to consider the balance of observation quality among different semantic objects and to present the relationships between them. The quadratic symmetric structure makes the joint contribution between object pairs order-independent, thus suitable for expressing the collaborative observation relationships between buildings, roads, and environmental elements in urban scenarios. Furthermore, this structure is highly compatible with the symmetric observation quality operator and symmetric semantic adjacency matrix constructed in this embodiment. Therefore, this embodiment introduces a quadratic collaborative observation objective function, simultaneously characterizing the benefits of independent observation and collaborative observation. The specific implementation is as follows:

[0081] Objective function definition: Combining the cognitive popularity vector Semantic adjacency matrix Individual observation quality matrix Construct the objective function: , When expanded, it can be represented as: , in, For object With object Association strength in semantic scene graphs and The two figures represent the level of public awareness for each other.

[0082] Benefit decomposition and design logic: The objective function includes two types of complementary observation benefits, perfectly matching the core requirement of perspective selection in urban scenarios: Independent observation benefit: When i=j, the diagonal term of the corresponding formula This describes the independent observational gain of a single object. This term is proportional to the square of the object's cognitive popularity, giving high cognitive popularity objects a greater weight in perspective optimization. This drives the algorithm to prioritize the high-quality presentation of high-popularity core objects, allowing the perspective to focus on the core of the scene rather than treating all objects equally.

[0083] Benefits of collaborative observation: when When, the off-diagonal terms of the corresponding formula This objective function characterizes the collaborative observational benefits of two related objects. If the two objects are closely connected in the semantic scene graph and both have high observational quality from the current perspective, this objective will significantly improve the overall score; conversely, if a perspective enhances the local subject but causes related objects to fall out of the field of view or be severely occluded, the corresponding joint benefit will be weakened. Thus, the objective function can automatically balance the expression of local subjects with the overall spatial narrative, ensuring that the final perspective not only highlights the core object but also preserves the shared presentation of related roads, building complexes, and environmental elements.

[0084] S5.4, two-stage optimization solution.

[0085] Defined on the candidate view surface Cooperative observation objective function Essentially, this is a non-convex optimization problem. Influenced by factors such as differences in building orientation, occlusion relationships, object spatial distribution, and terrain undulations, the objective function typically exhibits a multi-peak structure on the candidate surface. Directly performing a dense, continuous search across the entire candidate surface is computationally expensive; relying solely on local iterations easily leads to local optima. To balance global coverage with local optimization efficiency, this embodiment employs a two-stage solution framework: first, utilizing the structural prior obtained from pre-sequence semantic analysis, layered coarse sampling is performed on the candidate surface to select a small number of high-potential initial viewpoints; then, starting from these seeds, local continuous optimization under Riemann constraints is performed on the candidate surface to ultimately obtain the globally optimal viewpoint.

[0086] On the other hand, to reduce the interference of local occlusion jumps, fragmented geometric noise, and discrete sampling errors on single-point scoring, this embodiment does not directly use... Instead of using it as an optimization criterion, it further constructs its neighborhood stable form: , in, Indicates the candidate surface in terms of perspective The local neighborhood centered on, This represents the expectation within the neighborhood. This stable objective is present throughout the two-stage optimization process: the first stage uses... The candidate points are scored and the initial seeds are selected; the second stage is based on... The Euclidean gradient is calculated, step size acceptance and backtracking are performed, and it is used as the final comparison standard from the perspective of local extrema and global optimum.

[0087] (1) First stage: Semantic prior-guided hierarchical coarse sampling

[0088] The core objective of this stage is to leverage the semantic prior obtained from the preceding sequence to efficiently select high-potential initial seed perspectives, avoiding the high computational cost of global dense sampling, while ensuring global coverage and preventing the omission of high-quality perspectives. The specific implementation steps are as follows:

[0089] First, the orientation of the main facade of the core building is the most direct structural clue to the distribution of high-quality viewing angles. Assuming that the main facade orientations of high-core buildings in the scene form several dominant orientation clusters after clustering, high-density sampling zones are delineated on the candidate surface around these dominant orientations. This area is most likely to correspond to viewing directions with clear main facade expressions and prominent architectural structural features; therefore, a smaller angular step size is used for denser sampling to improve the ability to capture high-quality local peaks.

[0090] Secondly, for the remaining areas outside the aforementioned high-density sampling zone, low-density exploratory sampling is conducted with a larger step size to maintain basic traversal capability of the global space and avoid missing out on high-quality viewpoints that fall completely outside the prior knowledge of the main facade.

[0091] For any sampled candidate viewpoint In this embodiment, the stable target value is calculated. This is used to generate an initial candidate set; then, the candidates are sorted from highest to lowest score, and the top candidates are retained. Each perspective serves as a seed for the second stage of local optimization, and the optimal value of K ranges from 5 to 20.

[0092] In specific implementation, the core parameters include: azimuth angle cluster merging tolerance range of 20°~30°, high-density sampling band half-width range of 20°~25°, and maximum number of clusters range of 3~5.

[0093] (2) Second stage: Local optimization of Riemann constraints on candidate surfaces

[0094] The high-resolution seeds obtained in the first stage can significantly narrow the search range, but their positions are still limited by the discrete sampling resolution, making it difficult to directly reach the exact optimum. Therefore, this embodiment further refines the candidate viewpoint surface... The above methods involve performing local continuous optimization on these seeds to obtain precise local optimal solutions, and finally selecting the globally optimal perspective from them. The specific implementation steps are as follows:

[0095] Riemann gradient calculation: Since the viewpoint is strictly constrained to the candidate surface, this embodiment adopts the optimization framework of Riemann gradient ascent to ensure that the optimization process always satisfies the surface constraints.

[0096] From the current perspective of the k-th iteration First, the stable target is calculated using a finite difference approximation over the parameter domain of the candidate surface. The local rate of change is then calculated, and further combined with the first-order partial derivative of the surface parameterization, to recover it as the Euclidean gradient in the three-dimensional embedding space. ;

[0097] Subsequently, the candidate surface was recorded in The unit normal vector at is Then, by removing the normal component of the gradient through orthogonal projection, we obtain the value located in the current tangent space. Riemann gradient within: .

[0098] The purpose of this step is to restrict the update direction, which might otherwise cause the viewpoint to deviate from the candidate surface, to the tangential feasible direction on the surface, thereby ensuring that subsequent optimization always follows the geometric constraints of the candidate surface.

[0099] After obtaining the Riemann gradient, this embodiment normalizes it to the ascending direction of the current iteration and adaptively determines the step size using the Armijo line search strategy. Let the tangential update vector be... ,

[0100] The new candidate viewpoint is obtained through a retraction mapping on the surface, that is, first along the tangent space of the current point... Perform local displacement, and then project the updated results back onto the candidate surface using the nearest point. This yields the perspective for the next iteration. In each iteration, a new candidate viewpoint is accepted only if it satisfies the Armijo sufficient ascent condition; otherwise, the step size is gradually reduced and backtracking continues. This effectively suppresses oscillations caused by local occlusion and fragmented structures in complex urban scenes, making the optimization process focus more on stable upward trends rather than accidental local spikes. When the Riemann gradient magnitude is below a preset threshold, or when no effective step size satisfying the ascent condition can be found after backtracking, the current seed is considered to have converged to a local extremum, and the corresponding locally optimal viewpoint is obtained.

[0101] After performing the second stage of local optimization on all the seeds selected in the first stage, a set of locally convergent perspectives can be obtained. This embodiment compares the stable target values ​​corresponding to these local extrema. The highest scorer is selected as the globally optimal perspective. Output the corresponding camera pose parameters, including azimuth, elevation, observation distance, camera spatial position, and observation direction. Then, combine this with the minimum observation distance given earlier in this direction. The final camera pose is determined and the optimal viewing angle is output. Figure 7 The image displayed shows the rendered view corresponding to the optimal perspective obtained from the final solution.

[0102] In this embodiment, during line search and update, the optimal value for the sufficient ascent coefficient of the Armijo line search is selected. This ensures the stability of the iteration.

[0103] The implementation of the various embodiments of the present invention is based on programmed processing through a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a cognitive-driven optimal perspective selection system for large-scale urban scenarios. This system is used to execute the cognitive-driven optimal perspective selection method for large-scale urban scenarios in the above method embodiments.

[0104] The system includes: The semantic reconstruction module is configured to: acquire the original real-world 3D mesh, extract geometric primitives from it, and perform semantic segmentation and structural parsing using the geometric primitives as prior constraints to obtain a structured scene representation containing semantic attributes, structural information and topological relationships. The structured scene representation includes multiple semantic objects. The cognitive representation module is configured to: organize the semantic objects in the structured scene representation into a three-dimensional semantic scene graph, and quantify the cognitive core degree of each semantic object in the scene based on the three-dimensional semantic scene graph to obtain the cognitive heat of each semantic object; The viewpoint decision module is configured to: construct an adaptive candidate viewpoint surface, reduce the camera pose search to a direction optimization problem on the adaptive candidate viewpoint surface, construct a collaborative observation objective function based on the cognitive heat, and solve for the viewpoint that maximizes the collaborative observation objective function on the adaptive candidate viewpoint surface, which is then taken as the optimal viewpoint.

[0105] This invention provides a cognitive-driven optimal perspective selection system for large-scale urban scenarios. Addressing the current situation where existing technologies cannot meet the optimal perspective selection requirements for large-scale real-world urban scenarios, this system employs several modules to recover semantic objects and structural priors from the original unstructured grid, construct cognitive relationships between objects, and stably obtain the globally optimal perspective within the feasible observation domain.

[0106] It should be noted that the system embodiments provided by this invention, in addition to implementing the methods in the above method embodiments, are also used to implement the methods in other method embodiments provided by this invention. The difference lies only in setting corresponding functional modules, and their principles are basically the same as those of the above system embodiments provided by this invention. As long as those skilled in the art, based on the above system embodiments and referring to the specific technical solutions in other method embodiments, obtain corresponding technical means and technical solutions composed of these technical means by combining technical features, and improve the modules in the above system embodiments while ensuring the practicality of the technical solutions, they can obtain corresponding system-like embodiments for implementing the methods in other method-like embodiments. For example:

[0107] Based on the above system embodiments, as a preferred embodiment, the present invention provides a cognitive-driven optimal perspective selection system for large-scale urban scenarios, wherein the semantic reconstruction module further includes: Data preprocessing unit: responsible for reading the input real-world 3D mesh model, completing basic geometric feature calculations, cross-module stable signature construction, and establishing topological adjacency and spatial nearest neighbor relationships, providing a unified computing basis for all subsequent modules; Geometric primitive extraction unit: includes surface structure recognition subunit and line structure recognition subunit, which are responsible for extracting planar primitives and linear structure primitives from the original mesh, respectively, to provide prior constraints for subsequent semantic segmentation; Semantic Segmentation and Structure Parsing Unit: Includes vegetation recognition subunit, ground recognition subunit, building recognition subunit, building structure parsing subunit, terrain surface restoration subunit, and road recognition subunit. It is responsible for completing the semantic segmentation and objectification of the four core elements of the urban scene, parsing the internal structure of buildings, restoring continuous terrain surfaces, and outputting a structured set of semantic objects.

[0108] Based on the above system embodiments, as a preferred embodiment, this invention provides a cognitive-driven optimal perspective selection system for large-scale urban scenarios, wherein the cognitive representation module further includes: The scene graph construction unit and the cognitive heat calculation unit are responsible for organizing semantic objects into a three-dimensional semantic scene graph, calculating the comprehensive cognitive heat of each object, identifying the core area of ​​the scene, and providing cognitive priors for perspective decision-making.

[0109] Based on the above system embodiments, as a preferred embodiment, this invention provides a cognitive-driven optimal perspective selection system for large-scale urban scenarios, wherein the perspective decision module further includes: The candidate surface generation unit, view evaluation unit, and optimization solution unit are responsible for generating adaptive candidate view surfaces, constructing individual observation quality operators and collaborative observation objective functions, solving for the globally optimal view through a two-stage optimization strategy, and outputting the final camera pose parameters.

[0110] The modules of the above system can be deployed on the same computing device, or they can be deployed in a distributed and collaborative manner through file interfaces and network service interfaces to adapt to different engineering implementation scenarios.

[0111] This invention also provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor and the memory are communicatively connected via a bus. When the processor executes the computer program, it implements all the steps of the cognitive-driven optimal viewpoint selection method for large-scale urban scenarios described in any of the above embodiments.

[0112] Optionally, the electronic device may also include a network interface, which may include a wired interface and / or a wireless interface for data interaction with external devices. The electronic device may be a desktop computer, laptop computer, server, industrial control equipment, digital twin platform terminal, or other device with computing capabilities.

[0113] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements all the steps of the cognitive-driven optimal perspective selection method for large-scale urban scenarios described in any of the above embodiments.

[0114] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] In summary, this invention proposes a novel computational framework of "semantic reconstruction-cognitive representation-perspective decision-making." First, a bottom-up semantic reconstruction algorithm is designed to resolve unstructured meshes into semantic objects such as buildings, roads, and vegetation, each containing intrinsic structure. Second, Lynch's urban imagery theory is introduced to construct a three-dimensional semantic scene graph to understand the relationships between objects in the scene, and the core importance of each object in the scene is quantified from three dimensions: physical volume, topological connectivity, and semantic context. Then, an observation quality operator oriented towards individual objects is constructed, and a collaborative observation objective function integrating cognitive importance, individual observation quality, and semantic association is further established to characterize the joint presentation effect of multiple semantic objects. Finally, this invention employs a two-stage optimization strategy guided by the prior of the main facade on the adaptive candidate perspective surface, and introduces Riemann constraint optimization on the candidate surface in the local stage to solve for the globally stable optimal perspective.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A cognitive-driven optimal perspective selection method for large-scale urban scenarios, characterized in that, include: The original real-world 3D mesh is acquired, geometric primitives are extracted from it, and semantic segmentation and structural parsing are performed using the geometric primitives as prior constraints to obtain a structured scene representation containing semantic attributes, structural information and topological relationships. The structured scene representation includes multiple semantic objects. The semantic objects in the structured scene representation are organized into a three-dimensional semantic scene graph, and the cognitive core degree of each semantic object in the scene is quantified based on the three-dimensional semantic scene graph to obtain the cognitive popularity of each semantic object. An adaptive candidate view surface is constructed, and the camera pose search is reduced to a direction optimization problem on the adaptive candidate view surface. A collaborative observation objective function is constructed based on the cognitive heat. The view that maximizes the collaborative observation objective function is solved on the adaptive candidate view surface and is taken as the optimal view.

2. The cognitive-driven optimal perspective selection method for large-scale urban scenarios according to claim 1, characterized in that, The steps to obtain a structured scene representation also include: Extract planar primitives and linear structural primitives from the original real-world 3D mesh; Based on the aforementioned planar primitives and linear structural primitives, semantic segmentation is performed on urban scenes to identify four types of semantic objects: buildings, roads, vegetation, and ground. The intrinsic structure of the identified building objects is analyzed, and vegetation objects are merged and continuous terrain surfaces are restored.

3. The cognitive-driven optimal perspective selection method for large-scale urban scenarios according to claim 2, characterized in that, The steps to obtain cognitive popularity also include: Using the buildings, roads, and vegetation objects as nodes and the spatial semantic relationships between objects as edges, the three-dimensional semantic scene graph is constructed, wherein the edge weights integrate the physical proximity and semantic connectivity between objects. Based on the three-dimensional semantic scene graph, the cognitive heat of each semantic object is calculated from three dimensions: physical saliency, topological connectivity, and semantic contextual saliency. The core objects and core regions in the scene are then identified through the cognitive heat.

4. The cognitive-driven optimal perspective selection method for large-scale urban scenarios according to claim 1, characterized in that, Constructing an adaptive candidate view surface includes: For any observation direction, based on the preset camera field of view constraint, the minimum observation distance of each semantic object in that observation direction is calculated. The minimum observation distance is used to ensure that the semantic object is completely imaged within the view frustum. The maximum value of the minimum observation distance of all semantic objects in the observation direction is taken as the observation distance corresponding to the observation direction. A continuous and closed adaptive candidate view surface is generated from all observation directions and their corresponding observation distances, and the camera optical center is constrained on the adaptive candidate view surface.

5. The cognitive-driven optimal perspective selection method for large-scale urban scenarios according to claim 1, characterized in that, The objective function for collaborative observation is a quadratic function, and its expression is: ， in, From the current perspective, This is a heat vector composed of the cognitive heat of each semantic object. The semantic adjacency matrix is ​​generated from the three-dimensional semantic scene graph. It is a diagonal matrix composed of the individual observation quality of each semantic object from the current perspective; The diagonal term of the collaborative observation objective function is used to characterize the independent observation gains of each semantic object, while the off-diagonal term is used to characterize the collaborative observation gains between related semantic objects.

6. The cognitive-driven optimal perspective selection method for large-scale urban scenarios according to claim 5, characterized in that, The individual observation quality is obtained through weighted visibility calculation. For building objects, the visible area of ​​the main facade has a higher weight than that of other facades and the roof. For road objects, the visible area of ​​the side of the road closest to the observation direction has a higher weight than that of other parts.

7. The cognitive-driven optimal perspective selection method for large-scale urban scenarios according to claim 2, characterized in that, The viewpoint that maximizes the cooperative observation objective function is solved on the adaptive candidate viewpoint surface using a two-stage optimization strategy: In the first stage, semantic prior-guided hierarchical coarse sampling is carried out: based on the orientation of the main facade of the core building, a high-density sampling zone is delineated in the azimuth domain. Sampling is performed in the high-density sampling zone with a first angular step size, and sampling is performed in the area outside the high-density sampling zone with a second angular step size. The first angular step size is smaller than the second angular step size. The collaborative observation objective function value of each sampling viewpoint is calculated, and the multiple views with the highest scores are selected as the initial seed views. In the second stage, the initial seed view is subjected to Riemann-constrained local optimization on the adaptive candidate view surface. For each initial seed view, the view is iteratively updated along the Riemann gradient direction of the objective function in the tangent space of the surface until convergence is obtained to obtain the local optimal view. The view with the highest co-observation objective function value among all local optimal views is selected as the global optimal view.

8. A cognitive-driven optimal perspective selection system for large-scale urban scenarios, characterized in that, include: The semantic reconstruction module is configured to: acquire the original real-world 3D mesh, extract geometric primitives from it, and perform semantic segmentation and structural parsing using the geometric primitives as prior constraints to obtain a structured scene representation containing semantic attributes, structural information and topological relationships. The structured scene representation includes multiple semantic objects. The cognitive representation module is configured to: organize the semantic objects in the structured scene representation into a three-dimensional semantic scene graph, and quantify the cognitive core degree of each semantic object in the scene based on the three-dimensional semantic scene graph to obtain the cognitive heat of each semantic object; The viewpoint decision module is configured to: construct an adaptive candidate viewpoint surface, reduce the camera pose search to a direction optimization problem on the adaptive candidate viewpoint surface, construct a collaborative observation objective function based on the cognitive heat, and solve for the viewpoint that maximizes the collaborative observation objective function on the adaptive candidate viewpoint surface, which is then taken as the optimal viewpoint.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the cognitive-driven optimal perspective selection method for large-scale urban scenarios as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the cognitive-driven optimal perspective selection method for large-scale urban scenarios as described in any one of claims 1 to 7.

Images