News scene three-dimensional reconstruction and visualization method based on multi-source remote sensing data

Abstract

The invention provides a news scene three-dimensional reconstruction and visualization method based on multi-source remote sensing data, and relates to the technical field of three-dimensional reconstruction, and the method comprises the steps: extracting multi-modal data feature points through employing a spatial position weighted iterative nearest point algorithm and a tensor projection algorithm; after bidirectional attention deep neural network processing and local geometric consistency constraint screening, registration multi-source remote sensing data is generated, data enhancement is carried out through a variational auto-encoder network and a generative adversarial network, the enhanced data is used for depth information generation and texture feature migration, an initial three-dimensional grid model is generated based on a Poisson reconstruction algorithm, and the initial three-dimensional grid model is constructed; and carrying out topological optimization by combining multi-view projection consistency constraint and a grid optimization algorithm to obtain an optimized three-dimensional scene model, setting a space division function, carrying out space division, constructing a grade model and a depth region, generating a scene semantic segmentation map, and updating a visual scene.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional reconstruction and visualization method of news scenes based on multi-source remote sensing data. Background Art

[0002] 3D reconstruction technology aims to restore the 3D structure of the real world from 2D images or other data. It has been widely used in urban planning, disaster assessment, virtual reality and other fields. With the rapid development of remote sensing technology, 3D reconstruction based on multi-source remote sensing data has become a research hotspot. It can comprehensively utilize the information obtained by different sensors, such as the spectral information of optical images, the penetration ability of radar and the high-precision elevation information of lidar, so as to reconstruct the 3D scene more completely and accurately. Traditional remote sensing 3D reconstruction methods mainly rely on technologies such as stereo photogrammetry or structured light. However, these methods have certain limitations when dealing with complex scenes, especially news scenes that are dynamically changing and rich in information. There are still problems such as difficulty in fusing multi-source remote sensing data, low reconstruction accuracy of complex scenes, and low efficiency of scene visualization. Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0003] The embodiment of the present invention provides a three-dimensional reconstruction and visualization method of news scenes based on multi-source remote sensing data, which can at least solve some of the problems existing in the prior art.

[0004] A first aspect of an embodiment of the present invention provides a method for three-dimensional reconstruction and visualization of news scenes based on multi-source remote sensing data, comprising: Multi-source remote sensing data corresponding to the target news scene is collected through remote sensing satellites. Hierarchical feature descriptors are extracted based on the iterative closest point algorithm and tensor projection algorithm based on spatial position weighting and bidirectional attention deep neural network. The data is screened through a random sampling algorithm based on local geometric consistency constraints, and feature point pairs with projection errors greater than a preset projection threshold are eliminated and unified to global geographic coordinates to obtain registered multi-source remote sensing data. Enhanced remote sensing data is obtained through data enhancement using a variational autoencoder network and a noise-aware loss function combined with a generative adversarial network based on pixel-level adversarial discrimination. The enhanced remote sensing data is added to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map. The spatial attention feature map is added to the densely connected monocular depth estimation network to generate depth information. It is fused with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud. The initial three-dimensional mesh model is generated by the Poisson reconstruction algorithm. The channel attention feature map is added to the residual graph convolutional neural network for texture feature migration. The topology optimization is performed by combining the multi-view projection consistency constraint and the mesh optimization algorithm to obtain an optimized three-dimensional scene model. Calculate and optimize the curvature change value and normal vector change value of each mesh in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the mesh vertices in each space block and construct an edge collapse cost function to merge and simplify the mesh vertices to obtain four accuracy level models, construct a quadtree viewing cone divided into three depth areas, load the corresponding accuracy level model to obtain a visualized scene, extract local and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive position information and calculate the viewpoint transformation matrix to update the visualized scene for scene roaming.

[0005] In an optional embodiment, Multi-source remote sensing data corresponding to the target news scene is collected through remote sensing satellites. Hierarchical feature descriptors are extracted based on the iterative closest point algorithm and tensor projection algorithm based on spatial position weighting and bidirectional attention deep neural network. The feature point pairs with projection errors greater than the preset projection threshold are screened through a random sampling algorithm based on local geometric consistency constraints, and the feature points are unified to the global geographic coordinates to obtain the registered multi-source remote sensing data. The enhanced remote sensing data includes: Collect multi-source remote sensing data corresponding to the target news scene through remote sensing satellites, wherein the multi-source remote sensing data includes optical remote sensing image data, dual-polarization radar remote sensing data and multi-echo laser point cloud data; Add the multi-source remote sensing data to the iterative nearest point algorithm based on spatial position weighting, construct a spatial position weight matrix, construct a feature distance function according to the distance from the data point to the center of the scene, calculate the weight value, perform a nearest neighbor search on each data point through an octree spatial index structure to establish a nearest neighbor point set, calculate the position deviation vector between the data point and the nearest neighbor point set, multiply the position deviation vector by the spatial position weight matrix to obtain a weighted position deviation, repeat the iteration until the weighted position deviation is less than a preset threshold, and output the optimized data point set; Add the multi-source remote sensing data to the tensor projection algorithm, construct a characteristic covariance matrix through principal component analysis, determine the optimal projection direction by calculating eigenvalues ​​and eigenvectors, perform a tensor projection operation in the optimal projection direction, construct a local curvature calculation function and a normal vector change function for the projection results, and extract multimodal data feature points through curvature threshold segmentation and region growing algorithm; Adding the multimodal data feature points to a bidirectional attention deep neural network, the bidirectional attention deep neural network extracts multi-scale features through multi-layer convolution and pooling operations of an encoder, the multi-layer convolution includes a point convolution layer, an edge convolution layer and a surface convolution layer, and restores the features through deconvolution and upsampling operations of a decoder, and a bidirectional attention module is set between the encoder and the decoder, wherein the bidirectional attention module includes a self-attention mechanism for calculating the internal spatial correlation of feature points and a cross-attention mechanism for calculating the topological correlation between different feature points, and the feature mapping results of the self-attention mechanism and the cross-attention mechanism are integrated to obtain a hierarchical feature descriptor; The hierarchical feature descriptors are screened by a random sampling algorithm based on local geometric consistency constraints, candidate feature point pairs are determined by feature similarity calculation, feature point pairs are randomly selected as seed matching, initial geometric transformation relationships including rotation matrices and translation vectors are calculated based on the seed matching, local geometric consistency constraint checks are performed on the remaining feature point pairs to obtain projection errors, feature point pairs whose projection errors are greater than a preset projection threshold are eliminated, the random sampling process is repeated multiple times to select the geometric transformation relationship with the most internal point support, the multi-source remote sensing data are unified to global geographic coordinates based on the selected geometric transformation relationship, and registered multi-source remote sensing data are generated; Inputting the registered multi-source remote sensing data into a variational autoencoder network, the encoder of the variational autoencoder network maps the registered multi-source remote sensing data to a latent feature space through multiple layers of residual convolution, the decoder reconstructs the latent features into the original data form through deconvolution and jump connection, introduces random noise disturbance into the latent feature space, and constructs a noise-aware loss function to constrain the reconstruction result; A generative adversarial network based on pixel-level adversarial discrimination is constructed. The generative adversarial network adopts a densely connected fully convolutional structure to output a pixel-level authenticity score map. The decoder of the variational autoencoder network is used as a generator, and an adversarial loss function is constructed. The parameters of the generator and the generative adversarial network are optimized through a minimum-maximum game until a Nash equilibrium is reached to obtain enhanced remote sensing data.

[0006] In an optional embodiment, The hierarchical feature descriptor is screened by a random sampling algorithm based on local geometric consistency constraints, candidate feature point pairs are determined by feature similarity calculation, feature point pairs are randomly selected as seed matching, initial geometric transformation relationships including rotation matrices and translation vectors are calculated based on the seed matching, local geometric consistency constraint checks are performed on the remaining feature point pairs to obtain projection errors, feature point pairs whose projection errors are greater than a preset projection threshold are eliminated, the random sampling process is repeated multiple times to select the geometric transformation relationship with the most internal point support, and the multi-source remote sensing data is unified to global geographic coordinates based on the selected geometric transformation relationship, and the generation of registered multi-source remote sensing data includes: Constructing a local geometric feature extraction unit and a global semantic feature extraction unit, wherein the local geometric feature extraction unit extracts a local shape descriptor by constructing a feature point neighborhood and analyzing the distribution characteristics of points in the neighborhood, and the global semantic feature extraction unit extracts hierarchical semantic information from bottom to top through a multi-scale feature pyramid, and cascades the local shape descriptor and the hierarchical semantic information to form a feature vector; Constructing a feature similarity calculation module, which is based on the cosine similarity measurement method, introduces a spatial distance weighting term so that feature points with similar distances have a higher matching priority, and pairs feature descriptors with similarities higher than a preset threshold to form a set of candidate feature point pairs; Randomly select a feature point pair from the candidate feature point pair set as a seed match, build a geometric transformation estimation module based on the seed match, establish a mapping relationship between corresponding points through the geometric transformation estimation module to build a three-dimensional rigid body transformation model, combine an iterative optimization algorithm to decompose the rotation matrix in the transformation process into three basic rotation matrices and solve them, and solve the translation vector by minimizing the Euclidean distance between corresponding points; Constructing a geometric consistency verification module, the geometric consistency verification module includes a feature transformation unit and an error calculation unit, the feature transformation unit transforms the source feature point to the target coordinate system through the three-dimensional rigid body transformation model, the error calculation unit calculates the projection error between the transformed position and the actual target position, dynamically adjusts the projection error threshold according to the data distribution characteristics, and eliminates feature point pairs with projection errors greater than the projection error threshold; Constructing a random sampling iterative optimization module, wherein the random sampling iterative optimization module randomly selects different seeds to match and execute the geometric transformation estimation module and the geometric consistency verification module, counts the number of interior points that meet the constraint conditions, comprehensively considers the number of interior points and the stability of the geometric transformation to perform scoring, and selects the geometric transformation relationship with the highest score as the optimal geometric transformation relationship; A coordinate conversion module is constructed based on the optimal geometric transformation relationship. Based on the coordinate conversion module, the optical remote sensing image data is used as reference data. An affine transformation model considering imaging geometric characteristics is constructed for the dual-polarization radar remote sensing data, and transformation parameters are solved by a polynomial fitting method to achieve geometric correction. A three-dimensional rigid body transformation model is constructed for the multi-echo laser point cloud data, and transformation parameters are solved by an iterative optimization method. The optical remote sensing image data, the dual-polarization radar remote sensing data and the multi-echo laser point cloud data are uniformly converted to a global geographic coordinate system.

[0007] In an optional embodiment, The enhanced remote sensing data is added to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map. The spatial attention feature map is added to the densely connected monocular depth estimation network to generate depth information. It is fused with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud. The initial three-dimensional mesh model is generated by the Poisson reconstruction algorithm. The channel attention feature map is added to the residual graph convolutional neural network for texture feature migration. The optimized three-dimensional scene model is obtained by combining the multi-view projection consistency constraint and the mesh optimization algorithm for topology optimization. The following includes: Input the enhanced remote sensing data into the feature extraction network, extract multi-scale spatial features through a multi-layer cascade convolution structure and a batch normalization layer, use an adaptive weight calculation unit to assign spatial attention weights to each position, obtain channel descriptors through global pooling for information aggregation, perform multi-layer transformation on the channel descriptors to learn the inter-channel dependency relationship to generate channel attention weights, and adaptively fuse the spatial attention weights and the channel attention weights to generate a spatial attention feature map and a channel attention feature map; The spatial attention feature map is input into a densely connected monocular depth estimation network, a multi-layer cascade connection is established in the monocular depth estimation network by a jump method, a direct mapping relationship is established between shallow features and deep features, feature resolution is gradually restored through a multi-layer deconvolution structure, and a multi-scale feature pyramid structure is used to perform weight adaptive fusion of features at different levels to generate depth information; The depth information is sampled and a mapping relationship between pixel coordinates and three-dimensional space points is established. The confidence weight is calculated for the pre-acquired dual-polarization radar remote sensing data and multi-echo laser point cloud data based on the signal-to-noise ratio and spatial resolution. An adaptive weighted fusion strategy is constructed according to the spatial distance and feature similarity. The point clouds in the overlapping areas are fused to generate a sparse three-dimensional point cloud. The sparse three-dimensional point cloud is input into the Poisson reconstruction algorithm, the normal vector of each point is calculated by the feature decomposition method, the spatial subdivision granularity is dynamically adjusted according to the point cloud density distribution, the implicit surface expression is obtained by iteratively solving the Poisson equation, the grid structure is extracted by combining the adaptive threshold and Laplace smoothing is performed to generate the initial three-dimensional grid model, the channel attention feature map is input into the residual graph convolutional neural network, the node information is transferred by local neighborhood aggregation, the input feature and the transformed feature are added to construct the residual connection, the texture feature is adaptively captured by the graph attention mechanism, and the texture feature is mapped to the surface of the initial three-dimensional grid model by projection transformation to perform texture feature migration; Multiple observation perspectives are uniformly sampled in the spherical space, and the initial three-dimensional mesh model is projected onto each observation perspective plane to generate a multi-perspective projection image. The similarity between the multi-perspective projection image and the original image is calculated by feature descriptor matching to establish a multi-perspective projection consistency constraint. Based on the multi-perspective projection consistency constraint, the mesh vertex positions and connection relationships are optimized while maintaining the model geometric details to generate an optimized three-dimensional scene model.

[0008] In an optional embodiment, The sparse three-dimensional point cloud is input into the Poisson reconstruction algorithm, the normal vector of each point is calculated by the feature decomposition method, the spatial subdivision granularity is dynamically adjusted according to the point cloud density distribution, the implicit surface expression is obtained by iteratively solving the Poisson equation, the grid structure is extracted by combining the adaptive threshold and Laplace smoothing is performed to generate the initial three-dimensional grid model, the channel attention feature map is input into the residual graph convolutional neural network, the node information is transferred by local neighborhood aggregation, the input feature and the transformation feature are added to construct the residual connection, the texture feature is adaptively captured by the graph attention mechanism, and the texture feature is mapped to the surface of the initial three-dimensional grid model by projection transformation to perform texture feature migration, including: Input the sparse three-dimensional point cloud into the Poisson reconstruction algorithm, construct a point cloud neighbor search tree structure, use a spherical neighborhood search method to determine the local neighborhood point set corresponding to each point in the sparse three-dimensional point cloud, construct a covariance matrix based on the local neighborhood point set, calculate the normal vector of each point by the eigendecomposition method, construct a minimum spanning tree to establish the connection relationship between points, select the normal vector at the center of the point cloud as a reference direction, recursively traverse the minimum spanning tree to adjust the normal vector directions of all points to obtain a normal vector set; Calculate the outer bounding box of the point cloud as the initial spatial range, dynamically adjust the spatial subdivision granularity according to the point cloud density distribution, count the number of point clouds contained in each octree node and calculate the point density distribution, set an adaptive subdivision threshold based on the point density, divide the node with a point density greater than the adaptive subdivision threshold into eight child nodes and record the depth information and point set contained in each leaf node, merge the divided leaf nodes, and merge adjacent nodes with similar point density to obtain an optimized octree structure; A three-dimensional scalar field function is defined on the optimized octree structure, the normal vector set is converted into a gradient field constraint, an implicit surface expression is obtained by iteratively solving the Poisson equation, a solution hierarchy is constructed by adopting a multi-grid strategy, the Poisson equation is converted into a linear equation system by constructing a sparse matrix to represent the Laplace operator, an iterative optimization method is adopted to solve the linear equation system and the scalar values ​​of the grid nodes are updated to obtain a three-dimensional scalar field; Extracting the mesh structure in combination with the adaptive threshold, extracting the isosurface in the optimized octree structure based on the marching cube algorithm to generate the initial mesh, performing a topological check on the initial mesh and repairing the non-manifold edges and singular points, performing Laplace smoothing iteration to optimize the vertex positions, and simplifying the mesh based on the edge collapse criterion to obtain the initial three-dimensional mesh model; Input the channel attention feature map into the residual graph convolutional neural network, transfer node information through local neighborhood aggregation, define an adjacency matrix to describe the connection relationship between nodes, perform message aggregation based on the adjacency matrix, perform weighted combination of node features and neighborhood features, add the input features and the transformed features to construct a residual connection to obtain enhanced features; The texture features are adaptively captured through the graph attention mechanism, the feature statistics are obtained through the global pooling operation and nonlinear transformation is performed on the enhanced features, the correlation weights between different channels are learned to generate channel attention weights, the importance scores of different positions of the feature map are calculated to generate spatial attention weights, and the channel attention weights and the spatial attention weights are applied to the feature map to obtain the texture features; The texture features are mapped to the surface of the initial three-dimensional mesh model by projection transformation to perform texture feature migration, a mapping relationship between the vertices of the initial three-dimensional mesh model and the image pixels is established, the vertices are projected to the image plane according to the camera projection matrix, the texture coordinates are calculated by using the barycentric coordinate interpolation inside the mesh face, and a multi-view image is constructed. Figure 1 The consistency constraint is used to evaluate the texture mapping quality under different viewing angles, the optimal texture source is selected based on the graph cut optimization algorithm, and the texture refinement is dynamically adjusted according to the observation distance to obtain a textured 3D scene model.

[0009] In an optional embodiment, Calculate and optimize the curvature change value and normal vector change value of each grid in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the grid vertices in each space block and construct an edge collapse cost function to merge and simplify the grid vertices to obtain four precision level models, construct a quadtree view cone divided into three depth areas to load the corresponding precision level model to obtain a visualization scene, extract local features and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive location information and calculate the viewpoint transformation matrix to update the visualization scene for scene roaming and perform scene roaming, including: For each mesh face in the optimized three-dimensional scene model, project the normal vector of each mesh face to the local coordinate system of the vertex and calculate the weighted average to obtain the vertex normal vector, construct the local surface fitting equation of the vertex to calculate the principal curvature to obtain the curvature change value, traverse the adjacent vertices to calculate the normal vector angle change rate, combine the curvature change value with the normal vector angle change rate to form a vertex feature description, and establish an adjacency relationship graph that records the vertex feature description; Input the vertex feature description, vertex density distribution in the adjacency relationship graph, and feature point distribution ratio into the space partition evaluation function, recursively construct an octree with the scene bounding box as the initial space, calculate the evaluation function value of each node to be segmented, dynamically adjust the segmentation threshold according to the node level and the parent node evaluation value, and record the spatial range of each leaf node, the vertex feature description, and the mesh topology relationship to form a spatial index structure; For each leaf node in the spatial index structure, the sum of the squares of the distances from the vertex to the adjacent facets is used to construct a quadratic error matrix, the quadratic error matrix, the normal vector change rate, and the texture coordinate change are combined to construct an edge collapse cost function, the mesh edges in the node are traversed to calculate the optimal collapse position and stored in a priority queue, the edge collapse operation is iteratively performed according to the value of the collapse cost function, and the error matrix of the affected vertex and the cost value of the related edge are updated; Controlling the number of collapse iterations based on the edge collapse cost function, maintaining the continuity of the vertex feature description, generating four precision level models that retain 90%, 70%, 50%, and 30% of the original vertex number, constructing a quadtree structure corresponding to the viewing frustum, dividing the viewing frustum into a near view area, a mid-view area, and a far view area, setting an overlapping transition zone between adjacent areas, loading the two models with the highest precision among the four precision level models into the near view area, loading the model with the third highest precision into the mid-view area, and loading the model with the lowest precision into the far view area; Calculating vertex position interpolation based on the vertex feature description within the overlapping transition zone, performing smooth switching between models of different precisions to obtain a visualization scene, inputting the visualization scene into a dual-branch semantic segmentation network, extracting local features through multi-layer convolution and downsampling, and extracting global features through multi-level pooling; The local features are fused with the global features through a skip connection, the fused features are deconvolved to restore the spatial resolution to generate a semantic segmentation map, the semantic segmentation map is parsed to obtain the distribution of different category areas in the scene, the input news event description is associated with the corresponding category area, and the marked position is converted to the scene space coordinate system through line of sight projection; Obtain the position coordinates and operation instructions of the interactive device, convert the screen space coordinates to the normalized device coordinate system, construct the intersection position of the interactive ray calculation and the visualization scene, calculate the rotation matrix, translation vector and scaling factor of the viewpoint based on the intersection position, generate the viewpoint transformation path, and dynamically update the accuracy configuration of the model within the visible range and the display status of the annotation symbols according to the viewpoint position.

[0010] In an optional embodiment, For each leaf node in the spatial index structure, the sum of the squares of the distances from the vertex to the adjacent facets is used to construct a quadratic error matrix, the quadratic error matrix, the normal vector change rate, and the texture coordinate change amount are combined to construct an edge collapse cost function, the mesh edges in the node are traversed to calculate the optimal collapse position and stored in a priority queue, the edge collapse operation is iteratively performed according to the value of the collapse cost function, and the error matrix of the affected vertex and the cost value of the related edge are updated, including: Obtain the mesh information of each leaf node in the spatial index structure, establish an association table between vertices and adjacent facets, obtain the facet set and topological relationship to which each vertex belongs, calculate the facet normal vector and plane equation based on the facet set, calculate the sum of the squares of the distances from the vertex to the adjacent facets to construct a quadratic error matrix, wherein the quadratic error matrix represents the degree of influence on the adjacent facets when the vertex position changes; The quadratic error matrix, the normal vector change rate, and the texture coordinate change amount are combined to construct an edge collapse cost function, the shape deformation value is calculated based on the quadratic error matrix of the vertices at both ends of the edge, the normal vectors at both ends of the edge and the adjacent vertices are obtained to calculate the normal vector change rate, the vertex texture coordinates are analyzed to obtain the coordinate change amount, the shape deformation value, the normal vector change rate, and the texture coordinate change amount are normalized and then linearly combined to obtain the edge collapse cost function value; Traversing the mesh edges in the node to calculate the optimal collapse position, obtaining the facets adjacent to the edge to construct a local mesh area, selecting multiple candidate collapse points on the straight line where the edge is located, calculating the mesh error corresponding to each candidate point, selecting the position with the smallest mesh error and maintaining topological validity as the optimal collapse position, and storing the edge information and the optimal collapse position in a priority queue sorted by the cost function value; Extracting the edge with the smallest cost function value from the priority queue, obtaining the optimal collapse position corresponding to the current edge after verifying the validity of the edge, merging the two endpoints of the edge into a new vertex, updating the coordinates, normal vector and texture coordinates of the new vertex according to the optimal collapse position, updating the vertex index of the relevant facet, deleting the degenerate facet and adjusting the mesh connection relationship; Update the error matrix of the affected vertex, integrate the geometric relationship between the new vertex and the adjacent facets into the error matrix, recalculate the cost function value and the optimal collapse position for the edge connected to the new vertex, and store the updated edge information back into the priority queue to maintain the optimal simplification order.

[0011] A second aspect of an embodiment of the present invention provides a news scene 3D reconstruction and visualization system based on multi-source remote sensing data, comprising: The first unit is used to collect multi-source remote sensing data corresponding to the target news scene through remote sensing satellites, extract hierarchical feature descriptors based on the iterative closest point algorithm and tensor projection algorithm weighted by spatial position, and bidirectional attention deep neural network, screen them through the random sampling algorithm based on local geometric consistency constraints, remove feature point pairs with projection errors greater than the preset projection threshold, and unify them to global geographic coordinates to obtain registered multi-source remote sensing data, and enhance remote sensing data through the variational autoencoder network and noise-aware loss function combined with the generative adversarial network based on pixel-level adversarial discrimination; The second unit is used to add the enhanced remote sensing data to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map, add the spatial attention feature map to the densely connected monocular depth estimation network to generate depth information, fuse it with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud, generate an initial three-dimensional mesh model through the Poisson reconstruction algorithm, add the channel attention feature map to the residual graph convolutional neural network for texture feature migration, and combine the multi-view projection consistency constraint and the mesh optimization algorithm for topology optimization to obtain an optimized three-dimensional scene model; The third unit is used to calculate and optimize the curvature change value and normal vector change value of each mesh in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the mesh vertices in each space block and construct an edge collapse cost function to merge and simplify the mesh vertices to obtain four accuracy level models, construct a quadtree viewing cone divided into three depth areas, load the corresponding accuracy level model to obtain a visualized scene, extract local and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive position information and calculate the viewpoint transformation matrix to update the visualized scene for scene roaming and perform scene roaming.

[0012] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0013] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0014] In the present invention, multi-source remote sensing data (including satellite images, radar and laser point clouds) are used, and deep learning technology is combined for feature extraction, data enhancement and depth estimation, so that a more refined and realistic three-dimensional model of news scenes can be generated. Texture feature migration and grid optimization algorithms further enhance the visual effect of the model. Octree space division and quadtree frustum technology are used to dynamically load models of different precisions according to the viewpoint distance, thereby achieving efficient scene rendering and roaming, avoiding the computational burden brought by loading overly high-precision models, and annotating scenes through semantic segmentation technology, combined with interactive operations, which can highlight the location of news events, facilitate users to understand the scenes, and provide richer news information. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of a method for three-dimensional reconstruction and visualization of news scenes based on multi-source remote sensing data according to an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a news scene three-dimensional reconstruction and visualization system based on multi-source remote sensing data according to an embodiment of the present invention; Figure 3 A comparison diagram of simulation effects of three-dimensional model reconstruction of a news scene three-dimensional reconstruction and visualization method based on multi-source remote sensing data according to an embodiment of the present invention; Figure 4 This is a comparison diagram of the texture model simplification effect of the news scene three-dimensional reconstruction and visualization method based on multi-source remote sensing data in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0018] Figure 1 FIG. 1 is a flow chart of a method for three-dimensional reconstruction and visualization of news scenes based on multi-source remote sensing data according to an embodiment of the present invention. Figure 1 As shown, the method includes: S1. Collect multi-source remote sensing data corresponding to the target news scene through remote sensing satellites, extract hierarchical feature descriptors based on spatial position weighted iterative closest point algorithm and tensor projection algorithm, and bidirectional attention deep neural network, screen them through random sampling algorithm based on local geometric consistency constraints, remove feature point pairs with projection errors greater than the preset projection threshold, and unify them to global geographic coordinates to obtain registered multi-source remote sensing data, and enhance remote sensing data through variational autoencoder network and noise-aware loss function combined with generative adversarial network based on pixel-level adversarial discrimination; The iterative closest point algorithm based on spatial position weighting is an improved point cloud registration method, which assigns different weights to points in different areas to improve the registration accuracy. It is particularly suitable for point cloud data with non-uniform sampling or noise. The tensor projection algorithm is a high-dimensional data dimensionality reduction method. It extracts the main information by projecting high-order tensor data into a low-dimensional subspace. It can improve the computational efficiency while maintaining the data structure information. It is widely used in fields such as image processing and pattern recognition. The hierarchical feature descriptor is a multi-level feature expression method. It extracts the key features of the data at different scales and levels, so that it can better adapt to different resolutions. The random sampling algorithm based on local geometric consistency constraints is an optimized feature matching method. In the random sampling process, local geometric consistency constraints are combined to exclude erroneous matching point pairs and improve the accuracy of the algorithm. It is often used in 3D point cloud registration and feature matching tasks. The noise-aware loss function is a loss function designed for data noise robustness. By adaptively adjusting the penalty degree for noise points, the impact of noise on model training is reduced, and the stability and generalization ability of the model are improved. It is widely used in image denoising, target detection and deep learning training.

[0019] In an optional embodiment, Multi-source remote sensing data corresponding to the target news scene is collected through remote sensing satellites. Hierarchical feature descriptors are extracted based on the iterative closest point algorithm and tensor projection algorithm based on spatial position weighting and bidirectional attention deep neural network. The feature point pairs with projection errors greater than the preset projection threshold are screened through a random sampling algorithm based on local geometric consistency constraints, and the feature points are unified to the global geographic coordinates to obtain the registered multi-source remote sensing data. The enhanced remote sensing data includes: Collect multi-source remote sensing data corresponding to the target news scene through remote sensing satellites, wherein the multi-source remote sensing data includes optical remote sensing image data, dual-polarization radar remote sensing data and multi-echo laser point cloud data; Add the multi-source remote sensing data to the iterative nearest point algorithm based on spatial position weighting, construct a spatial position weight matrix, construct a feature distance function according to the distance from the data point to the center of the scene, calculate the weight value, perform a nearest neighbor search on each data point through an octree spatial index structure to establish a nearest neighbor point set, calculate the position deviation vector between the data point and the nearest neighbor point set, multiply the position deviation vector by the spatial position weight matrix to obtain a weighted position deviation, repeat the iteration until the weighted position deviation is less than a preset threshold, and output the optimized data point set; Add the multi-source remote sensing data to the tensor projection algorithm, construct a characteristic covariance matrix through principal component analysis, determine the optimal projection direction by calculating eigenvalues ​​and eigenvectors, perform a tensor projection operation in the optimal projection direction, construct a local curvature calculation function and a normal vector change function for the projection results, and extract multimodal data feature points through curvature threshold segmentation and region growing algorithm; Adding the multimodal data feature points to a bidirectional attention deep neural network, the bidirectional attention deep neural network extracts multi-scale features through multi-layer convolution and pooling operations of an encoder, the multi-layer convolution includes a point convolution layer, an edge convolution layer and a surface convolution layer, and restores the features through deconvolution and upsampling operations of a decoder, and a bidirectional attention module is set between the encoder and the decoder, wherein the bidirectional attention module includes a self-attention mechanism for calculating the internal spatial correlation of feature points and a cross-attention mechanism for calculating the topological correlation between different feature points, and the feature mapping results of the self-attention mechanism and the cross-attention mechanism are integrated to obtain a hierarchical feature descriptor; The hierarchical feature descriptors are screened by a random sampling algorithm based on local geometric consistency constraints, candidate feature point pairs are determined by feature similarity calculation, feature point pairs are randomly selected as seed matching, initial geometric transformation relationships including rotation matrices and translation vectors are calculated based on the seed matching, local geometric consistency constraint checks are performed on the remaining feature point pairs to obtain projection errors, feature point pairs whose projection errors are greater than a preset projection threshold are eliminated, the random sampling process is repeated multiple times to select the geometric transformation relationship with the most internal point support, the multi-source remote sensing data are unified to global geographic coordinates based on the selected geometric transformation relationship, and registered multi-source remote sensing data are generated; Inputting the registered multi-source remote sensing data into a variational autoencoder network, the encoder of the variational autoencoder network maps the registered multi-source remote sensing data to a latent feature space through multiple layers of residual convolution, the decoder reconstructs the latent features into the original data form through deconvolution and jump connection, introduces random noise disturbance into the latent feature space, and constructs a noise-aware loss function to constrain the reconstruction result; A generative adversarial network based on pixel-level adversarial discrimination is constructed. The generative adversarial network adopts a densely connected fully convolutional structure to output a pixel-level authenticity score map. The decoder of the variational autoencoder network is used as a generator, and an adversarial loss function is constructed. The parameters of the generator and the generative adversarial network are optimized through a minimum-maximum game until a Nash equilibrium is reached to obtain enhanced remote sensing data.

[0020] The octree spatial index structure is a hierarchical data structure for three-dimensional spatial data management. It recursively divides the three-dimensional space into eight sub-areas, which helps to efficiently store, index and query large-scale three-dimensional point cloud data. It is often used in computer vision, geographic information systems and robot navigation. The curvature threshold segmentation is an image or point cloud segmentation method based on geometric characteristics. By calculating curvature information and setting thresholds, regions with different curvature characteristics are distinguished. It is suitable for tasks such as object surface analysis, medical image processing and point cloud segmentation. The region growing algorithm is a segmentation method based on pixel or point neighborhood relationships. Select seed points and expand them to adjacent areas that meet specific similarity standards to form connected areas. It is widely used in image segmentation, point cloud clustering and medical image analysis. The seed matching is a feature matching strategy that selects a set of high-confidence matching points (seed points) in the data and then uses these matching points to guide the matching process of the entire data set to improve the accuracy and stability of the matching. It is often used in three-dimensional reconstruction and target tracking. The inliers refer to data points that are considered to conform to the transformation model and contribute to the final matching result during the feature matching or point cloud registration process. Compared with mismatched outliers, inliers usually have higher matching confidence.

[0021] Through remote sensing satellites, multi-source remote sensing data corresponding to the target news scene is obtained, including optical remote sensing image data, dual-polarization radar remote sensing data, and multi-echo laser point cloud data. For example, surface optical images, X-band dual-polarization radar data, and multi-echo laser radar point cloud data of a certain area can be obtained.

[0022] The acquired multi-source remote sensing data is input into the iterative nearest point algorithm based on spatial position weighting for preliminary registration. The spatial position weight matrix is ​​constructed, and the weight value is calculated based on the distance from the data point to the center of the scene. The closer the distance, the greater the weight. The octree spatial index structure is used to perform a nearest neighbor search for each data point and establish a nearest neighbor point set. The position deviation vector between the data point and the nearest neighbor point set is calculated, and the vector is multiplied by the spatial position weight matrix to obtain the weighted position deviation. The weighted position deviation is iteratively calculated until it is less than the preset threshold (for example, 0.5 pixels), and the optimized data point set is output.

[0023] The multi-source remote sensing data is input into the tensor projection algorithm for feature extraction. The feature covariance matrix is ​​constructed through principal component analysis, and the eigenvalues ​​and eigenvectors are calculated to determine the optimal projection direction. The tensor projection operation is performed in the optimal projection direction. The local curvature calculation function and the normal vector change function are constructed for the projection results. The feature points of multimodal data are extracted by setting the curvature threshold (for example, 0.01) and the regional growing algorithm.

[0024] The extracted multimodal data feature points are input into a bidirectional attention deep neural network. The encoder of the network extracts multi-scale features through multi-layer convolution and pooling operations (such as point convolution, edge convolution, and surface convolution). The decoder restores the features through deconvolution and upsampling operations. A bidirectional attention module is set between the encoder and the decoder, which includes a self-attention mechanism and a cross-attention mechanism. The self-attention mechanism calculates the spatial correlation within the feature points, and the cross-attention mechanism calculates the topological correlation between different feature points. The feature mapping results of the two attention mechanisms are fused to obtain a hierarchical feature descriptor.

[0025] The hierarchical feature descriptors are screened using a random sampling algorithm based on local geometric consistency constraints. First, candidate feature point pairs are determined by feature similarity calculation. Then, feature point pairs are randomly selected as seed matches, and the initial geometric transformation relationship including the rotation matrix and translation vector is calculated based on the seed match. The local geometric consistency constraint check is performed on the remaining feature point pairs to obtain the projection error. Feature point pairs with projection errors greater than a preset threshold (e.g., 1 pixel) are eliminated. The random sampling process is repeated multiple times to select the geometric transformation relationship with the most inliers. Based on the selected geometric transformation relationship, the multi-source remote sensing data is unified to the global geographic coordinates to generate registered multi-source remote sensing data.

[0026] The registered multi-source remote sensing data is input into the variational autoencoder network for data enhancement. The encoder of the variational autoencoder maps the data to the latent feature space through multiple layers of residual convolution, and the decoder reconstructs the latent features into the original data form through deconvolution and jump connections. Random noise perturbation is introduced into the latent feature space, and a noise-aware loss function is constructed to constrain the reconstruction result.

[0027] A generative adversarial network based on pixel-level adversarial discrimination is constructed, and a densely connected fully convolutional structure is used to output a pixel-level authenticity score map. The decoder of the variational autoencoder is used as the generator to construct an adversarial loss function. The parameters of the generator and the discriminator are optimized through a minimum-maximum game until a Nash equilibrium is reached to obtain enhanced remote sensing data.

[0028] In this embodiment, by fusing multi-source remote sensing data and deep learning technology, feature points can be extracted and matching relationships can be established more accurately, thereby improving the accuracy of remote sensing data registration. Through variational autoencoders and generative adversarial networks, noise can be effectively removed, missing information can be filled, and details can be enhanced, thereby improving the quality of remote sensing data. By fusing multi-source remote sensing data and deep learning technology, scene information can be acquired more comprehensively, thereby improving the ability to understand the scene.

[0029] In an optional embodiment, The hierarchical feature descriptor is screened by a random sampling algorithm based on local geometric consistency constraints, candidate feature point pairs are determined by feature similarity calculation, feature point pairs are randomly selected as seed matching, initial geometric transformation relationships including rotation matrices and translation vectors are calculated based on the seed matching, local geometric consistency constraint checks are performed on the remaining feature point pairs to obtain projection errors, feature point pairs whose projection errors are greater than a preset projection threshold are eliminated, the random sampling process is repeated multiple times to select the geometric transformation relationship with the most internal point support, and the multi-source remote sensing data is unified to global geographic coordinates based on the selected geometric transformation relationship, and the generation of registered multi-source remote sensing data includes: Constructing a local geometric feature extraction unit and a global semantic feature extraction unit, wherein the local geometric feature extraction unit extracts a local shape descriptor by constructing a feature point neighborhood and analyzing the distribution characteristics of points in the neighborhood, and the global semantic feature extraction unit extracts hierarchical semantic information from bottom to top through a multi-scale feature pyramid, and cascades the local shape descriptor and the hierarchical semantic information to form a feature vector; Constructing a feature similarity calculation module, which is based on the cosine similarity measurement method, introduces a spatial distance weighting term so that feature points with similar distances have a higher matching priority, and pairs feature descriptors with similarities higher than a preset threshold to form a set of candidate feature point pairs; Randomly select a feature point pair from the candidate feature point pair set as a seed match, build a geometric transformation estimation module based on the seed match, establish a mapping relationship between corresponding points through the geometric transformation estimation module to build a three-dimensional rigid body transformation model, combine an iterative optimization algorithm to decompose the rotation matrix in the transformation process into three basic rotation matrices and solve them, and solve the translation vector by minimizing the Euclidean distance between corresponding points; Constructing a geometric consistency verification module, the geometric consistency verification module includes a feature transformation unit and an error calculation unit, the feature transformation unit transforms the source feature point to the target coordinate system through the three-dimensional rigid body transformation model, the error calculation unit calculates the projection error between the transformed position and the actual target position, dynamically adjusts the projection error threshold according to the data distribution characteristics, and eliminates feature point pairs with projection errors greater than the projection error threshold; Constructing a random sampling iterative optimization module, wherein the random sampling iterative optimization module randomly selects different seeds to match and execute the geometric transformation estimation module and the geometric consistency verification module, counts the number of interior points that meet the constraint conditions, comprehensively considers the number of interior points and the stability of the geometric transformation to perform scoring, and selects the geometric transformation relationship with the highest score as the optimal geometric transformation relationship; A coordinate conversion module is constructed based on the optimal geometric transformation relationship. Based on the coordinate conversion module, the optical remote sensing image data is used as reference data. An affine transformation model considering imaging geometric characteristics is constructed for the dual-polarization radar remote sensing data, and transformation parameters are solved by a polynomial fitting method to achieve geometric correction. A three-dimensional rigid body transformation model is constructed for the multi-echo laser point cloud data, and transformation parameters are solved by an iterative optimization method. The optical remote sensing image data, the dual-polarization radar remote sensing data and the multi-echo laser point cloud data are uniformly converted to a global geographic coordinate system.

[0030] The three-dimensional rigid body transformation model is a mathematical model used to describe the rigid motion of three-dimensional objects, including translation and rotation transformations, but does not involve scaling or deformation. It is widely used in computer vision, robot navigation and medical imaging. The iterative optimization algorithm is a type of optimization method that continuously updates parameters and gradually approaches the optimal solution. It is often used to solve nonlinear problems. The affine transformation model that takes into account the geometric characteristics of imaging is a mathematical model used to represent spatial transformations in image processing and computer vision. It can describe geometric transformations such as rotation, scaling, translation and shearing, and is particularly suitable for remote sensing image correction and image registration tasks.

[0031] Construct a feature extraction unit, which includes a local geometric feature extraction part and a global semantic feature extraction part. The local geometric feature extraction part determines a feature point, selects a certain number of neighboring points around it, and analyzes the distribution of these neighboring points, such as the density, direction, and distance relationship between the points, so as to extract features that describe the local shape, such as divergence, curvature, and directional gradient histogram. The global semantic feature extraction part uses a multi-scale feature pyramid to gradually extract semantic information from the low-resolution layer of the image upward. Each layer extracts features at that scale, such as texture, color, edge, etc., and finally combines the features of each layer into a hierarchical semantic information descriptor. Finally, the local shape descriptor and the hierarchical semantic information descriptor are connected together to form a complete feature vector. For example, for a corner point of a building, the local geometric feature can describe the sharpness of the corner point, and the global semantic feature can describe the semantic category of the corner point belonging to the building.

[0032] Construct a feature similarity calculation module and use cosine similarity to measure the similarity between two feature vectors. In order to give feature points that are closer in space a higher matching priority, introduce a spatial distance weighting term to calculate the spatial distance between two feature points and convert it into a weight value. The closer the distance, the greater the weight. Then multiply the weight value by the cosine similarity to get the final similarity score. Pair feature descriptors with similarity scores higher than a preset threshold (e.g. 0.8) to form a set of candidate feature point pairs. For example, if the feature vectors of two building corners have a high similarity and their spatial distance is close, they will be paired into a candidate feature point pair.

[0033] A portion of feature point pairs are randomly selected from the candidate feature point pair set as seed matches. Based on these seed matches, a geometric transformation estimation module is constructed, a mapping relationship between corresponding points is established, and a three-dimensional rigid body transformation model is constructed, which includes two parts: rotation and translation. Through an iterative optimization algorithm, the rotation matrix is ​​decomposed into three basic rotation matrices, and the parameters of these three rotation matrices are solved respectively. At the same time, the translation vector is solved by minimizing the Euclidean distance between corresponding points. For example, assuming that three seed matching point pairs are selected, by minimizing the distance between the seed matching point pairs, a rotation matrix and a translation vector can be estimated to describe the geometric transformation relationship between the two data sets.

[0034] Construct a geometric consistency check module to transform the feature points in the source data into the coordinate system of the target data through the estimated three-dimensional rigid body transformation model. Calculate the projection error between the transformed position and the actual target position. Dynamically adjust the projection error threshold according to the data distribution characteristics, such as using statistical methods to calculate the distribution of errors and set the threshold based on the distribution. Eliminate feature point pairs whose projection errors are greater than the threshold. For example, if the distance between the transformed position of a feature point and the actual target position exceeds the set threshold, the point pair is considered to not meet the geometric consistency and is eliminated.

[0035] Construct a random sampling iterative optimization module and repeat the above random sampling, geometric transformation estimation and geometric consistency check process multiple times. Each iteration will select a different seed match and calculate the number of inliers that meet the constraints, that is, the number of feature point pairs with projection errors less than the threshold. The number of inliers and the stability of the geometric transformation are comprehensively considered for scoring, for example, the ratio of the number of inliers to the total number of feature points is used as the scoring criterion. The geometric transformation relationship with the highest score is selected as the optimal geometric transformation relationship.

[0036] Based on the obtained optimal geometric transformation relationship, a coordinate transformation module is constructed. Taking the optical remote sensing image data as the reference data, an affine transformation model considering the imaging geometric characteristics is constructed for the dual-polarization radar remote sensing data, and the transformation parameters are solved by the polynomial fitting method to achieve geometric correction. A three-dimensional rigid body transformation model is constructed for the multi-echo laser point cloud data, and the transformation parameters are solved by the iterative optimization method. Finally, the optical remote sensing image data, dual-polarization radar remote sensing data and multi-echo laser point cloud data are uniformly converted to the global geographic coordinate system.

[0037] In this embodiment, by combining local geometric features and global semantic features, and introducing spatial distance weighting and geometric consistency constraints, false matches can be effectively eliminated and the registration accuracy can be improved. Random sampling and iterative optimization strategies can effectively deal with problems such as noise and data missing, enhance the robustness of the algorithm, and automatically complete the registration of multi-source remote sensing data without human intervention, thereby improving the degree of automation.

[0038] S2. Add the enhanced remote sensing data to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map. Add the spatial attention feature map to the densely connected monocular depth estimation network to generate depth information. Fuse it with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud. Generate an initial three-dimensional mesh model through the Poisson reconstruction algorithm. Add the channel attention feature map to the residual graph convolutional neural network for texture feature migration. Combine the multi-view projection consistency constraint and the mesh optimization algorithm for topology optimization to obtain an optimized three-dimensional scene model. The densely connected monocular depth estimation network is a neural network that uses a dense connection strategy to improve the accuracy of monocular image depth estimation. By enhancing feature transfer and gradient flow, it alleviates the gradient vanishing problem of the deep network and improves the robustness of depth prediction and detail recovery capabilities. The Poisson reconstruction algorithm is a 3D reconstruction method based on the Poisson equation. It solves the Poisson equation by inputting the normal information of the point cloud to construct a smooth and coherent 3D surface. It is widely used in computer vision, reverse engineering, and medical image processing. The texture feature migration is a method for maintaining or enhancing texture information in image or 3D reconstruction tasks. The texture features are transferred between domains or scales, so that the target image or model is improved in terms of style consistency and detail richness. The multi-view projection consistency constraint is an optimization strategy for 3D reconstruction and depth estimation. By ensuring that the depth estimation or geometric features of projections from different viewpoints remain consistent, the accuracy and visual coherence of the reconstruction results are improved. It is suitable for multi-view stereo matching and structured light scanning. The mesh optimization algorithm is a class of methods for optimizing 3D mesh structures, including mesh simplification, topology optimization and adaptive resampling, to improve the quality of 3D models, reduce redundant data and optimize rendering efficiency, and is widely used in computer graphics and simulation.

[0039] In an optional embodiment, The enhanced remote sensing data is added to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map. The spatial attention feature map is added to the densely connected monocular depth estimation network to generate depth information. It is fused with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud. The initial three-dimensional mesh model is generated by the Poisson reconstruction algorithm. The channel attention feature map is added to the residual graph convolutional neural network for texture feature migration. The optimized three-dimensional scene model is obtained by combining the multi-view projection consistency constraint and the mesh optimization algorithm for topology optimization. The following includes: Input the enhanced remote sensing data into the feature extraction network, extract multi-scale spatial features through a multi-layer cascade convolution structure and a batch normalization layer, use an adaptive weight calculation unit to assign spatial attention weights to each position, obtain channel descriptors through global pooling for information aggregation, perform multi-layer transformation on the channel descriptors to learn the inter-channel dependency relationship to generate channel attention weights, and adaptively fuse the spatial attention weights and the channel attention weights to generate a spatial attention feature map and a channel attention feature map; The spatial attention feature map is input into a densely connected monocular depth estimation network, a multi-layer cascade connection is established in the monocular depth estimation network by a jump method, a direct mapping relationship is established between shallow features and deep features, feature resolution is gradually restored through a multi-layer deconvolution structure, and a multi-scale feature pyramid structure is used to perform weight adaptive fusion of features at different levels to generate depth information; The depth information is sampled and a mapping relationship between pixel coordinates and three-dimensional space points is established. The confidence weight is calculated for the pre-acquired dual-polarization radar remote sensing data and multi-echo laser point cloud data based on the signal-to-noise ratio and spatial resolution. An adaptive weighted fusion strategy is constructed according to the spatial distance and feature similarity. The point clouds in the overlapping areas are fused to generate a sparse three-dimensional point cloud. The sparse three-dimensional point cloud is input into the Poisson reconstruction algorithm, the normal vector of each point is calculated by the feature decomposition method, the spatial subdivision granularity is dynamically adjusted according to the point cloud density distribution, the implicit surface expression is obtained by iteratively solving the Poisson equation, the grid structure is extracted by combining the adaptive threshold and Laplace smoothing is performed to generate the initial three-dimensional grid model, the channel attention feature map is input into the residual graph convolutional neural network, the node information is transferred by local neighborhood aggregation, the input feature and the transformed feature are added to construct the residual connection, the texture feature is adaptively captured by the graph attention mechanism, and the texture feature is mapped to the surface of the initial three-dimensional grid model by projection transformation to perform texture feature migration; Multiple observation perspectives are uniformly sampled in the spherical space, and the initial three-dimensional mesh model is projected onto each observation perspective plane to generate a multi-perspective projection image. The similarity between the multi-perspective projection image and the original image is calculated by feature descriptor matching to establish a multi-perspective projection consistency constraint. Based on the multi-perspective projection consistency constraint, the mesh vertex positions and connection relationships are optimized while maintaining the model geometric details to generate an optimized three-dimensional scene model.

[0040] The multi-layer deconvolution structure is a deep learning architecture for image generation and feature reconstruction. It gradually restores low-resolution features to high resolution through multi-level deconvolution (or transposed convolution) layers, thereby generating more refined outputs, such as depth estimation, image super-resolution, and semantic segmentation. The point cloud density distribution refers to the distribution characteristics of point cloud data in three-dimensional space, which affects processes such as three-dimensional reconstruction, point cloud sampling and filtering. The performance of the point cloud processing algorithm can be optimized by estimating the local point cloud density. The implicit surface expression is a mathematical model for representing continuous three-dimensional surfaces. It defines isosurfaces in space through implicit functions and can be used for smooth surface reconstruction, shape modeling, and surface fitting in computer graphics. The Laplace smoothing is a geometric optimization method based on the Laplace operator. It iteratively adjusts the vertex positions to reduce noise and optimize surface smoothness. It is often used in point cloud denoising, mesh optimization, and shape reconstruction.

[0041] The enhanced remote sensing data (e.g., high-resolution satellite images after atmospheric correction, geometric correction, and image fusion, such as WorldView-4 image data, with a size of 1024x1024 pixels, including red, green, blue, and near-infrared bands) is input into the feature extraction network, and multi-scale spatial features are extracted using a multi-layer cascaded convolution structure and a batch normalization layer. For example, four convolution layers are used, with convolution kernel sizes of 7x7, 5x5, 3x3, and 3x3, respectively. Batch normalization is performed after each convolution layer, and an adaptive weight calculation unit is used to assign spatial attention weights to each position. For example, weights are assigned by calculating the similarity between the features of each position and the global average features, and information is aggregated by global pooling to obtain channel descriptors. Multi-layer transformations are performed on the channel descriptors (e.g., using two fully connected layers) to learn the dependencies between channels to generate channel attention weights, and the spatial attention weights and channel attention weights are adaptively fused (e.g., by weighted summation) to generate spatial attention feature maps and channel attention feature maps, both of which are 128x128 in size.

[0042] The spatial attention feature map is input into a densely connected monocular depth estimation network, and a multi-layer cascade connection is established by jumping to establish a direct mapping relationship between shallow features and deep features. For example, the output of the first convolutional layer is connected to the output of the third convolutional layer. The network gradually restores the feature resolution through a multi-layer deconvolution structure. For example, two deconvolution layers are used to restore the feature map size from 32x32 to 128x128. A multi-scale feature pyramid structure is used to adaptively fuse the weights of features at different levels to generate depth information. For example, the outputs of different deconvolution layers are weighted and summed to obtain the final depth information map with a size of 128x128.

[0043] Sample the depth information and establish a mapping relationship from pixel coordinates to three-dimensional spatial points. For example, convert pixel coordinates into three-dimensional coordinates based on the intrinsic and extrinsic matrix of the camera. Calculate confidence weights based on the signal-to-noise ratio and spatial resolution for pre-acquired dual-polarization radar remote sensing data (e.g., TerraSAR-X data, with the same coverage as satellite images) and multi-echo laser point cloud data (e.g., ALS data, with a point cloud density of 10 points / square meter). For example, the higher the signal-to-noise ratio and the higher the spatial resolution, the greater the weight. Construct an adaptive weighted fusion strategy based on spatial distance and feature similarity. For example, the closer the distance and the more similar the features, the greater the weight. Perform point cloud fusion on overlapping areas to generate a sparse three-dimensional point cloud. For example, weighted average the point cloud data from different data sources according to the weights to obtain a fused sparse three-dimensional point cloud containing 10,000 points.

[0044] Input the sparse 3D point cloud into the Poisson reconstruction algorithm. Calculate the normal vector of each point by the eigendecomposition method. For example, use the principal component analysis method to calculate the normal vector. Dynamically adjust the spatial subdivision granularity according to the point cloud density distribution. For example, the higher the density, the finer the subdivision granularity. Obtain the implicit surface expression by iteratively solving the Poisson equation. Combine the adaptive threshold to extract the mesh structure and perform Laplace smoothing to generate the initial 3D mesh model, which contains 5000 triangles.

[0045] The channel attention feature map is input into the residual graph convolutional neural network. Node information is transferred through local neighborhood aggregation. The input features are added to the transformed features to construct a residual connection. Texture features are adaptively captured through the graph attention mechanism. Texture features are mapped to the surface of the initial 3D mesh model using projection transformation for texture feature migration.

[0046] Uniformly sample multiple viewing angles in the spherical space (e.g., sample 8 viewing angles). Project the initial 3D mesh model onto each viewing angle plane to generate a multi-view projection image. Calculate the similarity between the multi-view projection image and the original image through feature descriptor matching (e.g., using SIFT feature descriptor) to establish a multi-view projection consistency constraint. Optimize the mesh vertex position and connection relationship based on the multi-view projection consistency constraint, while maintaining the model geometric details to generate an optimized 3D scene model.

[0047] In this embodiment, by fusing multi-source remote sensing data, the accuracy of three-dimensional scene modeling can be effectively improved, especially in complex scenes. Through the attention mechanism and residual graph convolutional neural network, the model's detail expression ability can be effectively enhanced, making the generated model more realistic. Through adaptive weight calculation and multi-scale feature fusion, the modeling efficiency can be effectively improved and the calculation time can be reduced.

[0048] In an optional embodiment, The sparse three-dimensional point cloud is input into the Poisson reconstruction algorithm, the normal vector of each point is calculated by the feature decomposition method, the spatial subdivision granularity is dynamically adjusted according to the point cloud density distribution, the implicit surface expression is obtained by iteratively solving the Poisson equation, the grid structure is extracted by combining the adaptive threshold and Laplace smoothing is performed to generate the initial three-dimensional grid model, the channel attention feature map is input into the residual graph convolutional neural network, the node information is transferred by local neighborhood aggregation, the input feature and the transformation feature are added to construct the residual connection, the texture feature is adaptively captured by the graph attention mechanism, and the texture feature is mapped to the surface of the initial three-dimensional grid model by projection transformation to perform texture feature migration, including: Input the sparse three-dimensional point cloud into the Poisson reconstruction algorithm, construct a point cloud neighbor search tree structure, use a spherical neighborhood search method to determine the local neighborhood point set corresponding to each point in the sparse three-dimensional point cloud, construct a covariance matrix based on the local neighborhood point set, calculate the normal vector of each point by the eigendecomposition method, construct a minimum spanning tree to establish the connection relationship between points, select the normal vector at the center of the point cloud as a reference direction, recursively traverse the minimum spanning tree to adjust the normal vector directions of all points to obtain a normal vector set; Calculate the outer bounding box of the point cloud as the initial spatial range, dynamically adjust the spatial subdivision granularity according to the point cloud density distribution, count the number of point clouds contained in each octree node and calculate the point density distribution, set an adaptive subdivision threshold based on the point density, divide the node with a point density greater than the adaptive subdivision threshold into eight child nodes and record the depth information and point set contained in each leaf node, merge the divided leaf nodes, and merge adjacent nodes with similar point density to obtain an optimized octree structure; A three-dimensional scalar field function is defined on the optimized octree structure, the normal vector set is converted into a gradient field constraint, an implicit surface expression is obtained by iteratively solving the Poisson equation, a solution hierarchy is constructed by adopting a multi-grid strategy, the Poisson equation is converted into a linear equation system by constructing a sparse matrix to represent the Laplace operator, an iterative optimization method is adopted to solve the linear equation system and the scalar values ​​of the grid nodes are updated to obtain a three-dimensional scalar field; Extracting the mesh structure in combination with the adaptive threshold, extracting the isosurface in the optimized octree structure based on the marching cube algorithm to generate the initial mesh, performing a topological check on the initial mesh and repairing the non-manifold edges and singular points, performing Laplace smoothing iteration to optimize the vertex positions, and simplifying the mesh based on the edge collapse criterion to obtain the initial three-dimensional mesh model; Input the channel attention feature map into the residual graph convolutional neural network, transfer node information through local neighborhood aggregation, define an adjacency matrix to describe the connection relationship between nodes, perform message aggregation based on the adjacency matrix, perform weighted combination of node features and neighborhood features, add the input features and the transformed features to construct a residual connection to obtain enhanced features; The texture features are adaptively captured through the graph attention mechanism, the feature statistics are obtained through the global pooling operation and nonlinear transformation is performed on the enhanced features, the correlation weights between different channels are learned to generate channel attention weights, the importance scores of different positions of the feature map are calculated to generate spatial attention weights, and the channel attention weights and the spatial attention weights are applied to the feature map to obtain the texture features; The texture features are mapped to the surface of the initial three-dimensional mesh model by projection transformation to perform texture feature migration, a mapping relationship between the vertices of the initial three-dimensional mesh model and the image pixels is established, the vertices are projected to the image plane according to the camera projection matrix, the texture coordinates are calculated by using the barycentric coordinate interpolation inside the mesh face, and a multi-view image is constructed. Figure 1 The consistency constraint is used to evaluate the texture mapping quality under different viewing angles, the optimal texture source is selected based on the graph cut optimization algorithm, and the texture refinement is dynamically adjusted according to the observation distance to obtain a textured 3D scene model.

[0049] The point cloud neighbor search tree structure is a spatial indexing method for accelerating point cloud data queries, which can be used for efficient nearest neighbor search, point cloud registration and three-dimensional feature matching. The local neighborhood point set refers to a set of spatial neighboring points relative to a target point in three-dimensional point cloud data, which is usually used to calculate local geometric features, normal estimation and point cloud filtering. The barycentric coordinate interpolation is an interpolation method based on triangle or polygonal meshes. It achieves smooth interpolation by calculating the barycentric coordinate weights of a point at its neighborhood vertices. It is widely used in computer graphics, finite element analysis and three-dimensional deformation modeling.

[0050] Preprocess the input sparse 3D point cloud. Construct a tree structure that can quickly search for neighboring points, such as a KD tree or an octree. With each point as the center, use a spherical neighborhood search to find all neighboring points within a certain radius around the point to form the local neighborhood point set of the point. Use these neighboring points to calculate the covariance matrix of the local area where the point is located. Perform eigenvalue decomposition on the covariance matrix, where the eigenvector corresponding to the minimum eigenvalue is the normal vector of the point. In order to ensure the consistency of the normal vector direction, construct a minimum spanning tree of the point cloud to connect the points. Select the normal vector of the center point of the point cloud as the reference direction, and adjust the normal vector direction of each point in turn along the minimum spanning tree to make it consistent with the reference direction, and finally obtain the normal vector set of all points. For example, assuming that the local neighborhood point set of point A includes points B, C, and D, the calculated covariance matrix is ​​decomposed by eigenvalue to obtain the normal vector n.

[0051] Calculate the bounding box of the point cloud and use it as the initial spatial range. Dynamically adjust the granularity of spatial division according to the density distribution of the point cloud. Divide the space into multiple small cubic units, count the number of point clouds contained in each unit, and calculate the point density. Set an adaptive division threshold, such as the average point density. For units with point density higher than the threshold, continue to divide them into eight smaller sub-units, and record the depth information of each leaf node and the point set it contains. In order to optimize the division results, merge adjacent leaf nodes with similar point density to finally obtain an optimized octree structure. Assume that the range of the point cloud bounding box is (0, 0, 0) to (100, 100, 100), the point density threshold is set to 10, and a unit contains 20 points, then continue to divide it.

[0052] Define a three-dimensional scalar field function on the optimized octree structure. Convert the normal vector set calculated previously into a gradient field constraint. Obtain the implicit surface expression by iteratively solving the Poisson equation. Use a multi-grid strategy to build a solution hierarchy. Convert the Poisson equation into a linear system of equations, and use an iterative optimization method to solve the system of equations, update the scalar values ​​of the grid nodes, and finally obtain a three-dimensional scalar field. For example, suppose the initial scalar value of a grid node is 0, and after one iteration, it is updated to 0.5.

[0053] Combined with the adaptive threshold, the marching cube algorithm is used to extract isosurfaces in the optimized octree structure to generate the initial mesh. The initial mesh is topologically checked to repair non-manifold edges and singular points. The Laplace smoothing algorithm is used to iteratively optimize the vertex positions to make the mesh smoother. According to the edge collapse criterion, the mesh is simplified and redundant triangles are removed to finally obtain the initial three-dimensional mesh model. For example, the coordinates of the three vertices of a triangle are (0, 0, 0), (1, 0, 0), (0, 1, 0), and after smoothing, they may become (0.1, 0.1, 0), (0.9, 0.1, 0), (0.1, 0.9, 0).

[0054] Input the channel attention feature map into the residual graph convolutional neural network. Transfer node information through local neighborhood aggregation. Define the adjacency matrix to describe the connection relationship between nodes. Based on the adjacency matrix, perform message aggregation and weighted combination of node features and their neighborhood features. Add the input features to the transformed features to build residual connections to obtain enhanced features. Adaptively capture texture features through the graph attention mechanism. Perform global pooling operations on the enhanced features to obtain feature statistics and perform nonlinear transformations. Learn the correlation weights between different channels to generate channel attention weights. Calculate the importance scores of different positions in the feature map and generate spatial attention weights. Apply the channel attention weights and spatial attention weights to the feature map to obtain texture features. Use projection transformation to map the texture features to the surface of the initial 3D mesh model. Establish a mapping relationship between the vertices of the initial 3D mesh model and the image pixels. According to the camera projection matrix, project the vertices to the image plane. For the interior of the mesh patch, use barycentric coordinate interpolation to calculate the texture coordinates. Construct a multi-view Figure 1 Consistency constraints are used to evaluate the texture mapping quality under different viewing angles. Based on the graph cut optimization algorithm, the optimal texture source is selected. According to the observation distance, the texture refinement is dynamically adjusted, and finally a 3D scene model with texture is obtained.

[0055] Figure 3This is a comparison diagram of the simulation effect of 3D model reconstruction based on the 3D reconstruction and visualization method of news scenes based on multi-source remote sensing data in an embodiment of the present invention. It shows the comparison of the simulation effect of 3D model reconstruction based on the same input point cloud (density of 235 points / m²). The upper left figure is the original point cloud data. It can be seen that the point cloud is unevenly distributed and has certain noise. The upper right figure is the reconstruction result of this technical solution. The model surface is smooth and natural, the edge features are clear, the reconstruction accuracy reaches 0.93, and it takes only 18.5 seconds to complete the reconstruction. The lower left figure is the result of traditional Poisson reconstruction. Although the overall shape is correct, there are some non-smooth areas on the surface. The accuracy is 0.85 and it takes 29.7 seconds. The lower right figure is the result of the March cube method. The edge details are lost, the surface smoothness is insufficient, the accuracy is only 0.79, and it takes 24.3 seconds. It can be clearly seen from the reconstruction effect that this technical solution not only retains the geometric details of complex structures, but also achieves higher surface smoothness. Especially at the edges and corners of objects, this technical solution can restore shape features more accurately, while other methods have varying degrees of detail loss or deformation.

[0056] In this embodiment, a high-precision 3D model is reconstructed from sparse point cloud data, and rich geometric details are retained. The surface shape of the point cloud can be effectively captured through adaptive space division and Poisson reconstruction, and a smooth and realistic 3D mesh is generated. The graph attention mechanism and residual graph convolutional neural network can be used to effectively extract and transfer texture features, so that the reconstructed 3D model has a realistic texture effect. Figure 1 Consistency constraints and adaptive texture fineness adjustment further improve the texture fit and visual quality. Through normal vector adjustment and topology repair, it can effectively handle noise and missing points in point cloud data and generate high-quality 3D models.

[0057] S3. Calculate and optimize the curvature change value and normal vector change value of each mesh in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the mesh vertices in each space block and construct an edge collapse cost function to merge and simplify the mesh vertices to obtain four accuracy level models, construct a quadtree viewing cone divided into three depth areas, load the corresponding accuracy level model to obtain a visualized scene, extract local and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive position information and calculate the viewpoint transformation matrix to update the visualized scene for scene roaming and perform scene roaming.

[0058] The octree space block is a spatial data structure that organizes and stores data by recursively dividing the three-dimensional space into eight sub-regions (i.e., octrees), so that point clouds, volume data, etc. in the three-dimensional space can be efficiently queried, accessed, and processed. The edge collapse cost function is an optimization function used in image or geometric mesh processing, which is used to measure the cost of the edge collapse operation. It usually considers the geometric accuracy, smoothness, and degree of structure retention after deformation to optimize processes such as mesh simplification, triangle subdivision, or surface reconstruction. The quadratic error matrix refers to a matrix obtained by calculating the sum of squares of errors. It is often used in optimization algorithms such as the least squares method to minimize the error of the objective function. It is usually applied to images. Processing, data fitting and regression analysis in machine learning, etc. The accuracy level model is a method model for measuring and controlling the accuracy of calculation or measurement. By defining multiple accuracy levels and standards, the quadtree frustum is a quadtree data structure for space partitioning and view management. The two-dimensional plane is divided into four regions by recursion. The dual-branch semantic segmentation network is a semantic segmentation model, usually composed of two parallel branch structures, which process different features or different scales of the image respectively. The image segmentation task is performed through a deep learning network. The viewpoint transformation matrix is ​​a mathematical matrix used to describe viewpoint transformation. The observation angle, projection direction and camera parameters are adjusted through coordinate transformation.

[0059] In an optional embodiment, Calculate and optimize the curvature change value and normal vector change value of each grid in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the grid vertices in each space block and construct an edge collapse cost function to merge and simplify the grid vertices to obtain four precision level models, construct a quadtree view cone divided into three depth areas to load the corresponding precision level model to obtain a visualization scene, extract local features and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive location information and calculate the viewpoint transformation matrix to update the visualization scene for scene roaming and perform scene roaming, including: For each mesh face in the optimized three-dimensional scene model, project the normal vector of each mesh face to the local coordinate system of the vertex and calculate the weighted average to obtain the vertex normal vector, construct the local surface fitting equation of the vertex to calculate the principal curvature to obtain the curvature change value, traverse the adjacent vertices to calculate the normal vector angle change rate, combine the curvature change value with the normal vector angle change rate to form a vertex feature description, and establish an adjacency relationship graph that records the vertex feature description; Input the vertex feature description, vertex density distribution in the adjacency relationship graph, and feature point distribution ratio into the space partition evaluation function, recursively construct an octree with the scene bounding box as the initial space, calculate the evaluation function value of each node to be segmented, dynamically adjust the segmentation threshold according to the node level and the parent node evaluation value, and record the spatial range of each leaf node, the vertex feature description, and the mesh topology relationship to form a spatial index structure; For each leaf node in the spatial index structure, the sum of the squares of the distances from the vertex to the adjacent facets is used to construct a quadratic error matrix, the quadratic error matrix, the normal vector change rate, and the texture coordinate change are combined to construct an edge collapse cost function, the mesh edges in the node are traversed to calculate the optimal collapse position and stored in a priority queue, the edge collapse operation is iteratively performed according to the value of the collapse cost function, and the error matrix of the affected vertex and the cost value of the related edge are updated; Controlling the number of collapse iterations based on the edge collapse cost function, maintaining the continuity of the vertex feature description, generating four precision level models that retain 90%, 70%, 50%, and 30% of the original vertex number, constructing a quadtree structure corresponding to the viewing frustum, dividing the viewing frustum into a near view area, a mid-view area, and a far view area, setting an overlapping transition zone between adjacent areas, loading the two models with the highest precision among the four precision level models into the near view area, loading the model with the third highest precision into the mid-view area, and loading the model with the lowest precision into the far view area; Calculating vertex position interpolation based on the vertex feature description within the overlapping transition zone, performing smooth switching between models of different precisions to obtain a visualization scene, inputting the visualization scene into a dual-branch semantic segmentation network, extracting local features through multi-layer convolution and downsampling, and extracting global features through multi-level pooling; The local features are fused with the global features through a skip connection, the fused features are deconvolved to restore the spatial resolution to generate a semantic segmentation map, the semantic segmentation map is parsed to obtain the distribution of different category areas in the scene, the input news event description is associated with the corresponding category area, and the marked position is converted to the scene space coordinate system through line of sight projection; Obtain the position coordinates and operation instructions of the interactive device, convert the screen space coordinates to the normalized device coordinate system, construct the intersection position of the interactive ray calculation and the visualization scene, calculate the rotation matrix, translation vector and scaling factor of the viewpoint based on the intersection position, generate the viewpoint transformation path, and dynamically update the accuracy configuration of the model within the visible range and the display status of the annotation symbols according to the viewpoint position.

[0060] The mesh patch refers to a plane segment obtained by mesh segmentation in three-dimensional modeling, which is often used to represent a discretized mesh of an object surface and is applied to fields such as computer graphics, finite element analysis, and three-dimensional modeling. The local surface fitting equation is a mathematical model for fitting a surface based on a local data point set. The equation of the local surface is obtained by minimizing the fitting error and is often used in tasks such as point cloud processing, image reconstruction, and three-dimensional surface fitting. The scene bounding box is a minimum bounding box used to enclose a three-dimensional object or scene, which is often used for collision detection, object detection, and scene representation in computer graphics. It improves computational efficiency by simplifying geometric shapes. The viewpoint transformation path refers to a path trajectory from one viewpoint to another in an image or three-dimensional scene, which is often used for perspective conversion, path planning, and navigation control in virtual reality in computer graphics.

[0061] Preprocess the three-dimensional scene model. For each mesh patch in the model, convert the normal vector of each patch to the local coordinate system where the vertex is located, and calculate the weighted average to obtain the vertex normal vector. For example, if a vertex is shared by three patches, the normal vector of the vertex is the weighted average of the normal vectors of the three patches, and the weight can be set to the patch area. Then, based on the local coordinate system of the vertex and the position information of the surrounding vertices, fit a local surface equation. By analyzing the surface equation, calculate the principal curvature at the vertex, and then obtain the curvature change value. For example, a quadratic polynomial can be used to fit the local surface, and the principal curvature can be calculated by the eigenvalue of the Hessian matrix. At the same time, traverse the adjacent vertices of the vertex and calculate the angle change rate of the normal vectors between them. Combine the calculated curvature change value and the normal vector angle change rate to form a feature description of the vertex. Finally, establish a graph structure to record the feature description of all vertices and the adjacency relationship between vertices.

[0062] The space is divided according to the vertex features, and the vertex feature description, vertex density distribution in the adjacency graph, and the distribution ratio of feature points are input into the space division evaluation function. The bounding box of the scene is used as the initial space, and the octree structure is recursively constructed. At each segmentation, the evaluation function value of each node to be segmented is calculated. The segmentation threshold is dynamically adjusted according to the level of the node and the evaluation value of the parent node. For example, for nodes with deeper levels or nodes with higher parent node evaluation values, a stricter segmentation threshold is set. After the segmentation is completed, each leaf node records its spatial range, the vertex feature description contained, and the mesh topology relationship to form a spatial index structure. Assume that the bounding box size of a scene is 10x10x10 and contains 1000 vertices. After octree partitioning, leaf nodes of different sizes such as 5x5x5, 2.5x2.5x2.5, etc. may be obtained.

[0063] For each leaf node in the spatial index structure, calculate the sum of the squares of the distances from each vertex in the node to the adjacent facets and construct a quadratic error matrix. Combine the error matrix, the rate of change of the normal vector, and the change in the texture coordinates to construct an edge collapse cost function. Traverse all mesh edges in the node, calculate the optimal position of each edge after collapse, and store the results in a priority queue. According to the value of the edge collapse cost function, take the edge with the lowest cost from the priority queue for collapse, and update the error matrix of the affected vertex and the cost value of the related edge. Repeat this process until the preset vertex number target is reached. For example, for a leaf node containing 100 vertices, you can set the target number of vertices to 50 and iterate the edge collapse operation.

[0064] The number of collapse iterations is controlled according to the edge collapse cost function, and four models with different precisions are generated. In order to maintain the continuity of vertex feature description, during the edge collapse process, the edges with less impact on vertex features are preferentially collapsed. Finally, four precision level models are generated, retaining 90%, 70%, 50% and 30% of the original number of vertices. Then, a quadtree structure corresponding to the viewing frustum is constructed, and the viewing frustum is divided into three areas: near view, mid view and far view, and overlapping transition zones are set between adjacent areas. The two models with the highest precision are loaded into the near view area, the model with the third highest precision is loaded into the mid view area, and the model with the lowest precision is loaded into the far view area.

[0065] In the overlapping transition zone, the interpolation of vertex positions is calculated based on vertex feature description to achieve smooth switching between models of different precisions, and finally a visual scene is obtained. The scene is input into the dual-branch semantic segmentation network, and local features are extracted through multi-layer convolution and downsampling operations, and global features are extracted through multi-level pooling operations. The local features and global features are fused through jump connections, and then deconvolution is performed to restore the spatial resolution to generate a semantic segmentation map.

[0066] The input news event description is associated with the corresponding category area, for example, the "fire" event is associated with the area labeled "building". The labeled position is transformed into the scene space coordinate system through line of sight projection.

[0067] Get the position coordinates and operation instructions of the interactive device, convert the screen space coordinates to the normalized device coordinate system, construct the interactive ray, and calculate its intersection position with the visualization scene. Based on the intersection position, calculate the viewpoint's rotation matrix, translation vector, and scaling factor to generate the viewpoint transformation path. Dynamically update the accuracy configuration of the model within the visible range and the display status of the annotation symbols according to the viewpoint position. For example, when the viewpoint is close to a certain area, load the high-precision model of the area and display the relevant annotation symbols.

[0068] In this embodiment, by adaptively loading models of different precisions based on viewpoint distance and scene complexity, the amount of rendering calculations is significantly reduced, the frame rate and smoothness of scene roaming are improved, and by optimizing the model, important geometric features and texture details are retained, and a smooth transition is made between models of different precisions, making the scene more realistic and enhancing the user's immersion. Combined with semantic segmentation technology, different objects in the scene can be accurately labeled, and news events can be associated with scene elements, making it convenient for users to quickly understand the location and contextual information of the event.

[0069] In an optional embodiment, For each leaf node in the spatial index structure, the sum of the squares of the distances from the vertex to the adjacent facets is used to construct a quadratic error matrix, the quadratic error matrix, the normal vector change rate, and the texture coordinate change amount are combined to construct an edge collapse cost function, the mesh edges in the node are traversed to calculate the optimal collapse position and stored in a priority queue, the edge collapse operation is iteratively performed according to the value of the collapse cost function, and the error matrix of the affected vertex and the cost value of the related edge are updated, including: Obtain the mesh information of each leaf node in the spatial index structure, establish an association table between vertices and adjacent facets, obtain the facet set and topological relationship to which each vertex belongs, calculate the facet normal vector and plane equation based on the facet set, calculate the sum of the squares of the distances from the vertex to the adjacent facets to construct a quadratic error matrix, wherein the quadratic error matrix represents the degree of influence on the adjacent facets when the vertex position changes; The quadratic error matrix, the normal vector change rate, and the texture coordinate change amount are combined to construct an edge collapse cost function, the shape deformation value is calculated based on the quadratic error matrix of the vertices at both ends of the edge, the normal vectors at both ends of the edge and the adjacent vertices are obtained to calculate the normal vector change rate, the vertex texture coordinates are analyzed to obtain the coordinate change amount, the shape deformation value, the normal vector change rate, and the texture coordinate change amount are normalized and then linearly combined to obtain the edge collapse cost function value; Traversing the mesh edges in the node to calculate the optimal collapse position, obtaining the facets adjacent to the edge to construct a local mesh area, selecting multiple candidate collapse points on the straight line where the edge is located, calculating the mesh error corresponding to each candidate point, selecting the position with the smallest mesh error and maintaining topological validity as the optimal collapse position, and storing the edge information and the optimal collapse position in a priority queue sorted by the cost function value; Extracting the edge with the smallest cost function value from the priority queue, obtaining the optimal collapse position corresponding to the current edge after verifying the validity of the edge, merging the two endpoints of the edge into a new vertex, updating the coordinates, normal vector and texture coordinates of the new vertex according to the optimal collapse position, updating the vertex index of the relevant facet, deleting the degenerate facet and adjusting the mesh connection relationship; Update the error matrix of the affected vertex, integrate the geometric relationship between the new vertex and the adjacent facets into the error matrix, recalculate the cost function value and the optimal collapse position for the edge connected to the new vertex, and store the updated edge information back into the priority queue to maintain the optimal simplification order.

[0070] The candidate collapse points are points selected for simplification or merging in geometric mesh or image processing, and are usually determined by calculating the surrounding environment, deformation cost and geometric features of each point, and are used for operations such as mesh simplification and point cloud compression. The topological validity means that during the topology optimization and mesh processing, the validity and connectivity of the spatial data structure or graphic model are maintained to avoid the appearance of structures that do not conform to physical or geometric constraints, so that the optimization results meet the actual application requirements.

[0071] Load the 3D mesh model into memory and build a spatial index structure, such as an octree or KD tree. The spatial index structure divides the model space into multiple leaf nodes, each of which contains a portion of the mesh data.

[0072] Traverse each leaf node in the spatial index structure. For each leaf node, obtain the mesh information contained therein, including vertex coordinates, facet connection relationships, etc. Establish an association table between vertices and adjacent faces, and record the facet set to which each vertex belongs and the topological relationship between them.

[0073] Based on the facet set to which each vertex belongs, the normal vector and plane equation of each facet are calculated, the sum of the squares of the distances from the vertex to the adjacent facets is calculated, and a quadratic error matrix is ​​constructed. This matrix is ​​used to characterize the degree of influence on the adjacent facets when the vertex position changes. For example, there are three facets around a vertex, and the sum of the squares of the distances to these three facets are 1, 4, and 9 respectively. The quadratic error matrix of the vertex can be expressed as [1, 4, 9].

[0074] Combine the quadratic error matrix, normal vector change rate, and texture coordinate change to construct the edge collapse cost function. For each edge, calculate the shape deformation value based on the quadratic error matrix of the vertices at both ends of the edge. Get the normal vectors of the two ends and adjacent vertices of the edge and calculate the normal vector change rate. Analyze the vertex texture coordinates to get the coordinate change. Normalize the shape deformation value, normal vector change rate, and texture coordinate change and linearly combine them to get the edge collapse cost function value. For example, assuming the shape deformation value is 0.8, the normal vector change rate is 0.5, the texture coordinate change is 0.2, and the weights are 0.5, 0.3, and 0.2 respectively, then the edge collapse cost function value is 0.5*0.8+0.3*0.5+0.2*0.2=0.59.

[0075] Traverse all mesh edges in the node and calculate the optimal collapse position. For each edge, obtain the facets adjacent to the edge and construct a local mesh area. Select multiple candidate collapse points on the line where the edge is located. For example, divide the edge into five equal parts and take these five equally divided points as candidate collapse points. Calculate the mesh error corresponding to each candidate point and select the position with the smallest mesh error and topological validity as the optimal collapse position. Store the edge information and the optimal collapse position in a priority queue sorted by the cost function value.

[0076] Extract the edge with the smallest cost function value from the priority queue. Verify the validity of the edge, for example, check whether collapsing the edge will produce degenerate triangles or flip normals. If the edge is valid, obtain the optimal collapse position corresponding to the current edge, merge the two endpoints of the edge into a new vertex, and update the coordinates, normal vector, and texture coordinates of the new vertex according to the optimal collapse position. Update the vertex index of the relevant facets, delete the degenerate facets, and adjust the mesh connection relationship.

[0077] Update the error matrix of the affected vertices. Integrate the geometric relationship between the new vertex and the adjacent facets into the error matrix. Recalculate the cost function value and the optimal collapse position for the edges connected to the new vertex, and store the updated edge information back into the priority queue to maintain the optimal simplification order. Repeat the above steps until the preset simplification goal is achieved, for example, the number of facets is reduced to a specified ratio.

[0078] Figure 4This is a comparison diagram of the texture model simplification effect of the news scene 3D reconstruction and visualization method based on multi-source remote sensing data in an embodiment of the present invention, showing the visual effect comparison of the texture model at an 80% simplification rate. The upper left is the original model (538,124 faces), the upper right is the simplification result of the technical solution (107,625 faces, texture distortion 0.275), the lower left is the simplification result of the QEM method (107,625 faces, texture distortion 0.463), and the lower right is the simplification result of the vertex clustering method (107,635 faces, texture distortion 0.798). It can be clearly seen from the visual effect that the technical solution performs best in maintaining the quality of texture mapping, with accurate texture boundary alignment and clear pattern details. The QEM method Although the approximate texture distribution is maintained, obvious distortion and blurring occur in densely textured areas (such as pattern boundaries). The vertex clustering rule leads to severe texture distortion, many details are completely lost, and boundary alignment is incorrect. From the data, the texture distortion degree (0.275) of this technical solution is 40.6% lower than that of the QEM method (0.463) and 65.5% lower than that of the vertex clustering method (0.798). Especially in the high-frequency texture areas and texture boundaries on the model surface, the preservation effect of this technical solution is more prominent, which fully proves the effectiveness of this technical solution incorporating the texture coordinate change into the edge collapse cost function, and can still maintain good texture mapping quality at a high simplification rate, which is suitable for application scenarios that require high-quality texture representation.

[0079] In this embodiment, the spatial index structure can be used to divide the grid model into multiple local areas, and the edge collapse operation can be performed independently in each area, thereby realizing parallel processing and improving simplification efficiency. By considering factors such as the quadratic error matrix, the rate of change of the normal vector, and the amount of change of the texture coordinates, the deformation of the model during the simplification process can be effectively controlled to maintain the visual quality of the model.

[0080] In an optional embodiment, Take the reconstruction and visualization of a city's production recovery as an example. Collect multi-source remote sensing data: use GF-2 to obtain 1-meter resolution multispectral images for object recognition; use GF-3 radar data to achieve all-weather observation; use GF-7 to obtain stereo mapping data for elevation extraction; and obtain meteorological satellite data such as Sunflower-8 for surface temperature inversion.

[0081] The data obtained by different sensors are registered by using the iterative closest point algorithm based on spatial position weighting, and the data are unified into the same coordinate system by using the tensor projection algorithm. The bidirectional attention deep neural network is used to extract multi-source data features and generate hierarchical feature descriptors, and the random sampling algorithm based on local geometric consistency constraints is used for feature matching and screening. The variational autoencoder network and the generative adversarial network based on pixel-level adversarial discrimination are used to enhance the data.

[0082] The enhanced data is subjected to feature extraction network to generate spatial attention feature map and channel attention feature map. The spatial attention feature map is used to generate depth information through a densely connected monocular depth estimation network. The depth information is fused with GF-3 radar data and GF-7 point cloud data to obtain a sparse 3D point cloud of urban buildings. The point cloud is converted into an initial 3D mesh model using the Poisson reconstruction algorithm, and then the channel attention feature map is used to transfer the texture of the model through the residual graph convolutional neural network. Topology optimization is performed by combining multi-view projection consistency constraints and mesh optimization algorithms.

[0083] To achieve hierarchical visualization, the curvature change and normal vector change of the mesh in the optimized model are calculated, and a spatial partitioning evaluation function is constructed to divide the scene into octree spatial blocks of different sizes. The quadratic error matrix of the mesh vertices in each spatial block is calculated and an edge collapse cost function is constructed. The mesh vertices are merged and simplified according to the cost function to generate four models with different accuracy levels. A quadtree frustum is constructed and divided into three depth regions: far, medium, and near, and models of corresponding accuracy levels are loaded in different regions.

[0084] The visualized scene is input into the dual-branch semantic segmentation network to extract local and global features, and the scene semantic segmentation map is generated through feature fusion, marking key areas such as industrial areas and transportation hubs. The system supports collecting user interaction location information and calculating the viewpoint transformation matrix to realize the scene roaming function.

[0085] Figure 2 FIG. 1 is a schematic diagram of the structure of a news scene 3D reconstruction and visualization system based on multi-source remote sensing data according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to collect multi-source remote sensing data corresponding to the target news scene through remote sensing satellites, extract hierarchical feature descriptors based on the iterative closest point algorithm and tensor projection algorithm weighted by spatial position, and bidirectional attention deep neural network, screen them through the random sampling algorithm based on local geometric consistency constraints, remove feature point pairs with projection errors greater than the preset projection threshold, and unify them to global geographic coordinates to obtain registered multi-source remote sensing data, and enhance remote sensing data through the variational autoencoder network and noise-aware loss function combined with the generative adversarial network based on pixel-level adversarial discrimination; The second unit is used to add the enhanced remote sensing data to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map, add the spatial attention feature map to the densely connected monocular depth estimation network to generate depth information, fuse it with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud, generate an initial three-dimensional mesh model through the Poisson reconstruction algorithm, add the channel attention feature map to the residual graph convolutional neural network for texture feature migration, and combine the multi-view projection consistency constraint and the mesh optimization algorithm for topology optimization to obtain an optimized three-dimensional scene model; The third unit is used to calculate and optimize the curvature change value and normal vector change value of each mesh in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the mesh vertices in each space block and construct an edge collapse cost function to merge and simplify the mesh vertices to obtain four accuracy level models, construct a quadtree viewing cone divided into three depth areas, load the corresponding accuracy level model to obtain a visualized scene, extract local and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive position information and calculate the viewpoint transformation matrix to update the visualized scene for scene roaming and perform scene roaming.

[0086] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0087] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0088] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional reconstruction and visualization method for news scenes based on multi-source remote sensing data, characterized in that: include: Multi-source remote sensing data corresponding to the target news scene is collected through remote sensing satellites. Hierarchical feature descriptors are extracted based on the iterative closest point algorithm and tensor projection algorithm based on spatial position weighting and bidirectional attention deep neural network. The data is screened through a random sampling algorithm based on local geometric consistency constraints, and feature point pairs with projection errors greater than a preset projection threshold are eliminated and unified to global geographic coordinates to obtain registered multi-source remote sensing data. Enhanced remote sensing data is obtained through data enhancement using a variational autoencoder network and a noise-aware loss function combined with a generative adversarial network based on pixel-level adversarial discrimination. The enhanced remote sensing data is added to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map. The spatial attention feature map is added to the densely connected monocular depth estimation network to generate depth information. It is fused with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud. The initial three-dimensional mesh model is generated by the Poisson reconstruction algorithm. The channel attention feature map is added to the residual graph convolutional neural network for texture feature migration. The topology optimization is performed by combining the multi-view projection consistency constraint and the mesh optimization algorithm to obtain an optimized three-dimensional scene model. The curvature change value and normal vector change value of each mesh in the three-dimensional scene model are calculated and optimized, and a space division evaluation function is constructed to divide the three-dimensional scene into octree space blocks of different sizes. The quadratic error matrix of the mesh vertices in each space block is calculated and an edge collapse cost function is constructed to merge and simplify the mesh vertices to obtain four precision level models. A quadtree viewing cone is constructed to divide it into three depth areas and load the corresponding precision level models to obtain a visualized scene. Local and global features are extracted through a dual-branch semantic segmentation network, and a scene semantic segmentation map is generated through fusion upsampling and the location of news events is marked. The interactive position information is collected and the viewpoint transformation matrix is ​​calculated to update the visualized scene for scene roaming.

2. The method according to claim 1, characterized in that Multi-source remote sensing data corresponding to the target news scene is collected through remote sensing satellites. Hierarchical feature descriptors are extracted based on the iterative closest point algorithm and tensor projection algorithm based on spatial position weighting and bidirectional attention deep neural network. The feature point pairs with projection errors greater than the preset projection threshold are screened through a random sampling algorithm based on local geometric consistency constraints, and the feature points are unified to the global geographic coordinates to obtain the registered multi-source remote sensing data. The enhanced remote sensing data includes: Collect multi-source remote sensing data corresponding to the target news scene through remote sensing satellites, wherein the multi-source remote sensing data includes optical remote sensing image data, dual-polarization radar remote sensing data and multi-echo laser point cloud data; Add the multi-source remote sensing data to the iterative nearest point algorithm based on spatial position weighting, construct a spatial position weight matrix, construct a feature distance function according to the distance from the data point to the center of the scene, calculate the weight value, perform a nearest neighbor search on each data point through an octree spatial index structure to establish a nearest neighbor point set, calculate the position deviation vector between the data point and the nearest neighbor point set, multiply the position deviation vector by the spatial position weight matrix to obtain a weighted position deviation, repeat the iteration until the weighted position deviation is less than a preset threshold, and output the optimized data point set; Add the multi-source remote sensing data to the tensor projection algorithm, construct a characteristic covariance matrix through principal component analysis, determine the optimal projection direction by calculating eigenvalues ​​and eigenvectors, perform a tensor projection operation in the optimal projection direction, construct a local curvature calculation function and a normal vector change function for the projection results, and extract multimodal data feature points through curvature threshold segmentation and region growing algorithm; Adding the multimodal data feature points to a bidirectional attention deep neural network, the bidirectional attention deep neural network extracts multi-scale features through multi-layer convolution and pooling operations of an encoder, the multi-layer convolution includes a point convolution layer, an edge convolution layer and a surface convolution layer, and restores the features through deconvolution and upsampling operations of a decoder, and a bidirectional attention module is set between the encoder and the decoder, wherein the bidirectional attention module includes a self-attention mechanism for calculating the internal spatial correlation of feature points and a cross-attention mechanism for calculating the topological correlation between different feature points, and the feature mapping results of the self-attention mechanism and the cross-attention mechanism are integrated to obtain a hierarchical feature descriptor; The hierarchical feature descriptors are screened by a random sampling algorithm based on local geometric consistency constraints, candidate feature point pairs are determined by feature similarity calculation, feature point pairs are randomly selected as seed matching, initial geometric transformation relationships including rotation matrices and translation vectors are calculated based on the seed matching, local geometric consistency constraint checks are performed on the remaining feature point pairs to obtain projection errors, feature point pairs whose projection errors are greater than a preset projection threshold are eliminated, the random sampling process is repeated multiple times to select the geometric transformation relationship with the most internal point support, the multi-source remote sensing data are unified to global geographic coordinates based on the selected geometric transformation relationship, and registered multi-source remote sensing data are generated; Inputting the registered multi-source remote sensing data into a variational autoencoder network, the encoder of the variational autoencoder network maps the registered multi-source remote sensing data to a latent feature space through multiple layers of residual convolution, the decoder reconstructs the latent features into the original data form through deconvolution and jump connection, introduces random noise disturbance into the latent feature space, and constructs a noise-aware loss function to constrain the reconstruction result; A generative adversarial network based on pixel-level adversarial discrimination is constructed. The generative adversarial network adopts a densely connected fully convolutional structure to output a pixel-level authenticity score map. The decoder of the variational autoencoder network is used as a generator, and an adversarial loss function is constructed. The parameters of the generator and the generative adversarial network are optimized through a minimum-maximum game until a Nash equilibrium is reached to obtain enhanced remote sensing data.

3. The method according to claim 2, characterized in that The hierarchical feature descriptor is screened by a random sampling algorithm based on local geometric consistency constraints, candidate feature point pairs are determined by feature similarity calculation, feature point pairs are randomly selected as seed matching, initial geometric transformation relationships including rotation matrices and translation vectors are calculated based on the seed matching, local geometric consistency constraint checks are performed on the remaining feature point pairs to obtain projection errors, feature point pairs whose projection errors are greater than a preset projection threshold are eliminated, the random sampling process is repeated multiple times to select the geometric transformation relationship with the most internal point support, and the multi-source remote sensing data is unified to global geographic coordinates based on the selected geometric transformation relationship, and the generation of registered multi-source remote sensing data includes: Constructing a local geometric feature extraction unit and a global semantic feature extraction unit, wherein the local geometric feature extraction unit extracts a local shape descriptor by constructing a feature point neighborhood and analyzing the distribution characteristics of points in the neighborhood, and the global semantic feature extraction unit extracts hierarchical semantic information from bottom to top through a multi-scale feature pyramid, and cascades the local shape descriptor and the hierarchical semantic information to form a feature vector; Constructing a feature similarity calculation module, which is based on the cosine similarity measurement method, introduces a spatial distance weighting term so that feature points with similar distances have a higher matching priority, and pairs feature descriptors with similarities higher than a preset threshold to form a set of candidate feature point pairs; Randomly select a feature point pair from the candidate feature point pair set as a seed match, build a geometric transformation estimation module based on the seed match, establish a mapping relationship between corresponding points through the geometric transformation estimation module to build a three-dimensional rigid body transformation model, combine an iterative optimization algorithm to decompose the rotation matrix in the transformation process into three basic rotation matrices and solve them, and solve the translation vector by minimizing the Euclidean distance between corresponding points; Constructing a geometric consistency verification module, the geometric consistency verification module includes a feature transformation unit and an error calculation unit, the feature transformation unit transforms the source feature point to the target coordinate system through the three-dimensional rigid body transformation model, the error calculation unit calculates the projection error between the transformed position and the actual target position, dynamically adjusts the projection error threshold according to the data distribution characteristics, and eliminates feature point pairs with projection errors greater than the projection error threshold; Constructing a random sampling iterative optimization module, wherein the random sampling iterative optimization module randomly selects different seeds to match and execute the geometric transformation estimation module and the geometric consistency verification module, counts the number of interior points that meet the constraint conditions, comprehensively considers the number of interior points and the stability of the geometric transformation to perform scoring, and selects the geometric transformation relationship with the highest score as the optimal geometric transformation relationship; A coordinate conversion module is constructed based on the optimal geometric transformation relationship. Based on the coordinate conversion module, the optical remote sensing image data is used as reference data. An affine transformation model considering imaging geometric characteristics is constructed for the dual-polarization radar remote sensing data, and transformation parameters are solved by a polynomial fitting method to achieve geometric correction. A three-dimensional rigid body transformation model is constructed for the multi-echo laser point cloud data, and transformation parameters are solved by an iterative optimization method. The optical remote sensing image data, the dual-polarization radar remote sensing data and the multi-echo laser point cloud data are uniformly converted to a global geographic coordinate system.

4. The method according to claim 1, characterized in that: The enhanced remote sensing data is added to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map. The spatial attention feature map is added to the densely connected monocular depth estimation network to generate depth information. It is fused with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud. The initial three-dimensional mesh model is generated by the Poisson reconstruction algorithm. The channel attention feature map is added to the residual graph convolutional neural network for texture feature migration. The optimized three-dimensional scene model is obtained by combining the multi-view projection consistency constraint and the mesh optimization algorithm for topology optimization. The following includes: Input the enhanced remote sensing data into the feature extraction network, extract multi-scale spatial features through a multi-layer cascade convolution structure and a batch normalization layer, use an adaptive weight calculation unit to assign spatial attention weights to each position, obtain channel descriptors through global pooling for information aggregation, perform multi-layer transformation on the channel descriptors to learn the inter-channel dependency relationship to generate channel attention weights, and adaptively fuse the spatial attention weights and the channel attention weights to generate a spatial attention feature map and a channel attention feature map; The spatial attention feature map is input into a densely connected monocular depth estimation network, a multi-layer cascade connection is established in the monocular depth estimation network by a jump method, a direct mapping relationship is established between shallow features and deep features, feature resolution is gradually restored through a multi-layer deconvolution structure, and a multi-scale feature pyramid structure is used to perform weight adaptive fusion of features at different levels to generate depth information; The depth information is sampled and a mapping relationship between pixel coordinates and three-dimensional space points is established. The confidence weight is calculated for the pre-acquired dual-polarization radar remote sensing data and multi-echo laser point cloud data based on the signal-to-noise ratio and spatial resolution. An adaptive weighted fusion strategy is constructed according to the spatial distance and feature similarity. The point clouds in the overlapping areas are fused to generate a sparse three-dimensional point cloud. The sparse three-dimensional point cloud is input into the Poisson reconstruction algorithm, the normal vector of each point is calculated by the feature decomposition method, the spatial subdivision granularity is dynamically adjusted according to the point cloud density distribution, the implicit surface expression is obtained by iteratively solving the Poisson equation, the grid structure is extracted by combining the adaptive threshold and Laplace smoothing is performed to generate the initial three-dimensional grid model, the channel attention feature map is input into the residual graph convolutional neural network, the node information is transferred by local neighborhood aggregation, the input feature and the transformed feature are added to construct the residual connection, the texture feature is adaptively captured by the graph attention mechanism, and the texture feature is mapped to the surface of the initial three-dimensional grid model by projection transformation to perform texture feature migration; Multiple observation perspectives are uniformly sampled in the spherical space, and the initial three-dimensional mesh model is projected onto each observation perspective plane to generate a multi-perspective projection image. The similarity between the multi-perspective projection image and the original image is calculated by feature descriptor matching to establish a multi-perspective projection consistency constraint. Based on the multi-perspective projection consistency constraint, the mesh vertex positions and connection relationships are optimized while maintaining the model geometric details to generate an optimized three-dimensional scene model.

5. The method according to claim 4, characterized in that The sparse three-dimensional point cloud is input into the Poisson reconstruction algorithm, the normal vector of each point is calculated by the feature decomposition method, the spatial subdivision granularity is dynamically adjusted according to the point cloud density distribution, the implicit surface expression is obtained by iteratively solving the Poisson equation, the grid structure is extracted by combining the adaptive threshold and Laplace smoothing is performed to generate the initial three-dimensional grid model, the channel attention feature map is input into the residual graph convolutional neural network, the node information is transferred by local neighborhood aggregation, the input feature and the transformation feature are added to construct the residual connection, the texture feature is adaptively captured by the graph attention mechanism, and the texture feature is mapped to the surface of the initial three-dimensional grid model by projection transformation to perform texture feature migration, including: Input the sparse three-dimensional point cloud into the Poisson reconstruction algorithm, construct a point cloud neighbor search tree structure, use a spherical neighborhood search method to determine the local neighborhood point set corresponding to each point in the sparse three-dimensional point cloud, construct a covariance matrix based on the local neighborhood point set, calculate the normal vector of each point by the eigendecomposition method, construct a minimum spanning tree to establish the connection relationship between points, select the normal vector at the center of the point cloud as a reference direction, recursively traverse the minimum spanning tree to adjust the normal vector directions of all points to obtain a normal vector set; Calculate the outer bounding box of the point cloud as the initial spatial range, dynamically adjust the spatial subdivision granularity according to the point cloud density distribution, count the number of point clouds contained in each octree node and calculate the point density distribution, set an adaptive subdivision threshold based on the point density, divide the node with a point density greater than the adaptive subdivision threshold into eight child nodes and record the depth information and point set contained in each leaf node, merge the divided leaf nodes, and merge adjacent nodes with similar point density to obtain an optimized octree structure; A three-dimensional scalar field function is defined on the optimized octree structure, the normal vector set is converted into a gradient field constraint, an implicit surface expression is obtained by iteratively solving the Poisson equation, a solution hierarchy is constructed by adopting a multi-grid strategy, the Poisson equation is converted into a linear equation system by constructing a sparse matrix to represent the Laplace operator, an iterative optimization method is adopted to solve the linear equation system and the scalar values ​​of the grid nodes are updated to obtain a three-dimensional scalar field; Extracting the mesh structure in combination with the adaptive threshold, extracting the isosurface in the optimized octree structure based on the marching cube algorithm to generate the initial mesh, performing a topological check on the initial mesh and repairing the non-manifold edges and singular points, performing Laplace smoothing iteration to optimize the vertex positions, and simplifying the mesh based on the edge collapse criterion to obtain the initial three-dimensional mesh model; Input the channel attention feature map into the residual graph convolutional neural network, transfer node information through local neighborhood aggregation, define an adjacency matrix to describe the connection relationship between nodes, perform message aggregation based on the adjacency matrix, perform weighted combination of node features and neighborhood features, add the input features and the transformed features to construct a residual connection to obtain enhanced features; The texture features are adaptively captured through the graph attention mechanism, the feature statistics are obtained through the global pooling operation and nonlinear transformation is performed on the enhanced features, the correlation weights between different channels are learned to generate channel attention weights, the importance scores of different positions of the feature map are calculated to generate spatial attention weights, and the channel attention weights and the spatial attention weights are applied to the feature map to obtain the texture features; A projection transformation is used to map the texture features to the surface of the initial three-dimensional mesh model for texture feature migration, a mapping relationship between the vertices of the initial three-dimensional mesh model and the image pixels is established, the vertices are projected to the image plane according to the camera projection matrix, the texture coordinates are calculated inside the mesh facets by using barycentric coordinate interpolation, a multi-view consistency constraint is constructed to evaluate the texture mapping quality under different viewing angles, the optimal texture source is selected based on the graph cut optimization algorithm, and the texture fineness is dynamically adjusted according to the observation distance to obtain a textured three-dimensional scene model.

6. The method according to claim 1, characterized in that Calculate and optimize the curvature change value and normal vector change value of each grid in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the grid vertices in each space block and construct an edge collapse cost function to merge and simplify the grid vertices to obtain four precision level models, construct a quadtree view cone divided into three depth areas to load the corresponding precision level model to obtain a visualization scene, extract local features and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive location information and calculate the viewpoint transformation matrix to update the visualization scene for scene roaming and perform scene roaming, including: For each mesh face in the optimized three-dimensional scene model, project the normal vector of each mesh face to the local coordinate system of the vertex and calculate the weighted average to obtain the vertex normal vector, construct the local surface fitting equation of the vertex to calculate the principal curvature to obtain the curvature change value, traverse the adjacent vertices to calculate the normal vector angle change rate, combine the curvature change value with the normal vector angle change rate to form a vertex feature description, and establish an adjacency relationship graph that records the vertex feature description; Input the vertex feature description, vertex density distribution in the adjacency relationship graph, and feature point distribution ratio into the space partition evaluation function, recursively construct an octree with the scene bounding box as the initial space, calculate the evaluation function value of each node to be segmented, dynamically adjust the segmentation threshold according to the node level and the parent node evaluation value, and record the spatial range of each leaf node, the vertex feature description, and the mesh topology relationship to form a spatial index structure; For each leaf node in the spatial index structure, the sum of the squares of the distances from the vertex to the adjacent facets is used to construct a quadratic error matrix, the quadratic error matrix, the normal vector change rate, and the texture coordinate change are combined to construct an edge collapse cost function, the mesh edges in the node are traversed to calculate the optimal collapse position and stored in a priority queue, the edge collapse operation is iteratively performed according to the value of the collapse cost function, and the error matrix of the affected vertex and the cost value of the related edge are updated; Controlling the number of collapse iterations based on the edge collapse cost function, maintaining the continuity of the vertex feature description, generating four precision level models that retain 90%, 70%, 50%, and 30% of the original vertex number, constructing a quadtree structure corresponding to the viewing frustum, dividing the viewing frustum into a near view area, a mid-view area, and a far view area, setting an overlapping transition zone between adjacent areas, loading the two models with the highest precision among the four precision level models into the near view area, loading the model with the third highest precision into the mid-view area, and loading the model with the lowest precision into the far view area; Calculating vertex position interpolation based on the vertex feature description within the overlapping transition zone, performing smooth switching between models of different precisions to obtain a visualization scene, inputting the visualization scene into a dual-branch semantic segmentation network, extracting local features through multi-layer convolution and downsampling, and extracting global features through multi-level pooling; The local features are fused with the global features through a skip connection, the fused features are deconvolved to restore the spatial resolution to generate a semantic segmentation map, the semantic segmentation map is parsed to obtain the distribution of different category areas in the scene, the input news event description is associated with the corresponding category area, and the marked position is converted to the scene space coordinate system through line of sight projection; Obtain the position coordinates and operation instructions of the interactive device, convert the screen space coordinates to the normalized device coordinate system, construct the intersection position of the interactive ray calculation and the visualization scene, calculate the rotation matrix, translation vector and scaling factor of the viewpoint based on the intersection position, generate the viewpoint transformation path, and dynamically update the accuracy configuration of the model within the visible range and the display status of the annotation symbols according to the viewpoint position.

7. The method according to claim 6, characterized in that For each leaf node in the spatial index structure, the sum of the squares of the distances from the vertex to the adjacent facets is used to construct a quadratic error matrix, the quadratic error matrix, the normal vector change rate, and the texture coordinate change amount are combined to construct an edge collapse cost function, the mesh edges in the node are traversed to calculate the optimal collapse position and stored in a priority queue, the edge collapse operation is iteratively performed according to the value of the collapse cost function, and the error matrix of the affected vertex and the cost value of the related edge are updated, including: Obtain the mesh information of each leaf node in the spatial index structure, establish an association table between vertices and adjacent facets, obtain the facet set and topological relationship to which each vertex belongs, calculate the facet normal vector and plane equation based on the facet set, calculate the sum of the squares of the distances from the vertex to the adjacent facets to construct a quadratic error matrix, wherein the quadratic error matrix represents the degree of influence on the adjacent facets when the vertex position changes; The quadratic error matrix, the normal vector change rate, and the texture coordinate change amount are combined to construct an edge collapse cost function, the shape deformation value is calculated based on the quadratic error matrix of the vertices at both ends of the edge, the normal vectors at both ends of the edge and the adjacent vertices are obtained to calculate the normal vector change rate, the vertex texture coordinates are analyzed to obtain the coordinate change amount, the shape deformation value, the normal vector change rate, and the texture coordinate change amount are normalized and then linearly combined to obtain the edge collapse cost function value; Traversing the mesh edges in the node to calculate the optimal collapse position, obtaining the facets adjacent to the edge to construct a local mesh area, selecting multiple candidate collapse points on the straight line where the edge is located, calculating the mesh error corresponding to each candidate point, selecting the position with the smallest mesh error and maintaining topological validity as the optimal collapse position, and storing the edge information and the optimal collapse position in a priority queue sorted by the cost function value; Extracting the edge with the smallest cost function value from the priority queue, obtaining the optimal collapse position corresponding to the current edge after verifying the validity of the edge, merging the two endpoints of the edge into a new vertex, updating the coordinates, normal vector and texture coordinates of the new vertex according to the optimal collapse position, updating the vertex index of the relevant facet, deleting the degenerate facet and adjusting the mesh connection relationship; Update the error matrix of the affected vertex, integrate the geometric relationship between the new vertex and the adjacent facets into the error matrix, recalculate the cost function value and the optimal collapse position for the edge connected to the new vertex, and store the updated edge information back into the priority queue to maintain the optimal simplification order.

8. A news scene three-dimensional reconstruction and visualization system based on multi-source remote sensing data, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect multi-source remote sensing data corresponding to the target news scene through remote sensing satellites, extract hierarchical feature descriptors based on the iterative closest point algorithm and tensor projection algorithm weighted by spatial position, and bidirectional attention deep neural network, screen them through the random sampling algorithm based on local geometric consistency constraints, remove feature point pairs with projection errors greater than the preset projection threshold, and unify them to global geographic coordinates to obtain registered multi-source remote sensing data, and enhance remote sensing data through the variational autoencoder network and noise-aware loss function combined with the generative adversarial network based on pixel-level adversarial discrimination; The second unit is used to add the enhanced remote sensing data to the feature extraction network to obtain the spatial attention feature map and the channel attention feature map, add the spatial attention feature map to the densely connected monocular depth estimation network to generate depth information, fuse it with the dual-polarization radar remote sensing data and the multi-echo laser point cloud data to obtain a sparse three-dimensional point cloud, generate an initial three-dimensional mesh model through the Poisson reconstruction algorithm, add the channel attention feature map to the residual graph convolutional neural network for texture feature migration, and combine the multi-view projection consistency constraint and the mesh optimization algorithm for topology optimization to obtain an optimized three-dimensional scene model; The third unit is used to calculate and optimize the curvature change value and normal vector change value of each mesh in the three-dimensional scene model, construct a space division evaluation function to divide the three-dimensional scene into octree space blocks of different sizes, calculate the quadratic error matrix of the mesh vertices in each space block and construct an edge collapse cost function to merge and simplify the mesh vertices to obtain four accuracy level models, construct a quadtree viewing cone divided into three depth areas, load the corresponding accuracy level model to obtain a visualized scene, extract local and global features through a dual-branch semantic segmentation network, generate a scene semantic segmentation map through fusion upsampling and mark the location of news events, collect interactive position information and calculate the viewpoint transformation matrix to update the visualized scene for scene roaming and perform scene roaming.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images