Surveying and mapping geography multivariate data fusion processing method

Abstract

The invention discloses a surveying and mapping geography multivariate data fusion processing method. The method comprises the following steps: S1, data preprocessing; s2, constructing a sparse representation model; s3, estimating posterior distribution, and assuming a data generation mode; s4, carrying out sparse coefficient fusion, and carrying out joint modeling by using a multi-task sparse coding model; s5, enhancing relevance, and constructing a semantic network graph model; and S6, performing global optimization and output. According to the method, the defects of a traditional method in processing diversified scales can be effectively overcome in the data preprocessing link, the data quality is greatly improved, local optimum is avoided and the convergence speed is increased through self-adaptive step length adjustment and a random restart mechanism when the sparse representation model is constructed, and meanwhile, in the global optimization and output stage, the algorithm is simple and convenient to operate. A multi-objective evolutionary algorithm is adopted, multiple optimization objectives can be considered at the same time, an optimization strategy is flexibly adjusted according to an actual application scene, deep data fusion is achieved, and the comprehensive utilization value of surveying and mapping geography multivariate data is improved.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping geographic data processing technology, and in particular to a surveying and mapping geographic multivariate data fusion processing method. Background Art

[0002] In today's surveying and mapping geographic information field, with the rapid development of technology, the sources of geographic data are becoming increasingly rich, covering satellite remote sensing data, aerial photogrammetry data, ground measurement data and other types. In order to fully and accurately describe geographic spatial information and achieve in-depth understanding and analysis of geographical phenomena, multivariate data fusion has become an inevitable trend. By integrating data from different data sources, the advantages of each data source can be fully utilized to make up for the shortcomings of a single data source, thereby improving the accuracy, reliability and integrity of surveying and mapping geographic information.

[0003] However, there are many problems in the existing surveying and mapping multivariate data fusion processing methods. In the data preprocessing stage, traditional scale standardization methods are often unable to effectively handle the diverse data scales, resulting in uneven data quality, affecting subsequent data processing and analysis. In terms of feature extraction, conventional methods are difficult to mine deep potential features in the data and cannot fully express the intrinsic information of the data, limiting the effect of data fusion.

[0004] When building sparse representation models, traditional methods have shortcomings in dictionary matrix construction and iterative optimization. The iterative optimization method with a fixed step size is prone to falling into local optimal solutions and has a slow convergence speed, resulting in low model construction efficiency and failure to accurately reflect the internal structure of the data.

[0005] When estimating the posterior distribution, traditional methods lack in-depth consideration of the actual characteristics of the data, making it difficult to accurately approximate the true posterior distribution, affecting the robustness and accuracy of data fusion. In the sparse coefficient fusion link, the existing methods lack flexibility and pertinence in the weight allocation of different data sources, and are unable to give full play to the advantages of each data source, making it difficult to achieve deep data fusion.

[0006] In addition, in terms of enhancing the relevance of data sources, the traditional graph model construction and update methods are relatively simple, and cannot fully consider the complex spatiotemporal characteristics and semantic relationships between data sources, and cannot adapt to the dynamic changes of data in a timely manner. In the global optimization and output stage, traditional methods often cannot take into account multiple optimization goals at the same time, resulting in optimization results that are difficult to meet the diverse needs of practical applications. Summary of the invention

[0007] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a method for fusion processing of surveying and mapping geographic multivariate data.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for fusing and processing multi-source geospatial data, the method comprising the following steps:

[0010] S1. Data preprocessing: Obtain multi-source geospatial data, remove noise, and use quantile normalization to obtain preprocessed data x i ″. After preliminary feature screening, use a generative adversarial network to extract features;

[0011] S2. Construct a sparse representation model: Represent the preprocessed data x i ″ as a linear combination of a dictionary matrix d i and a sparse coefficient matrix α i . Initialize using prior knowledge, adopt an adaptive step size and set an adjustment range during iterative optimization, combine the second derivative, and add a random restart mechanism;

[0012] S3. Estimate the posterior distribution: Assume the data generation method, set the prior distribution, use variational inference to approximate the true posterior, fuse reinforcement learning, adopt hierarchical rewards and adjust the policy regularly;

[0013] S4. Sparse coefficient fusion: Jointly model using a multi-task sparse coding model, assign weights according to the importance of data sources, perform parallel optimization and solution, combine the attention mechanism, and calculate weights comprehensively considering multiple factors;

[0014] S5. Enhance relevance: Construct a semantic network graph model, adjust the weight matrix W, introduce graph regularization and increase structural constraints, and update the graph model according to data changes or error thresholds;

[0015] S6. Global optimization and output: Introduce the sparse coefficient matrix after graph theory regularization into a semidefinite programming model, set the objective function according to actual needs, solve using a multi-objective evolutionary algorithm, adjust parameters according to the problem scale, and monitor objective optimization.

[0016] Preferably, the quantile normalization calculation method in step S1 is:

[0017]

[0018] where min(x i ) and max(x i ) are the minimum and maximum values of the data source x i , respectively.

[0019] Preferably, the dictionary matrix in step S2 is constructed using the K-SVD algorithm.

[0020] Preferably, in step S3, it is assumed that the preprocessed data source x i ″ is composed of the dictionary matrix d i and the sparse coefficient matrix α iGenerated, and the noise of the data source follows a Gaussian distribution with zero mean and variance σ 2 of x i = d i α i + ∈ i , where d is the dictionary matrix i and α is the sparse coefficient matrix i Specify the Gaussian distribution as the prior distribution.

[0021] Preferably, the optimization objective function of multi-task sparse coding in step S4 is:

[0022]

[0023] By optimizing this objective function, first fix the sparse coefficients of other tasks and optimize the sparse coefficient α of each task j , and use the gradient descent method to solve it.

[0024] Preferably, the graph model in step S5 is:

[0025] G = (V, E)

[0026] where V represents the data source nodes and E represents the connection relationship between the nodes.

[0027] Preferably, the calculation formula of the weight matrix W in step S5 is:

[0028]

[0029] where, w ij is each element of the weight matrix W, representing the similarity between the data sources x i ″ and the data source x j ″, σ is a parameter controlling the similarity sensitivity between the data sources, ||x i ″ - x j ″|| 2 represents the Euclidean distance between the data sources x i ″ and x j ″.

[0030] Preferably, the objective function in step S6 is min Z trace(Z), which contains the sparse representation error term and the graph regularization term. In the objective function form, Z is a positive semi-definite matrix, and z ij represents the correlation between the sparse coefficients α i and α j . Set the regularization term z ij in the objective function to the square form of the Euclidean distance between the sparse coefficients

[0031] The present invention has the following beneficial effects:

[0032] 1. In the data preprocessing stage of the present invention, the quantile normalization method is used first, combined with a generative adversarial network to mine deep features, which can effectively overcome the defects of traditional methods in dealing with diverse scales, greatly improve the data quality, provide a more accurate data basis for subsequent analysis, and improve the accuracy of surveying and mapping geographic information.

[0033] 2. When constructing the sparse representation model of the present invention, through the adaptive step size adjustment and random restart mechanism, it avoids falling into local optima and speeds up the convergence rate. Compared with the traditional fixed-step iteration method, it can significantly shorten the model construction time, improve work efficiency, and make the processing of surveying and mapping geographic data more efficient.

[0034] 3. In the process of estimating the posterior distribution and enhancing the relevance of data sources, the present invention fully considers the actual characteristics of the data, spatio-temporal and semantic relationships, and dynamically updates the graph model, enabling the model to better adapt to the dynamic changes of the data, overcoming the limitations of traditional methods in this regard, enhancing the adaptability and robustness of the model, and ensuring the stability and reliability of the data fusion results.

[0035] 4. In the global optimization and output stage, the present invention adopts a multi-objective evolutionary algorithm, which can take into account multiple optimization objectives at the same time, break through the bottleneck that traditional methods cannot meet diverse requirements, can flexibly adjust the optimization strategy according to the actual application scenario, achieve deep data fusion, and enhance the comprehensive utilization value of multi-source surveying and mapping geographic data. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of a method for fusing and processing multi-source surveying and mapping geographic data proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0038] Embodiment

[0039] Refer to Figure 1 , which shows a flowchart of a method for fusing and processing multi-source surveying and mapping geographic data provided by an embodiment of the present invention. The method includes the following steps:

[0040] S1. Data preprocessing, obtaining multi-source surveying data monitored by multiple sensors such as satellite remote sensing data, aerial photogrammetry data, and ground survey data, removing noise, and obtaining preprocessed data x i ″, and the quantile normalization calculation method is:

[0041]

[0042] After preliminary feature screening, a generative adversarial network (GAN) is used to extract features. The specific method is as follows: the screened data source is input into the generative adversarial network (GAN). The GAN consists of a generator and a discriminator. The generator, based on the input data source, uses a neural network structure to learn the data distribution and generate simulated data features. The discriminator then distinguishes between the features of the input real data source and the simulated data features generated by the generator, and adjusts its own parameters through the backpropagation algorithm to improve the discrimination ability. In order to generate data closer to the real features, the generator will also continuously adjust its own parameters. During the continuous adversarial training process of the two, the generator gradually becomes able to generate more representative and unique latent features. These features are extracted and used for subsequent data processing processes, thus completing the feature extraction step based on GAN and improving the feature expression ability of multi-source surveying and mapping geography data.

[0043] S2. Construct a sparse representation model to represent the preprocessed data x i ″ as a linear combination of the dictionary matrix d i and the sparse coefficient matrix α i . Initialize using prior knowledge, adopt an adaptive step size and set an adjustment range during iterative optimization, combine the second derivative, and add a random restart mechanism.

[0044] Among them, the dictionary matrix is constructed using the K-SVD algorithm and applied in the following way:

[0045] First, when initializing the dictionary matrix and the sparse coefficient matrix using prior knowledge, randomly select some preprocessed data samples as initial dictionary atoms. These atoms form the initial dictionary matrix D0, whose dimension needs to be adapted to subsequent calculations, generally d×k, where d is the data feature dimension and k is the number of dictionary atoms;

[0046] In the iterative optimization stage, in each iteration, first fix the dictionary matrix D, use an existing algorithm (such as the orthogonal matching pursuit algorithm) to solve the sparse coefficient matrix α, and update the dictionary matrix D. For each atom d i in the dictionary matrix D, find all the data sources x corresponding to the non-zero sparse coefficients related to this atom j , construct the residual matrix R j = x j - ∑ l≠i d l α lj , that is, the remaining part after removing the contributions of other atoms to the data source x j . Perform singular value decomposition (SVD) on the residual matrix R j , obtain the singular vector corresponding to the largest singular value, and replace the original dictionary atom d with this vectori , thereby updating the dictionary matrix D, and repeating this process until the dictionary matrix D converges or reaches the preset number of iterations.

[0047] By applying the K-SVD algorithm in step S2, the dictionary matrix can more accurately reflect the internal structure of the data, improving the processing effect of the sparse representation model on the surveying and mapping geographic multi-source data.

[0048] S3. Estimate the posterior distribution, assume the data generation method, and assume that the preprocessed data source x i ″ is generated by the dictionary matrix d i and the sparse coefficient matrix α, and the noise of the data source follows a Gaussian distribution with zero mean and variance σ 2 x i = d i α i + ∈ i ; Set the prior distribution, specify the Gaussian distribution as the prior distribution for the dictionary matrix d i and the sparse coefficient matrix α i , approximate the true posterior with variational inference, fuse reinforcement learning, adopt hierarchical rewards and adjust the policy regularly;

[0049] S4. Sparse coefficient fusion, jointly model with a multi-task sparse coding model, and the optimization objective function of multi-task sparse coding is:

[0050]

[0051] By optimizing this objective function, first fix the sparse coefficients of other tasks and optimize the sparse coefficient α of each task i , and use the gradient descent method to solve it; allocate weights according to the importance of the data source, solve the optimization in parallel, and combine the attention mechanism to calculate the weights considering multiple factors.

[0052] S5. Enhance the relevance, construct a semantic network graph model, and the graph model is:

[0053] G=(V,E)

[0054] where V represents the data source nodes and E represents the connection relationship between the nodes; adjust the weight matrix W, and the calculation formula of the weight matrix W is:

[0055]

[0056] where, w ij Each element of the weight matrix W represents the similarity between the data source x i ″ and the data source x j ″, σ is a parameter that controls the similarity sensitivity between the data sources, ||x i ″ - xj ″|| 2 Denote the Euclidean distance between the data source x i ″ and x j ″;

[0057] Introduce graph regularization and add structural constraints. The form of the graph regularization term is:

[0058]

[0059] where α i and α j are the sparse coefficient matrices corresponding to the data sources x i ″ and x j ″ respectively. This regularization term will make the sparse coefficients corresponding to similar data sources more similar. Subsequently, to further enhance the correlation between data sources, structural constraints can be added to the sparse coefficient matrix α i such as restricting the sparsity range of the sparse coefficients;

[0060] Update the graph model according to data changes or error thresholds, and set the judgment criteria for data changes and error thresholds. When the data source changes, such as new data collection or update of existing data, calculate the difference in key features of the data source before and after the change. If the change amplitude of the key features exceeds the pre-set threshold (such as 10%), it is considered that the data has changed significantly, and the update of the graph model is triggered. During the data fusion process, calculate the error between the fusion result and the expected result. When the error exceeds the set error threshold, the update of the graph model is also triggered. When updating the graph model, recalculate the weight matrix, and re-optimize the objective function of multi-task sparse coding according to the new weight matrix and structural constraint conditions to adapt to the data changes and ensure the stability and accuracy of the fusion effect.

[0061] S6. Global optimization and output. Introduce the sparse coefficient matrix after graph theory regularization into the semi-definite programming model, and set the objective function according to actual needs. The objective function is min Z trace(Z), which includes the sparse representation error term and the graph regularization term. In the form of the objective function, Z is a positive semi-definite matrix, and z ij represents the correlation between the sparse coefficients α i and α j . Set the regularization term z ij in the objective function as the square of the Euclidean distance between the sparse coefficients z ij =∥α i -α j ∥ 2 2; Solve it with a multi-objective evolutionary algorithm, adjust the parameters according to the problem scale and monitor the objective optimization;

[0062] The solution of the multi-objective evolutionary algorithm includes the following steps:

[0063] First, a group of initial solutions are randomly generated as the population, and each solution represents a possible value of the sparse coefficient matrix α. The selection of the population size is crucial and is generally determined according to the complexity of the problem and computing resources. For example, for a relatively complex mapping and geographical data fusion problem, the population size can be set to 100 - 500. The sparse coefficient matrix α in each solution needs to satisfy certain constraint conditions, such as the sparsity range and structural constraints mentioned above.

[0064] Subsequently, for each solution, calculate its performance indicators on each objective, that is, the fitness value. For example, calculate the sparse representation error and data source correlation index corresponding to each solution. Based on these fitness values, non - dominated sorting is performed on the solutions in the population. Non - dominated sorting is one of the core operations of the multi - objective evolutionary algorithm. It divides the solutions in the population into different ranks. Solutions in the same rank are non - dominated with each other, that is, there is no situation where one solution is better than another in all objectives. The lower the rank, the better the performance of the solution.

[0065] Secondly, a new generation of population is generated through genetic operations such as selection, crossover, and mutation. The selection operation usually adopts methods such as tournament selection, and selects solutions with better fitness from the population as parents. The crossover operation is to exchange some genes of two parent solutions to generate new offspring solutions. For example, the elements of the sparse coefficient matrix α can be exchanged in the way of single - point crossover or multi - point crossover. The mutation operation is to randomly change the genes of some offspring solutions to increase the diversity of the population and prevent the algorithm from falling into local optima. For example, some elements in the sparse coefficient matrix are randomly perturbed.

[0066] Finally, repeat steps such as calculating fitness and genetic operations to continuously evolve the population. In each iteration, retain the better solutions in the previous generation and pass them to the next generation. At the same time, monitor the optimization of each objective. When a certain objective reaches the optimal or near - optimal state, appropriately adjust the search direction of the algorithm. For example, the parameters of the genetic operation can be dynamically adjusted to increase the search intensity for other unoptimized objectives. When the preset termination conditions are met, such as reaching the maximum number of iterations, convergence of the objective function, etc., the algorithm stops and outputs the optimal solution or a set of non - dominated solutions as the final sparse coefficient matrix α to complete global optimization and output.

[0067] In summary, in the data preprocessing stage, after acquiring multi-source surveying and mapping data, the present invention combines multiple denoising algorithms to remove noise, uses quantile standardization to eliminate scale differences, and screens features by calculating correlation coefficients, and then uses generative adversarial networks (GANs) to mine deep potential features. When constructing a sparse representation model, the dictionary matrix is constructed using the K-SVD algorithm, initialized using prior knowledge, and an adaptive step size adjustment mechanism is used during iterative optimization in combination with second-order derivatives to set upper and lower limits on the step size. A random restart mechanism is also introduced to avoid falling into local optimality and accelerate convergence.

[0068] When estimating the posterior distribution, we make reasonable assumptions about the data generation method and prior distribution, apply variational inference technology, integrate reinforcement learning ideas, and adopt a hierarchical reward mechanism. We give rewards based on the closeness of the variational distribution to the true posterior distribution and the speed of model convergence, and adjust the strategy regularly to improve the accuracy of inference. In the sparse coefficient fusion step, we jointly model the multi-task sparse coding model, assign weights according to the importance and relevance of the data source, optimize the solution using parallel computing technology and gradient descent method, and combine the attention mechanism to comprehensively consider the spatial resolution, temporal resolution and data acquisition confidence of the data source to calculate the weights, and achieve deep fusion.

[0069] In terms of enhancing relevance, a semantic network graph model is constructed, the weight matrix is adjusted in combination with domain knowledge, a graph regularization term is introduced and structural constraints on the sparse coefficient matrix are added, such as limiting the sparsity range, updating the graph model weight matrix when the change in key data features exceeds 10% or the fusion error exceeds the threshold, and during global optimization and output, the sparse coefficient matrix after graph theory regularization is introduced into the semidefinite programming model, the objective function is set according to actual needs, and a multi-objective evolutionary algorithm is used to solve it. The population size and evolution parameters are set according to the scale and complexity of the problem, the optimization status of each objective is monitored in real time, and the search direction is dynamically adjusted.

[0070] Compared with the existing technology, the method of the present invention has significant beneficial effects, improves the data processing accuracy, and provides a more accurate data basis for analysis; speeds up the model construction speed and improves work efficiency; enhances the adaptability and robustness of the model and can better adapt to dynamic changes in data; realizes multi-objective global optimization and meets diversified application needs, effectively solves the shortcomings of existing methods, and improves the processing level and application value of surveying and mapping geographic information.

[0071] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for fusing and processing multi-source geospatial data, characterized in that The method includes the following steps: S1. Data preprocessing: Obtain multi-source surveying and mapping data, remove noise, and use quantile normalization to obtain preprocessed data x″ i , after preliminary feature screening, use a generative adversarial network to extract features; S2. Construct a sparse representation model to represent the preprocessed data x″ i as a linear combination of the dictionary matrix d i and the sparse coefficient matrix α i Initialize using prior knowledge. When iteratively optimizing, adopt an adaptive step size and set an adjustment range. Combine the second derivative and add a random restart mechanism; S3. Estimate the posterior distribution, assume the data generation method, set the prior distribution, approximate the true posterior with variational inference, fuse reinforcement learning, adopt hierarchical rewards and adjust the policy regularly; S4. Sparse coefficient fusion, jointly model with a multi-task sparse coding model, assign weights according to the importance of data sources, optimize and solve in parallel, combine the attention mechanism, and calculate weights comprehensively considering multiple factors; S5. Enhance the relevance, construct a semantic network graph model, adjust the weight matrix W, introduce graph regularization and increase structural constraints, and update the graph model according to data changes or error thresholds; S6. Global optimization and output, introduce the sparse coefficient matrix after graph theory regularization into the semi-definite programming model, set the objective function according to actual needs, solve with a multi-objective evolutionary algorithm, adjust parameters according to the problem scale, and monitor the objective optimization.

2. A method for processing the fusion of multiple surveying and mapping geographic data according to claim 1, characterized in that, The quantile normalization calculation method in step S1 is as follows: where min(x i ) and max(x i ) are the minimum and maximum values of the data source x i , respectively.

3. A method for processing the fusion of surveying and mapping geographical multi-source data according to claim 1, characterized in that, The dictionary matrix in step S2 is constructed using the K-SVD algorithm.

4. A method for processing the fusion of surveying and mapping geographic multi-source data according to claim 1, characterized in that In the step S3, it is assumed that the preprocessed data source is x″ i is generated by the dictionary matrix d i and the sparse coefficient matrix α i , and the noise of the data source follows a Gaussian distribution with zero mean and variance σ 2 x i = d i α i + ∈ i . A Gaussian distribution is specified as the prior distribution for the dictionary matrix d i and the sparse coefficient matrix α i .

5. A method for processing the fusion of surveying and mapping geographical multi-source data according to claim 1, characterized in that, The optimization objective function of multi-task sparse coding in step S4 is: By optimizing this objective function, first fix the sparse coefficients of other tasks Optimize the sparse coefficient α of each task i , and use the gradient descent method to solve it.

6. The method for processing and fusing multi-source geospatial data according to claim 1, characterized in that, The graph model in step S5 is: G = (V, E) where V represents the data source nodes and E represents the connection relationships between the nodes.

7. A method for processing the fusion of surveying and mapping geographical multi-source data according to claim 1, characterized in that, The calculation formula for the weight matrix W in step S5 is: where, w ij is each element of the weight matrix W, representing the similarity between the data source x″ i and the data source x″ j ; σ is a parameter controlling the similarity sensitivity between data sources, ||x″ i - x″ j || 2 represents the Euclidean distance between the data source x″ i and x″ j .

8. A method for processing and fusing multi-source geospatial data according to claim 1, characterized in that In the step S6, the objective function is min Z trace(Z), which includes a sparse representation error term and a graph regularization term. In the form of the objective function, Z is a positive semi-definite matrix, and z ij represents the sparse coefficient α i and α j the correlation between them. The regularization term z in the objective function ij is set to the square of the Euclidean distance between the sparse coefficients

Images