A surveying and mapping results management method and system based on big data

Abstract

The present invention provides a surveying and mapping achievement management method and system based on big data, the method comprising: using unmanned aerial vehicles, vehicle-mounted laser radars and sensors to acquire geospatial surveying and mapping data, and recording the time, location and equipment information of the surveying and mapping data in real time; classifying the surveying and mapping data according to geographical location, acquisition time and data type based on an online linear optimization regularizer, and uploading the classified surveying and mapping data to a distributed storage system; performing data cleaning, format unification and quality verification on the stored surveying and mapping data to generate standardized surveying and mapping data; constructing a knowledge base based on a large language model (LLM), performing semantic analysis and feature annotation on the standardized surveying and mapping data using the geospatial operators in the knowledge base, and establishing a spatial index for the standardized surveying and mapping data; receiving a user search request, and performing a privacy protection search on the standardized surveying and mapping data based on the spatial index.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping technology, and in particular to a surveying and mapping results management method and system based on big data. Background Art

[0002] With the rapid development of 3S (GIS, RS, GPS), drones, lidar and other technologies, geographic information systems and surveying and mapping industries have gradually become the core support for the development of the digital economy. However, traditional surveying and mapping technologies have many bottlenecks in results management, including low data management efficiency, cumbersome file filing, waste of storage space and difficulty in data retrieval.

[0003] The current surveying and mapping data management process mainly has the following problems:

[0004] 1. Information is scattered and difficult to integrate: Due to the lack of unified standards for surveying and mapping data from different sources and formats, information integration is difficult, resulting in low data utilization;

[0005] 2. Single results management model: The organization and archiving of surveying and mapping results mainly rely on manual operations, which are complicated and prone to errors.

[0006] Therefore, there is an urgent need to provide a surveying and mapping results management method and system based on big data to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide a surveying and mapping results management method and system based on big data, aiming to solve the problem of low efficiency in surveying and mapping data management in the prior art.

[0008] To achieve the above purpose, the technical solution provided by the present invention is:

[0009] In the first aspect, a surveying and mapping achievement management method based on big data is provided, which is used to realize the archiving and access management of surveying and mapping achievements, including:

[0010] Use drones, vehicle-mounted lidar and sensors to acquire geospatial mapping data, and record the time, location and device information of said mapping data in real time;

[0011] Based on an online linear optimization regularizer, the surveying and mapping data are classified according to geographical location, collection time and data type, and the classified surveying and mapping data are uploaded to a distributed storage system;

[0012] Performing data cleaning, format unification and quality verification on the stored surveying and mapping data to generate standardized surveying and mapping data;

[0013] Constructing a knowledge base based on a large language model (LLM), using geospatial operators in the knowledge base to perform semantic analysis and feature annotation on the standardized surveying and mapping data, and establishing a spatial index of the standardized surveying and mapping data;

[0014] A user search request is received, and based on the spatial index, a privacy protection search is performed on the standardized surveying and mapping data.

[0015] Optionally, the regularizer based on online linear optimization classifies the surveying and mapping data according to geographical location, collection time and data type, including:

[0016] Based on the spatiotemporal distribution characteristics of the surveying and mapping data, constructing an edge geodetic set, wherein the edge geodetic set includes measurement point coordinates, acquisition time, and data type information;

[0017] Determining a regularization optimization objective function, wherein the objective function is used to evaluate data classification accuracy and computing resource consumption;

[0018] Set regularization constraints, including spatiotemporal consistency constraints and classification boundary constraints;

[0019] Based on the regularization constraint conditions, the regularization optimization objective function is solved through online optimization to obtain the optimal classification parameters, and the edge geodetic set is classified according to the optimal classification parameters.

[0020] Optionally, solving the regularized optimization objective function through online optimization includes:

[0021] Calculating a weight coefficient of each measuring point in the edge geodetic set, wherein the weight coefficient is related to the spatial distribution density and time correlation of the measuring points;

[0022] A regularization term is constructed based on the weight coefficient to prevent overfitting;

[0023] Iteratively optimizing the objective function using a gradient descent method until convergence or a preset number of iterations is reached;

[0024] After the iterative optimization of the objective function is completed, the optimal classification parameters are obtained so as to classify the edge geodetic set in real time according to the optimal classification parameters.

[0025] Optionally, the building of a knowledge base based on a large language model LLM includes:

[0026] Select a pre-trained large language model and collect professional documents in the geospatial field as training corpus;

[0027] Using the training corpus to fine-tune the pre-trained large language model to obtain a language model adapted to the geospatial domain;

[0028] Automatically extracting geographic information based on the language model, the geographic information including definitions, parameter descriptions, and usage examples of geospatial operators;

[0029] An operator function index is established, and the extracted geographic information is stored in the knowledge base according to functional categories.

[0030] Optionally, the using of geospatial operators in the knowledge base to perform semantic analysis and feature annotation on the standardized surveying and mapping data includes:

[0031] Retrieving an operator function associated with the standardized mapping data from the knowledge base;

[0032] Calling the relevant operator function to process the standardized surveying and mapping data to obtain a data feature vector;

[0033] Perform semantic clustering based on the data feature vector to generate semantic labels;

[0034] The semantic tags are associated with the standardized surveying and mapping data to construct a semantically enhanced spatial index.

[0035] Optionally, performing privacy protection retrieval on the standardized surveying and mapping data includes:

[0036] The differential privacy parameter ε is set according to the data sensitivity to generate corresponding noise perturbations for different categories of surveying and mapping data;

[0037] Building a Matroid constraint set based on system resource constraints, wherein the system resource constraints include computing resource constraints, storage resource constraints, and bandwidth resource constraints;

[0038] Combining the differential privacy mechanism and Matroid constraint optimization method, a privacy-preserving retrieval strategy is constructed;

[0039] The user's search request is processed according to the privacy protection search strategy, and the search results that meet the privacy protection requirements are returned.

[0040] Optionally, the differential privacy mechanism and the Matroid constraint optimization method are combined to construct a privacy protection retrieval strategy, including:

[0041] Build a private learning index and perform differential privacy processing on search keywords;

[0042] Transform the system resource allocation problem into a submodule optimization problem based on Matroid constraints;

[0043] A greedy algorithm is used to solve the submodule optimization problem and obtain a resource allocation solution;

[0044] The privacy-preserving retrieval strategy is constructed based on the resource allocation scheme and the privatized learning index to perform data retrieval.

[0045] Optionally, constructing the privacy protection retrieval strategy based on the resource allocation scheme and the privatized learning index to perform data retrieval includes:

[0046] Receive user's search request, extract search conditions, and generate search tasks;

[0047] Matching in a private learning index according to the search criteria;

[0048] Scheduling system resources to perform the retrieval task according to the resource allocation plan;

[0049] The retrieval results are processed with differential privacy and then returned to the user.

[0050] Optionally, the method further comprises:

[0051] Evaluate the privacy protection level of search results;

[0052] Dynamically adjust the differential privacy parameter ε and Matroid constraints based on the evaluation results;

[0053] Record retrieval logs, including resource usage and privacy protection effects;

[0054] The retrieval strategy is optimized based on the retrieval log.

[0055] In a second aspect, the present invention also provides a surveying and mapping results management system based on big data, which is used to realize the archiving and access management of surveying and mapping results, and the system comprises:

[0056] An acquisition module is used to acquire geospatial mapping data using drones, vehicle-mounted laser radars and sensors, and to record the time, location and device information of the mapping data in real time;

[0057] A classification module, which is used to classify the surveying and mapping data according to geographical location, collection time and data type based on an online linear optimization regularizer, and upload the classified surveying and mapping data to a distributed storage system;

[0058] A generation module, used to perform data cleaning, format unification and quality verification on the stored surveying and mapping data to generate standardized surveying and mapping data;

[0059] A semantic analysis and feature annotation module, which is used to construct a knowledge base based on a large language model (LLM), perform semantic analysis and feature annotation on the standardized surveying and mapping data using the geospatial operators in the knowledge base, and establish a spatial index for the standardized surveying and mapping data;

[0060] The retrieval module is used to receive a user retrieval request and perform a privacy protection retrieval on the standardized surveying and mapping data based on the spatial index.

[0061] The beneficial effects of the present invention are:

[0062] 1. By introducing an online linear optimization regularizer for data classification, the management efficiency of surveying and mapping data is improved;

[0063] 2. Using a knowledge base based on a large language model (LLM) for semantic analysis and feature annotation improves the accuracy of data retrieval;

[0064] 3. Combining the differential privacy mechanism and Matroid constraint optimization method for retrieval, it protects data privacy while ensuring retrieval efficiency;

[0065] 4. It realizes the automated and intelligent management of the entire life cycle of surveying and mapping results, significantly reducing the manual operation costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 A flowchart of a surveying and mapping results management method provided by an embodiment of the present invention;

[0068] Figure 2 A flowchart of data classification based on an online linear optimization regularizer in an embodiment of the present invention;

[0069] Figure 3 A schematic diagram of the edge geodetic set construction process in an embodiment of the present invention;

[0070] Figure 4 A flowchart of online optimization for solving the regularized optimization objective function in an embodiment of the present invention;

[0071] Figure 5 A flowchart for constructing a knowledge base based on a large language model LLM in an embodiment of the present invention;

[0072] Figure 6A flowchart of semantic analysis and feature annotation in an embodiment of the present invention;

[0073] Figure 7 A flowchart of a privacy protection retrieval in an embodiment of the present invention;

[0074] Figure 8 It is a flow chart of the combination of the differential privacy mechanism and the Matroid constraint optimization method in an embodiment of the present invention;

[0075] Fig. 9 A schematic diagram of the retrieval strategy construction process in an embodiment of the present invention;

[0076] Fig.10 A schematic diagram of the structure of the surveying and mapping results management system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0078] The embodiment of the present invention provides a surveying and mapping results management method and system based on big data, which is used to realize the archiving and reference management of surveying and mapping results. Figure 1 , the method comprises the following steps:

[0079] Step S10: Use drones, vehicle-mounted laser radars and sensors to obtain geospatial mapping data, and record the time, location and equipment information of the mapping data in real time.

[0080] Specifically, the collection of geospatial mapping data is achieved through the following methods:

[0081] 1) Equip drones with high-precision cameras and lidar equipment to collect terrain data;

[0082] 2) Use vehicle-mounted LiDAR systems to measure roads and buildings;

[0083] 3) Deploy a ground sensor network to collect local refined data.

[0084] During the collection process, the system automatically records information such as the collection time, geographic coordinates, and device model used for each data point.

[0085] Step S20: Based on the online linear optimization regularizer, the surveying and mapping data are classified according to geographical location, collection time and data type, and the classified surveying and mapping data are uploaded to the distributed storage system.

[0086] like Figure 2 As shown, the step includes:

[0087] S21: Based on the spatiotemporal distribution characteristics of the surveying and mapping data, construct an edge geodetic set, wherein the edge geodetic set includes measurement point coordinates, acquisition time, and data type information.

[0088] In S21, the construction of the edge geodetic set is the basis for the classification of surveying and mapping data. The system first performs a projection transformation on the original coordinates of each measuring point and unifies them into the local coordinate system. In order to establish the spatial correlation between the measuring points, the system uses the K nearest neighbor algorithm to calculate the neighborhood characteristics of each measuring point. At the same time, the system converts the acquisition time into a unified timestamp format and calculates the time interval between adjacent measuring points. For the marking of data types, the system has established a multi-level classification system including terrain, building, road, water system, etc. In addition, the system will also record auxiliary information such as the type of acquisition equipment and accuracy level of each measuring point, which will be used for subsequent weight calculations.

[0089] like Figure 3 As shown in Figure 2, the construction process of the edge geodetic set includes:

[0090] 1) Extract the three-dimensional coordinate information of the measurement point;

[0091] 2) Record the acquisition timestamp of each measurement point;

[0092] 3) Mark data types (such as terrain, buildings, roads, etc.);

[0093] 4) Establish spatial correlation between measurement points.

[0094] S22: Determine a regularized optimization objective function, where the objective function is used to evaluate data classification accuracy and computing resource consumption.

[0095] In S22, the design of the regularized optimization objective function adopts the multi-objective optimization concept. The classification accuracy evaluation item uses the cross entropy loss function to measure the accuracy of the classification result; the computing resource consumption evaluation item considers factors such as CPU usage, memory usage and storage space, and uses a weighted summation method for comprehensive evaluation; the spatiotemporal consistency constraint item is constructed by calculating the category difference and time continuity of adjacent measurement points. The final expression of the objective function is:

[0096] F = α * L_classification + β * L_resource + γ * L_consistency

[0097] Among them, α, β, and γ are weight coefficients, and the optimal values ​​are determined by cross-validation method.

[0098] The design of the objective function takes the following factors into consideration:

[0099] 1) Classification accuracy evaluation item;

[0100] 2) Computational resource consumption evaluation items;

[0101] 3) Space-time consistency constraints.

[0102] S23: Set regularization constraints, including spatiotemporal consistency constraints and classification boundary constraints.

[0103] In S23, the setting of regularization constraints ensures the physical rationality of the classification results. The spatiotemporal consistency constraint requires that the categories of adjacent measurement points should be continuous in space to avoid isolated classification results; the classification boundary constraint stipulates that the transition area between different categories should be smooth to prevent significant classification breaks. Specifically, the system sets the following constraints:

[0104] The category difference between adjacent measurement points should not exceed the preset threshold;

[0105] The category changes in the time series should conform to the actual observation rules;

[0106] The classification results in the same area should satisfy the uniformity of spatial distribution;

[0107] The curvature of class boundaries should change smoothly.

[0108] S24: Based on the regularization constraint conditions, the regularization optimization objective function is solved through online optimization to obtain the optimal classification parameters, and the edge geodetic set is classified according to the optimal classification parameters.

[0109] In S24, the online optimization solution process adopts an improved stochastic gradient descent algorithm. First, the system calculates the weight coefficient based on the spatial distribution density of the measurement points. The denser the area, the higher the weight. The time correlation is quantified by calculating the autocorrelation function of the time series of the measurement points. The construction of the regularization term adopts a combination of the L1 norm and the L2 norm, which not only ensures the sparsity of the model but also prevents overfitting. During the optimization process, the system adopts an adaptive learning rate strategy to dynamically adjust the step size according to the convergence of the objective function. When the change in the objective function value for multiple consecutive iterations is less than the preset threshold or reaches the maximum number of iterations, the optimization process ends. The optimal classification parameters finally obtained include the weight coefficients and classification thresholds of each feature dimension, and the system classifies the data in the edge geodetic set in real time based on this.

[0110] For example, in a comprehensive surveying and mapping mission in a certain city, the system collected multiple types of data including building outlines, road networks, and terrain elevations. For a survey area of ​​about 2 square kilometers, the drone conducted aerial surveys between 9 and 11 a.m. and obtained about 50GB of oblique photography data; at the same time, the vehicle-mounted lidar conducted supplementary measurements on the ground and obtained about 30GB of point cloud data. The system first divides the survey area into 100-meter-by-100-meter grid cells based on geographic location, and then organizes the data in time series within each grid according to the data collection time, and finally classifies and organizes it according to the data type (image, point cloud, vector, etc.). After regularization, the raw data is efficiently organized and stored in a distributed file system for subsequent processing and call-up.

[0111] like Figure 4 As shown in Figure 2, the process of solving the objective function includes:

[0112] 1) calculating the weight coefficient of each measuring point in the edge geodetic set, wherein the weight coefficient is related to the spatial distribution density and time correlation of the measuring points;

[0113] 2) constructing a regularization term based on the weight coefficient to prevent overfitting;

[0114] 3) Iteratively optimizing the objective function using a gradient descent method until convergence or a preset number of iterations is reached;

[0115] 4) After the iterative optimization of the objective function is completed, the optimal classification parameters are obtained so as to classify the edge geodetic set in real time according to the optimal classification parameters.

[0116] The above process is described in detail as follows:

[0117] In the process of solving the objective function, it is first necessary to calculate the weight coefficient of each measurement point in the edge geodetic set. For each measurement point, the system uses the kernel density estimation method to calculate its local spatial distribution density. Specifically, with the current measurement point as the center, set the search radius R (usually 5%-10% of the measurement area scale), count the number of measurement points within the range, and use the Gaussian kernel function for density weighting. At the same time, the system calculates the temporal autocorrelation coefficient of the measurement point to evaluate its importance in the time series. The final weight coefficient is obtained by a weighted combination of spatial density and temporal correlation, that is:

[0118] w_i = α * D_spatial(i) + (1-α) * C_temporal(i),

[0119] Where α is the balance factor, which is usually 0.6-0.8.

[0120] After obtaining the weight coefficients, the system constructs a regularization term to prevent the model from overfitting. The regularization term takes the form of an elastic network (Elastic Net), combining the advantages of the L1 norm and the L2 norm. The L1 norm term promotes the sparsity of the model and can automatically select features; the L2 norm term ensures the stability of the model and avoids excessive parameter values. The expression of the regularization term is:

[0121] R(θ) = λ 1 ||θ|| 1 + λ2||θ|| 2 2 ,

[0122] where λ 1 and λ 2 is a hyperparameter that controls the strength of regularization, and the optimal value is determined by cross-validation. The typical range is λ 1 ∈[0.01, 0.1],λ 2 ∈[0.001, 0.01].

[0123] During the optimization process, the system uses stochastic gradient descent with momentum term for iteration. Each iteration first randomly selects a mini-batch of samples and calculates the gradient of the loss function with respect to the model parameters. The momentum term records the historical gradient information, which can accelerate convergence and help escape the local optimal solution. Specifically, the parameter update formula is:

[0124] v_t = μv_{t-1} + η∇L(θ), θ_t = θ_{t-1} - v_t,

[0125] Where μ is the momentum coefficient (usually 0.9), and η is the learning rate. The learning rate is dynamically adjusted using the cosine annealing strategy, with the initial value set to 0.01, and gradually decreases as the number of iterations increases. When the relative change rate of the loss function for 10 consecutive iterations is less than 10^-6, the optimization is considered to have converged; or when the number of iterations reaches the preset maximum value (such as 1000 times), the optimization is terminated.

[0126] After the optimization is completed, the system obtains the optimal classification parameters, including the feature transformation matrix W and the classification threshold vector b. For each new data point x in the edge geodetic set, the system calculates its feature vector f = Wx and obtains the probability distribution of each category through the softmax function:

[0127] p_k = exp(f_k + b_k) / Σexp(f_i + b_i).

[0128] The category with the highest probability is selected as the classification result of the measurement point. In order to improve the efficiency of real-time processing, the system adopts a streaming computing framework to organize data points into data blocks according to spatial indexes to achieve parallel processing. In addition, the system also maintains a parameter cache pool to regularly update the optimal classification parameters to ensure the timeliness and accuracy of the classification results.

[0129] Step S30: performing data cleaning, format unification and quality verification on the stored surveying and mapping data to generate standardized surveying and mapping data.

[0130] The specific implementation of this step includes:

[0131] 1) Remove outliers and duplicate data;

[0132] 2) Unify data format and coordinate system;

[0133] 3) Conduct data integrity and accuracy checks;

[0134] 4) Generate surveying and mapping datasets that meet standards.

[0135] Step S40: construct a knowledge base based on a large language model (LLM), use the geospatial operators in the knowledge base to perform semantic analysis and feature annotation on the standardized surveying and mapping data, and establish a spatial index for the standardized surveying and mapping data.

[0136] like Figure 5 As shown, the step includes:

[0137] S41: Select a pre-trained large language model and collect professional documents in the field of geospatial as training corpus;

[0138] In S41, the system first selects a pre-trained large language model as the basic model. Specifically, the system uses a Transformer architecture model with a parameter volume of 100B, which has been pre-trained on a large amount of general text including Wikipedia, academic papers, etc. For the collection of professional documents in the field of geospatial, the system has established a multi-source data acquisition mechanism: extracting professional terms and technical procedures from the standard specification documents of the surveying and mapping industry; obtaining the definition and usage of spatial operators from the API documentation of GIS software; extracting practical cases and application scenarios from surveying and mapping project reports. After format unification and quality filtering, these documents form a professional field training corpus of about 50GB. At the same time, the system also constructs a geospatial knowledge graph, which contains more than 100,000 concept nodes and 500,000 relationship edges to enhance the model's domain knowledge understanding ability.

[0139] S42: fine-tuning the pre-trained large language model using the training corpus to obtain a language model adapted to the geographic space domain;

[0140] In S42, the system uses a hierarchical fine-tuning strategy to adaptively train the pre-trained model. First, middle-tuning is performed, using general documents in the geospatial field to train the model to understand professional terms and basic concepts; then task-tuning is performed, focusing on optimizing specific tasks such as spatial operator extraction and semantic understanding. During the fine-tuning process, a progressive learning rate strategy is adopted. The underlying parameters use a smaller learning rate (10^-5) to maintain basic language understanding capabilities, and the high-level parameters use a larger learning rate (10^-4) to accelerate task adaptation. To prevent catastrophic forgetting, a knowledge distillation mechanism is introduced, and the original pre-trained model is used as a teacher model to guide the fine-tuning process. In addition, the system also designs a special loss function, which includes three parts: domain vocabulary understanding loss, spatial relationship reasoning loss, and operator semantic consistency loss. The fine-tuning process is carried out on 8 GPUs, using mixed precision training, with a single round of training time of about 24 hours, and a total of 50 rounds of training until the performance of the validation set converges.

[0141] The following are detailed definitions so that those skilled in the art can better understand:

[0142] Overview of Fine-tuning Strategy The system adopts a hierarchical fine-tuning strategy, which is divided into two main stages: intermediate-level fine-tuning and task-level fine-tuning. This hierarchical approach allows the model to gradually adapt to the specific needs of the geospatial domain.

[0143] In this stage, the system is trained using common documents in the field of geospatial information. For example, using surveying and mapping textbooks, GIS manuals and other documents, the model is taught professional terms such as "contours", "projection transformation", and "spatial reference". This is like asking a general translator to first focus on learning professional vocabulary in the field of surveying and mapping to establish basic cognition.

[0144] The second stage of the task-level fine-tuning phase is to optimize specific tasks. For example, the model is trained to recognize spatial operation instructions such as "calculate the distance between two points" and "generate a buffer zone", and to understand complex query statements such as "find all buildings within 500 meters of a certain point". This is equivalent to further learning how to solve practical problems on the basis of mastering professional vocabulary.

[0145] The learning rate strategy system uses differentiated learning rate settings. The underlying parameters use a smaller learning rate of 10^-5, which can maintain the model's original language understanding ability; the high-level parameters use a larger learning rate of 10^-4 to speed up the model's adaptation to new tasks. This is like learning a new skill, which requires both maintaining the basics and quickly mastering new knowledge.

[0146] The system introduces a knowledge distillation mechanism to prevent catastrophic forgetting, using the original pre-trained model as a teacher model. This is like having an experienced mentor by your side to guide you when learning new knowledge, ensuring that you do not forget the basic knowledge you have mastered.

[0147] The loss function design system designs a loss function with three parts:

[0148] Domain vocabulary understanding loss: evaluates the model’s mastery of surveying and mapping terminology

[0149] Spatial relationship reasoning loss: measures the model’s ability to understand spatial location relationships

[0150] Operator semantic consistency loss: Test the accuracy of the model’s understanding of spatial analysis operations

[0151] Training implementation details

[0152] The actual training was performed on 8 GPUs, using mixed precision training to improve efficiency. Each round of training took about 24 hours, and a total of 50 rounds of training were conducted. This is equivalent to about 50 days of intensive learning for the model, and the training was terminated only when the performance on the validation set no longer improved significantly.

[0153] In the model training implementation phase, the system uses high-specification hardware configuration and sophisticated training strategies. The training platform is equipped with 8 GPUs, each with 80GB of video memory, and achieves efficient data transmission between GPUs. The server is also equipped with a high-performance CPU and large-capacity memory, providing sufficient computing resources for data preprocessing and model management.

[0154] In order to improve training efficiency, the system implements a mixed precision training strategy of FP16 and FP32. Under this strategy, key calculations such as the attention layer and normalization layer use FP32 precision to ensure calculation stability, while other layers use FP16 precision to increase training speed and optimize video memory usage. For example, during forward propagation, matrix multiplication operations use FP16 precision, while gradient calculations and parameter updates use FP32 precision. At the same time, a dynamic loss scaling mechanism is introduced to prevent numerical underflow problems that may occur in FP16 calculations.

[0155] In terms of training batch settings, the system configures a batch size of 32 for each GPU and uses gradient accumulation technology to update parameters every 4 cumulative steps. This setting makes the actual equivalent batch size reach 1024 (32 * 8 * 4), which not only ensures the stability of training but also improves computing efficiency. At the same time, the system monitors various key indicators in the training process in real time, including training loss, verification loss, accuracy of each subtask, GPU resource utilization, and the changing trends of learning rate and gradient.

[0156] In terms of training time arrangement, each round of training takes about 24 hours, including 2 hours of data processing, 20 hours of model training, 1.5 hours of validation evaluation, and 0.5 hours of model checkpoint preservation. The entire training process lasts for 50 rounds and takes about 50 days. The system performs a complete validation set evaluation every 10 rounds to ensure the effectiveness of the training. At the same time, a strict checkpoint preservation strategy is adopted. A checkpoint is saved at the end of each round of training, retaining the 5 most recent checkpoints and the 3 checkpoints with the best validation set performance. These checkpoints contain not only model parameters, but also optimizer status, training progress information, and evaluation indicator records.

[0157] The system sets multiple training termination conditions, including no significant improvement in the validation set performance for five consecutive rounds, reaching the preset upper limit of 50 rounds of training, or the validation set accuracy reaching the preset target of 95%. In order to further optimize the training performance, the system has taken optimization measures in data loading, calculation process and memory usage. In terms of data preloading, multi-process data loading and pre-fetch cache mechanism are used; in terms of calculation optimization, gradient checkpoint optimization and attention calculation optimization are implemented; in terms of memory optimization, technologies such as gradient accumulation and video memory reuse are adopted.

[0158] Through this complete training implementation plan, the system can efficiently complete the model fine-tuning work, while ensuring the training quality and maximizing the utilization of hardware resources. After each round of training, the system will generate a detailed training report, recording various performance indicators, resource usage and optimization suggestions, providing an important reference for subsequent training optimization. This refined training management ensures that the model can be steadily iterated and optimized towards the expected goal, and ultimately achieve the expected training effect.

[0159] S43: Automatically extracting geographic information based on the language model, the geographic information including definitions, parameter descriptions, and usage examples of geospatial operators;

[0160] In S43, the language model automatically extracts geographic information based on the attention mechanism and semantic understanding. The system designs a multi-level extraction framework: first, named entity recognition technology is used to identify key information such as spatial operator names and parameter names in the text; then semantic role labeling technology is used to analyze the input-output relationship and parameter constraints of the operator; finally, the call dependency and combination pattern between operators are mined through relationship extraction technology. To improve the accuracy of extraction, the system introduces a rule-based post-processing mechanism, including professional terminology standardization, parameter type verification, and example validity verification. At the same time, the system also establishes a quality assessment mechanism for the operator knowledge base, which ensures that the accuracy of extracted information reaches more than 95% through a combination of cross-validation and manual spot checks. For complex spatial analysis operators, the system also automatically generates a visual calculation flow chart to help understand the calculation process and intermediate results.

[0161] S44: Establish an operator function index and store the extracted geographic information in the knowledge base according to functional categories.

[0162] In S44, the system uses a multi-dimensional index structure to organize the operator function library. First, a hierarchical directory structure is established according to the functional category, including spatial query category, topological analysis category, spatial statistics category and other major categories; then, a multi-level index is established under each category according to the attributes such as operation complexity, applicable scenarios, and data scale. The index structure is implemented using a B+ tree to support efficient operator search and update operations. At the same time, the system establishes a semantic vector space for operators, and each operator represents its functional characteristics through a 300-dimensional dense vector, supporting fuzzy queries based on semantic similarity. In addition, the system also maintains the association relationship network between operators, and records commonly used operator combination patterns and best practice cases. In order to support version management and evolution tracking of operators, the system implements a version control mechanism based on Git, and each operator update will generate a new version number and change log. The access interface of the knowledge base adopts a RESTful architecture, supports multiple forms of query and retrieval operations, and optimizes the access performance of hot operators through a cache mechanism.

[0163] For example, in the geographic surveying and mapping work of a smart city project, the system needs to process and analyze a variety of surveying and mapping data including buildings, road networks, terrain features, etc. The following is the specific processing process:

[0164] 1. Knowledge base construction example:

[0165] The system selected GPT-4 as the basic language model and collected about 20GB of professional documents including "Surveying and Mapping Specifications", "GIS Technical Manual", "Spatial Analysis Methods" as training corpus. After 3 weeks of fine-tuning training, the model's accuracy in understanding professional terms in the field of geospatial space reached 92%. For example, the model can accurately understand professional concepts such as "buffer analysis", "spatial interpolation", "Thyssen polygons", and master their operation rules.

[0166] 2. Semantic analysis example:

[0167] For the survey data of a newly built high-rise building, the system first calls the "Building Feature Extraction" operator from the knowledge base. The operator processes the data through the following steps:

[0168] Extract the geometric features of the building (height, area, volume, etc.);

[0169] Analyze the structural characteristics of the building (number of floors, type of facade, etc.);

[0170] Identify the functional attributes of the building (residential, commercial, office, etc.);

[0171] Calculate the spatial relationship between the building and its surroundings;

[0172] 3. Feature annotation case:

[0173] When processing a city’s road network data, the system performed the following annotation process:

[0174] Basic attribute annotation of road section:

[0175] Road grade: main road;

[0176] Pavement type: asphalt concrete;

[0177] Design speed: 60km / h;

[0178] Direction of traffic: two-way traffic;

[0179] Road network topology relationship annotation:

[0180] Connection relationship with intersecting roads;

[0181] Locations of important facilities along the route;

[0182] Traffic control information;

[0183] Spatial and temporal feature annotation:

[0184] Peak hour capacity;

[0185] seasonal usage characteristics;

[0186] Historical renovation records;

[0187] 4. Spatial index construction example:

[0188] Taking the surveying and mapping data of a commercial area as an example, the system constructs a multi-level spatial index:

[0189] First layer: spatial grid index (100m×100m);

[0190] The second layer: feature type index (buildings, roads, green spaces, etc.);

[0191] The third layer: semantic feature index (purpose, attributes, association relationship);

[0192] Specific application case: When a user needs to query "commercial office buildings with a height of more than 100 meters newly built in the CBD core area in 2023", the system can quickly locate the target area through spatial indexing, filter qualified buildings through semantic tags, and finally return detailed information of 10 qualified buildings within 0.5 seconds, including:

[0193] 3D models of buildings;

[0194] Basic attribute data (building area, floor height, construction time, etc.);

[0195] Supporting facilities information;

[0196] Surrounding environment analysis;

[0197] At the same time, the system also automatically generates a heat map of the spatial distribution of these buildings, which intuitively shows their distribution characteristics in the urban space. The surveying and mapping data of these buildings are provided to users after differential privacy processing, which not only ensures the availability of data, but also ensures the security of sensitive information (such as internal structure, security facilities, etc.).

[0198] This intelligent semantic analysis and annotation system significantly improves the management efficiency and use value of surveying and mapping data. For example, under the traditional manual processing method, it takes 3-5 working days to complete the annotation and indexing of surveying and mapping data of a commercial area. After adopting this system, the same workload can be completed in 4-6 hours, the accuracy rate is improved by 15%, and the retrieval efficiency is improved by more than 300%.

[0199] This processing flow is not only applicable to building data, but can also be extended to other types of surveying and mapping data, such as municipal facilities, underground pipelines, green space systems, etc., providing a reliable data foundation for smart city construction.

[0200] like Figure 6 As shown, in S40, the process of using the knowledge base for semantic analysis and feature annotation includes:

[0201] 1) retrieving an operator function related to the standardized surveying and mapping data from the knowledge base;

[0202] 2) calling the relevant operator function to process the standardized surveying and mapping data to obtain a data feature vector;

[0203] 3) Performing semantic clustering based on the data feature vector to generate semantic labels;

[0204] 4) Associating the semantic tags with the standardized surveying and mapping data to construct a semantically enhanced spatial index.

[0205] The steps 1)-4) are discussed in detail below:

[0206] Retrieving relevant operator functions from the knowledge base is the first step in data processing. The system adopts a multi-strategy retrieval mechanism. First, operators are preliminarily screened by matching data types. For example, for point cloud data, the system will prioritize the set of operator functions related to point cloud processing. Then, deep matching is performed based on data features, and the most relevant operator is selected by calculating the semantic similarity between data attributes and operator function descriptions. The system also considers the frequency of use and historical performance of operators, and builds an operator importance scoring mechanism based on the PageRank algorithm. For example, for building outline data, the system may retrieve core operators such as "building segmentation", "external contour extraction", and "height calculation", and will also find related auxiliary operators such as "shadow analysis" and "viewshed analysis".

[0207] Calling operator functions to process standardized surveying and mapping data is a key step in feature extraction. The system has established an operator call management framework to uniformly manage the operator call process through the task scheduler. For each operator, the system automatically optimizes its parameter configuration, such as dynamically adjusting the processing batch size according to the data scale and selecting the most suitable algorithm variant according to the data characteristics. During the data processing process, the system adopts a pipeline architecture to combine multiple operators into a processing chain to achieve parallel processing of data. The output of each operator is converted into a standardized feature vector, the vector dimension is usually between 128-512, and it is stored in a sparse coding manner. For example, when processing the three-dimensional model data of a building, the system will extract geometric features (such as height, area, volume, etc.), topological features (such as adjacency, connectivity, etc.) and semantic features (such as architectural functions, style features, etc.) in turn, and finally form a comprehensive feature vector.

[0208] Semantic clustering based on feature vectors is the core step in generating semantic labels. The system adopts a hierarchical clustering strategy. First, the HDBSCAN algorithm is used for density clustering to automatically determine the optimal number of clusters; then the fine-grained K-Means++ algorithm is applied to each cluster for further division. The two dimensions of spatial distance and semantic similarity are considered in the clustering process, and the distance between samples is calculated by weighted combination. In order to improve the clustering quality, the system introduces a feature importance evaluation module based on the attention mechanism to automatically learn the weights of different feature dimensions. After clustering is completed, the system uses a professional terminology library and knowledge graph to generate semantic labels for each cluster. The labels contain multiple levels, with detailed descriptions from basic categories (such as "residential buildings") to specific attributes (such as "high-rise residential buildings" and "garden houses").

[0209] The last key step is to associate semantic tags with surveying and mapping data and build a semantically enhanced spatial index. The system adopts a hybrid index structure, combining the spatial indexing capability of the R-tree and the semantic retrieval advantage of the inverted index. When building the index, the surveying and mapping data is first organized into an R-tree structure according to the spatial position, and each node contains the minimum circumscribed rectangle of all features within its coverage; then an inverted index of semantic tags is established for each leaf node to support fast retrieval based on semantics. The index structure adopts a dynamic update mechanism, which can reflect data changes in real time. The system also establishes a tag association network to record the relationship between different semantic tags, supporting semantic reasoning and similarity retrieval. For example, when querying "landmark buildings in commercial areas", the system can automatically expand to related concepts such as "high-rise office buildings" and "commercial complexes" through semantic associations, thereby improving the recall rate of retrieval. In addition, the system also maintains a cache mechanism for hot data, establishes memory indexes for frequently accessed data, and significantly improves retrieval performance.

[0210] Step S50: receiving a user search request, and performing a privacy protection search on the standardized surveying and mapping data based on the spatial index.

[0211] like Figure 7 As shown, the step includes:

[0212] S51: Setting the differential privacy parameter ε according to the data sensitivity, and generating corresponding noise perturbations for different categories of surveying and mapping data;

[0213] In S51, the setting of the differential privacy parameter ε is a dynamic optimization process. The system first performs a sensitivity analysis on the surveying and mapping data, and divides the data into multiple sensitivity levels according to the data type, usage scenario, and security level. For example, the sensitivity of ordinary terrain data is low, and a larger ε value (such as ε=1.0) can be set; while the surveying and mapping data around military facilities is more sensitive, and a smaller ε value (such as ε=0.1) needs to be set. For each type of data, the system uses an adaptive noise generation mechanism: for numerical attributes, Laplace distribution is used to generate noise, and the noise size is inversely proportional to the ε value; for categorical attributes, exponential distribution is used to generate noise. The system also establishes a hierarchical strategy for noise injection, using different perturbation intensities for different dimensions of the data to ensure that the availability of the data is retained to the greatest extent while protecting privacy. For example, for building data, large noise may be added to specific coordinates, while only slight perturbations are added to the basic attributes of the building (such as the number of floors, purpose, etc.).

[0214] S52: constructing a Matroid constraint set based on system resource constraints, wherein the system resource constraints include computing resource constraints, storage resource constraints, and bandwidth resource constraints;

[0215] In S52, the construction of Matroid constraint sets takes into account the multi-dimensional resource constraints of the system. In terms of computing resource constraints, the system monitors indicators such as CPU utilization, memory usage, and GPU load to establish upper limit constraints on resource usage; in terms of storage resource constraints, factors such as disk capacity, I / O bandwidth, and cache space are considered to build efficiency constraints for storage access; in terms of network bandwidth constraints, parameters such as data transmission rate, number of connections, and response time are monitored to set capacity constraints for network communication. These constraints are formally described through Matroid mathematical structures to ensure the feasibility of resource allocation schemes. The system uses a dynamic programming algorithm to calculate the set of feasible solutions that meet all constraints, and selects the optimal solution through a greedy strategy. For example, when the system load is high, the resource quotas of some non-critical queries may be reduced to ensure the normal execution of important tasks.

[0216] S53: Combining differential privacy mechanism and Matroid constraint optimization method to build a privacy-preserving retrieval strategy;

[0217] In S53, the construction of privacy-preserving retrieval strategy adopts a two-layer optimization framework. At the differential privacy level, the system uses an exponential mechanism to design a query perturbation strategy to randomize query conditions and result sets; at the Matroid constraint level, the resource allocation problem is transformed into a submodular optimization problem, and the optimal resource allocation solution is obtained through iterative solution. The system also introduces a query rewriting mechanism to decompose complex queries into multiple subqueries, each of which independently applies differential privacy protection, and finally merges the results in a secure combination. In order to handle continuous query scenarios, the system implements a budget allocation mechanism to reasonably allocate privacy budgets and avoid privacy leaks caused by multiple queries. For example, for continuous queries of the same user, the system will dynamically adjust the ε value of each query to ensure that the overall privacy protection strength remains unchanged.

[0218] S54: Process the user's search request according to the privacy protection search strategy, and return the search results that meet the privacy protection requirements.

[0219] In S54, the processing of retrieval requests adopts a multi-stage pipeline architecture. First, a security audit is performed on the user request to verify the user's authority and the legitimacy of the query; then the query conditions are converted according to the privacy protection policy and necessary noise is added; then the system resources are scheduled to execute the query according to the resource allocation plan; finally, the query results are post-processed, including data desensitization, format conversion, and quality assessment. The system optimizes the access performance of hot data through a caching mechanism and establishes pre-calculated indexes for frequently queried data sets. In order to ensure the availability of query results, the system also implements a result verification mechanism to evaluate the accuracy and consistency of noise data through statistical analysis methods. For example, when querying the building density of a certain area, the system not only returns the density value after adding noise, but also provides a credible interval to help users understand the uncertainty range of the data.

[0220] The above four steps form a complete privacy-preserving retrieval framework that can protect data privacy while ensuring the efficiency and availability of retrieval services. During the operation of the entire framework, the system continuously collects performance indicators and user feedback, optimizes parameter configurations through machine learning methods, and achieves continuous improvement of retrieval services. For example, the system may find that the noise setting of a certain type of query is too large, resulting in distorted results, and it will automatically adjust the relevant parameters to improve data accuracy while ensuring privacy protection.

[0221] Taking the underground pipe network surveying and mapping data retrieval of a smart city as an example, the system implements a comprehensive privacy protection retrieval mechanism. A staff member of the municipal planning department needs to query the distribution of underground pipe networks in a specific area, including information such as pipe network type, burial depth, and distribution density. This type of data has high security requirements because it not only involves the security of urban infrastructure, but also may involve the location information of important surrounding facilities.

[0222] In response to this retrieval requirement, the system first sets differential privacy parameters based on data sensitivity. For general network type information (such as water supply, drainage, gas, etc.), the system sets relatively loose privacy parameters (ε=0.8); for specific spatial location information, the system adopts stricter privacy protection (ε=0.2); and for network data involving important facilities, the most stringent privacy protection level (ε=0.05) is used. For example, for the location information of a trunk water supply network, the system will add a random offset of ±5 meters to ensure that the true location information is not accurately obtained.

[0223] In terms of resource scheduling, the system builds a Matroid constraint set based on the current load conditions. Assuming that the current system CPU utilization is 75%, the available memory is 30GB, and the network bandwidth margin is 200Mbps, the system sets the following constraints accordingly:

[0224] A single query takes up to 10% of CPU resources

[0225] The upper limit of memory usage is 8GB;

[0226] The data transmission rate does not exceed 50Mbps;

[0227] Based on these constraints, the system uses a greedy algorithm to calculate the optimal resource allocation plan. For example, the query task is decomposed into multiple subtasks, each of which processes a 500m x 500m grid area, to achieve parallel processing to improve efficiency.

[0228] In the actual retrieval process, the system adopts a multi-level privacy protection strategy:

[0229] 1. Spatial fuzzy processing:

[0230] Convert precise coordinates into grid code;

[0231] Add random disturbances to the locations of pipe network nodes;

[0232] Simplify the topology of the pipe network connection relationship;

[0233] 2. Attribute information protection:

[0234] Convert the specific pipe diameter value into an interval range;

[0235] Add random noise to the burial depth information;

[0236] Blurring the precise time of laying into year intervals;

[0237] 3. Statistical information processing:

[0238] Perform segmented fuzzification on the pipe network density data;

[0239] Add reasonable noise disturbance;

[0240] Provide confidence intervals rather than exact values;

[0241] For example, a query returns the following result:

[0242] Area: Eastern Business District;

[0243] Total length of pipe network: about 4500±200 meters;

[0244] Average burial depth: 2.5-3.0 meters;

[0245] Pipeline network density: medium (0.15-0.20 km / km2);

[0246] Main types: water supply, gas, electricity;

[0247] The system also implements a dynamic privacy protection effect evaluation mechanism. By monitoring query logs, the system found that the query frequency in some hot spots is high, and there may be a risk of inferring real information through multiple queries. In response to this situation, the system automatically adjusts the protection strategy:

[0248] Reduce the accuracy of a single query;

[0249] Increase the intensity of the noise disturbance;

[0250] Limit the number of queries in a short period of time;

[0251] Dynamically adjust the data desensitization level;

[0252] Through this complete privacy-preserving retrieval mechanism, the system successfully balances data availability and security. In actual applications, this mechanism has increased the retrieval efficiency of underground pipe network data by 200%, while ensuring the security of sensitive information and reducing the risk of leakage by 85%. User feedback shows that although the data accuracy has been reduced, the impact on actual work is small, and 95% of users believe that the retrieval results still meet their business needs. This retrieval scheme based on differential privacy has been successfully extended to other types of surveying and mapping data protection, such as the management of sensitive data such as terrain around military facilities and distribution of important infrastructure.

[0253] like Figure 8 As shown in Figure 2, the process of building a retrieval strategy by combining the differential privacy mechanism and the Matroid constraint optimization method includes:

[0254] 1) Build a private learning index and perform differential privacy processing on search keywords;

[0255] 2) Convert the system resource allocation problem into a submodule optimization problem based on Matroid constraints;

[0256] 3) Using a greedy algorithm to solve the submodule optimization problem and obtain a resource allocation solution;

[0257] 4) Constructing the privacy-preserving retrieval strategy based on the resource allocation scheme and the privatized learning index.

[0258] Specifically, in the stage of receiving user search requests, the system adopts a multimodal request parsing framework. First, the system uses the natural language processing module to semantically understand the user input and identify the core search intent and constraints. For example, when the user enters "find commercial buildings higher than 100 meters in the CBD area newly built in 2023", the system will identify multi-dimensional search conditions such as time conditions (2023), spatial range (CBD area), height threshold (>100 meters) and building type (commercial building). The system will also normalize the search conditions and convert non-standard expressions into a standard format within the system. At the same time, based on historical query logs and user portraits, the system will intelligently complete and optimize the search conditions, such as automatically adding commonly used filtering conditions or sorting rules. Finally, the system encapsulates these processed search conditions into standard search task objects, containing complete information such as query type, data range, filtering conditions, sorting rules, etc.

[0259] In the private learning index matching stage, the system implements a multi-level matching strategy. First, the system uses local sensitive hashing (LSH) technology to quickly locate potential matching data sets at a coarse-grained level. Then, the semantic similarity between the query conditions and the index items is calculated through the vector space model, and the most relevant index items are selected for precise matching. In order to improve the matching accuracy, the system adopts a deep learning model based on the attention mechanism, which can automatically identify the importance weights of different retrieval conditions. For example, for building queries, the weight of location information may be higher than the construction time. The system also maintains a dynamically updated synonym library and term mapping table, which can handle retrieval conditions with different expressions. During the matching process, the system will record the intermediate results and matching paths to facilitate subsequent result interpretation and optimization.

[0260] In the resource scheduling execution phase, the system adopts a distributed task scheduling framework. First, the system evaluates the resource requirements of the retrieval task, including indicators such as computational complexity, data access volume, and expected response time. Then, based on the current system load status and resource allocation plan, the system calculates the optimal task decomposition plan through a dynamic programming algorithm. For large-scale retrieval tasks, the system will decompose them into multiple subtasks for parallel execution. For example, for cross-regional spatial queries, the system will divide subtasks according to spatial grids, and each subtask is responsible for processing the data of a grid area. The system also implements an adaptive load balancing mechanism that can dynamically adjust resource allocation according to the execution situation. For abnormal situations that occur during the execution process, the system provides a fault-tolerant mechanism to ensure the reliability of the query through task retries and backup execution.

[0261] In the differential privacy processing return phase, the system implements a complete privacy protection chain. First, a sensitivity analysis is performed on the search results to identify sensitive fields and attributes that need to be protected. Then, according to the sensitivity of different fields, the system adopts a hierarchical differential privacy processing strategy. For highly sensitive numerical attributes (such as specific coordinates), the system uses the Laplace mechanism to add random noise; for categorical attributes, the system uses an exponential mechanism for randomization. The system also implements a result consistency verification mechanism to ensure that the data after adding noise still meets business constraints. For example, the height of a building should remain within a reasonable range after adding noise. Finally, the system generates a privacy protection effect report, which includes data quality assessment indicators and privacy protection strength indicators to help users understand the credibility of the results. In order to optimize the user experience, the system also provides a visual display of the results, presenting the data distribution characteristics through intuitive charts, and identifying the degree of uncertainty of the data through color coding and other methods.

[0262] The above four steps are interconnected through strict logical relationships to form a complete privacy-preserving retrieval process. The system continuously monitors and analyzes the execution of each step, continuously optimizes the processing strategy, and improves the efficiency and security of the retrieval service. For example, the system regularly analyzes user feedback and query logs to identify frequently occurring query patterns, and optimizes the index structure and resource allocation strategy accordingly to achieve continuous improvement in service quality.

[0263] like Fig. 9 As shown in Figure 1, the process of data retrieval based on the retrieval strategy includes:

[0264] 1) Receive the user's search request, extract the search conditions, and generate the search task;

[0265] In the retrieval request receiving stage, the system implements a multi-level request processing architecture. First, the user input is preprocessed by the request parser, including basic operations such as removing special characters, standardizing formats, and word segmentation. Then, a BERT-based semantic understanding model is used to convert unstructured user queries into standardized retrieval conditions. The system supports complex condition combinations. For example, for complex queries such as "find buildings within 500 meters of a key facility since January 2023 and with a building height of more than 80 meters", the system can accurately identify retrieval elements in multiple dimensions such as time conditions (since January 2023), spatial conditions (within 500 meters), and attribute conditions (height>80 meters). At the same time, the system will also verify the legitimacy of the retrieval conditions to ensure that the logical relationship between the conditions is correct, and intelligently complete some ambiguous conditions. Finally, the system generates a standardized retrieval task object containing complete information such as query type, data range, filtering conditions, and sorting rules.

[0266] 2) matching in the private learning index according to the search condition;

[0267] In the process of private learning index matching, the system adopts a multi-level index structure. The outermost layer is a spatial index based on geographic grids, which divides the entire survey area into regular grid units; the second layer is a classification index based on data types, which establishes a special index structure for different types of surveying and mapping data (such as buildings, roads, terrain, etc.); the innermost layer is an attribute index, which establishes an inverted index for specific attributes of each data type. The system uses an improved KNN algorithm for similarity matching, and selects the most relevant candidate set by calculating the cosine similarity between the retrieval conditions and the index items. In order to improve matching efficiency, the system implements a caching mechanism to cache the index items of hot data in memory. For example, for frequently queried commercial building data, the system will load its index information into memory first. In addition, the system also quickly filters out data that does not meet the conditions through Bloom filters to reduce unnecessary detailed matching calculations.

[0268] 3) Scheduling system resources to perform the retrieval task according to the resource allocation plan;

[0269] In the resource scheduling execution phase, the system establishes a priority-based task scheduling framework. First, the system evaluates the resource requirements of the retrieval task, including computing resource requirements (CPU, memory, GPU, etc.), storage resource requirements (disk I / O, cache, etc.), and network resource requirements (bandwidth, number of connections, etc.). Then, based on the current system load status, the system uses a dynamic time slice allocation algorithm for resource scheduling. For large-scale retrieval tasks, the system uses the MapReduce model to decompose tasks and divide the retrieval task into multiple subtasks that are executed in parallel. Each subtask is assigned an independent resource quota, and the system ensures that resource usage does not exceed the preset limit through real-time monitoring. At the same time, the system implements a dynamic adjustment mechanism for task priorities, so that queries that are sensitive to response time can obtain higher processing priority.

[0270] 4) The search results are processed with differential privacy and then returned to the user.

[0271] In the differential privacy processing return phase, the system implements an adaptive privacy protection mechanism. First, the system performs a sensitivity assessment on the search results, and different types of data fields are assigned different privacy protection levels. For highly sensitive location information, the system uses the ε-differential privacy mechanism to add Laplace noise, and the noise size is proportional to the sensitivity and inversely proportional to the ε value. For statistical data (such as building density, height distribution, etc.), the system uses the histogram differential privacy algorithm to achieve privacy protection while maintaining data distribution characteristics. The system also implements a consistency check mechanism for query results to ensure that the data after adding noise still meets the business logic constraints. For example, for the spatial relationship data of buildings, the system maintains the correctness of the topological relationship. Finally, the system generates a privacy protection report, which records the noise addition process and protection effect in detail, helping administrators evaluate and optimize privacy protection strategies. At the same time, the system also provides users with a visual result display interface, which intuitively presents query results through different view modes (such as map display, statistical charts, etc.), and expresses the degree of uncertainty of the data through clear visual prompts.

[0272] In addition, this embodiment also includes the evaluation and optimization process of the search results:

[0273] 1) Evaluate the privacy protection level of search results;

[0274] In the privacy protection evaluation stage, the system implements a multi-dimensional evaluation framework. First, the system evaluates the uncertainty of the data through information entropy analysis methods, and calculates the information loss between the original data and the data after adding noise. For example, for building location data, the system calculates the change in spatial distribution entropy before and after coordinate disturbance; for attribute data, the degree of change in the numerical range and distribution characteristics is analyzed. At the same time, the system also implements a privacy leakage risk assessment model based on differential privacy, which evaluates the possibility of data reconstruction by simulating the attacker's reasoning process. For example, the system simulates the scenario of inferring the true value through multiple queries, and calculates the success rate and accuracy. In addition, the system has also established user experience evaluation indicators, including the usability score of the query results, data quality satisfaction, etc., and collects feedback through a combination of questionnaires and automated testing.

[0275] 2) Dynamically adjust the differential privacy parameter ε and Matroid constraints based on the evaluation results;

[0276] In the dynamic parameter adjustment stage, the system adopts an adaptive parameter optimization strategy. Based on the evaluation results, the system uses a reinforcement learning algorithm to dynamically adjust the differential privacy parameter ε to find the best balance between protecting privacy and maintaining data availability. For example, when the query frequency of a certain type of data is abnormally high, the system will automatically reduce the corresponding ε value to increase the privacy protection strength; when it is found that the distortion of some query results is too high and affects the use, the system will appropriately relax the privacy protection parameters. For Matroid constraints, the system implements a dynamic adjustment mechanism based on load prediction. By analyzing the system resource usage trend and query pattern, predicting future resource demand, and adjusting resource allocation constraints accordingly. For example, during business peaks, the system will optimize constraints to improve resource utilization efficiency; during off-peak periods, it may relax constraints to provide better service quality.

[0277] 3) Record retrieval logs, including resource usage and privacy protection effects;

[0278] In the retrieval log recording stage, the system has established a comprehensive log management system. The complete life cycle of each retrieval request is recorded in detail, including basic information such as request time, user information, query conditions, resource consumption, execution time, and result set size. For resource usage, the system records time series data of detailed indicators such as CPU utilization, memory usage, I / O load, and network bandwidth. In terms of privacy protection effect, key indicators such as noise addition parameters, data distortion, and privacy budget consumption are recorded. The system adopts a distributed log storage architecture, efficiently manages massive log data through a time series database, and implements a full-text retrieval function based on ElasticSearch, supporting complex log analysis queries. At the same time, the system also implements a visual monitoring interface for log data, which displays the system operation status and performance indicators through real-time charts.

[0279] 4) Optimizing the retrieval strategy based on the retrieval log.

[0280] In the retrieval strategy optimization phase, the system implements a closed-loop optimization mechanism. By deeply mining the retrieval logs, the system uses machine learning algorithms to identify query patterns and resource usage patterns. For example, common query types are discovered through cluster analysis, and resource demand fluctuations are predicted through time series analysis. Based on these analysis results, the system automatically optimizes the index structure, such as establishing finer-grained indexes for frequently accessed data and pre-calculating hot query conditions. In terms of resource allocation strategy, the system adjusts the task scheduling algorithm based on historical performance data and optimizes task decomposition and parallel processing solutions. For privacy protection strategies, the system continuously adjusts the parameter configuration and noise addition strategy of the differential privacy mechanism by analyzing privacy leakage risks and user feedback. For example, the system may find that the privacy protection of certain data fields is excessive and affects normal business use, and will appropriately adjust the relevant parameters. In addition, the system has also established a policy evaluation mechanism to regularly evaluate the optimization effect and generate optimization recommendation reports to help administrators make decisions.

[0281] Through the continuous cycle of these four stages, the system forms an adaptive privacy-preserving retrieval framework that can continuously improve and optimize according to actual operation conditions to enhance service quality and security. For example, in a smart city project, through this mechanism, the system successfully reduced query response time by 30%, while maintaining 99.9% privacy protection effectiveness, significantly improving user satisfaction.

[0282] like Fig.10 As shown, the embodiment of the present invention also provides a surveying and mapping results management system based on big data, which is used to realize the archiving and reference management of surveying and mapping results. The system includes:

[0283] The acquisition module 101 is used to acquire geospatial surveying and mapping data using drones, vehicle-mounted laser radars and sensors, and to record the time, location and device information of the surveying and mapping data in real time;

[0284] A classification module 102, which is used to classify the surveying and mapping data according to geographical location, acquisition time and data type based on an online linear optimization regularizer, and upload the classified surveying and mapping data to a distributed storage system;

[0285] A generation module 103 is used to perform data cleaning, format unification and quality verification on the stored surveying and mapping data to generate standardized surveying and mapping data;

[0286] The semantic analysis and feature annotation module 104 is used to construct a knowledge base based on a large language model LLM, perform semantic analysis and feature annotation on the standardized surveying and mapping data using the geospatial operators in the knowledge base, and establish a spatial index of the standardized surveying and mapping data;

[0287] The retrieval module 105 is used to receive a user retrieval request and perform a privacy protection retrieval on the standardized surveying and mapping data based on the spatial index.

[0288] The above modules can be implemented in the form of hardware or software functional modules. The modules interact with each other through the data bus to achieve full-process intelligent management of surveying and mapping results.

[0289] The protection scope of the present invention includes but is not limited to the above embodiments. The protection scope of the present invention shall be subject to the appended claims.

Claims

1. A surveying and mapping results management method based on big data, characterized in that: Used to realize the archiving and access management of surveying and mapping results, the method includes: using unmanned aerial vehicles, vehicle-mounted laser radars and sensors to obtain geospatial surveying and mapping data, and recording the time, location and equipment information of the surveying and mapping data in real time; based on an online linear optimization regularizer, classifying the surveying and mapping data according to geographical location, acquisition time and data type, and uploading the classified surveying and mapping data to a distributed storage system; performing data cleaning, format unification and quality verification on the stored surveying and mapping data to generate standardized surveying and mapping data; constructing a knowledge base based on a large language model (LLM), using the geospatial operators in the knowledge base to perform semantic analysis and feature annotation on the standardized surveying and mapping data, and establishing a spatial index for the standardized surveying and mapping data; receiving a user search request, and performing a privacy protection search on the standardized surveying and mapping data based on the spatial index; The regularizer based on online linear optimization classifies the surveying and mapping data according to geographic location, acquisition time and data type, including: constructing an edge geodetic set based on the spatiotemporal distribution characteristics of the surveying and mapping data, the edge geodetic set including measurement point coordinates, acquisition time and data type information; determining a regularization optimization objective function, the objective function being used to evaluate data classification accuracy and computing resource consumption; setting regularization constraints, including spatiotemporal consistency constraints and classification boundary constraints; based on the regularization constraints, solving the regularization optimization objective function through online optimization to obtain optimal classification parameters, and classifying the edge geodetic set according to the optimal classification parameters; Among them, solving the regularized optimization objective function through online optimization includes: calculating the weight coefficient of each measurement point in the edge geodetic set, the weight coefficient is related to the spatial distribution density and time correlation of the measurement point; constructing a regularization term based on the weight coefficient to prevent overfitting; using the gradient descent method to iteratively optimize the objective function until convergence or reaching a preset number of iterations; after the iterative optimization of the objective function is completed, obtaining the optimal classification parameters, so as to perform real-time classification of the edge geodetic set according to the optimal classification parameters.

2. The method according to claim 1, characterized in that The construction of a knowledge base based on a large language model (LLM) includes: selecting a pre-trained large language model and collecting professional documents in the field of geospatial space as training corpus; using the training corpus to fine-tune the pre-trained large language model to obtain a language model adapted to the field of geospatial space; automatically extracting geographic information based on the language model, the geographic information including definitions, parameter descriptions and usage examples of geospatial operators; establishing an operator function index, and storing the extracted geographic information in the knowledge base according to functional categories.

3. The method according to claim 2, characterized in that The method of using the geospatial operators in the knowledge base to perform semantic analysis and feature annotation on the standardized surveying and mapping data includes: retrieving operator functions related to the standardized surveying and mapping data from the knowledge base; calling the related operator functions to process the standardized surveying and mapping data to obtain data feature vectors; performing semantic clustering based on the data feature vectors to generate semantic labels; and associating the semantic labels with the standardized surveying and mapping data to construct a semantically enhanced spatial index.

4. The method according to claim 1, characterized in that: The privacy-preserving retrieval of the standardized surveying and mapping data includes: setting a differential privacy parameter ε according to data sensitivity to generate corresponding noise disturbances for different categories of surveying and mapping data; constructing a Matroid constraint set based on system resource constraints, wherein the system resource constraints include computing resource constraints, storage resource constraints, and bandwidth resource constraints; combining a differential privacy mechanism and a Matroid constraint optimization method to construct a privacy-preserving retrieval strategy; processing a user's retrieval request according to the privacy-preserving retrieval strategy, and returning a retrieval result that meets the privacy protection requirements.

5. The method according to claim 4, characterized in that The differential privacy mechanism and the Matroid constraint optimization method are combined to construct a privacy-preserving retrieval strategy, including: constructing a privatized learning index and performing differential privacy processing on retrieval keywords; converting the system resource allocation problem into a submodule optimization problem based on Matroid constraints; using a greedy algorithm to solve the submodule optimization problem to obtain a resource allocation plan; and constructing the privacy-preserving retrieval strategy based on the resource allocation plan and the privatized learning index to perform data retrieval.

6. The method according to claim 5, characterized in that The privacy-preserving retrieval strategy is constructed based on the resource allocation scheme and the privatized learning index to perform data retrieval, including: receiving a user's retrieval request, extracting retrieval conditions, and generating a retrieval task; matching the retrieval conditions in the privatized learning index; scheduling system resources to execute the retrieval task according to the resource allocation scheme; and returning the retrieval results to the user after differential privacy processing.

7. The method according to claim 4, characterized in that The method also includes: evaluating the privacy protection degree of the retrieval results; dynamically adjusting the differential privacy parameter ε and the Matroid constraint conditions according to the evaluation results; recording the retrieval log, including resource usage and privacy protection effect; and optimizing the retrieval strategy based on the retrieval log.

8. A surveying and mapping results management system based on big data, characterized in that: Used to realize the archiving and access management of surveying and mapping results, the system includes: an acquisition module, used to use drones, vehicle-mounted laser radars and sensors to acquire geospatial surveying and mapping data, and to record the time, location and equipment information of the surveying and mapping data in real time; a classification module, used to classify the surveying and mapping data according to geographical location, acquisition time and data type based on an online linear optimization regularizer, and upload the classified surveying and mapping data to a distributed storage system; a generation module, used to perform data cleaning, format unification and quality verification on the stored surveying and mapping data, and generate standardized surveying and mapping data; a semantic analysis and feature annotation module, used to build a knowledge base based on a large language model (LLM), use the geospatial operators in the knowledge base to perform semantic analysis and feature annotation on the standardized surveying and mapping data, and establish a spatial index for the standardized surveying and mapping data; a retrieval module, used to receive user retrieval requests, and perform privacy protection retrieval on the standardized surveying and mapping data based on the spatial index; The classification module is used to classify the surveying and mapping data according to geographical location, acquisition time and data type based on an online linear optimization regularizer, including: constructing an edge geodetic set based on the spatiotemporal distribution characteristics of the surveying and mapping data, the edge geodetic set including measurement point coordinates, acquisition time and data type information; determining a regularization optimization objective function, the objective function being used to evaluate data classification accuracy and computing resource consumption; setting regularization constraints, including spatiotemporal consistency constraints and classification boundary constraints; based on the regularization constraints, solving the regularization optimization objective function through online optimization to obtain optimal classification parameters, and classifying the edge geodetic set according to the optimal classification parameters; Among them, solving the regularized optimization objective function through online optimization includes: calculating the weight coefficient of each measurement point in the edge geodetic set, the weight coefficient is related to the spatial distribution density and time correlation of the measurement point; constructing a regularization term based on the weight coefficient to prevent overfitting; using the gradient descent method to iteratively optimize the objective function until convergence or reaching a preset number of iterations; after the iterative optimization of the objective function is completed, obtaining the optimal classification parameters, so as to perform real-time classification of the edge geodetic set according to the optimal classification parameters.

Images