A large map overlay scene analysis method based on data splitting

By using data splitting and parallel processing, the problems of low computational efficiency and memory overflow in large-scale graphics overlay calculations were solved, achieving efficient and stable graphics overlay processing.

CN118332057BActive Publication Date: 2026-11-10中国国土勘测规划院 +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410346626.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-26
Publication Date
2026-11-10
Estimated Expiration
2044-03-26

AI Technical Summary

Technical Problem

Existing technologies struggle to guarantee computational efficiency and stability when handling large-scale image overlays, easily leading to memory overflows. Traditional divide-and-conquer algorithms consume excessive resources on large datasets.

Method used

The method of data splitting and parallel processing is adopted. The graphic data is split into several subsets through format conversion and divide-and-conquer strategy, and then the subsets are overlaid using multi-process parallel processing and the final results are merged.

Benefits of technology

It improves the processing performance of large-scale graphic overlay, avoids memory overflow, and enhances computational efficiency and stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118332057B_ABST
    Figure CN118332057B_ABST
Patent Text Reader

Abstract

The application provides a large-layer superimposed scene analysis method based on data splitting, and belongs to the technical field of geospatial data analysis, and comprises the following steps: acquiring graphic data to be superimposed, and performing format conversion on the graphic data to obtain a plurality of layers to be superimposed; splitting different layers to be superimposed according to the layer size of each layer to be superimposed and according to a divide-and-conquer strategy to obtain a plurality of sub-data sets; controlling the process of each sub-data set in a geographic database workspace to perform superimposition processing on the corresponding sub-data set, wherein the superimposition processing of all processes is multi-process parallel processing; and merging all superimposition processing results to obtain a final superimposition result and outputting and displaying the final superimposition result. Through the ideas of data partial divide-and-conquer and parallel processing, the processing performance is improved and memory overflow is avoided when processing large-element-quantity graphic superimposition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of geospatial data analysis technology, and in particular to a method for analyzing large-layer overlay landscapes based on data splitting. Background Technology

[0002] When processing image overlays, existing technologies experience a sharp decline in performance, and may even produce memory overflow errors, when the number of features in multiple input data sets is very large. Figure 1 As shown, for operations such as intersection and clipping, the relationship between the number of features and processing time is not a simple linear one. For example, suppose layer A has 100,000 features and layer B has 1 million features. The time taken to perform the intersection operation on the two layers is T. When the number of features in layer A becomes 1 million while the number of features in layer B remains the same, the time taken to perform the intersection operation on the two layers is not simply a 10-fold increase. Depending on the computer configuration, the time taken may be 10T, or it may be 100T, or it may not even produce a result. The efficiency, stability, and reliability of the calculation cannot be guaranteed.

[0003] To address this situation, a divide-and-conquer algorithm is generally used, such as... Figure 2 As shown. However, traditional divide-and-conquer algorithms consume a lot of resources when splitting data, especially for large datasets (because the pruning operation involves fragmentation), and this splitting may be unnecessary for certain spatial operations (such as the union operation).

[0004] Therefore, this invention proposes a method for analyzing large-layer overlay images based on data splitting. Summary of the Invention

[0005] This invention provides a method for analyzing large-layer overlay images based on data splitting. By using the ideas of partial data divide-and-conquer and parallel processing, it can improve processing performance and avoid memory overflow when processing large-scale image overlays.

[0006] This invention proposes a method for analyzing large-layer overlay landscapes based on data splitting, including:

[0007] Step 1: Obtain the graphic data to be overlaid, and convert the format of the graphic data to obtain several layers to be overlaid;

[0008] Step 2: Based on the size of each layer to be overlaid, and following a divide-and-conquer strategy, split the different layers to be overlaid to obtain several subsets.

[0009] Step 3: Control the process in the geodatabase workspace where each subset of data sets is located to perform overlay processing on the corresponding subset of data sets. The overlay processing of all processes is a multi-process parallel processing.

[0010] Step 4: Merge all the overlay results to obtain the final overlay result and output it for display.

[0011] Preferably, the graphic data is format-converted to obtain several layers to be overlaid, including:

[0012] Each piece of graphic data is parsed to obtain the corresponding set of data attributes, wherein the data attributes include: individual data attributes, combined data attributes, and hidden data attributes;

[0013] The format of the corresponding graphic data is obtained from the combination-format mapping table based on the data attribute set, and it is determined whether there is a format conversion mapping table in the layer processing platform that matches the format. If not, a format conversion strategy for the corresponding graphic data is created in the layer processing platform.

[0014] Read the data length of the corresponding graphic data;

[0015] If the data length is greater than the preset length, then the number of nested formats of the corresponding graphic data is determined;

[0016]

[0017] Where sc represents the corresponding data length; yc represents the corresponding preset length; Δ1(yc) represents the nested program capacity function under the unit nested space based on the preset length; n1 represents the number of attribute categories contained in the data attribute set of the corresponding graphic data; J j1 This represents the data length under the j1-th attribute category in the corresponding image data; Δ2(j1, J j1 ) represents the relationship between the j1th attribute category and J j1 Related nested program capacity functions under unit nested space; zc j1 This represents the class weight of the j1-th attribute category in the corresponding image data; ceiling indicates the rounding up sign; S1 represents the number of nested formats in the corresponding graphic data;

[0018] The format conversion strategy is nested and optimized according to the format conversion program that matches each nesting in the specified number of nestings, and the corresponding graphic data is converted according to the optimized strategy.

[0019] Otherwise, the corresponding graphic data is converted according to the format conversion strategy described above;

[0020] If it exists, convert the format of the corresponding image data according to the matching format conversion mapping table;

[0021] Several layers to be overlaid are obtained based on the results of all format conversions.

[0022] Preferably, based on the layer size of each layer to be overlaid, and following a divide-and-conquer strategy, the different layers to be overlaid are split into several subsets, including:

[0023] Obtain historical layer overlay logs to determine the layer splitting threshold;

[0024] Layers whose size is smaller than the layer splitting threshold are designated as split layers, while layers whose size is not smaller than the layer splitting threshold are designated as non-split layers.

[0025] The maximum and optimal data processing volume of the workspace for the corresponding geodatabase specified by each process are determined as the calculation range of the corresponding workspace. At the same time, the data processing efficiency coefficient of the corresponding workspace is obtained based on the historical data processing logs of the corresponding workspace.

[0026] The first space allocated to each unsplit layer is determined by matching the first computational cost of each unsplit layer, the layer display clarity requirements of each unsplit layer, the computational range of all groups, and the data processing efficiency coefficient.

[0027] Obtain the first remaining computational cost and the reasonable coefficient of the first remaining computational cost for each first space;

[0028] The total computational cost of each split layer and the maximum number of split layer blocks determined according to the layer splitting threshold are determined separately and matched with the obtained result array of each first space in turn to set the belonging space label for each split layer block;

[0029] Check if the total computational cost of all layer blocks under the same spatial label is less than the first remaining computational cost. If so, keep the same spatial label unchanged.

[0030] If not, perform normal distribution analysis on the individual attribution calculation amount of each attribution layer block under the same attribution space label, and reduce the range of normal distribution probability according to the absolute difference between the total attribution calculation amount and the first remaining calculation amount until the total interval calculation amount of the normal distribution interval is less than the first remaining calculation amount. At this time, set the attribution layer blocks outside the normal distribution interval to a pending state.

[0031] After all the spatial labels have been analyzed, the spatial labels of the undetermined layer blocks are sorted according to the label weight and quantity weight of the undetermined layer blocks, and spatial matching is performed again on the sorted layer blocks until the matching is completed.

[0032] The final layer results involved in each workspace are treated as a subset of the dataset.

[0033] Preferably, historical layer overlay logs are obtained to determine the layer splitting threshold, including:

[0034] Extract the first size of all historical layer blocks involved in each process from the historical layer overlay log;

[0035] The first size is determined by different normal distribution probabilities to obtain the mean size under each normal distribution probability. At the same time, the second size that appears most frequently among all the first sizes is extracted.

[0036] The effective size of the corresponding process is calculated based on the average size and the second largest size.

[0037]

[0038] Where Yd represents the effective size of the corresponding process; d2 represents the corresponding second size; m1 represents the number of normally distributed probabilities; R i1 This represents the mean value of the magnitude based on the probability of the i1th normal distribution. This represents the variance of the absolute value of the difference between all adjacent means after sorting according to the probability of a normal distribution.

[0039] Plot curves for all valid sizes, and perform curve fitting on the plotted curves by randomly selecting n02-1 points n01 times;

[0040] The average of all fitted values ​​after curve fitting is used as the layer splitting threshold.

[0041] Preferably, all overlay processing results are merged to obtain the final overlay result, including:

[0042] Based on the graphic to which the layer belongs and the location of the layer involved in each overlay processing result, the indication information of the single overlay layer for each graphic is obtained.

[0043] According to the merging requirements and in conjunction with the instruction information, each overlay processing result is merged to obtain the final overlay result.

[0044] Preferably, merging all overlay processing results to obtain the final overlay result and outputting it for display also includes:

[0045] Obtain the user's display requirements, and then highlight each individual overlay layer in the final overlay result according to the display requirements before displaying it.

[0046] Preferably, after merging all overlay processing results to obtain the final overlay result and outputting it for display, the process also includes:

[0047] Receive operation commands input by the user on the graphics processing platform;

[0048] If the operation command is related to clearing the running space, then the running information generated during the overlay process will be cleared.

[0049] Preferably, the reasonable coefficients for obtaining the first remaining computational quantity in each first space include:

[0050] The process of non-splitting layers allocated in each historical processing of the first space is captured, and the processing process is parsed to obtain the normal processing flow and abnormal processing flow under the corresponding historical processing conditions. Combined with the remaining amount of historical space after each historical processing, the process array of the corresponding processing process is constructed.

[0051] Based on the process array, the reasonable coefficient of the first remaining computational quantity corresponding to the first space is calculated;

[0052]

[0053] Where m2 represents the total number of process arrays; N1 i2 This represents the number of steps in the normal processing flow within the i2th process array; Cg1 i2 This represents the number of steps in the exception handling process within the i2-th process array; Q1 i2 K1 represents the processing weight of the exception handling process in the i2th process array; β1 represents the conversion coefficient based on the number of steps; β2 represents the conversion coefficient based on the amount of space computation; K1 i2 K0 represents the remaining amount of historical space in the i2th process array; K0 represents the total space processing amount corresponding to the first space; Δ1 represents the coefficient fine-tuning amount; m3 represents the amount of space that satisfies the condition. The number of arrays corresponding to exception handling processes greater than 1 / 2; Sg1 i2 Hx represents the processing volume of the exception handling process in the i2th process array; Hx represents the reasonable coefficient corresponding to the first space.

[0054] When the reasonable coefficient is less than the preset coefficient, the minimum allowable correction coefficient for each exception handling process is determined, and the reasonable coefficient is corrected upwards.

[0055]

[0056] Among them, yx1 i2 This represents the minimum allowable correction coefficient for the exception handling process in the i2th process array; (yx1) i2 ) min Represents all yx1 i2 The minimum value in the equation; β3 represents the standardized coefficient for the correction factor; Hx0 represents the coefficient after upward correction;

[0057] If the coefficient after upward correction is still less than the preset coefficient Yx0, then replace it with the reasonable coefficient corresponding to the first space of 0;

[0058] If the reasonable coefficient is not less than the preset coefficient, then the reasonable coefficient remains unchanged.

[0059] Compared with the prior art, the beneficial effects of this application are as follows:

[0060] By employing the principles of data divide-and-conquer and parallel processing, we can improve processing performance and avoid memory overflow when handling large-scale graphic overlays.

[0061] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0062] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0063] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0064] Figure 1 This is a result of overlaying prior art processed graphics in an embodiment of the present invention;

[0065] Figure 2 This is a diagram showing the result of using the traditional divide-and-conquer algorithm in an embodiment of the present invention;

[0066] Figure 3 This is a flowchart of a large-layer overlay scene analysis method based on data splitting in an embodiment of the present invention;

[0067] Figure 4 This is a diagram showing the result of the divide-and-conquer approach in an embodiment of the present invention;

[0068] Figure 5 This is a flowchart of parallel computing in an embodiment of the present invention. Detailed Implementation

[0069] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0070] This invention proposes a method for analyzing large-layer overlay landscapes based on data splitting, such as... Figure 3 As shown, it includes:

[0071] Step 1: Obtain the graphic data to be overlaid, and convert the format of the graphic data to obtain several layers to be overlaid;

[0072] Step 2: Based on the size of each layer to be overlaid, and following a divide-and-conquer strategy, split the different layers to be overlaid to obtain several subsets.

[0073] Step 3: Control the process in the geodatabase workspace where each subset of data sets is located to perform overlay processing on the corresponding subset of data sets. The overlay processing of all processes is a multi-process parallel processing.

[0074] Step 4: Merge all the overlay results to obtain the final overlay result and output it for display.

[0075] In this embodiment, the present invention adopts a divide-and-conquer approach for vector layer overlay analysis scenarios, but only splits relatively smaller layers while leaving relatively larger layers unsplit. Figure 4 As shown, this reduces the consumption of computing resources.

[0076] To process these subsets more efficiently, multi-processing is used, with each process handling one subset. This introduces the problem of synchronous data writing, especially in single-machine scenarios without a server-side database. To address this, different geodatabase workspaces can be assigned to different processes. Each split unit, along with the larger layer, occupies its own workspace and process for multi-process parallel computation. This approach effectively reduces the load on individual processes and leverages the parallel computing capabilities of modern multi-core processors, such as... Figure 5 As shown.

[0077] In this embodiment, the graphic data to be superimposed refers to the graphic data that needs to be superimposed. The superimposed layer can be obtained by converting the format of the graphic data. The superimposed layer is an image in image format. Each graphic data to be superimposed corresponds to an image, hereinafter referred to as graphic data, and each graphic data contains the information involved in the corresponding image.

[0078] In this embodiment, the graphic data is, for example, in character format. The corresponding overlay layer can be obtained by converting the character format to JPG format, and the conversion between formats is an existing technology.

[0079] In this embodiment, the layer size refers to the amount of information involved in the corresponding overlay layer.

[0080] In this embodiment, the divide-and-conquer strategy refers to splitting smaller layers while leaving larger ones unsplit, thereby reducing the consumption of computing resources.

[0081] In this embodiment, a subset of data refers to a combination of multiple split layers and unsplit layers.

[0082] For example, if layer 1 is split into layer 01 and layer 02, and layer 2 is not split, then the combination of layer 2 and layer 01 constitutes a subset of data.

[0083] In this embodiment, the purpose of setting up the workspace of the geodatabase is to provide a processing space for the subset of data, which facilitates overlay processing in the space and improves processing performance.

[0084] In this embodiment, multi-process parallel processing refers to processes in different workspaces processing a single block, thereby improving processing efficiency.

[0085] In this embodiment, the merging of overlay processing results is to combine the results from different workspaces corresponding to all layers to be overlaid, and the final result is the merging of all layers.

[0086] The beneficial effects of the above technical solution are: by using the ideas of partial data divide-and-conquer and parallel processing, it can improve processing performance and avoid memory overflow when processing large-scale graphic overlays.

[0087] This invention proposes a method for analyzing large-layer overlay images based on data splitting. The method involves converting the format of the graphic data to obtain several layers to be overlaid, including:

[0088] Each piece of graphic data is parsed to obtain the corresponding set of data attributes, wherein the data attributes include: individual data attributes, combined data attributes, and hidden data attributes;

[0089] The format of the corresponding graphic data is obtained from the combination-format mapping table based on the data attribute set, and it is determined whether there is a format conversion mapping table in the layer processing platform that matches the format. If not, a format conversion strategy for the corresponding graphic data is created in the layer processing platform.

[0090] Read the data length of the corresponding graphic data;

[0091] If the data length is greater than the preset length, then the number of nested formats of the corresponding graphic data is determined;

[0092]

[0093] Where sc represents the corresponding data length; yc represents the corresponding preset length; Δ1(yc) represents the nested program capacity function under the unit nested space based on the preset length; n1 represents the number of attribute categories contained in the data attribute set of the corresponding graphic data; J j1 This represents the data length under the j1-th attribute category in the corresponding image data; Δ2(j1,Jj1 ) represents the relationship between the j1th attribute category and J j1 Related nested program capacity functions under unit nested space; zc j1 This represents the class weight of the j1-th attribute category in the corresponding image data; ceiling indicates the rounding up sign; S1 represents the number of nested formats in the corresponding graphic data;

[0094] The format conversion strategy is nested and optimized according to the format conversion program that matches each nesting in the specified number of nestings, and the corresponding graphic data is converted according to the optimized strategy.

[0095] Otherwise, the corresponding graphic data is converted according to the format conversion strategy described above;

[0096] If it exists, convert the format of the corresponding image data according to the matching format conversion mapping table;

[0097] Several layers to be overlaid are obtained based on the results of all format conversions.

[0098] In this embodiment, for example, if the graphic data is in format 1, but the platform requires format 2, then converting the graphic data from format 1 to format 2 will result in the corresponding layer to be overlaid.

[0099] In this embodiment, during the format conversion process, the platform will have a partial format conversion mapping table. When the platform is missing a certain format conversion, it needs to be created to ensure the reasonable conversion of all formats and provide a basis for subsequent layer segmentation.

[0100] In this embodiment, a separate data attribute refers to data that exists independently in the corresponding graphic data, that is, data that exists independently is regarded as a separate data attribute.

[0101] Combined data attributes refer to data that does not exist in isolation within the corresponding graphic data; that is, data that does not exist in isolation is considered as combined data attributes.

[0102] Hidden data attributes refer to data in the corresponding graphic data that represents implicit meaning; that is, data that represents implicit meaning is considered hidden data attributes.

[0103] In this embodiment, the data attribute set includes different attribute combinations and format conversion mapping tables that match the combinations from the combination-format mapping table. The table contains conversion methods for different formats predetermined by experts, which facilitates direct conversion.

[0104] In this embodiment, when there is no corresponding mapping table, a new space needs to be created. However, in order to ensure running performance, the space occupancy is set based on the principle of minimization. At this time, it is necessary to calculate the number of related format nesting to ensure the effectiveness of format conversion and minimize the impact on processing performance.

[0105] In this embodiment, the data length refers to the character length of the corresponding data.

[0106] In this embodiment, the preset length is pre-set.

[0107] In this embodiment, the unit nesting space refers to the space size occupied based on a preset length yc. For example, if the preset length is 100, the unit nesting space corresponding to the preset length is 10.

[0108] In this embodiment, for example, the data length is 120, of which the data length for a single data attribute category is 30, the data length for a combined data attribute category is 80, and the data length for a hidden data attribute category is 10.

[0109] In this embodiment, for example, if the number of format nesting for image data 1 is 2, then the format conversion strategy is adjusted by format nesting. The format conversion program matched by the nesting is obtained based on the format itself, attributes, platform required format, and program database. This database contains various different formats and format matching attributes, as well as matching programs. The program nesting setting is an existing technology, mainly to save the program's space occupied for format conversion. Nesting can also effectively reduce the multiple runs of repetitive programs, thereby indirectly reducing the amount of data generated during the operation.

[0110] In this embodiment, for example, the format conversion strategy operates as follows:

[0111] Fixed code -- Sub-strategy 01;

[0112] Fixed code -- Sub-strategy 02;

[0113] Fixed code -- Sub-strategy 03;

[0114] The optimized nested execution is as follows:

[0115] Fixed code;

[0116] Nested -- Sub-strategy 01;

[0117] Nested -- Sub-strategy 02;

[0118] Nested -- Sub-strategy 03;

[0119] The number of nested programs and the amount of execution are much smaller than the number of programs with fixed code and the amount of execution.

[0120] The beneficial effects of the above technical solution are: by checking whether the mapping table exists, it effectively ensures that the data is converted according to the specified format, which facilitates subsequent layer analysis. The purpose of the nested setting is to reduce the memory space occupied by log information generated during the format conversion process, and also to further improve processing performance.

[0121] This invention proposes a method for analyzing large-layer overlay images based on data splitting. According to the size of each layer to be overlaid, and following a divide-and-conquer strategy, different layers to be overlaid are split to obtain several subsets, including:

[0122] Obtain historical layer overlay logs to determine the layer splitting threshold;

[0123] Layers whose size is smaller than the layer splitting threshold are designated as split layers, while layers whose size is not smaller than the layer splitting threshold are designated as non-split layers.

[0124] The maximum and optimal data processing volume of the workspace for the corresponding geodatabase specified by each process are determined as the calculation range of the corresponding workspace. At the same time, the data processing efficiency coefficient of the corresponding workspace is obtained based on the historical data processing logs of the corresponding workspace.

[0125] The first space allocated to each unsplit layer is determined by matching the first computational cost of each unsplit layer, the layer display clarity requirements of each unsplit layer, the computational range of all groups, and the data processing efficiency coefficient.

[0126] Obtain the first remaining computational cost and the reasonable coefficient of the first remaining computational cost for each first space;

[0127] The total computational cost of each split layer and the maximum number of split layer blocks determined according to the layer splitting threshold are determined separately and matched with the obtained result array of each first space in turn to set the belonging space label for each split layer block;

[0128] Check if the total computational cost of all layer blocks under the same spatial label is less than the first remaining computational cost. If so, keep the same spatial label unchanged.

[0129] If not, perform normal distribution analysis on the individual attribution calculation amount of each attribution layer block under the same attribution space label, and reduce the range of normal distribution probability according to the absolute difference between the total attribution calculation amount and the first remaining calculation amount until the total interval calculation amount of the normal distribution interval is less than the first remaining calculation amount. At this time, set the attribution layer blocks outside the normal distribution interval to a pending state.

[0130] After all the spatial labels have been analyzed, the spatial labels of the undetermined layer blocks are sorted according to the label weight and quantity weight of the undetermined layer blocks, and spatial matching is performed again on the sorted layer blocks until the matching is completed.

[0131] The final layer results involved in each workspace are treated as a subset of the dataset.

[0132] In this embodiment, the historical layer overlay log is extracted from the historical database and contains the block size of different layer blocks under different processes.

[0133] In this embodiment, the layer splitting threshold is used to set a threshold for dividing the layer size.

[0134] In this embodiment, the maximum and optimal data processing amounts can be directly obtained by matching the data amount database corresponding to the process-space. The calculation range is (optimal data processing amount, maximum data processing amount), with the aim of satisfying the optimal data processing amount as much as possible to achieve effective matching of layer blocks in the future.

[0135] In this embodiment, the historical data processing log refers to the amount of data processed in the corresponding workspace after each processing, thus obtaining the data processing efficiency coefficient = 2 / 3 × processing efficiency of the processed data amount / processing efficiency of the optimal processed data amount + 1 / 3 × processing efficiency of the processed data amount / processing efficiency of the maximum processed data amount.

[0136] In this embodiment, the first computational load is determined based on the amount of information in the corresponding layer.

[0137] In this embodiment, the purpose of requiring clear layer display is to determine the display effectiveness of the layers. Generally, the higher the processing efficiency, the higher the corresponding data processing effectiveness coefficient.

[0138] In this embodiment, the first computational load matches the computational range, and the requirement for clear layer display matches the effective coefficient of data processing. At this time, the most matching space should be used as the first space of the corresponding non-split layer, which is the workspace.

[0139] In this embodiment, the first remaining computational cost is the total computational cost of the first space minus the first computational cost of the corresponding allocated non-split layer.

[0140] In this embodiment, the obtained result array is: first remaining computation amount, reasonable coefficient.

[0141] In this embodiment, the maximum number of split layer blocks refers to the number of layer blocks obtained after splitting the layer according to the layer splitting threshold. It can be based on the threshold and the corresponding splitting strategy (pre-set) to directly split the corresponding layer. For example, the layer can be split into 3 blocks according to the threshold.

[0142] In this embodiment, the total computational load is determined by the information involved in the corresponding layer; the more information there is, the greater the total computational load.

[0143] In this embodiment, matching with the obtained result array refers to matching the amount of computation and matching the reasonable coefficient. The larger the reasonable coefficient, the higher the probability of allocation, and thus the more it matches the assigned space of the split layer block.

[0144] In this embodiment, the total attribution calculation is obtained by adding the calculations of each attribution layer block under the same attribution space label.

[0145] In this embodiment, the computational cost per attribution refers to the computational cost involved in a single attribution layer block.

[0146] In this embodiment, the purpose of normal distribution analysis is to statistically analyze all individual attribution calculations using a normal distribution. For example, the normal distribution is based on a range selection with 80% as the basis. The purpose of narrowing is to ensure that the total interval calculation amount corresponding to the normal distribution probability is less than the first remaining calculation amount. For example, after analyzing according to the absolute difference, the range is further narrowed. At this time, for example, the corresponding probability will drop to 70%. The attribution layer blocks involved in the interval corresponding to this 70% are in a pending state. The purpose of narrowing is to ensure that the total interval calculation amount is exactly less than the first remaining calculation amount. The total interval calculation amount refers to the sum of all individual attribution calculation amounts in the corresponding interval.

[0147] In this embodiment, the label weight is obtained based on the layer-label-weight mapping table, which includes different layers, layer-set labels, and pre-set weights for the labels by experts.

[0148] The quantity weight is obtained based on the label-quantity-weight mapping table, which includes different labels, the number of labels for the corresponding labels, and the weights that experts pre-set for the quantity.

[0149] In this embodiment, the sorting is performed by sorting the results by label weight × quantity weight, which facilitates subsequent orderly matching.

[0150] The beneficial effects of the above technical solution are as follows: by obtaining logs to determine the splitting threshold and then dividing the layer size, the non-split layers are allocated to the space based on the matching of computational load and data processing efficiency coefficients. Then, by setting labels for the split layers based on the remaining computational load of the space and reasonable coefficients, and then using normal distribution analysis to achieve secondary allocation of the assigned layer blocks, not only is the processing efficiency guaranteed, but the layer block assignment problem is also effectively guaranteed, indirectly improving the processing performance.

[0151] This invention proposes a method for analyzing large-layer overlay landscapes based on data splitting, which involves obtaining historical layer overlay logs and determining layer splitting thresholds, including:

[0152] Extract the first size of all historical layer blocks involved in each process from the historical layer overlay log;

[0153] The first size is determined by different normal distribution probabilities to obtain the mean size under each normal distribution probability. At the same time, the second size that appears most frequently among all the first sizes is extracted.

[0154] The effective size of the corresponding process is calculated based on the average size and the second largest size.

[0155]

[0156] Where Yd represents the effective size of the corresponding process; d2 represents the corresponding second size; m1 represents the number of normally distributed probabilities; R i1 This represents the mean value of the magnitude based on the probability of the i1th normal distribution. This represents the variance of the absolute value of the difference between all adjacent means after sorting according to the probability of a normal distribution.

[0157] Plot curves for all valid sizes, and perform curve fitting on the plotted curves by randomly selecting n02-1 points n01 times;

[0158] The average of all fitted values ​​after curve fitting is used as the layer splitting threshold.

[0159] In this embodiment, the first size of the historical layer block can be extracted from the log. The normal distribution probability can be 60%, 70%, or 80%. At this time, all the first sizes contained under different distribution probabilities can be obtained respectively, and then the average value can be calculated to obtain the average size.

[0160] The beneficial effects of the above technical solution are: by extracting the size from the logs, the effective size mean and the second size are determined by different normal distribution probabilities, which provides a basis for subsequent calculation of the effective size. Furthermore, by using curve fitting, the reliability of the layer splitting threshold is further guaranteed, which provides an effective basis for subsequent layer size division.

[0161] This invention proposes a method for analyzing large-layer overlay images based on data splitting, which merges all overlay processing results to obtain the final overlay result, including:

[0162] Based on the graphic to which the layer belongs and the location of the layer involved in each overlay processing result, the indication information of the single overlay layer for each graphic is obtained.

[0163] According to the merging requirements and in conjunction with the instruction information, each overlay processing result is merged to obtain the final overlay result.

[0164] In this embodiment, for example, the subset 1 includes a non-split layer 1 and a split layer block 02;

[0165] Subdataset 2 includes unsplit layer 2 and split layer block 03;

[0166] Furthermore, split layer block 03 and split layer block 02 belong to split layer 3;

[0167] In other words, split layer blocks 03 and 02 belong to split layer 3. The layer belonging position refers to the position occupied by 02 and 03 based on split layer 3. For example, split layer 3 also contains split layer block 01. In this case, the indication information of a single overlay layer for the graphic of split layer 3 includes the relevant belonging.

[0168] In this embodiment, the merging requirement refers to merging the overlay processing results, which can be done directly.

[0169] The beneficial effects of the above technical solution are: by determining the belonging graphic and the belonging position of the layer, and then merging them through indication information, effective guidance is achieved, and effective overlay results are obtained.

[0170] This invention proposes a method for analyzing large-layer overlay images based on data splitting, which merges all overlay processing results to obtain the final overlay result and outputs it for display. It also includes:

[0171] Obtain the user's display requirements, and then highlight each individual overlay layer in the final overlay result according to the display requirements before displaying it.

[0172] In this embodiment, the salience label is a color label.

[0173] The beneficial effect of the above technical solution is that it enables effective display by highlighting different layers.

[0174] This invention proposes a method for analyzing large-layer overlay images based on data splitting. After merging all overlay processing results to obtain the final overlay result and outputting it for display, it also includes:

[0175] Receive operation commands input by the user on the graphics processing platform;

[0176] If the operation command is related to clearing the running space, then the running information generated during the overlay process will be cleared.

[0177] The beneficial effect of the above technical solution is that by clearly defining the operating space and ensuring its free space, processing efficiency can be indirectly improved.

[0178] This invention proposes a method for analyzing large-layer overlay landscapes based on data splitting, obtaining reasonable coefficients for the first remaining computational quantity in each first space, including:

[0179] The process of non-splitting layers allocated in each historical processing of the first space is captured, and the processing process is parsed to obtain the normal processing flow and abnormal processing flow under the corresponding historical processing conditions. Combined with the remaining amount of historical space after each historical processing, the process array of the corresponding processing process is constructed.

[0180] Based on the process array, the reasonable coefficient of the first remaining computational quantity corresponding to the first space is calculated;

[0181]

[0182]

[0183] Where m2 represents the total number of process arrays; N1 i2 This represents the number of steps in the normal processing flow within the i2th process array; Cg1 i2 This represents the number of steps in the exception handling process within the i2-th process array; Q1 i2 K1 represents the processing weight of the exception handling process in the i2th process array; β1 represents the conversion coefficient based on the number of steps; β2 represents the conversion coefficient based on the amount of space computation; K1 i2 K0 represents the remaining amount of historical space in the i2th process array; K0 represents the total space processing amount corresponding to the first space; Δ1 represents the coefficient fine-tuning amount; m3 represents the amount of space that satisfies the condition. The number of arrays corresponding to exception handling processes greater than 1 / 2; Sg1 i2 Hx represents the processing volume of the exception handling process in the i2th process array; Hx represents the reasonable coefficient corresponding to the first space.

[0184] When the reasonable coefficient is less than the preset coefficient, the minimum allowable correction coefficient for each exception handling process is determined, and the reasonable coefficient is corrected upwards.

[0185]

[0186] Among them, yx1 i2 This represents the minimum allowable correction coefficient for the exception handling process in the i2th process array; (yx1) i2 ) min Represents all yx1 i2 The minimum value in the equation; β3 represents the standardized coefficient for the correction factor; Hx0 represents the coefficient after upward correction;

[0187] If the coefficient after upward correction is still less than the preset coefficient Yx0, then replace it with the reasonable coefficient corresponding to the first space of 0;

[0188] If the reasonable coefficient is not less than the preset coefficient, then the reasonable coefficient remains unchanged.

[0189] In this embodiment, for example, the process includes 10 processing steps, of which there are 7 normal processing steps and 3 abnormal processing steps, all of which can be directly obtained from the process log analysis.

[0190] In this embodiment, the process array = [the remaining amount of historical space for normal data flow and abnormal handling process].

[0191] In this embodiment, the preset coefficient is pre-set.

[0192] In this embodiment, the minimum allowable correction coefficient of the exception handling process is the corresponding range obtained by matching the exception handling process-process exception correction range table, and the minimum value in the range is the minimum allowable correction coefficient.

[0193] The beneficial effects of the above technical solution are: by statistically analyzing the normal processing flow, abnormal processing flow, and historical space remaining quantity during the processing, a reasonable coefficient can be effectively calculated, providing a basis for subsequent matching. Furthermore, by correcting upwards, the possibility of the coefficient being less than the preset coefficient due to calculation errors can be effectively avoided, thereby improving the reliability of the analysis. Setting it to 0 is to directly treat the corresponding space as a mismatch case. Compared with making a judgment after comparing the numerical values, this reduces the amount of calculation and indirectly improves performance.

[0194] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for analyzing large-layer overlay landscapes based on data splitting, characterized in that, include: Step 1: Obtain the graphic data to be overlaid, and convert the format of the graphic data to obtain several layers to be overlaid; Step 2: Based on the size of each layer to be overlaid, and following a divide-and-conquer strategy, split the different layers to be overlaid to obtain several subsets. Step 3: Control the process in the geodatabase workspace where each subset of data sets is located to perform overlay processing on the corresponding subset of data sets. The overlay processing of all processes is a multi-process parallel processing. Step 4: Merge all the overlay results to obtain the final overlay result and output it for display; Specifically, based on the size of each layer to be overlaid, and following a divide-and-conquer strategy, different layers to be overlaid are split into several subsets, including: Obtain historical layer overlay logs to determine the layer splitting threshold; Layers whose size is smaller than the layer splitting threshold are designated as split layers, while layers whose size is not smaller than the layer splitting threshold are designated as non-split layers. The maximum and optimal data processing volume of the workspace for the corresponding geodatabase specified by each process are determined as the calculation range of the corresponding workspace. At the same time, the data processing efficiency coefficient of the corresponding workspace is obtained based on the historical data processing logs of the corresponding workspace. The first space allocated to each unsplit layer is determined by matching the first computational cost of each unsplit layer, the layer display clarity requirements of each unsplit layer, the computational range of all groups, and the data processing efficiency coefficient. Obtain the first remaining computational cost and the reasonable coefficient of the first remaining computational cost for each first space; The total computational cost of each split layer and the maximum number of split layer blocks determined according to the layer splitting threshold are determined separately and matched with the obtained result array of each first space in turn to set the belonging space label for each split layer block; Check if the total computational cost of all layer blocks under the same spatial label is less than the first remaining computational cost. If so, keep the same spatial label unchanged. If not, perform normal distribution analysis on the individual attribution calculation amount of each attribution layer block under the same attribution space label, and reduce the range of normal distribution probability according to the absolute difference between the total attribution calculation amount and the first remaining calculation amount until the total interval calculation amount of the normal distribution interval is less than the first remaining calculation amount. At this time, set the attribution layer blocks outside the normal distribution interval to a pending state. After all the spatial labels have been analyzed, the spatial labels of the undetermined layer blocks are sorted according to the label weight and quantity weight of the undetermined layer blocks, and spatial matching is performed again on the sorted layer blocks until the matching is completed. The final layer results involved in each workspace are treated as a subset of the dataset.

2. The method for analyzing large-layer overlay images based on data splitting according to claim 1, characterized in that, The graphic data is format-converted to obtain several layers to be overlaid, including: Each piece of graphic data is parsed to obtain the corresponding set of data attributes, wherein the data attributes include: individual data attributes, combined data attributes, and hidden data attributes; The format of the corresponding graphic data is obtained from the combination-format mapping table based on the data attribute set, and it is determined whether there is a format conversion mapping table in the layer processing platform that matches the format. If not, a format conversion strategy for the corresponding graphic data is created in the layer processing platform. Read the data length of the corresponding graphic data; If the data length is greater than the preset length, then the number of nested formats of the corresponding graphic data is determined; ; in, Indicates the corresponding data length; Indicates the corresponding preset length; This function represents the capacity of nested programs within a unit nested space of a preset length. This indicates the number of attribute categories contained in the data attribute set of the corresponding graphic data; Indicates the first image in the corresponding image data Data length under each attribute category; Indicates the relationship with the first Each attribute category and Related nested program capacity functions within a single nested space; Indicates the first image in the corresponding image data The category weight of each attribute category; S1 indicates the rounding up symbol; S1 indicates the number of nested formats in the corresponding graphic data. The format conversion strategy is nested and optimized according to the format conversion program that matches each nesting in the specified number of nestings, and the corresponding graphic data is converted according to the optimized strategy. Otherwise, the corresponding graphic data is converted according to the format conversion strategy described above; If it exists, convert the format of the corresponding image data according to the matching format conversion mapping table; Several layers to be overlaid are obtained based on the results of all format conversions.

3. The method for analyzing large-layer overlay images based on data splitting according to claim 1, characterized in that, Obtain historical layer overlay logs to determine layer splitting thresholds, including: Extract the first size of all historical layer blocks involved in each process from the historical layer overlay log; The first size is determined by different normal distribution probabilities to obtain the mean size under each normal distribution probability. At the same time, the second size that appears most frequently among all the first sizes is extracted. The effective size of the corresponding process is calculated based on the average size and the second largest size. ; in, Indicates the effective size of the corresponding process; This indicates the corresponding second size; Indicates the number of probabilities in a normal distribution; Indicates based on the first The mean of the magnitudes under a normal distribution probability; This represents the variance of the absolute value of the difference between all adjacent means after sorting according to the probability of a normal distribution. Plot curves for all valid sizes, and perform curve fitting on the plotted curves by randomly selecting n02-1 points n01 times; The average of all fitted values ​​after curve fitting is used as the layer splitting threshold.

4. The method for analyzing large-layer overlay images based on data splitting according to claim 1, characterized in that, All overlay results are merged to obtain the final overlay result, including: Based on the graphic to which the layer belongs and the location of the layer involved in each overlay processing result, the indication information of the single overlay layer for each graphic is obtained. According to the merging requirements and in conjunction with the instruction information, each overlay processing result is merged to obtain the final overlay result.

5. The method for analyzing large-layer overlay images based on data splitting according to claim 4, characterized in that, The process of merging all overlay results to obtain the final overlay result and then outputting and displaying it includes: Obtain the user's display requirements, and then highlight each individual overlay layer in the final overlay result according to the display requirements before displaying it.

6. The method for analyzing large-layer overlay landscapes based on data splitting according to claim 1, characterized in that, After merging all the overlay results to obtain the final overlay result and outputting it for display, the following is also included: Receive operation commands input by the user on the graphics processing platform; If the operation command is related to clearing the running space, then the running information generated during the overlay process will be cleared.

Citation Information

Patent Citations

  • Mineral resource multi-information processing method based on GIS and system thereof

    CN103942841A

  • Parallel method and system for vector space overlay analysis in geographic information system

    CN107066542A