A spatial data processing method, apparatus, medium, and device

Abstract

The application discloses a kind of spatial data processing method, device, medium and equipment, belong to geoscience data processing technical field.The present application is first by processing object classification to original spatial data, is divided into vector data and grid or remote sensing data respectively processing, improve processing efficiency and accuracy, then use multidimensional quality confidence to distinguish whether to use automatic processing, improve intelligent processing level and guarantee the accurate screening of abnormal situation, finally through the data of each link and its derivative relationship, establish traceability, so that the output preview of data is clearer, the traceability of data reapplication is stronger, effectively improve the processing effect of geoscience spatial data.

Description

Technical Field

[0001] This invention belongs to the field of geoscience data processing technology, specifically relating to a spatial data processing method, apparatus, medium, and equipment. Background Technology

[0002] Geological surveys, mineral exploration, remote sensing interpretation, and 3D intelligent mineral prospecting prediction require long-term collection of multi-source heterogeneous geospatial data. This type of data includes vector data such as Shapefile, GeoJSON, and GeoPackage, as well as raster or remote sensing data such as GeoTIFF, ENVI, Erdas Imagine, PCI PIX, JPEG2000, and Sentinel-2 SAFE. It also includes auxiliary materials such as borehole coordinates, sampling points, profiles, columnar sections, documents, and compressed archives.

[0003] In practical engineering, the aforementioned data often comes from different organizations, different software environments, and historical results from different periods. Their file encoding, coordinate systems, spatial extents, data formats, rendering methods, and archiving structures are not uniform. To integrate this data into a unified database or WebGIS service platform, preprocessing tasks such as file type identification, Chinese attribute reading, coordinate correction, raster rendering determination, format conversion, and thumbnail or map tile generation are typically required.

[0004] Existing technologies use fixed processes or manual configuration methods for processing. While this approach is feasible for small amounts of data, it performs poorly when accessing large-scale geoscience data from multiple mining areas in multiple batches. This can easily lead to problems such as low processing efficiency, high error rates, poor preview effects, and lack of data traceability. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a spatial data processing method, apparatus, medium and device.

[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of the present invention provide a spatial data processing method, comprising the following steps: The raw spatial data is classified according to the type of object being processed to obtain first target data and second target data; wherein, the first target data is vector data and the second target data is raster data or remote sensing data; Perform a first processing on the first target data to obtain first processed data, and perform a second processing on the second target data to obtain second processed data; Based on the multi-dimensional quality confidence scores and automatic processing thresholds of the first and second processed data, the target processing method is selected and executed to obtain the target data. A tracing relationship is established based on the original spatial data, the first processed data, the second processed data, the target data, and their derived relationships; Data output is based on raw spatial data, first-processed data, second-processed data, multi-dimensional quality confidence, and traceability relationships.

[0007] In one possible implementation of the first aspect, performing a first process on the first target data to obtain first processed data includes: Multiple encoding attempts are performed on the first target data, and the target encoding is obtained based on the garbled character suspicion score and semantic hit score.

[0008] In one possible implementation of the first aspect, performing a first process on the first target data to obtain first processed data includes: Gaussian-Krüger band prefix detection and repair were performed on the first target data to obtain the detection and repair results. If the detection and repair results are abnormal, expandable candidates are generated and sorted according to the target requirements; Based on the sorting results, a re-repair plan is obtained.

[0009] In one possible implementation of the first aspect, the second target data undergoes a second process to obtain second processed data, including: The second target data is classified to obtain categorical and continuous data. Based on categorical and continuous data, COG derived files and Web map tiles are generated.

[0010] In one possible implementation of the first aspect, COG derived files and Web map tiles are generated based on categorical and continuous data, including: COG state detection and tile range calculation are performed based on classified data and continuous data respectively to obtain detection results and calculation results. If the detection result is negative, COG overview layer resampling is performed to generate COG derived files; for classified data, nearest neighbor resampling is used, and for continuous data, triple convolution resampling is used. Based on the calculation results, reprojection sampling is performed to generate Web map tiles; for categorical data, the nearest neighbor reprojection sampling method is used, and for continuous data, the bilinear reprojection sampling method is used.

[0011] In one possible implementation of the first aspect, before classifying the raw spatial data according to the type of object being processed to obtain the first target data and the second target data, the method further includes: The file extensions, archive members, and accompanying files of the original files are scanned to determine the objects to be processed and obtain the original spatial data.

[0012] In one possible implementation of the first aspect, the multi-dimensional quality confidence includes encoding detection confidence, coordinate repair confidence, raster classification confidence, and master file selection confidence.

[0013] In a second aspect, embodiments of the present invention provide a spatial data processing apparatus, comprising: The classification module is used to classify the raw spatial data according to the type of the object being processed, and to obtain first target data and second target data; wherein, the first target data is vector data, and the second target data is raster data or remote sensing data; The processing module is configured to perform a first processing on the first target data and obtain the first processed data, and to perform a second processing on the second target data and obtain the second processed data. The confidence module is used to select the target processing method and obtain the target data based on the multi-dimensional quality confidence of the first and second processed data and the automatic processing threshold. A module is established to create a tracing relationship based on the original spatial data, the first processed data, the second processed data, the target data, and their derived relationships. The output module is used to output data based on the original spatial data, the first processed data, the second processed data, the multi-dimensional quality confidence level, and the traceability relationship.

[0014] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, implements the spatial data processing method provided in any of the first aspects above.

[0015] Fourthly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein: Memory is used to store computer programs; The processor is used to load and execute computer programs to cause the electronic device to perform spatial data processing methods as provided in any of the first aspects above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention discloses a spatial data processing method, apparatus, medium, and device. The method includes classifying raw spatial data according to the type of the object being processed to obtain first target data and second target data; wherein the first target data is vector data and the second target data is raster data or remote sensing data; performing a first processing on the first target data to obtain first processed data, and performing a second processing on the second target data to obtain second processed data; selecting a target processing method and executing it to obtain target data based on the multi-dimensional quality confidence and automatic processing threshold of the first and second processed data; establishing a traceability relationship based on the raw spatial data, the first processed data, the second processed data, the target data, and their derivation relationships; and outputting data based on the raw spatial data, the first processed data, the second processed data, the multi-dimensional quality confidence, and the traceability relationship. This invention first classifies the raw spatial data into vector data and raster or remote sensing data for separate processing, improving processing efficiency and accuracy. Then, it uses multi-dimensional quality confidence to distinguish whether automatic processing is used, improving the level of intelligent processing and ensuring accurate screening of anomalies. Finally, it establishes traceability relationships through the data and their derivation relationships in each stage, making the data output preview clearer and the traceability capability of data reapplication stronger, effectively improving the processing effect of geospatial data. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the electronic device structure for the hardware operating environment involved in this invention; Figure 2 A flowchart illustrating the spatial data processing method provided by the present invention; Figure 3 A flowchart illustrating the automatic detection and garbled character scoring and correction of vector Chinese encoding in the spatial data processing method provided by this invention; Figure 4 A flowchart illustrating the automatic detection and repair of Gauss-Kruger code prefixes in the spatial data processing method provided by this invention; Figure 5 A schematic diagram of the process of single-band raster classification / continuity determination and COG / slice adaptive generation in the spatial data processing method provided by the present invention; Figure 6 A flowchart illustrating one embodiment of the spatial data processing method provided by the present invention; Figure 7 A schematic diagram of the spatial data processing device provided by the present invention.

[0018] Wherein: 101-processor, 102-communication bus, 103-network interface, 104-user interface, 105-memory. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] See attached document Figure 1 , attached Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. The communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 105 may be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as at least one disk storage device. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0022] Those skilled in the art will understand that the appendix Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0023] As attached Figure 1 As shown, the memory 105, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a space data processing device.

[0024] In the appendix Figure 1 In the electronic device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in this invention can be set in the electronic device, and the electronic device calls the spatial data processing device stored in the memory 105 through the processor 101 and executes the spatial data processing method provided in the embodiment of this invention.

[0025] See attached document Figure 2 Based on the hardware device described in the foregoing embodiments, embodiments of the present invention provide a spatial data processing method, comprising the following steps: S10: Classify the original spatial data according to the type of the object being processed to obtain first target data and second target data; wherein, the first target data is vector data and the second target data is raster data or remote sensing data.

[0026] In the specific implementation process, the raw spatial data is classified according to the different object types, into primary target data (vector data) and secondary target data (raster or remote sensing data). Before this, in order to reduce the operation of manually selecting bands from complex catalogs and accurately extract the processing object from the file, multi-faceted scanning processing of the main processing object can be performed. That is, before classifying the raw spatial data according to the processing object type and obtaining the primary and secondary target data, the method also includes: The file extensions, archive members, and accompanying files of the original files are scanned to determine the objects to be processed and obtain the original spatial data.

[0027] In the specific implementation process, the system first receives raw files, such as single files or archive packages uploaded by users. It then scans the file extensions, internal members of the archive, and accompanying files to determine the target for processing. When the uploaded object is an archive package: it is decompressed to a temporary directory, its internal members are scanned, and candidate files are collected in groups according to formats such as GeoTIFF (TIFF raster image format with georeferenced information), ENVI (remote sensing raster data format based on header description), Erdas / PCI (raster data format generated by geoscientific software such as ERDAS IMAGINE or PCIGeomatics), and JPEG2000 (image compression and storage format based on the JPEG 2000 standard). Accompanying files with the same name, such as .hdr, .prj, .tfw, .aux.xml, and mtd_*.xml, are also identified. When JPEG2000 files exist in the archive: further identify whether they belong to the Sentinel-2 SAFE structure (Sentinel 2 standard archive product structure), check whether the path contains directory features such as .safe / , granule, img_data, etc., and assign priority to candidate files according to filenames: candidates with filenames containing tci_10m have the highest priority, followed by candidates with filenames containing tci, then ordinary JPEG2000 candidates conforming to the SAFE standard directory, and other JPEG2000 candidates have the lowest priority; select the main file in descending order of "priority, file size", and collect metadata XML from the same Sentinel-2SAFE root directory as companion files.

[0028] When the archive contains a geoscientific engineering catalog or a mixed results catalog: further perform engineering archive master file scoring. For Shapefile file groups (vector spatial data file groups composed of .shp, .dbf, .shx, etc. files), generate a completeness score based on the integrity, name matching relationship, file size, and spatial reference existence of .shp, .dbf, .shx, .prj, and .cpg files; for ENVI data, generate a pairing score based on whether .dat and .hdr files have the same name pairing, and whether .hdr contains the number of rows and columns, the number of bands, the data type, and map information; for MPJ ​​files (geographic information engineering project files), engineering catalogs, or map results catalogs, generate an engineering master file score based on the path hierarchy of the engineering master file, associated maps, transformed results, original files, and explanatory documents; for cases where original files, transformed results, and attachments exist simultaneously within the same archive, identify them as master data, derived results, or attachments based on path keywords, file format, companion relationships, and list fields.

[0029] When multiple valid candidates exist in the archive and the master file cannot be uniquely identified: instead of failing directly, a list of candidate master files, candidate scores, conflict reasons, and unpacking suggestions are generated. For example, when the same archive contains vector results, raster results, and document descriptions, a "multi-master type conflict" exception code is output, and the user is advised to archive the original data, transformed results, and attachments separately. When candidate files belong to the same project directory and have a clear master-slave relationship, the project master file is treated as the primary processing object, and other files are recorded as companion files or attachments.

[0030] Based on the aforementioned implementation method, the output includes the main processing file path, source format, list of accompanying files, candidate data types, candidate scores, conflict exception codes, and archive splitting suggestions.

[0031] S20: Perform a first processing on the first target data and obtain first processed data, and perform a second processing on the second target data and obtain second processed data.

[0032] In the specific implementation process, after clarifying the processing objects and their types through the aforementioned steps, different categories of data are processed separately according to two parallel processes: vector data undergoes the first processing, and raster data or remote sensing data undergoes the second processing. Specifically, the first processing of the first target data to obtain the first processed data includes: Multiple encoding attempts are performed on the first target data, and the target encoding is obtained based on the garbled character suspicion score and semantic hit score.

[0033] In the specific implementation process, refer to the appendix. Figure 3 The first step involves automatic detection and garbled character scoring and correction of vector Chinese encoding. When processing vector data such as Shapefiles, multiple encoding attempts and garbled character scoring are performed to automatically select the optimal encoding. First, construct a candidate encoding list. Prioritize reading the .cpg file with the same name as Shapefile, extract the declared encodings, and add that encoding to the top of the candidate list. Then add common encodings such as UTF-8, UTF-8-sig, GB18030, GBK, CP936, and Latin-1. Further expansion to Big5, UTF-16, etc., can be performed based on the project region. Standardize and deduplicate the encoding names, for example, normalize GB2312 to GB18030, and normalize 936 or ANSI 936 to GBK.

[0034] Secondly, attempt to read the vector file using each encoding; sequentially read the vector file using the candidate encodings, and after each successful read, score the field names and string field samples in the result for potential garbled characters, specifically: If a field name or field value contains the Unicode replacement character "U+FFFD", it is considered suspected garbled text. If a field name or field value contains two or more Latin-1 supplementary characters "U+00A1" to "U+00FF" but does not contain any CJK unified kanji "U+4E00" to "U+9FFF", it is also considered suspected garbled text. Garbled field names have a greater impact on the understanding of subsequent data, so each occurrence of garbled characters in a field name is scored. For string field values, only the first 20 non-empty values ​​of each column are sampled for detection, and each suspected garbled value is scored. The score is based on the field name being higher than the field value. For example, a field name is scored 2-5 points for each occurrence, while the field value is scored 1 point.

[0035] Then, select the optimal encoding; if the garbled character suspicion score of a certain read is 0, immediately adopt the encoding result; if all successful read results have some garbled character suspicion, adopt the read result with the lowest score; if all encodings fail to read, throw the last read exception and record the reason for the failure.

[0036] Semantic-assisted scoring of geoscientific fields is implemented. A geoscientific field dictionary and synonym rules are established, including but not limited to field terms such as "lithology," "structure," "mineralization," "age," "strata," "hole number," "drill hole number," "depth," "starting depth," "ending depth," "grade," "sample number," "coordinates," "east coordinates," "north coordinates," "elevation," "section," "map sheet," and "ore body," as well as corresponding pinyin, English abbreviations, or historical field aliases. For each candidate code reading result, the number of field name matches with the geoscientific field dictionary, the number of synonym matches, and the completeness of field combinations are statistically analyzed, and a semantic hit score is generated. The geoscientific field dictionary can be expanded according to mineral type, region, or project stage. For example, gold mining projects can add field terms such as "alteration," "silicification," "pyrite," "gold grade," "Au," "Ag," and "Cu"; drilling projects can add field terms such as "final hole depth," "hole inclination," "azimuth," "number of runs," and "recovery rate"; remote sensing interpretation projects can add field terms such as "anomaly level," "alteration information," "band combination," and "index name."

[0037] Finally, a comprehensive score is used for selection. The scrambling suspicion score and semantic hit score are combined to form a comprehensive coding score. The lower the scrambling suspicion score, the better, and the higher the geoscientific semantic hit score, the better. When the scrambling suspicion scores of two candidate codes are close, the result with the higher geoscientific semantic hit score is selected first. When a candidate code has a significantly high scrambling suspicion score, it is not adopted even if it has a small number of semantic hits. The final code, scrambling suspicion score, semantic hit field, and coding selection confidence are output.

[0038] Through the above steps, this invention can automatically select a better encoding for Chinese geoscientific attribute tables without relying on users to manually specify the encoding. It also improves the interpretability of encoding selection by utilizing the semantics of geoscientific fields, with field name weighting higher than field value weighting, which better meets the actual requirement of "prioritizing correct field structure" when geoscientific data is entered into the database.

[0039] In one embodiment, performing a first process on the first target data to obtain first processed data includes: Gaussian-Krüger band prefix detection and repair were performed on the first target data to obtain the detection and repair results. If the detection and repair results are abnormal, expandable candidates are generated and sorted according to the target requirements; Based on the sorting results, a re-repair plan is obtained.

[0040] In the specific implementation process, refer to the appendix. Figure 4 The first step is automatic detection and repair of Gauss-Kruger zone prefixes. Coordinate anomaly repair is performed on both vector data and borehole coordinate data. When the vector data has a coordinate reference system and the result is empty or exceeds the reasonable latitude and longitude range after coordinate transformation to WGS84, Gauss-Kruger zone prefix fault tolerance repair is triggered. First, obtain the original geometric bounding box, read the minimum and maximum X values. If the maximum X value is less than 1,000,000, it is determined that there is no banded prefix, and the repair is skipped.

[0041] Read the coordinate reference frame text and determine if it contains Gaussian-Kruger or transverse Mercator projection features such as Gauss or transverse_mercator. If not, skip the repair process.

[0042] Secondly, extract the false_easting parameter; if this parameter is greater than or equal to 1,000,000, it means that the coordinate system definition may already have a built-in zone offset, and the coordinate value prefix will no longer be stripped.

[0043] Then, the central meridian parameter (central_meridian) is extracted, and candidate zone numbers are generated. Candidate zones of 6 degrees are calculated using the formula zone=round((central_meridian+3.0) / 6.0); candidates of 3 degrees are calculated using the formula zone=round(central_meridian / 3.0). Simultaneously, candidate zone numbers are inferred from the coordinate values ​​themselves using the formula zone = floor(minx / 1,000,000).

[0044] For each candidate zone number, calculate the zone number prefix value: prefix = zone × 1,000,000, where prefix represents the million-digit coordinate prefix value corresponding to the Gauss-Kruger zone number, and zone represents the candidate zone number; and calculate the stripped local coordinates: local_minx = minx - prefix and local_maxx = maxx - prefix. If the stripped local coordinates are all within a reasonable range of [-500,000, 1,500,000], then the candidate zone number is confirmed as valid.

[0045] Finally, subtract zone×1,000,000 from the X coordinates of all geometric elements to obtain the repaired geometric objects. Then, re-execute the coordinate transformation to WGS84 and record the repair zone number, original coordinate system, and repair description in the preprocessed metadata for traceability.

[0046] For borehole coordinate data, the Y-coordinate is converted to a string. If it starts with 16 and the value is greater than or equal to 10,000,000, it is considered a mis-prefixed Y-coordinate. The first two digits 16 are removed, and the remaining part is converted to a floating-point number as the cleaned Y-coordinate. The original Y-coordinate, the cleaned Y-coordinate, and the processing description are retained. Subsequently, according to the exploration network to which the borehole belongs or the default coordinate system, the cleaned projected coordinates are converted to WGS84 latitude and longitude for map retrieval and unified spatial indexing.

[0047] Based on the above-mentioned band number repair, further coordinate anomaly expansion detection is performed to cover common coordinate entry and coordinate system configuration issues in geoscientific data: First, X / Y coordinate interchange detection is performed; coordinate transformation is performed with the original X / Y order and the interchanged Y / X order respectively, and the transformation results are compared with the study area boundary, mining area boundary, administrative area range or project preset latitude and longitude range; if the interchanged result falls into the target range and the original order result exceeds the target range, then X / Y interchange candidate repair schemes are generated.

[0048] Secondly, anomaly detection of meter / kilometer units; detection of the magnitude and spatial range of projected coordinate values. When the coordinate value or distance between points differs from the expected magnitude of the target coordinate system by about 1000 times, unit repair candidates are generated by multiplying by 1000 or dividing by 1000 respectively, and it is verified whether the repaired result falls within the study area.

[0049] Next, candidate coordinate systems are selected. A set of candidate coordinate systems is constructed based on the data source, exploration network name, project area, central meridian, EPSG code, .prj text, and study area boundary. Examples include Beijing 1954 Gauss-Kruger, Xi'an 1980 Gauss-Kruger, CGCS2000 Gauss-Kruger, and WGS84 UTM. Each candidate coordinate system is transformed and scored based on whether it falls within the study area, whether its spatial morphology is preserved, and whether its point, line, and surface scales are reasonable.

[0050] Outlier coordinate identification: For point sets, boreholes, or sampling point data, calculate the distance between the transformed point and the center of the overall point set, the project boundary, and neighboring points. If individual points deviate significantly from the main range, mark them as outliers. Instead of directly modifying the original coordinates, output an outlier exception code and review suggestions.

[0051] Finally, candidate repairs are sorted. A candidate repair list is formed by removing band numbers, swapping X / Y coordinates, converting units, and transforming coordinate systems. This list is then sorted according to the study area coverage, coordinate range rationality, spatial morphology preservation, field semantic matching, and historical project rule coverage. Automatic repair is performed when the highest-scoring candidate exceeds the automatic repair threshold; when the highest score is within the pending confirmation range, candidate repair schemes are output for manual confirmation.

[0052] During the above processing, exception codes and processing status can be recorded in the preprocessed metadata or processing logs, classifying issues into exception codes and risk levels, for example: COORDINATE_REPAIR_CANDIDATES indicates candidate solutions for coordinate repair; OUTLIER_COORDINATE indicates outlier coordinates; ENCODING_LOW_CONFIDENCE indicates low confidence in character encoding. RASTER_CLASSIFICATION_AMBIGUOUS indicates that the raster classification / continuity determination result is unclear; ARCHIVE_MULTI_PRIMARY indicates that there are multiple possible master data files in the archive package; MISSING_SIDECAR_FILE indicates that a necessary or recommended companion file is missing.

[0053] This invention generates candidate zone numbers from both the CRS central meridian and the coordinate values ​​themselves, and verifies their reasonableness through the stripped local coordinate range, avoiding reliance on a single coordinate system parameter or manual experience. When conventional coordinate transformation fails or the result is empty, automatic fault-tolerant repair can be performed, improving the success rate of historical surveying data import.

[0054] In one embodiment, the second target data is subjected to a second processing to obtain second processed data, including: The second target data is classified to obtain categorical and continuous data. Based on categorical and continuous data, COG derived files and Web map tiles are generated.

[0055] In the specific implementation process, refer to the appendix. Figure 5 The second processing step involves single-band raster classification / continuity determination and adaptive COG / slice generation. When the data being processed is single-band raster data, it automatically determines whether it should be processed as classified or continuous data. First, the built-in color table is detected; an attempt is made to read the built-in color table of the first band. If the color table exists, the raster is determined to be classified data, and the pixel values ​​and color mapping in the color table are used as the classification rendering configuration.

[0056] Secondly, integer approximation detection is performed. If there is no built-in color table, the first band data after sampling is read, and invalid values ​​such as non-digit NaN and infinite Inf are removed. For each valid pixel value, the distance between it and the nearest integer is calculated, and the 95th percentile of all distance values ​​is calculated.

[0057] Then, the number of unique values ​​is detected; if the 95th percentile is less than 0.05, it means that most pixel values ​​are close to integers. The number of unique values ​​is counted for the rounded pixel values. If the number of unique values ​​does not exceed the preset upper limit of 32, it is determined to be classified data and the color is cyclically assigned according to the preset classification color palette; otherwise, it is determined to be continuous data. In other embodiments, the integer distance percentile can be the 90th to 99th percentile, the distance threshold can be 0.01 to 0.1, and the upper limit of the number of unique values ​​can be 16, 32, 64 or other preset values.

[0058] Finally, continuous data processing: if the integer approximation condition is not met, or the number of unique values ​​exceeds the threshold, it is determined to be continuous data, and 2% to 98% percentile stretching is used as the default rendering configuration.

[0059] This step outputs rendering modes such as categorical or grayscale, category color tables, number of unique values, stretching parameters, and band statistics.

[0060] Based on the raster classification / continuous attribute determination results, adaptively generate COG derived files and Web map tiles: First, COG status detection: determine if the source file is already a Cloud Optimized GeoTIFF. If the source file is a GeoTIFF that has been tiled and has an overview layer, or is explicitly marked as a COG, then the source file is directly recorded as a usable COG.

[0061] Then, COG conversion is performed; if the source file is not COG, automatic conversion is performed. The conversion parameters include: the driver type is COG, the block size is 512×512 pixels, the compression algorithm is lossless LZW compression, and large files use the BigTIFF=IF_SAFER strategy, that is, when the output file may exceed the traditional TIFF file size limit, the BigTIFF large file format is automatically adopted and multi-threaded compression is enabled.

[0062] In one embodiment, COG derived files and Web map tiles are generated based on categorical data and continuous data, including: COG state detection and tile range calculation are performed based on classified data and continuous data respectively to obtain detection results and calculation results. If the detection result is negative, COG overview layer resampling is performed to generate COG derived files; for classified data, nearest neighbor resampling is used, and for continuous data, triple convolution resampling is used. Based on the calculation results, reprojection sampling is performed to generate Web map tiles; for categorical data, the nearest neighbor reprojection sampling method is used, and for continuous data, the bilinear reprojection sampling method is used.

[0063] In the specific implementation process, adaptive selection of resampling for the overview layer is performed. If the output is categorical data, the COG overview layer uses nearest neighbor resampling to preserve category boundaries; if the output is continuous data, cubic convolutional resampling is used to obtain a smoother continuous variation effect. Web map tile generation: The tile range under Web Mercator EPSG:3857 is calculated based on the requested z / x / y tile coordinates, and the source raster is reprojected and sampled to 256×256 pixels. In other implementations, 512×512 pixels can also be used. The map target coordinate system can be other map display coordinate systems depending on the business platform. If the rendering mode is categorical data, the nearest neighbor method is used for reprojection sampling; if the rendering mode is continuous or multi-band data, the bilinear method is used. The final output is an RGBA PNG tile, i.e., a PNG image with an alpha channel, with invalid areas set to transparent.

[0064] Therefore, the semantic type of raster data will affect the COG overview layer and the way Web tiles are generated, avoiding the smoothing of classification maps by continuous resampling, and also avoiding the discretization of continuous data.

[0065] S30: Based on the multi-dimensional quality confidence of the first and second processed data and the automatic processing threshold, select the target processing method to execute and obtain the target data.

[0066] In the specific implementation process, quality evaluation results, namely multi-dimensional quality confidence scores, can be generated at each step. These results are then summarized into a preprocessing and pre-inspection report, changing the existing processing methods that only output success or failure results. This approach achieves evaluation from multiple dimensions, automatically distinguishing data that is difficult to judge manually. Specifically, the multi-dimensional quality confidence scores include encoding detection confidence, coordinate restoration confidence, raster classification confidence, and master file selection confidence, among which: Encoding confidence score: The encoding confidence score is generated based on factors such as the difference between the comprehensive scores of the best and second-best encodings, whether the garbled character suspicion score is 0, the number of semantic hits in geoscientific fields, and whether the .cpg declaration is consistent with the final encoding.

[0067] Coordinate repair confidence: The coordinate confidence is generated based on factors such as whether the repaired result falls within the boundary of the study area, the difference between candidate repair schemes, whether the coordinate range is reasonable, whether the spatial morphology is maintained, and whether the source of the coordinate system candidate is reliable.

[0068] Raster classification confidence: The classification confidence is generated based on the existence of a built-in color table, the difference between the integer distance to the 95th percentile and the threshold, the number of unique values, whether the category area distribution is reasonable, and the file metadata description.

[0069] Master file selection confidence: The master file selection confidence is generated based on the candidate file priority difference, companion file integrity, project directory structure, file size, metadata existence and list field consistency.

[0070] For each exception code, actionable suggestions are provided, such as supplementing the .prj file, splitting the archive package, confirming the coordinate system, confirming the classification threshold, or manually selecting the master file.

[0071] When a certain confidence level is higher than the automatic processing threshold, the best candidate result is automatically adopted, and the basis for automatic processing is recorded. When the confidence level is in the pending confirmation range, a list of candidate solutions is output, showing the user the candidate encoding, candidate coordinate repair, candidate main file or candidate raster type, and displaying the score, hit rule and expected impact of each candidate. After user confirmation, subsequent preprocessing continues according to the confirmation result. When the confidence level is lower than the rejection threshold or there is a serious conflict, the formal entry into the database is stopped, and the error reason and repair suggestions are output.

[0072] The results of manual confirmation are written into the project rule base. When the same project, the same data source, the same exploration network, or the same file naming rule appears again, the priority of the same rule is increased, thereby forming an iterative semi-automatic preprocessing capability.

[0073] S40: Establish a tracing relationship based on the original spatial data, the first processed data, the second processed data, the target data, and their derived relationships.

[0074] In the specific implementation process, after geospatial data undergoes encoding correction, coordinate restoration, format conversion, thumbnail generation, COG conversion, and tile service publication, it will generate multiple derived results. Existing technologies usually only save the final file or preview address. This invention establishes a traceability relationship between the original file, the restoration result, and the derived result by using the original data and the processing data of each step, providing a more reliable foundation for the subsequent application of data, such as auditing, reproduction, and determination of responsibility for the quality of results.

[0075] First, generate resource identifiers or object identifiers for the original file, the repaired intermediate file, the COG derived file, the thumbnail, the Web tile service, the attribute preview result, and the coordinate cleaning result, respectively.

[0076] Secondly, record the type of derivation relationship, such as "derived from the original file", "generated from the coordinate repair result", "associated as a borehole attachment", "as a member of the same project directory", "generated as a preview cache", etc.

[0077] Then, record the processing parameters for each derived result, including the final code, coordinate repair scheme, raster classification judgment parameters, COG transformation parameters, resampling method, slice coordinate system, human verifier and verification time.

[0078] Finally, when the original file, preprocessing algorithm, or manual verification rules change, the preview file, COG file, slice service, or spatial index that needs to be regenerated can be located based on the derivation relationship to avoid inconsistencies between historical results and current rules.

[0079] Relationships can be traced and stored in relational database tables, graph databases, resource relationship tables, or metadata JSON structures, as long as they can record the original object, derived object, relationship type, processing parameters, and processing time.

[0080] S50: Data output is based on raw spatial data, first-processed data, second-processed data, multi-dimensional quality confidence, and traceability relationships.

[0081] In the specific implementation process, after the aforementioned processing is completed, multi-dimensional outputs are achieved, such as: Data source identification results: Includes main file, source format, accompanying files, data type, and candidate file selection instructions. Vector preview results: Includes correctly read attribute fields, sample values, semantic hit rate of geoscientific fields, geometric type, spatial extent, source coordinate system, display coordinate system, and coordinate repair instructions. Raster preview results: Includes source coordinate system, display extent, band information, classification / continuity determination results, color table or stretching parameters, COG status, tile service address, and thumbnail. Coordinate cleaning traceability results: Includes original coordinates, cleaned coordinates, target latitude and longitude, candidate repair schemes, and repair instructions. Quality pre-inspection results: Includes confidence level, anomaly code, risk level, and repair suggestions for each step. Derivative result traceability results: Includes the relationship between original files, intermediate results, COG files, thumbnails, tile services, and manual confirmation records.

[0082] This invention first classifies the raw spatial data into vector data and raster or remote sensing data for separate processing, improving processing efficiency and accuracy. Then, it uses multi-dimensional quality confidence to distinguish whether automatic processing is used, improving the level of intelligent processing and ensuring accurate screening of anomalies. Finally, it establishes traceability relationships through the data and their derivation relationships in each stage, making the data output preview clearer and the traceability capability of data reapplication stronger, effectively improving the processing effect of geospatial data.

[0083] See attached document Figure 6 In the attached Figure 6 The present invention will be further described in the embodiments shown below: First, the user uploads a geospatial file or archive package. After identifying candidate files and determining the processing object, the process proceeds to score candidate files and select the main archive file. Once the main processing object is determined, the type of the processing object is judged. If it is vector data, the following steps are executed in sequence: automatic detection of Chinese vector encoding and semantic scoring of geospatial fields, automatic detection and candidate repair of coordinate anomalies, and generation of vector preview and spatial metadata. If it is raster or remote sensing data, the following steps are executed in sequence: raster or remote sensing data main file identification, automatic determination of single-band raster classification / continuous attributes, COG conversion, and adaptive generation of web map tiles.

[0084] Each processing step is evaluated, generating multi-dimensional quality confidence scores. Combined with anomaly codes and repair suggestions, it is determined whether the confidence score reaches the automatic processing threshold. If it does, the best processing result is automatically adopted; otherwise, candidate solutions are output for manual confirmation, and the confirmation results are written into the project rule base. Finally, a traceability relationship between the original file, the repair result, and the derived result is established, and multi-dimensional data such as metadata, preview results, COG files, slice services, quality reports, and traceability chains are implemented.

[0085] See attached document Figure 7Based on the same inventive concept as in the foregoing embodiments, this embodiment of the invention also provides a spatial data processing apparatus, comprising: The classification module is used to classify the raw spatial data according to the type of the object being processed, and to obtain first target data and second target data; wherein, the first target data is vector data, and the second target data is raster data or remote sensing data; The processing module is configured to perform a first processing on the first target data and obtain the first processed data, and to perform a second processing on the second target data and obtain the second processed data. The confidence module is used to select the target processing method and obtain the target data based on the multi-dimensional quality confidence of the first and second processed data and the automatic processing threshold. A module is established to create a tracing relationship based on the original spatial data, the first processed data, the second processed data, the target data, and their derived relationships. The output module is used to output data based on the original spatial data, the first processed data, the second processed data, the multi-dimensional quality confidence level, and the traceability relationship.

[0086] Those skilled in the art should understand that the division of the various modules in the embodiments is merely a logical functional division. In actual applications, they can be fully or partially integrated into one or more actual carriers. These modules can be implemented entirely in software through processing unit calls, entirely in hardware, or a combination of software and hardware. It should be noted that each module in the spatial data processing device in this embodiment corresponds one-to-one with each step in the spatial data processing method in the aforementioned embodiments. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned spatial data processing method, which will not be repeated here.

[0087] Based on the same inventive concept as in the foregoing embodiments, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, implements the spatial data processing method provided in the embodiments of the present invention.

[0088] Based on the same inventive concept as in the foregoing embodiments, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein: Memory is used to store computer programs; The processor is used to load and execute computer programs to enable electronic devices to perform spatial data processing methods as provided in embodiments of the present invention.

[0089] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a device including one or any combination of the above-mentioned memories. The computer may be a variety of computing devices, including smart terminals and servers.

[0090] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0091] As an example, executable instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0092] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a multimedia terminal device (which may be a mobile phone, computer, television receiver, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0094] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A spatial data processing method, characterized in that, Includes the following steps: The raw spatial data is classified according to the type of object being processed to obtain first target data and second target data; wherein, the first target data is vector data and the second target data is raster data or remote sensing data; The first target data is subjected to a first processing to obtain first processed data, and the second target data is subjected to a second processing to obtain second processed data; Based on the multi-dimensional quality confidence and automatic processing threshold of the first and second processed data, the target processing method is selected and executed to obtain the target data. A tracing relationship is established based on the original spatial data, the first processed data, the second processed data, the target data, and their derivation relationships; Data output is performed based on the original spatial data, the first processed data, the second processed data, the multi-dimensional quality confidence level, and the traceability relationship.

2. The spatial data processing method according to claim 1, characterized in that, The first processing of the first target data to obtain the first processed data includes: Multiple encoding attempts are performed on the first target data, and the target encoding is obtained based on the garbled character suspicion score and semantic hit score.

3. The spatial data processing method according to claim 1, characterized in that, The first processing of the first target data to obtain the first processed data includes: Gaussian-Krüger band prefix detection and repair are performed on the first target data to obtain the detection and repair results; If the detection and repair results are abnormal, an expanded candidate list is generated and sorted according to the target requirements; Based on the sorting results, a re-repair plan is obtained.

4. The spatial data processing method according to claim 1, characterized in that, The second processing of the second target data to obtain second processed data includes: The second target data is classified to obtain categorical data and continuous data; Based on the categorized data and the continuous data, COG derived files and Web map tiles are generated.

5. The spatial data processing method according to claim 4, characterized in that, The process of generating COG derived files and Web map tiles based on the categorized data and the continuous data includes: COG state detection and tile range calculation are performed based on the classified data and the continuous data, respectively, to obtain detection results and calculation results. If the detection result is negative, COG overview layer resampling is performed to generate COG derived files; wherein, the nearest neighbor resampling method is used for classified data, and the cubic convolution resampling method is used for continuous data. Based on the calculation results, reprojection sampling is performed to generate Web map tiles; wherein, in the case of classified data, the nearest neighbor reprojection sampling method is used, and in the case of continuous data, the bilinear reprojection sampling method is used.

6. The spatial data processing method according to claim 1, characterized in that, Before classifying the raw spatial data according to the type of the object being processed to obtain the first target data and the second target data, the method further includes: The file extensions, archive members, and accompanying files of the original file are scanned to determine the processing objects and obtain the original spatial data.

7. The spatial data processing method according to claim 1, characterized in that, The multi-dimensional quality confidence includes encoding detection confidence, coordinate repair confidence, raster classification confidence, and master file selection confidence.

8. A spatial data processing device, characterized in that, include: The classification module is used to classify the raw spatial data according to the type of the object being processed, and to obtain first target data and second target data; wherein, the first target data is vector data, and the second target data is raster data or remote sensing data; The processing module is configured to perform a first processing on the first target data and obtain first processed data, and to perform a second processing on the second target data and obtain second processed data; The confidence module is used to select a target processing method and obtain target data based on the multi-dimensional quality confidence of the first processed data and the second processed data and the automatic processing threshold. A module is established to establish a tracing relationship based on the original spatial data, the first processed data, the second processed data, the target data, and their derivation relationships; The output module is used to output data based on the original spatial data, the first processed data, the second processed data, the multi-dimensional quality confidence score, and the traceability relationship.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it implements the spatial data processing method as described in any one of claims 1-7.

10. An electronic device, characterized in that, Including processor and memory, of which: The memory is used to store computer programs; The processor is used to load and execute the computer program to cause the electronic device to perform the spatial data processing method as described in any one of claims 1-7.

Images