An AI-ready dataset construction method and device for cryosphere multi-source data

Abstract

The present application relates to the field of cryosphere discipline, data science and artificial intelligence technology, and particularly relates to a kind of AI ready dataset construction method and device for cryosphere multi-source data.The method forms a basic dataset by standardizing cryosphere multi-source data;Then object-pixel mapping, pixel-grid mapping, object-grid mapping and time mapping are established, and multi-scale collaborative construction is performed according to the above mappings, respectively obtaining spatial corresponding results, time phase screening results, semantic consistency constraint results and checking results;According to the above four results, cryosphere multi-source data is fused to obtain AI ready dataset for cryosphere multi-source data.The present application realizes the batch processing and automated AI ready processing of multi-source heterogeneous cryosphere scientific data at the data center level, and can be widely applied to cryosphere research tasks such as glacier change monitoring, snow identification and permafrost discrimination.

Description

Technical Field

[0001] This invention relates to the fields of cryospheric science, data science and artificial intelligence, specifically to a method and apparatus for constructing AI-ready datasets for multi-source cryospheric data. Background Technology

[0002] AI-ready data refers to datasets that can be efficiently, securely, and reproducibly used to train, evaluate, and deploy machine learning and artificial intelligence models. Data quality is crucial to model training results; if data contains a large amount of noise, errors, or redundancy, the model will learn incorrect features and relationships, leading to performance degradation. High-quality data can provide models with more accurate and reliable learning samples, helping them learn more realistic features and relationships, thereby improving their performance and generalization ability. AI-ready data emphasizes data accessibility, understandability, interoperability, high quality, and efficient management to provide a solid foundation for artificial intelligence applications. With the exponential growth of large model parameter scales and the continuous expansion of application scenarios, easily accessible high-quality publicly available text data on the internet is quickly being consumed. Although traditional data centers, especially scientific data centers, store large amounts of raw data and data products, this data is often not AI-ready and cannot be directly used for model training.

[0003] In the field of cryosphere geoscience, there are particularly prominent issues such as data fragmentation, inconsistent storage standards, and varying quality. Cryosphere scientific data includes data from field observations, surveys, numerical simulations, testing and analysis, remote sensing monitoring and inversion, and statistical analysis. It typically exhibits complex characteristics such as strong spatiotemporal dependence, multi-source heterogeneity, varying resolutions, and massive data volumes. Therefore, it is necessary to strengthen data governance from the source to improve quality and the degree of sharing. Unlike general geoscience data, cryosphere data also has significant characteristics such as seasonal snow cover, ablation season variations, freeze-thaw transitions, strong topographic constraints, and ice-snow-water phase changes. In the same thematic task, object-level glacier / glacial lake boundary data, pixel-level remote sensing observation data, and raster-level climate-driven or numerical simulation data are often involved simultaneously. Different data have significant differences in spatial scale, temporal granularity, semantic granularity, and physical meaning. Therefore, the AI ​​readiness of cryosphere data not only requires that individual datasets be formatted in a standardized and quality-controllable manner, but also requires that data at the object, pixel, and raster scales be organized in a unified manner within a multi-scale collaborative framework to form a standardized dataset that can be directly discovered, filtered, combined, trained, and inferred by artificial intelligence models. In order to better support cryosphere models and artificial intelligence applications, it is urgent to gradually make existing cryosphere scientific data AI-ready, and promote the comprehensive adaptive transformation of data resources from application architecture to service model.

[0004] Currently, research on AI readiness primarily focuses on how data resources can promote AI technology innovation, or how the construction of data infrastructure itself can achieve innovation. Research on AI-ready data technologies generally targets only a specific type of data or a single dataset, studying data cleaning, imputation, spatiotemporal alignment, feature processing, and annotation processes. Domain data centers store thousands of datasets, each with different characteristics, inconsistent metadata, data formats, and data quality. This is especially true in the cryosphere research field, where data sources are diverse, formats are complex, spatiotemporal features are distinct, spatiotemporal resolutions are inconsistent, and data volumes are enormous. Achieving an "AI-ready" state for each dataset requires a complex process that is inefficient and costly. The cryosphere field has long faced a core problem that has not yet been effectively solved by existing technologies: object-level cataloging data (such as glacier boundary vectors), pixel-level remote sensing observation data, and raster-level climate-driven or numerical simulation data are processed independently when constructing AI training datasets. This requires manual object-by-object matching, time-by-time relative alignment, and variable-by-variable correlation, lacking a collaborative mechanism that enables the joint organization, joint screening, and joint construction of data at all three scales within the same technical framework.

[0005] In existing technologies, such as establishing the correspondence between objects, pixels, and grids one by one through spatial overlay of Geographic Information Systems (GIS) or scripts, although it is possible to establish mappings for individual objects or single datasets, it is not feasible for manual or semi-automatic methods when facing thousands of datasets, hundreds of thousands of object records, and multi-temporal observations in data centers. This is because it is impossible to establish mappings in batches, update them dynamically, and verify them across scales. In particular, in the cryosphere, unique processes such as seasonal snow cover, ablation season changes, freeze-thaw transition, and cloud-snow mixing further exacerbate the difficulty of timeliness matching and semantic consistency contradictions between data at different scales, making it difficult to extend existing conventional methods to large-scale, multi-type collaborative construction scenarios at the data center level. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a method and apparatus for constructing an AI-ready dataset for multi-source cryosphere data. The method standardizes multi-source cryosphere data to form a basic dataset; then, it establishes object-cell mapping, cell-grid mapping, object-grid mapping, and temporal mapping. Based on these mappings, it performs multi-scale collaborative construction to obtain spatial correspondence results, temporal filtering results, semantic consistency constraint results, and inspection results. Based on these four results, the multi-source cryosphere data is fused to obtain an AI-ready dataset for multi-source cryosphere data. This invention achieves batch and automated AI-ready processing of multi-source heterogeneous cryosphere scientific data at the data center level, and can be widely applied to cryosphere research tasks such as glacier change monitoring, snow cover identification, and permafrost discrimination.

[0007] This invention adopts the following technical solution: a method for constructing an AI-ready dataset for multi-source cryosphere data, comprising: Acquire multi-source cryosphere data and perform standardization processing to obtain a basic dataset; the multi-source data includes at least tabular data, vector data, and raster data. Based on the basic dataset, obtain object units and object attribute information for object-level data, pixel units and pixel attribute information for pixel-level data, and grid units and grid attribute information for grid-level data; Establish object-cell mapping based on the spatial correspondence between object units and cell units; establish cell-grid mapping based on the spatial correspondence between cell units and grid units; establish object-grid mapping based on the spatial correspondence between object units and grid units. Establish a time mapping based on object attribute information of object-level data, pixel attribute information of pixel-level data, and grid attribute information of grid-level data; The set of cell units corresponding to the object unit is obtained according to the object-cell mapping, the set of grid units corresponding to the cell unit is obtained according to the cell-grid mapping, and the set of grid units corresponding to the object unit is obtained according to the object-grid mapping, thus obtaining the spatial correspondence result; The object unit, the pixel unit, and the grid unit are unified in time scale according to the time mapping to obtain unit data after time scale unification; and the phase of the unit data after time scale unification is filtered based on the physical constraint rules of the cryosphere melting period, snow cover period, or freeze-thaw transition period to obtain phase filtering results. Obtain the change label of the object unit based on the object attribute information, obtain the spectral change trend of the pixel unit based on the pixel attribute information, and obtain the climate-driven change trend of the grid unit based on the grid attribute information. The change label of the object unit is compared with the spectral change trend of the corresponding pixel unit and the climate-driven change trend of the grid unit for logical consistency. If the change label is inconsistent with the direction of the spectral change trend or the climate-driven change trend, the object unit is marked as a semantically inconsistent sample with the corresponding pixel unit and grid unit. Otherwise, it is marked as a consistent sample. All marking results are output to obtain the semantic consistency constraint result. Perform primary key conflict checks, spatial mismatch checks, and temporal mismatch checks on object-cell mapping, cell-mesh mapping, and object-mesh mapping, and obtain the check results; Based on the spatial correspondence results, temporal filtering results, semantic consistency constraint results, and inspection results, the multi-source data of the cryosphere are fused to obtain an AI-ready dataset for multi-source data of the cryosphere.

[0008] Furthermore, the standardization process specifically includes: format standardization, metadata construction, quality control, and feature processing.

[0009] Furthermore, the process of acquiring multi-source cryosphere data and standardizing it, including format standardization, metadata construction, quality control, and feature generation for tabular data, is as follows: The records in the table data are used as the first processing unit, where: The format standardization process includes: constructing a corresponding composite primary key based on the table type; unifying the time expression format, spatial location expression method, quality status mark, and missing value encoding rules of the first processing unit; and mapping the field names, units, classification codes, and attribute meanings from different sources in the first processing unit to a unified field set, measurement unit system, encoding system, and semantic tag system, respectively. The metadata construction includes: labeling the attribute meanings of each field in the first processing unit as unified semantic tags, supplementing relation anchor fields in each record, and generating corresponding metadata description files; The quality control includes: performing quality detection on the first processing unit that has duplicate records, compound primary key conflicts, out-of-order records, and values ​​exceeding the preset physical range; and performing marking, correction, elimination, and completion operations according to the quality detection results. The characterization process includes: deriving seasonal representation, ablation period identifier, and time-series change statistical features based on the first processing unit after the above processing, and outputting standardized table files and corresponding record index tables, object-record association tables, and time window index tables.

[0010] Furthermore, construct the corresponding composite primary key based on the table type, specifically as follows: The table types include site observation tables, object attribute tables, and statistical time series tables; For station observation tables, construct a composite primary key using station identifier and observation time; For tables with object attributes, a composite primary key is constructed using the object identifier; For statistical time series tables, a composite primary key is constructed using region identifier, time identifier, and variable identifier.

[0011] Furthermore, the process of acquiring multi-source cryosphere data and performing standardization processing, including format standardization, metadata construction, quality control, and feature generation for raster data, is as follows: Using the pixel units and grid units in the raster data as the second processing units, where: The format standardization process includes: unifying the spatial reference and spatial index of each second processing unit, resampling the spatial resolution of raster data from different sources, aligning the time axis, and mapping each band or variable to a unified naming convention. The metadata construction includes: generating an element index table and a time index mapping table corresponding to the second processing unit, and establishing a corresponding metadata description file; The quality control includes: identifying cloud, snow and shadow regions in the second processing unit based on quality bands or mask files and performing mask processing, and performing interpolation or marking on missing measurement regions, and unifying the missing measurement value encoding for each band; The feature processing includes: deriving spectral features and spatial texture features from the quality-controlled pixel units; deriving terrain constraint features from the DEM raster data spatially registered with the pixel units; deriving temporal variation features from the multi-period grid unit sequence after quality control; and adding temporal constraint markers; and outputting a standardized raster file.

[0012] Furthermore, the process of acquiring multi-source cryosphere data and performing standardization processing, including format standardization, metadata construction, quality control, and feature generation for vector data, is as follows: The object unit in the vector data is used as the third processing unit, where: The format standardization process includes: unifying the coordinate reference system and attribute field naming rules of the third processing unit, and performing legalization repair on the geometric objects in the third processing unit; The metadata construction includes: supplementing the object identifier in the attribute information corresponding to the third processing unit, generating an object attribute table and an object adjacency table, and establishing a corresponding metadata description file; The quality control includes: checking the integrity of the attribute information of the third processing unit and performing topology repair on the geometric objects in the third processing unit; The feature processing includes: extracting features from three levels: geometry, attributes, and spatial relationships; converting the third processing unit into a unit attribute table; rasterizing the boundaries of the third processing unit; organizing the relationships between the third processing units into a unit relationship table; and outputting a standardized vector file.

[0013] Furthermore, the metadata description file constructed from the metadata specifically includes: basic metadata, file organization metadata, attribute metadata, semantic metadata, and relation mapping metadata; The basic metadata includes description file name, version, reference identifier, spatiotemporal resolution, data volume, and publication time; The file organization metadata includes descriptions of file type, file name, data format, projection coordinate system, and file path; The attribute metadata includes attribute name, data type, attribute meaning, and attribute domain, and marks the scale type to which the fields in the cryosphere multi-source data belong. The semantic metadata includes the task semantic types supported by the basic dataset, as well as the applicable objects, applicable regions, and applicable time phases corresponding to each task semantic type; the task semantic types include at least one or more of classification tasks, recognition tasks, and prediction tasks; The relational mapping metadata includes spatial and temporal correspondences between object-level data, cell-level data, and grid-level data, and includes at least: object identifier, cell identifier, grid identifier, spatial coverage relationship, temporal correspondence, aggregation method, sampling method, and quality constraint information.

[0014] Furthermore, when filtering the temporal phase of the unit data after unifying the time scale according to the preset rules, the preset rules adopt at least one of the following: the melting season filtering rule, the snow accumulation period removal rule, and the freeze-thaw conversion filter.

[0015] Furthermore, the object attribute information includes object identifier and observation time; the cell attribute information includes cell spatial index and imaging time; and the grid attribute information includes grid spatial index and driving time step.

[0016] Furthermore, the present invention also proposes an AI-ready dataset construction device for multi-source cryosphere data, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the AI-ready dataset construction method for multi-source cryosphere data as described above.

[0017] The beneficial effects of this invention are as follows: This invention categorizes cryosphere scientific data into three types: tabular data, vector data, and raster data, and establishes differentiated processes for format standardization, metadata construction, quality control, and feature processing for each type. This enables the conversion of raw data into a unified, quality-controllable, and semantically clear basic dataset, solving the problem of low efficiency in processing thousands of datasets one by one in existing technologies. Furthermore, the metadata constructed by this invention includes basic metadata, file organization metadata, attribute metadata, semantic metadata, and relational mapping metadata, enabling the dataset to be automatically discovered, filtered, combined, and invoked by artificial intelligence models. Further, by establishing object-cell mapping and cell-grid mapping... The invention incorporates object-level, cell-level, and raster-level data into a unified indexing system through object-level, object-level, and temporal mapping, enabling multi-scale collaborative construction across four dimensions: spatial collaboration, temporal collaboration, semantic collaboration, and collaborative verification. This solves the core problems of existing technologies, such as the independent processing of data at each of the three scales, the need for manual object-by-object matching, temporal alignment, and variable-by-variable association. Furthermore, this invention establishes a complete thematic dataset construction process for typical cryosphere tasks such as glacier change monitoring, snow cover identification, permafrost discrimination, and glacial lake identification. It also introduces cryosphere-specific physical constraints such as ablation season screening, cloud and snow obfuscation control, and terrain shadow correction, making the generated thematic datasets more effective and applicable in specific domain tasks. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a method for constructing an AI-ready dataset for multi-source cryosphere data according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an AI-ready dataset construction device for multi-source cryosphere data according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] A flowchart illustrating a method for constructing an AI-ready dataset for multi-source cryosphere data according to an embodiment of the present invention is shown below. Figure 1 As shown, it includes: We acquired multi-source cryosphere data and performed standardization processing to obtain the basic dataset; In this embodiment of the invention, cryosphere multi-source data refers to scientific data generated from different observation methods, sensors, and simulation methods in cryosphere scientific research. Based on differences in data organization, cryosphere multi-source data can be categorized into three types: tabular data, vector data, and raster data. Tabular data includes structured data with records as the basic unit, such as station observation records and statistical tables; vector data includes geographic vector data with object elements as the basic unit, such as glacier catalogs, glacier-lake boundaries, and watershed divisions; and raster data includes raster data with pixels or grids as the basic unit, such as remote sensing images, topographic data, and reanalysis data.

[0022] Standardization processing specifically includes format standardization, metadata construction, quality control, and feature processing. Format standardization addresses the consistency of data storage formats and coding standards; metadata construction addresses the discoverability, understandability, and callability of data; quality control addresses the accuracy, completeness, and consistency of data; and feature processing addresses the computability and modelability of data. After these four stages, the raw data is transformed into a basic dataset that can be directly used in subsequent processes.

[0023] Since cryosphere multi-source data includes tabular data, vector data, and raster data, and the standardization processing methods for each type of data are different, the standardization processing methods for each data type are described separately in this embodiment of the invention: It should be noted that the object-level data, pixel-level data, and grid-level data proposed in this embodiment of the invention are classifications based on the spatial scale type of data units, which belong to a different classification dimension than tabular data, vector data, and raster data. Object-level data, pixel-level data, and grid-level data are used to describe the spatial granularity and semantic hierarchy of data units, while tabular data, vector data, and raster data describe the storage and organization of data. The two can coexist; for example, regular grid data belongs to raster data in terms of data format, but belongs to grid-level data in terms of scale classification; remote sensing imagery belongs to raster data in terms of data format, but belongs to pixel-level data in terms of scale classification. Through the above distinction, raster data, as a higher-level concept of data format, includes both pixel-level and grid-level data, while pixel-level and grid-level, as lower-level concepts of spatial scale classification, point to different spatial granularities and physical meanings, and there is no inclusion or overlap between the two at the scale level.

[0024] For tabular data, the records in the table are used as the first processing unit, where: The format standardization process includes: constructing corresponding composite primary keys based on table types, including station observation tables, object attribute tables, and statistical time series tables; for station observation tables, constructing composite primary keys using station identifiers and observation times; for object attribute tables, constructing composite primary keys using object identifiers; and for statistical time series tables, constructing composite primary keys using region identifiers, time identifiers, and variable identifiers. The hierarchy identifier is used to distinguish records from different observation levels or depths at the same station and observation time, such as soil temperature at different depths or wind speed at different altitudes; the source identifier is used to distinguish different data sources for the same variable at the same time in the same region, such as reanalysis data and station interpolation data. Through the aforementioned auxiliary primary key fields, it is ensured that each record can be uniquely tracked and referenced throughout its entire data lifecycle; the time representation format, spatial location representation method, quality status marking, and missing value encoding rules of the first processing unit are unified; the field names, units, classification codes, and attribute meanings from different sources in the first processing unit are mapped to a unified field set, unit of measurement system, encoding system, and semantic tag system respectively; metadata construction includes: marking the attribute meanings of each field in the first processing unit as unified semantic tags, supplementing relation anchor fields in each record, and generating corresponding metadata description files; quality control includes: performing quality checks on the first processing units with duplicate records, compound primary key conflicts, out-of-order records, and values ​​exceeding the preset physical range, and performing marking, correction, removal, and completion operations according to the quality check results; feature processing includes: deriving seasonal representations, ablation period identifiers, and time-series change statistical features from the first processing units processed above, and outputting standardized table files and corresponding record index tables, object-record association tables, and time window index tables.

[0025] For raster data, the pixel units and grid units in the raster data are used as the second processing units, where: Format standardization includes: unifying the spatial reference and spatial index of each second processing unit, resampling the spatial resolution of raster data from different sources, aligning the time axis, and mapping each band or variable to a unified naming convention; metadata construction includes: generating element index tables and time index mapping tables corresponding to the second processing units, and establishing corresponding metadata description files; quality control includes: identifying cloud, snow, and shadow regions in the second processing units based on quality bands or mask files and performing mask processing, performing interpolation or marking on missing measurement regions, and unifying the encoding of missing measurement values ​​for each band; feature processing includes: deriving spectral features, spatial texture features, and terrain constraint features from the quality-controlled pixel units, and based on the quality... After quantity control, the temporal variation characteristics of the grid cells are derived. Specifically, the multi-period grid cell sequence is organized as follows: the grid cell data of each time step are sorted according to a unified time axis to form a three-dimensional data cube (spatial dimension × spatial dimension × time dimension) with the grid spatial index as the primary key and the time step as the dimension. For each grid cell location, the time series of the driving variable at that location is extracted along the time axis, and the temporal variation statistical characteristics are calculated based on the time series. These temporal variation statistical characteristics include the rate of change, trend direction, abnormal event markers, and deviation from the multi-year average state, etc., which are then used for subsequent cross-scale semantic consistency comparison with object-level change labels, and temporal constraint markers are added. A standardized raster file is output.

[0026] For vector data, the object units within the vector data are used as the third processing unit, where: The format standardization process includes: unifying the coordinate reference system and attribute field naming rules of the third processing unit, and performing legalization repair on the geometric objects in the third processing unit; metadata construction includes: supplementing object identifiers in the attribute information corresponding to the third processing unit, generating object attribute tables and object adjacency relationship tables, and establishing corresponding metadata description files; quality control includes: checking the integrity of the attribute information of the third processing unit, and performing topology repair on the geometric objects in the third processing unit; feature processing includes: extracting features from three levels: geometry, attributes, and spatial relationships, converting the third processing unit into a unit attribute table, rasterizing the boundaries of the third processing unit, organizing the relationships between the third processing units into a unit relationship table, and outputting a standardized vector file.

[0027] In practical implementation, the multi-source cryosphere data acquired uses the Second China Glacier Inventory dataset as an example. This dataset extracts glacier boundaries based on Landsat TM / ETM+ remote sensing satellite data and extracts glacier elevation attributes based on Space Shuttle Radar Topography Mission (SRTM) V4 data. A total of 42,370 glaciers were interpreted, covering a total area of ​​43,102.58 km². The data is stored in Shapefile (SHP) format, and the attribute data includes glacier name, glacier code, watershed code, mountain range name, administrative division, glacier latitude and longitude, glacier area, area accuracy, glacier perimeter, maximum glacier elevation, minimum glacier elevation, average glacier elevation, median area height, average aspect, and average slope.

[0028] The specific process of performing format standardization, metadata construction, quality control, and feature processing on vector data is as follows: During the format standardization process, the data from China's second glacier inventory was stored in SHP format, with the World Geodetic System 1984 (WGS-84) geographic coordinate system and the Universal Transverse Mercator (UTM) projection. The original vector files were preserved using the standard storage format, coordinate system, and projection for vector data, and a unified data directory structure and index files were added. Specifically, object units in the vector data were used as the third processing unit. The unified coordinate reference system was the WGS-84 UTM projection, and the naming rules for attribute fields were standardized. For example, fields with the same meaning from different data sources were given the same name. Geometric object legalization and repair were performed, including detecting and repairing self-intersecting polygons and deleting duplicate nodes, to ensure that the geometric representation of each object unit was topologically valid.

[0029] For metadata construction, in this embodiment of the invention, object identifiers are supplemented into the attribute information corresponding to the third processing unit, using the glacier code as the unique object identifier to ensure that each glacier vector feature can be uniquely tracked throughout the entire data lifecycle. Furthermore, an object attribute table and an object adjacency table are generated. The object attribute table uses the object identifier as a composite primary key and records the geometric statistics of each object unit, such as area, perimeter, shape index, and other attribute fields, such as glacier name, glacier code, and watershed code, as well as partition fields, such as mountain range affiliation and administrative division affiliation. The object adjacency table records the spatial adjacency, containment, and intersection relationships between object units. Then, a corresponding metadata description file is established, which includes basic metadata, file organization metadata, attribute metadata, semantic metadata, and relationship mapping metadata.

[0030] For quality control, this embodiment of the invention first checks the integrity of the attribute information of each object unit and deletes records with empty watershed codes. Then, it checks fields such as glacier area, glacier longitude, glacier latitude, moraine area, glacier perimeter, maximum glacier elevation, minimum glacier elevation, average glacier elevation, average glacier slope, and average glacier aspect, marking, correcting, or removing data records with empty values, values ​​exceeding preset physical ranges, or semantically inconsistent values. For example, the glacier area value must be positive, and the maximum elevation must be greater than the minimum elevation. Topology repair is performed on geometric objects, including detecting and repairing empty geometry, duplicate geometry, and topologically incorrect objects, ensuring that object boundaries do not cause ambiguity in subsequent spatial analysis.

[0031] For feature processing, this embodiment of the invention extracts features from three levels: geometric, attribute, and spatial relationship. Geometric features include area, perimeter, shape index, and centroid coordinates, used to characterize the morphological attributes of the glacier; attribute features include glacier area, maximum elevation, minimum elevation, average elevation, average slope, and average aspect, used to characterize the topographic attributes of the glacier; spatial relationship features include latitude and longitude location, mountain system affiliation, watershed affiliation, and subordinate relationships with higher-level regional units, used to characterize the glacier's location in the regional geographical pattern. The third processing unit is transformed into a unit attribute table, the boundaries of the third processing unit are rasterized, the relationships between the third processing units are organized into a unit relationship table, a standardized vector file is output, and finally, the above processing forms the basic dataset.

[0032] Based on the basic dataset, obtain object units and object attribute information for object-level data, pixel units and pixel attribute information for pixel-level data, and grid units and grid attribute information for grid-level data; In this embodiment of the invention, an object unit refers to a basic data unit in vector data that carries the geometric shape and attribute information of an independent geographic entity. In cryosphere scientific data, an object unit corresponds to an independent geographic entity with a clear spatial boundary, such as a glacier, a glacial lake, or a permafrost region. Object units are stored in the form of vector features, containing geometric information such as vector graphics like points, lines, and polygons, and attribute information such as physical, geographical, or statistical feature key-value pairs. Simultaneously, the object attribute information includes at least an object identifier and an observation time. The object identifier is used to uniquely identify an object unit during data storage, retrieval, and cross-scale association, while the observation time records the data observation or cataloging time corresponding to that object unit.

[0033] A pixel is the smallest spatial unit in raster data that carries remote sensing observations. It has definite spatial coordinates, pixel size, and observation values. Each pixel records information such as surface reflectance, radiance, or inversion parameters at that location. Pixel attribute information includes at least the pixel spatial index and imaging time. The pixel spatial index is used to uniquely identify and retrieve a pixel in the spatial dimension. The imaging time records the date of acquisition of the remote sensing image corresponding to the pixel and is the basis for the time scale of pixel-level data in time coordination.

[0034] A grid cell refers to the basic spatial unit in regular raster data that carries climate driving variables or numerical simulation results. It has definite spatial coordinates, grid size, and driving variable values. In cryosphere scientific data, a grid cell corresponds to a regular grid point in reanalysis data or climate model output. Each grid cell records atmospheric driving information such as temperature, precipitation, radiation, or wind speed at that location. Grid attribute information includes at least the grid spatial index and the driving time step. The grid spatial index is used to uniquely identify and retrieve a grid cell in the spatial dimension. The driving time step records the temporal resolution of the driving data corresponding to that grid cell, such as the specific year, month, or date corresponding to the grid point's annual data in reanalysis data. This is used to align with the object observation time and pixel imaging time in subsequent time mapping processes and is the basis for the time scale of grid-level data in time coordination.

[0035] In practical implementation, for each object unit, this embodiment of the invention reads a vector data file from the basic dataset, traverses each vector element in the file, and identifies each vector element as an object unit. For the second national glacier inventory data of China, each glacier vector element is an object unit, and each object unit is assigned a unique object identifier throughout the entire data lifecycle. This identifier is taken from the glacier coding field already present in the data source. At the same time as identifying the object unit, the object attribute information corresponding to the object unit is extracted from the attribute table of the vector elements, including fields such as object identifier, observation time, object name, object category, area, perimeter, elevation, slope, aspect, administrative division, and watershed.

[0036] For a pixel unit, this embodiment of the invention reads a raster data file from a base dataset, treating each pixel in the raster data as a pixel unit. Each pixel unit has defined spatial coordinates and pixel size, and its spatial location is determined by the coordinate reference system and geographic transformation parameters of the raster data. For multi-band raster data, multiple band values ​​at the same spatial location together constitute the observation vector of that pixel unit. Simultaneously with determining the pixel unit, the corresponding pixel attribute information, including the pixel spatial index and imaging time, is extracted.

[0037] For grid cells, in this embodiment of the invention, a raster data file is read from the basic dataset, and each regular grid in the raster data is taken as a grid cell. Each grid cell has a defined spatial coordinate and grid size, and its spatial position is determined by the coordinate reference system of the raster data and the grid definition parameters. Grid cells are usually organized in the form of latitude and longitude grids or projected grids. At the same time as determining the grid cell, the grid attribute information corresponding to the grid cell is extracted, including the grid spatial index and the driving time step.

[0038] Establish object-cell mapping based on the spatial correspondence between object units and cell units; establish cell-grid mapping based on the spatial correspondence between cell units and grid units; establish object-grid mapping based on the spatial correspondence between object units and grid units. In this embodiment of the invention, spatial correspondence refers to the overlay, inclusion, or intersection relationships of data units at different scales in the spatial dimension. Object units have irregular vector boundaries, while pixel units and grid units have regular grid structures, and the three may have multiple corresponding forms in space. Object-pixel mapping uses vector object boundaries as constraints and determines the pixel set corresponding to each object unit through spatial overlay analysis. This mapping records the association between object identifiers and corresponding pixel identifiers, as well as the pixel coverage ratio and effective pixel count of each object. The purpose of this mapping is to establish a spatial association between object-level attributes and pixel-level observation features. Pixel-grid mapping addresses the inconsistency between remote sensing pixel resolution and climate-driven grid resolution. It determines the grid unit to which each pixel unit belongs through spatial overlay analysis. This mapping records the association between pixel identifiers and grid identifiers, as well as the resampling method, aggregation method, and error constraints. The purpose of this mapping is to spatially connect pixel-level observation information with raster-level driving information. Object-mesh mapping determines one or more mesh cells covered by each object cell through overlay analysis of object boundaries and regular meshes. This mapping records the association between object identifiers and mesh identifiers, as well as area weights, distance weights, and elevation correction parameters. Thus, the driving variable values ​​of each mesh cell are aggregated to the object level through methods such as area-weighted aggregation, distance-weighted aggregation, or elevation correction aggregation. These aggregation methods are conventional techniques in this field, and the specific aggregation method used can be determined according to task requirements and data types.

[0039] In practical implementation, taking the glacier cataloging vector data, Landsat remote sensing image data, and the fifth-generation European Centre for Medium-Range Weather Forecasts (ERA5) reanalysis data involved in the glacier change monitoring mission as examples, the specific establishment process of the three mappings is given as follows: The process of establishing the object-cell mapping is as follows: Using the glacier vector boundary from the second national glacier inventory data for China as a constraint, spatial overlay analysis is performed between this vector boundary and the cell grid of the Landsat remote sensing image. For each glacier object, all cell units covered by its boundary are determined, and the coverage ratio of each cell unit within the glacier boundary is recorded. For boundary cells, the inclusion of the cell in the glacier object's cell set is determined based on the coverage ratio: if the coverage ratio is greater than 50%, it is included in the set; if the coverage ratio is less than or equal to 50%, it is excluded. For cases where the glacier area is smaller than the area of ​​a single cell, the cell with the largest coverage ratio is selected and included in the set to ensure that small glaciers have at least one effective cell representation. If the glacier is covered by only one cell, that cell is directly included in the set. Simultaneously, the number of effective cells and quality markers are recorded, and an object-cell mapping table is output, using the object identifier as a composite primary key, recording the cell identifier list, cell coverage ratio, number of effective cells, and quality markers corresponding to each glacier object.

[0040] The process of establishing the pixel-grid mapping is as follows: The pixel grid of the Landsat remote sensing image is spatially overlaid with the grid of the ERA5 reanalysis data. For each pixel, the grid cell containing its center point is determined, and this pixel is mapped to the corresponding grid cell. Since pixel resolution is usually higher than grid resolution, a grid cell typically contains hundreds to thousands of pixels. The resampling method and aggregation method are recorded. Resampling methods can include nearest neighbor resampling, bilinear interpolation, or cubic convolution interpolation. Aggregation methods can include mean aggregation, median aggregation, or weighted average. A pixel-grid mapping table is output, using the pixel identifier as a composite primary key, recording the grid identifier and resampling parameters corresponding to each pixel.

[0041] The process of establishing the object-mesh mapping is as follows: The glacier vector boundary is spatially overlaid with the mesh of the ERA5 reanalysis data. For each glacier object, all mesh cells covered by its boundary are determined. The area weight of each mesh cell within the glacier boundary is calculated, which is the ratio of the area of ​​the mesh cell falling within the glacier boundary to the total area of ​​that mesh cell. Simultaneously, distance weights and elevation correction parameters are recorded. The distance weight is obtained by normalizing the inverse of the distance from the center point of each mesh cell to the centroid of the glacier, and the elevation correction parameter is obtained by the difference between the average elevation of the mesh cells and the average elevation of the glacier. Finally, an object-mesh mapping table is output, using the object identifier as a composite primary key, recording the mesh identifier list, area weight list, distance weight list, and elevation correction parameters corresponding to each glacier object.

[0042] Establish a time mapping based on object attribute information of object-level data, pixel attribute information of pixel-level data, and grid attribute information of grid-level data; In this embodiment of the invention, time mapping can incorporate three types of time systems—object observation period, pixel imaging time, and grid-driven time step—into a unified expression framework. Since the time base, time resolution, and time expression format of the three types of data are different, the object observation time may be in the unit of years, the pixel imaging time in the unit of days, and the grid-driven time step in the unit of months or days. Therefore, time mapping establishes a time correspondence, enabling the three to be compared and matched on the same time axis. At the same time, time mapping records the phase selection rules, time window length, time interpolation method, and quality constraints.

[0043] In practical implementation, the process of establishing time mapping is illustrated using glacier cataloging data (observation time in years), Landsat remote sensing images (imaging time in days), and ERA5 reanalysis data (driving time step in months) involved in the glacier change monitoring mission as examples.

[0044] First, a unified timeline reference is established, using days as the unified time scale. The observation time of each glacier catalog data is converted into a unified time marker for the middle of the ablation season of that year. The imaging time of each Landsat remote sensing image is directly mapped to the unified timeline. The driving time step of each ERA5 reanalysis data is converted into a unified time marker for the 15th of that month. The middle of the ablation season is determined as follows: For areas with daily temperature observations or reanalysis data, the period when the daily average temperature is continuously above 0°C is taken as the ablation season, and the middle date of that period is taken as the middle of the ablation season. For areas lacking temperature data, a typical middle date of the ablation season is preset based on the latitude zone and altitude range of the study area, and dynamic adjustments of ±15 days are allowed according to the specific climate anomalies of the year.

[0045] Then, establish the correspondence between the object observation time and the pixel imaging time: For each glacier object, determine the time window of the melting season of that year based on its observation time, and select the best quality Landsat remote sensing image from the time window, such as a remote sensing image with cloud cover of less than 10%, as the matching image for the glacier object. If there is no usable image in the melting season of that year, the time window is extended to one year before and after, and the image closest to the middle of the melting season is selected.

[0046] Then, establish the correspondence between pixel imaging time and grid driving time step: For each pixel unit, determine the corresponding grid driving time step according to its imaging time. The grid driving time step can be the monthly average data of the month in which the imaging time is located. If the imaging time is June 15, then match the grid monthly average data of June of that year.

[0047] The final output is a time mapping table, which records the observation time stamps of each glacier object, the imaging time stamps of each pixel unit, the driving time step stamps of each grid unit, and the time correspondence and matching parameters between the object, pixel, and grid.

[0048] Multi-scale collaborative construction based on object-cell mapping, cell-mesh mapping, object-mesh mapping, and temporal mapping includes: The set of cell units corresponding to the object unit is obtained based on the object-cell mapping, the set of grid units corresponding to the cell unit is obtained based on the cell-grid mapping, and the set of grid units corresponding to the object unit is obtained based on the object-grid mapping, thus obtaining the spatial correspondence result; In this embodiment of the invention, the multi-scale collaboration achieved in this step is spatial collaboration. Its essence lies in incorporating data units that originally belonged to three independent spatial reference systems into the same index chain through a three-level mapping relationship. Any object unit can quickly locate its corresponding set of pixel units and set of grid units through this index chain, thereby avoiding the spatial correspondence operation of manual object-by-object matching.

[0049] Based on the time mapping, the object unit, pixel unit, and grid unit are unified in time scale to obtain the unit data after time scale unification; and the time phase of the unit data after time scale unification is filtered according to preset rules to obtain the time phase filtering results. In this embodiment of the invention, the multi-scale collaboration achieved in this step is temporal collaboration. Its essence lies in first unifying the different time bases of the three types of data onto the same time axis through time mapping, and then filtering on the unified time axis according to the cryosphere's unique temporal constraints. The preset rules can employ at least one of the following: ablation season filtering rules, snow cover period removal rules, and freeze-thaw transition filtering. The ablation season filtering rules prioritize observational data from periods with weaker snow cover interference during the ablation season or periods with less snow cover interference, reducing the interference of seasonal snow cover on glacier boundary identification. The snow cover period removal rules exclude phases with snow cover ratios exceeding a preset threshold to avoid confusion between snow cover and glaciers. The freeze-thaw transition filtering rules filter out unstable samples during periods when the daily average temperature fluctuates around 0°C, preventing noise from sudden changes in surface state during freeze-thaw cycles.

[0050] Based on object attribute information, cell attribute information, and grid attribute information, consistency constraints are applied to the basic dataset to obtain semantic consistency constraint results. In this embodiment of the invention, the multi-scale collaboration achieved in this step is semantic collaboration. Its essence lies in determining the type of each data by the scale label in the attribute metadata, namely object level, cell level or raster level, and using the semantic constraint information in the relation mapping metadata to impose logical consistency constraints on the classification labels and attribute fields of each scale data. For example, the change labels of glacier objects are advance, stable and retreat, which need to be logically consistent with the spectral change trend of the corresponding pixel and the temperature or precipitation change direction of the corresponding grid to avoid introducing sample noise due to semantic mismatch.

[0051] Perform primary key conflict checks, spatial mismatch checks, and temporal mismatch checks on object-cell mapping, cell-mesh mapping, and object-mesh mapping, and obtain the check results; In this embodiment of the invention, the multi-scale collaboration implemented in this step is a collaborative verification, which essentially involves performing cross-scale consistency checks on three types of mapping tables. The primary key conflict check is used to confirm the uniqueness and integrity of the composite primary key in each mapping table; the spatial mismatch check is used to verify whether the overlay accuracy between the object boundary, the pixel spatial location, and the grid spatial range meets the preset tolerance; and the temporal mismatch check is used to verify whether the correspondence between the object observation period, the pixel imaging time, and the grid-driven time step meets the preset time window.

[0052] In practical implementation, this embodiment of the invention, based on the object-cell mapping, cell-grid mapping, and object-grid mapping established above, as well as the time mapping, takes the glacier change monitoring task as an example to illustrate the specific implementation process of multi-scale collaborative construction, as follows: First, spatial collaborative operations are performed. Based on the object-cell mapping, for each glacier object, the list of corresponding cell identifiers and coverage ratios are read from the mapping table. The spatial index and remote sensing band values ​​of each cell unit are obtained, and a remote sensing feature vector for the glacier object is constructed. This remote sensing feature vector may include the mean, standard deviation, and normalized difference index of each band. Next, based on the cell-grid mapping, for each cell unit, the grid identifier and resampling parameters corresponding to that cell are read from the mapping table. The spatial index and climate variable values ​​of each grid unit are obtained. Finally, based on the object-grid mapping, for each glacier object, the list of corresponding grid identifiers, area weights, distance weights, and elevation correction parameters are read from the mapping table. The driving variable values ​​of each grid unit are aggregated to the object level using area-weighted averaging to obtain the background climate characteristics of each glacier object. After these three operations are completed, the spatial correspondence results are output.

[0053] Then, a time-coordinated operation is performed. Based on the time mapping, the observation time of each glacier object, the imaging time of each pixel unit, and the driving time step of each grid unit are unified onto the diurnal time axis. For each glacier object, the ablation season time window is determined according to its observation year. Landsat remote sensing images with a cloud cover ratio of less than 10% are selected from this time window. Phases with a snow cover ratio of more than 30% are removed, and images of freeze-thaw transition periods with a daily average temperature between -1℃ and 1℃ are filtered out. This average temperature can be dynamically adjusted according to the region / altitude. For glacier objects corresponding to observation phases that are removed due to phase selection, they are marked as having no valid matching data for that phase. Finally, the phase selection results are output.

[0054] Then, semantic co-operation is performed to compare the change labels (advancing, stable, and retreating) in the glacier change monitoring task with pixel-level spectral features and raster-level temperature-driven features for consistency. For glaciers marked as retreating, it is checked whether the spectral features of their corresponding pixels show a decreasing reflectance trend. If the reflectance does not show a decreasing trend, it is marked as semantically inconsistent and needs to be reviewed. For glaciers marked as advancing, it is checked whether the temperature-driven features of their corresponding grids show a cooling trend or an increasing precipitation trend. If the temperature does not show a cooling trend or the precipitation does not show an increasing trend, it is marked as semantically inconsistent and needs to be reviewed. Finally, the semantic consistency constraint results are output.

[0055] In a specific embodiment of the present invention, for the snow recognition task, the process of obtaining its semantic consistency constraint result is as follows: the snow classification label of the pixel unit is compared with the multi-temporal spectral features of the pixel unit. If the pixel is marked as snow but the corresponding visible light reflectance is lower than the preset snow threshold or the near-infrared reflectance is higher than the preset threshold, it is marked as semantically inconsistent and needs to be reviewed; if the pixel is marked as non-snow but the corresponding normalized difference snow index is continuously higher than the preset snow index threshold, it is also marked as semantically inconsistent.

[0056] For the permafrost identification task, the process of obtaining its semantic consistency constraint results is as follows: the classification label of the active layer thickness of the permafrost in the grid cell is compared with the multi-year geothermal change trend and surface type of the grid cell. If the grid cell is marked as permafrost but the corresponding annual average geothermal temperature is consistently higher than 0℃ or the active layer thickness exceeds the preset range of permafrost area, it is marked as semantically inconsistent; if the grid cell is marked as seasonal permafrost but the corresponding winter geothermal temperature is consistently lower than the preset temperature threshold, it is marked as semantically inconsistent.

[0057] Finally, the semantic consistency determination results of each object unit and its corresponding pixel unit and grid unit in the glacier change monitoring task, snow cover identification task and permafrost discrimination task are output respectively. Each determination result contains a mark of semantic consistency or semantic inconsistency, which serves as a semantic consistency constraint result. This result is used in subsequent fusion steps to retain the associations that pass the consistency determination and to remove or mark the associations that fail the consistency determination.

[0058] Finally, a collaborative verification operation is performed. This involves checking the object-cell mapping table for composite primary key conflicts and spatial mismatches; checking the cell-grid mapping table for composite primary key conflicts and spatial mismatches; checking the object-grid mapping table for primary key conflicts; and performing temporal mismatch checks across the mappings to verify whether the observation year of the glacier object matches the imaging year of the corresponding cell (with a matching error not exceeding one year). It also verifies whether the imaging month of the cell matches the driving month of the corresponding grid (with a matching error not exceeding one month). The final check results are then output.

[0059] Based on the spatial correspondence results, temporal filtering results, semantic consistency constraint results, and inspection results, the multi-source data of the cryosphere are fused to obtain an AI-ready dataset for multi-source data of the cryosphere.

[0060] In this embodiment of the invention, multi-source data that has undergone spatial, temporal, semantic, and collaborative verification is organized under a unified object identifier or spatial unit. For object-level machine learning tasks, vector object features, remote sensing spectral features, texture features, digital elevation model (DEM) terrain features, station observation features, and grid-driven features are concatenated by field to form a sample feature table. For deep learning-based recognition tasks, multi-channel raster sample blocks are constructed with the object boundary as the center. Each channel carries remote sensing bands, terrain factors, and auxiliary driving factors, respectively. After the above processing, an AI-ready dataset that can be efficiently, securely, and reproducibly used for training, evaluating, and deploying machine learning and artificial intelligence models can be obtained. This dataset has the characteristics of standardized format, complete metadata, controllable quality, clear spatiotemporal relationships, and semantic consistency, and can be directly discovered, filtered, combined, trained, and inferred by artificial intelligence models.

[0061] In practical implementation, this embodiment of the invention takes a glacier change monitoring task as an example to illustrate the process of fusing multi-source data from the cryosphere, as follows: First, for each glacier object that is verified as valid through the inspection results, a sample record is created. This sample record includes: object identifier, observation time, geometric features, topographic features, spectral features, texture features, climate background features, and change labels. All sample records are merged into a sample feature table and output as a comma-separated values ​​(CSV) format file.

[0062] Secondly, for the deep learning-based identification model in the glacier change monitoring task, a fixed-size local window is cropped from the remote sensing image with the boundary of each glacier object as the center to construct a multi-channel raster sample block. Each channel sequentially carries the red band reflectance, green band reflectance, blue band reflectance, near-infrared band reflectance, short-wave infrared band reflectance, normalized differential snow cover index, normalized vegetation index, slope layer, aspect layer, elevation layer, temperature background layer, and precipitation background layer of visible light. Each sample block is associated and stored with the corresponding glacier object identifier and change label. The specific normalization method is selected according to the subsequent model requirements, such as range normalization, standard deviation normalization, and quantile normalization, at least one of these methods.

[0063] Secondly, a corresponding metadata description file is created for the AI-ready dataset, recording the dataset name, version, generation time, spatial range, time range, task type, field names and meanings, channel names and physical units. A quality control file is generated, recording the number of samples that passed the inspection, the number of outlier samples removed and the reasons for removal, and the pass rate for each type of inspection. A version tracking file is generated, recording the process identifier for this dataset generation, the version number of the underlying dataset used, and the processing parameters.

[0064] Finally, the above processing results in an AI-ready dataset for glacier change monitoring tasks.

[0065] This invention further provides an AI-ready dataset construction device for multi-source cryosphere data, such as... Figure 2 As shown, the device includes a processor and a memory, in which a computer program is stored. When the processor executes the computer program, it implements the steps of any of the above-described methods for constructing AI-ready datasets for multi-source cryosphere data.

[0066] During operation, the device acquires multi-source data from the cryosphere through a data acquisition module. This multi-source data includes at least tabular, vector, and raster data. A standardization module performs format standardization, metadata construction, quality control, and feature processing on the tabular, vector, and raster data respectively, forming a basic dataset. A feature extraction module extracts object units and object attribute information from the basic dataset, pixel units and pixel attribute information from the pixel-level data, and grid units and grid attribute information from the grid-level data. A mapping establishment module establishes object-pixel mappings based on the spatial correspondence between object units and pixel units, and establishes mappings based on pixel units and grid units. A pixel-grid mapping is established based on the spatial correspondence between object units and grid units. An object-grid mapping is then established based on the spatial correspondence between object units and grid units, and a temporal mapping is established based on object attribute information, pixel attribute information, and grid attribute information. A multi-scale collaborative construction module performs multi-scale collaborative construction based on the object-pixel mapping, pixel-grid mapping, object-grid mapping, and temporal mapping to obtain spatial correspondence results, temporal filtering results, semantic consistency constraint results, and inspection results. A fusion generation module fuses the cryosphere multi-source data based on the spatial correspondence results, temporal filtering results, semantic consistency constraint results, and inspection results to generate an AI-ready dataset for cryosphere multi-source data. The specific functions of each module are completely consistent with the corresponding steps in the aforementioned method embodiments, and will not be repeated here.

[0067] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing an AI-ready dataset for multi-source cryosphere data, characterized in that, include: Acquire multi-source cryosphere data and perform standardization processing to obtain a basic dataset; the multi-source data includes at least tabular data, vector data, and raster data. Based on the basic dataset, obtain object units and object attribute information for object-level data, pixel units and pixel attribute information for pixel-level data, and grid units and grid attribute information for grid-level data; Establish object-cell mapping based on the spatial correspondence between object units and cell units; establish cell-mesh mapping based on the spatial correspondence between cell units and grid units; Establish an object-mesh mapping based on the spatial correspondence between object cells and mesh cells; Establish a time mapping based on object attribute information of object-level data, pixel attribute information of pixel-level data, and grid attribute information of grid-level data; The set of cell units corresponding to the object unit is obtained according to the object-cell mapping, the set of grid units corresponding to the cell unit is obtained according to the cell-grid mapping, and the set of grid units corresponding to the object unit is obtained according to the object-grid mapping, thus obtaining the spatial correspondence result; The object unit, the pixel unit, and the grid unit are unified in time scale according to the time mapping to obtain unit data after time scale unification; and the phase of the unit data after time scale unification is filtered based on the physical constraint rules of the cryosphere melting period, snow cover period, or freeze-thaw transition period to obtain phase filtering results. Obtain the change label of the object unit based on the object attribute information, obtain the spectral change trend of the pixel unit based on the pixel attribute information, and obtain the climate-driven change trend of the grid unit based on the grid attribute information. The change label of the object unit is compared with the spectral change trend of the corresponding pixel unit and the climate-driven change trend of the grid unit for logical consistency. If the change label is inconsistent with the direction of the spectral change trend or the climate-driven change trend, the object unit is marked as a semantically inconsistent sample with the corresponding pixel unit and grid unit. Otherwise, it is marked as a consistent sample. All marking results are output to obtain the semantic consistency constraint result. Perform primary key conflict checks, spatial mismatch checks, and temporal mismatch checks on object-cell mapping, cell-mesh mapping, and object-mesh mapping, and obtain the check results; Based on the spatial correspondence results, temporal filtering results, semantic consistency constraint results, and inspection results, the multi-source data of the cryosphere are fused to obtain an AI-ready dataset for multi-source data of the cryosphere.

2. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 1, characterized in that: The standardization process specifically includes: format standardization, metadata construction, quality control, and feature processing.

3. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 2, characterized in that: The specific process of acquiring multi-source cryosphere data and performing standardization, including format standardization, metadata construction, quality control, and feature processing of tabular data, is as follows: The records in the table data are used as the first processing unit, where: The format standardization process includes: constructing a corresponding composite primary key based on the table type; unifying the time expression format, spatial location expression method, quality status mark, and missing value encoding rules of the first processing unit; and mapping the field names, units, classification codes, and attribute meanings from different sources in the first processing unit to a unified field set, measurement unit system, encoding system, and semantic tag system, respectively. The metadata construction includes: labeling the attribute meanings of each field in the first processing unit as unified semantic tags, supplementing relation anchor fields in each record, and generating corresponding metadata description files; The quality control includes: performing quality detection on the first processing unit that has duplicate records, compound primary key conflicts, out-of-order records, and values ​​exceeding the preset physical range; and performing marking, correction, elimination, and completion operations according to the quality detection results. The characterization process includes: deriving seasonal representation, ablation period identifier, and time-series change statistical features based on the first processing unit after the above processing, and outputting standardized table files and corresponding record index tables, object-record association tables, and time window index tables.

4. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 3, characterized in that: Construct a corresponding composite primary key based on the table type, specifically as follows: The table types include site observation tables, object attribute tables, and statistical time series tables; For station observation tables, construct a composite primary key using station identifier and observation time; For tables with object attributes, a composite primary key is constructed using the object identifier; For statistical time series tables, a composite primary key is constructed using region identifier, time identifier, and variable identifier.

5. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 2, characterized in that: The specific process of acquiring multi-source cryosphere data and performing standardization processing, including format standardization, metadata construction, quality control, and feature processing of raster data, is as follows: Using the pixel units and grid units in the raster data as the second processing units, where: The format standardization process includes: unifying the spatial reference and spatial index of each second processing unit, resampling the spatial resolution of raster data from different sources, aligning the time axis, and mapping each band or variable to a unified naming convention. The metadata construction includes: generating an element index table and a time index mapping table corresponding to the second processing unit, and establishing a corresponding metadata description file; The quality control includes: identifying cloud, snow and shadow regions in the second processing unit based on quality bands or mask files and performing mask processing, and performing interpolation or marking on missing measurement regions, and unifying the missing measurement value encoding for each band; The feature processing includes: deriving spectral features and spatial texture features from the quality-controlled pixel units; deriving terrain constraint features from the DEM raster data spatially registered with the pixel units; deriving temporal variation features from the multi-period grid unit sequence after quality control; and adding temporal constraint markers; and outputting a standardized raster file.

6. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 2, characterized in that: The specific process of acquiring multi-source cryosphere data and performing standardization, including format standardization, metadata construction, quality control, and feature processing of vector data, is as follows: The object unit in the vector data is used as the third processing unit, where: The format standardization process includes: unifying the coordinate reference system and attribute field naming rules of the third processing unit, and performing legalization repair on the geometric objects in the third processing unit; The metadata construction includes: supplementing the object identifier in the attribute information corresponding to the third processing unit, generating an object attribute table and an object adjacency table, and establishing a corresponding metadata description file; The quality control includes: checking the integrity of the attribute information of the third processing unit and performing topology repair on the geometric objects in the third processing unit; The feature processing includes: extracting features from three levels: geometry, attributes, and spatial relationships; converting the third processing unit into a unit attribute table; rasterizing the boundaries of the third processing unit; organizing the relationships between the third processing units into a unit relationship table; and outputting a standardized vector file.

7. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 2, characterized in that: The metadata description file constructed from the metadata specifically includes: basic metadata, file organization metadata, attribute metadata, semantic metadata, and relation mapping metadata; The basic metadata includes description file name, version, reference identifier, spatiotemporal resolution, data volume, and publication time; The file organization metadata includes descriptions of file type, file name, data format, projection coordinate system, and file path; The attribute metadata includes attribute name, data type, attribute meaning, and attribute domain, and marks the scale type to which the fields in the cryosphere multi-source data belong. The semantic metadata includes the task semantic types supported by the basic dataset, as well as the applicable objects, applicable regions, and applicable time phases corresponding to each task semantic type; the task semantic types include at least one or more of classification tasks, recognition tasks, and prediction tasks; The relational mapping metadata includes spatial and temporal correspondences between object-level data, cell-level data, and grid-level data, and includes at least: object identifier, cell identifier, grid identifier, spatial coverage relationship, temporal correspondence, aggregation method, sampling method, and quality constraint information.

8. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 1, characterized in that: When filtering the temporal phase of unit data after unifying the time scale according to preset rules, the preset rules adopt at least one of the following: ablation season filtering rules, snow cover period removal rules, and freeze-thaw conversion filtering.

9. The method for constructing an AI-ready dataset for multi-source cryosphere data according to claim 1, characterized in that: The object attribute information includes the object identifier and observation time; the cell attribute information includes the cell spatial index and imaging time; the grid attribute information includes the grid spatial index and driving time step.

10. An apparatus for constructing AI-ready datasets for multi-source cryosphere data, comprising a processor and a memory, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements a method for constructing an AI-ready dataset for multi-source cryosphere data as described in any one of claims 1 to 9.

Images