A remote sensing big data processing method and system based on block and parallel machine learning

Abstract

The application discloses a kind of based on block and parallel machine learning's remote sensing big data processing method and system.The processing method includes obtaining remote sensing big data;Remote sensing big data is blocked based on preset rule, and multiple sub-region data are obtained;Multiple sub-region data are parallelly input to pre-constructed remote sensing image prediction model, and the corresponding each sub remote sensing image information of each sub-region data is obtained, wherein the remote sensing image prediction model includes multiple sub-models, and each sub-region data is selected from multiple sub-models based on preset screening rule to select a sub-model for independent training and prediction;Each sub remote sensing image information corresponding to each sub-region data is integrated to obtain target remote sensing image.Multiple sub-region data are parallelly input to sub-model, not only can handle a large amount of data, improve the overall operation efficiency, but also better capture spatial heterogeneity, improve remote sensing image prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and recognition technology, and in particular to a remote sensing big data processing method and system based on block-based and parallel machine learning. Background Technology

[0002] With the formation of a multi-platform remote sensing observation system (including satellites, drones, and ground-based radars), global Earth observation data is experiencing exponential growth. According to the latest statistics from the European Space Agency's (ESA) Copernicus program, the Sentinel series satellites alone are adding an astonishing 20TB of data daily. Furthermore, spatial resolution has reached sub-meter levels; for example, the WorldView-4 satellite has a resolution of 0.31m. The expansion of spectral dimensions is also remarkable; for instance, the EnMAP hyperspectral satellite has expanded to over 400 bands. This spatiotemporally continuous multimodal data provides unprecedented observational capabilities for global change research. However, traditional data processing paradigms are struggling to cope with the challenges brought about by the explosion of three-dimensional (spatial-spectral-temporal) data.

[0003] 1. The mismatch between data throughput and computing architecture

[0004] Current mainstream serial processing architectures (such as ENVI, ERDAS, and other commercial software) suffer from a significant memory wall effect: a single hyperspectral image (1024×1024×224 bands) already occupies as much as 2GB of memory in float32 format, while global-scale analysis often requires processing on the order of 10^24 bytes. 6 The image blocks are often segmented using fixed granularity (e.g., 512×512 pixels), but this does not take into account the anisotropic distribution of ground features, such as the coexistence of high-frequency details in urban areas and low-frequency textures in farmland areas. This leads to the following problems: coarse-grained segmentation causes cross-block feature segmentation in complex areas, disrupting the contextual relationships of algorithms such as convolutional neural networks; fine-grained segmentation causes uneven thread load in homogeneous areas, resulting in waiting overhead of more than 30% during MPI parallel computation.

[0005] 2. The Coupling Dilemma Between Algorithm Complexity and Computation Granularity

[0006] The successful application of deep learning models (such as U-Net and Transformer) in remote sensing interpretation has led to a superlinear increase in computational graph complexity. Experiments show that ResNet-50 requires 1.3 × 10^6 operations per forward propagation when processing 1m resolution urban imagery. 10 Sub-floating-point operations. Existing parallel solutions tend to be polarized:

[0007] Global parallelization: Distributed training using data parallelism (such as the Horovod framework) can expand the batch size, but it cannot overcome the limitation of single-card memory and ignores the impact of spatial heterogeneity on the model's generalization ability.

[0008] Local fragmentation: Although thread-level parallelism based on OpenMP can accelerate the processing of a single image, it lacks a cross-node load balancing mechanism, and the task scheduling loss can reach 45% in a cluster of 100+ nodes.

[0009] 3. Spatial heterogeneity and scale mismatch of modeling units

[0010] Existing machine learning methods often use whole-scene imagery to train global models (e.g., random forest classification), but neglect the spatial non-stationary nature of land surface parameters. Taking forest carbon storage retrieval as an example, there are significant differences in the spectral-biomass relationship between tropical rainforests and northern forests (R0). 2 The difference is 0.32), while the global model will produce systematic bias in heterogeneous regions (RMSE increases by 1.8Mg / ha). Although some studies have attempted to segment images into superpixels for local modeling, they have not established a mapping relationship between block parameters and computational topology, resulting in: block boundary effects causing 17% to 23% feature leakage; lack of knowledge transfer mechanisms between local models, and a 300% increase in repeated training overhead.

[0011] In summary, current research often views the three technical dimensions of partitioning strategies, parallel computing, and machine learning in isolation, lacking system-level collaborative optimization. This fragmented design leads to a triple constraint on remote sensing intelligent processing: the memory wall, the communication wall, and the accuracy wall. Summary of the Invention

[0012] The main objective of this invention is to provide a remote sensing big data processing method and system based on block-based and parallel machine learning, thereby overcoming the shortcomings of existing technologies. To achieve the aforementioned objective, the technical solution adopted by this invention includes:

[0013] The first aspect of this invention provides a remote sensing big data processing method based on block-based and parallel machine learning, the specific steps of which include:

[0014] Acquiring remote sensing big data;

[0015] Based on preset rules, the remote sensing big data is divided into blocks to obtain multiple sub-region data;

[0016] Multiple sub-region data are input in parallel into a pre-constructed remote sensing image prediction model to obtain each sub-remote sensing image information corresponding to each sub-region data. The remote sensing image prediction model includes multiple sub-models. Each sub-region data is selected from multiple sub-models for independent training and prediction based on a preset selection rule.

[0017] The target remote sensing image is obtained by integrating the data of each sub-region and the corresponding remote sensing image information.

[0018] In some more specific implementation schemes, the remote sensing big data is preprocessed, and the preprocessing includes at least reprojection, resampling, filling invalid values, and image overlay of the remote sensing big data.

[0019] In some more specific implementations, the remote sensing big data is divided into blocks based on preset rules to obtain multiple sub-region data, including:

[0020] Preset spatial resolution threshold and geographic range threshold;

[0021] Based on spatial resolution thresholds and geographical range thresholds, the remote sensing big data is initially screened to obtain the regional remote sensing dataset of the study area, which serves as sub-regional data.

[0022] Furthermore, the method for processing the remote sensing big data also includes:

[0023] Preset spectral characteristics and normalized vegetation index;

[0024] Based on spectral characteristics, the remote sensing big data is divided into first-level segments to obtain images of various land types;

[0025] Based on the normalized vegetation index, the land cover images after the first level of classification are divided into a second level to obtain environmental remote sensing datasets under different growth conditions or different topography of each land cover image.

[0026] Based on spatial resolution thresholds and geographic range thresholds, environmental remote sensing datasets under different growth conditions or different topographic features are uniformly divided into blocks under the same land cover imagery to obtain multiple sub-region data.

[0027] In some more specific implementation schemes, the training method for the pre-built remote sensing image prediction model includes:

[0028] Define the random forest model;

[0029] Acquire remote sensing image training data and remote sensing big data source. The remote sensing image training data includes environmental remote sensing training datasets for multiple land types. The remote sensing big data source includes at least meteorological, vegetation, topographic, and soil environmental information related to the information of the elements to be simulated. The information of the elements to be simulated includes latitude and longitude that correspond to the location information of the remote sensing images and the density of the elements to be simulated that quantifies the number of elements.

[0030] The random forest model is trained using remote sensing image training data, remote sensing big data sources, and preset hyperparameters through a model tuning wrapper function. The hyperparameters include at least the number of features, sample size, sampling with replacement, node size, number of decision trees, and node splitting rules.

[0031] Cross-validation was used to determine the remote sensing image prediction model, which includes multiple sub-models, each of which corresponds one-to-one with an environmental remote sensing training dataset for multiple land types.

[0032] Furthermore, the data from multiple sub-regions are input in parallel into a pre-constructed remote sensing image prediction model to obtain the information of each sub-remote sensing image corresponding to each sub-region data. Specific steps include:

[0033] For each sub-region of data, each cell is traversed, and the sub-model corresponding to the sub-region of data is selected based on land type, vegetation growth information and / or landform information.

[0034] Input a sub-region of data, and use an if statement to filter out the sub-models corresponding to the sub-region of data;

[0035] The sub-remote sensing image information corresponding to the sub-region data is obtained through sub-model calculation.

[0036] In some more specific implementation schemes, the method for processing remote sensing big data further includes:

[0037] Set up receiving and summarizing nodes;

[0038] The receiving node receives remote sensing big data;

[0039] Based on remote sensing big data, the total amount of data in the sub-region is determined to be M, and the total amount of computing clusters required is N, where M≤N;

[0040] Data from M sub-regions is distributed to N computing clusters for parallel computation. Once all N computing clusters have completed their computations, the results from the N computing clusters are aggregated through the aggregation node.

[0041] In some more specific implementations, the step of integrating the remote sensing image information corresponding to each sub-region data to obtain a remote sensing image includes:

[0042] Scan the image data file corresponding to each sub-remote sensing image information and read the geographical location information from the header file;

[0043] Integrate all geographic location information to construct virtual image files, which will serve as a directory index;

[0044] According to the directory index, the image data file corresponding to each sub-remote sensing image information is read and integrated to obtain the remote sensing image.

[0045] Furthermore, after integrating the remote sensing image information corresponding to each of the sub-region data to obtain the remote sensing image, the process further includes:

[0046] Based on image optimization algorithms, outliers are eliminated according to the actual situation of remote sensing images.

[0047] The "factual situation" here refers to the process of making predictions using remote sensing images. For example, if we want to predict the density of carbon in the soil, the density of soil carbon cannot be negative in reality. Therefore, when analyzing large amounts of remote sensing data, if negative values ​​are found, we must take measures to correct this error. The usual practice is to replace these negative values ​​with zero to ensure the accuracy of the data and the validity of the prediction results.

[0048] A second aspect of the present invention provides a remote sensing big data processing system based on block-based and parallel machine learning, comprising:

[0049] The acquisition module is used to acquire remote sensing big data;

[0050] The segmentation module is used to divide the remote sensing big data into blocks based on preset rules to obtain multiple sub-region data.

[0051] The remote sensing image prediction module is used to input multiple sub-region data into a pre-constructed remote sensing image prediction model in parallel to obtain each sub-remote sensing image information corresponding to each sub-region data; wherein, the remote sensing image prediction model includes multiple sub-models, and each sub-region data selects one sub-model from the multiple sub-models for independent training and prediction based on a preset selection rule;

[0052] The remote sensing image integration module is used to integrate the remote sensing image information corresponding to each of the sub-region data to obtain the target remote sensing image.

[0053] Compared with the prior art, the advantages of the present invention include at least the following:

[0054] First, this invention provides a remote sensing big data processing method based on block partitioning and parallel machine learning. This method deeply integrates and synergistically optimizes block partitioning strategies, parallel computing, and machine learning to form a complete processing chain. It fully leverages the synergistic effects of each technology to achieve efficient processing and high-precision analysis of remote sensing data, overcoming the limitations of separate applications in existing technologies. The remote sensing big data is partitioned into multiple sub-regions, allowing each sub-region to be processed independently, reducing the computational load per operation and improving processing efficiency. The multiple sub-regions are distributed to multiple computing clusters for parallel computation, significantly shortening computation time and improving efficiency when processing large-scale data. Each sub-region employs machine learning and other predictive methods, greatly improving computational accuracy.

[0055] Second, the remote sensing big data processing method based on block and parallel machine learning provided by the present invention inputs multiple sub-region data into a pre-constructed remote sensing image prediction model in parallel. By improving the prediction accuracy in the local area through independent modeling, it can better capture spatial heterogeneity and improve the prediction accuracy of remote sensing images in the local area.

[0056] Third, the present invention provides a remote sensing big data processing method based on block segmentation and parallel machine learning. Based on spatial positioning technology and combined with geometric correction and interpolation algorithms, it integrates the remote sensing image information corresponding to each sub-region data to ensure the spatial consistency of block data during stitching, minimize spatial position errors, and achieve seamless high-precision image stitching. Attached Figure Description

[0057] Figure 1 This is a flowchart of a remote sensing big data processing method based on block-based and parallel machine learning according to an embodiment of the present invention.

[0058] Figure 2 This is a flowchart of a remote sensing big data processing method based on block-based and parallel machine learning according to an embodiment of the present invention;

[0059] Figure 3 This is a comparison chart of the accuracy of the predicted SOC density and the reserved verification SOC density of each model provided in the embodiments of the present invention;

[0060] Figure 4 This is a comparison chart of the computation time of block processing and overall non-block processing at different resolutions provided in the embodiments of the present invention;

[0061] Figure 5 This is a comparison chart of the computation time for block processing and overall non-block processing under different numbers of bands provided in the embodiments of the present invention;

[0062] Figure 6This is a schematic diagram of the structure of a remote sensing big data processing system based on block-based and parallel machine learning provided in an embodiment of the present invention.

[0063] Explanation of reference numerals in the attached figures:

[0064] 11. Acquisition module; 12. Segmentation module; 13. Remote sensing image prediction module; 14. Remote sensing image integration module. Detailed Implementation

[0065] In view of the shortcomings of the prior art, the inventors of this invention, through long-term research and extensive practice, have proposed the technical solution of this invention. The following will further explain and illustrate the technical solution, its implementation process, and its principles.

[0066] Please refer to Figures 1-2 This embodiment provides a remote sensing big data processing method based on block-based and parallel machine learning for simulating SOC (soil organic carbon) carbon storage, including:

[0067] S100: Acquire remote sensing big data.

[0068] As can be understood, remote sensing big data refers to large-scale, high-resolution, multi-temporal, and multispectral Earth observation data acquired through remote sensing technologies (such as satellites and drones). This Earth observation data encompasses various information including surface features, environmental changes, and meteorology, and is characterized by its large volume, multiple dimensions, and long time span. It is widely used in fields such as environmental monitoring, resource management, and disaster early warning. Here, remote sensing big data refers to remote sensing data sources, including information on meteorology, vegetation, topography, and soil.

[0069] Before processing large-scale remote sensing data in blocks, preprocessing is essential. Specifically, this involves first cropping images based on the study area boundaries; then reprojecting the images, transforming them from one coordinate system to another to suit specific application requirements or analytical standards. Additionally, resampling is necessary, which involves changing the image resolution to match the specific pixel sizes required for the analysis. Invalid or missing values ​​may exist in the data, so these need to be filled in, typically using the average of surrounding pixels or other statistical methods to estimate missing data. Finally, image overlay is a crucial step, allowing multiple image layers to be combined for complex analyses. Through preprocessing, the quality and accuracy of large-scale remote sensing data can be ensured.

[0070] S200: Based on preset rules, the remote sensing big data is divided into blocks to obtain multiple sub-region data. This block-based processing of remote sensing big data includes two steps: classification and block division.

[0071] First, let's introduce the classification:

[0072] Spatial consistency can be understood as the ability to process spatial information. In image processing, spatial consistency represents local structure and spatial relationships. For example, in image segmentation tasks, spatial consistency requires accurate identification of object boundaries in an image and maintaining coherence and stability between different regions, thus providing a good foundation for subsequent parallel computing and local modeling. To improve the spatial consistency of data in each sub-region, this application performs a classification operation on the preprocessed remote sensing big data before segmenting it into blocks. Specifically, this includes:

[0073] Preset spectral characteristics and normalized vegetation index;

[0074] Based on spectral characteristics, the remote sensing big data is divided into first-level segments to obtain images of various land types;

[0075] Based on the normalized vegetation index, the land cover images after the first level of classification are divided into a second level to obtain environmental remote sensing datasets under different growth conditions or different topographic features of each land cover image.

[0076] Pre-defined spectral features. First-level segmentation is performed based on spectral features to ensure that the main land cover types (e.g., grassland, forest, farmland) within each region remain singular or similar, thus achieving the integrity and stability of spectral characteristics within the region and reducing interference effects caused by an excessive number of "mixed pixels." When segmenting regions, further subdivision can be based on main vegetation types or ecosystem types to ensure a relatively balanced state in spectral and geographic information across regions. This segmentation strategy helps to deeply understand the impact of different ecosystems or vegetation types on spectral characteristics, thereby improving the accuracy of remote sensing image analysis. Specific segmentation strategies are as follows: Vegetation is broadly classified according to vegetation category or ecosystem type. For example, grassland, forest, and cultivated land. For instance, vegetation can be divided into main categories such as grassland, forest, and cultivated land. Subsequently, by applying NDVI (Normalized Difference Vegetation Index), various growth states of vegetation are further subdivided, reflecting key information such as the degree of grassland degradation.

[0077] Based on the Normalized Difference Vegetation Index (NDVI), different NDVI thresholds are used for classification within the same vegetation species, such as NDVI less than 0.1, between 0.1 and 0.3, and greater than 0.3. This method effectively identifies and assesses important ecological indicators such as grassland degradation, forest density, and growth status. It also allows for the estimation and processing of carbon information for a specific type of vegetation under different growth states within a specific geographic area. For vegetation with consistent growth states (or similar spectral characteristics), centralized carbon information estimation can effectively improve the accuracy of the inversion process and help reduce the resulting uncertainties. Specifically, if land cover types are too diverse and complex within the same area, it can lead to significant noise and errors in subsequent feature extraction or modeling. To overcome this problem, a block-based strategy combining vegetation type and spectral information can be adopted. This method divides the land surface into sample blocks with more uniform spectral characteristics, making these blocks more effective for model training. Furthermore, it helps improve the efficiency of model training and the robustness of the final results, making the model's predictions and analyses more accurate and reliable.

[0078] Next, we will introduce the specific block division process, including:

[0079] Based on spatial resolution thresholds and geographic range thresholds, environmental remote sensing datasets (subclasses) under different growth conditions or different topographic features are uniformly divided into blocks under the same land cover image (major category) to obtain multiple sub-region data.

[0080] Block partitioning is the process of dividing large-scale remote sensing big data into multiple smaller blocks or sub-regions according to preset rules. Each sub-region contains local image metadata to facilitate subsequent parallel computing and processing. Intuitively, when an image is magnified to a certain extent, you will see blocks; these blocks are the image metadata. Spatial resolution refers to the size or dimensions of the smallest unit that can be distinguished in detail on remote sensing big data, used to characterize the ability to distinguish details of ground targets. Generally speaking, the smaller the ground area represented by the image metadata, the higher the spatial resolution, and the richer and more accurate the information expressed. Block partitioning the remote sensing big data allows sub-region data to be processed independently, reducing the computational load of a single operation and improving processing efficiency.

[0081] First, a spatial resolution threshold is preset. This threshold is set to ensure that the resolution used for data processing and analysis is no lower than the highest quality standard of similar products globally or within a specific region. This means that if the spatial resolution of the highest-quality soil carbon products globally or within China currently reaches 250 meters, then in this scheme, the spatial resolution threshold must be maintained at least at 250 meters when processing each segment to ensure the accuracy and quality of data processing. By maintaining the same spatial resolution threshold as mainstream or high-precision data, this scheme guarantees the accuracy and comparability of the final results. If the resolution is reduced, the amount of detailed information contained in the data will decrease, leading to a decline in the scientific value and market competitiveness of the product.

[0082] Secondly, preset geographical range thresholds. Taking into account product requirements and available computing resources (such as memory, CPU threads, storage space, etc.), set geographical range thresholds for the blocks to avoid computational and storage burdens caused by excessive data volume.

[0083] The maximum processing limit for each data block is set at 17 megabytes, which can be considered an "equivalent" scale in terms of data volume or pixel size. On a computer configured with 24 processing threads, if each thread is responsible for processing one data block, the maximum memory requirement is calculated as follows: 300 images multiplied by 17 megabytes per image, then multiplied by 24 threads, resulting in 122,400 megabytes, or approximately 122 GB. This calculated memory requirement is within the manageable range of a typical office computer's memory capacity (128 GB), thus ensuring the smooth execution of processing tasks. When performing data block processing, both the number and size of data blocks need to be considered. This is to avoid setting data blocks that are too large, which could lead to memory overflow or slow down the calculation process; at the same time, it is also to avoid data blocks that are too small. While this can reduce the processing time of a single data block, it increases the number of data blocks to manage and the frequency of input / output (I / O) operations, which can actually slow down the overall processing speed.

[0084] By setting spatial resolution thresholds and geographical range thresholds, remote sensing big data of the same state can be processed in blocks. This can not only meet the scientific research requirements for high precision and robustness, but also make full use of limited computing and storage resources, ensuring processing efficiency and feasibility.

[0085] S300: Input the data from multiple sub-regions in parallel into a pre-constructed remote sensing image prediction model to obtain the remote sensing image information corresponding to each sub-region data. The remote sensing image prediction model includes multiple sub-models, and for each sub-region data, based on preset selection rules, one sub-model is selected from the multiple sub-models for independent training and prediction.

[0086] As is understandable, machine learning is a technique that automatically learns and identifies patterns from data by building mathematical models, and then makes predictions and decisions. In remote sensing big data processing, machine learning can be used to analyze and interpret image data, improving the prediction accuracy of covariates. Common machine learning algorithms include regression analysis, decision trees, and neural networks. Existing machine learning methods typically perform global modeling of the entire image, making it difficult to achieve accurate modeling and analysis in spatially heterogeneous local areas, resulting in a significant decrease in prediction accuracy in complex terrains or ecological environments.

[0087] In this scheme, the training method for the pre-built remote sensing image prediction model includes:

[0088] Define the random forest model;

[0089] Acquire remote sensing image training data and soil organic carbon (SOC) density data source. The remote sensing image training data includes remote sensing training datasets of various environments (such as meteorology, vegetation, soil, and topography) for multiple land types (forests, grasslands, farmland, etc.). The SOC data source includes at least latitude and longitude, depth, and SOC density.

[0090] The random forest model is trained using remote sensing image training data, remote sensing big data sources, and preset hyperparameters through a model tuning wrapper function. The hyperparameters include at least the number of features, sample size, sampling with replacement, node size, number of decision trees, and node splitting rules.

[0091] Cross-validation was used to determine the remote sensing image prediction model. The remote sensing image prediction model includes multiple sub-models (grassland SOC density estimation model, forest SOC density estimation model, farmland SOC density estimation model, and other land cover SOC density estimation models), which is block-to-major category modeling; it can also be further subdivided into different growth conditions / topography models for each land cover, which is block-to-minor category modeling.

[0092] The model training process in this application mainly involves adjusting six hyperparameters, including the number of features, sample size, sampling with replacement, node size, number of decision trees, and node splitting rules. The hyperparameters are iteratively optimized using the `tunerange` function in the R package. Finally, cross-validation is used to determine the final model parameters and model. In this implementation scheme, based on the segmented data, machine learning models such as random forests are independently applied for local modeling to improve prediction accuracy within local ranges.

[0093] In this scheme, multiple sub-region data are input in parallel into a pre-constructed remote sensing image prediction model to obtain the remote sensing image information corresponding to each sub-region data. Specific steps include:

[0094] In each sub-region of data, each cell is traversed, and the corresponding sub-model is selected by using an if statement based on the vegetation category, growth information (e.g., NDVI>0.3), or topographic information (e.g., slope>20°).

[0095] Finally, input a sub-region data into the sub-model corresponding to the sub-region data to obtain the sub-remote sensing image information corresponding to the sub-region data.

[0096] Within different vegetation types, further distinctions are made between different vegetation growth patterns or topographic features to establish prediction models for a particular vegetation under different growth conditions or different topographic features. This increases the number of zoning areas and makes data processing more complex.

[0097] In this scheme, a corresponding (pre-established) density estimation model is selected based on the vegetation growth or topographic information of each sub-region. Because the vegetation information and growth are similar, the environmental data of each block are similar. Based on this block division, the model is trained separately for each sub-region. The model for each sub-region is independent and optimized specifically for the specific environmental conditions and vegetation characteristics within that sub-region. In this way, after the models are trained, they can more accurately predict the soil carbon content within their respective regions. By independently modeling the sub-region data, high-precision prediction within a local range is achieved. Specifically, this invention uses a random forest model to independently train and predict each block of data, enabling the model to better capture the spatial characteristics and heterogeneity within the block. This local modeling method significantly improves the prediction accuracy in complex environments, especially in the accurate simulation of carbon storage, where the effect is particularly remarkable.

[0098] In some implementations, the random forest model can also use other machine learning models. For example, SVM (Support Vector Machine) or GBDT (Gradient Boosting Decision Tree) models can be employed. These SVM and GBDT models can be applied to independent modeling of segmented data, especially in scenarios with strong nonlinear characteristics, thereby improving prediction accuracy.

[0099] S400: Integrate the remote sensing image information corresponding to each of the sub-region data to obtain a remote sensing image.

[0100] Integration is the process of combining multiple sub-region data points processed into a single, complete remote sensing image. During integration, it is crucial to ensure the accurate spatial stitching of each sub-region to avoid misalignment or overlap. Existing integration methods are prone to spatial location errors during data stitching, leading to geographical discontinuities or misalignments in the remote sensing image, thus affecting the accuracy of the overall analysis. In other words, the analysis and processing of segmented sub-region data can easily introduce spatial location errors, impacting the final stitching accuracy. Although some stitching algorithms improve this through geometric correction and spatial interpolation, the lack of effective spatial positioning techniques means that the stitched image still suffers from some misalignment or discontinuity, affecting the accuracy of subsequent analysis.

[0101] To address the shortcomings of existing technologies, in some more specific implementation schemes, the integration of each sub-remote sensing image information corresponding to each sub-region data to obtain a remote sensing image includes the following specific steps:

[0102] Scan the image data file corresponding to each sub-remote sensing image information and read the geographical location information from the header file;

[0103] Integrate all geographic location information to construct virtual image files, which will serve as a directory index;

[0104] According to the directory index, the image data of the image data file corresponding to each sub-remote sensing image information is read and written into the target image data file for integration to obtain the remote sensing image.

[0105] This application can be performed using the `gdal` function in the R language. Specifically, the `gdalbui ldvrt()` function automatically identifies and reads all image data files ending in `.tif` within the current working folder. These image data files typically contain geographic raster data. Through the `gdalbuildvrt()` function, these scattered geographic raster data files are integrated to construct a single virtual image file, i.e., a virtual raster file. This virtual raster file is a virtual raster dataset that can contain information from multiple source data files, but it does not actually store the merged data itself. Instead, it records the stitching information of each source file in an XML format, dynamically reading information from the original data source when needed. This recording method is essentially equivalent to providing a "directory index" for subsequent stitching processing. Virtual raster files are usually very small in size because they only record source data information, including the source file paths, georeferences, bands, projections, and the spatial relationships between the files. Since this process only generates an index and does not require copying or writing large amounts of data to disk, it can achieve rapid processing of thousands (+) of images. This approach significantly improves data processing efficiency, especially when dealing with large amounts of geographic data. It can significantly reduce waiting time, simplify data management, and improve data processing efficiency.

[0106] The `gdal_translate()` function reads image metadata from each raster dataset according to the predefined stitching rules in the directory index and integrates it into a single target image data file. Furthermore, `gdal_translate()` allows users to perform various processing operations on the output target image data file as needed, including but not limited to data compression, storage format conversion, and setting the `nodata` value. These operations ensure that the final generated target image data file is not only spatially contiguous but also more efficient in data processing and better suited to specific requirements.

[0107] This technical solution introduces precise spatial positioning technology, combined with geometric correction and interpolation algorithms, which effectively eliminates spatial position errors generated during the block processing, ensuring that the data of each sub-region can maintain spatial consistency when stitching, and ensuring that the spatial position of the sub-region data is accurately aligned during stitching, thus achieving seamless high-precision image stitching and providing high-precision basic data support for subsequent remote sensing data analysis.

[0108] Preferably, the method of integrating the remote sensing image information corresponding to each of the sub-region data to obtain a remote sensing image further includes: using a graph optimization algorithm to eliminate outliers based on the actual situation of the remote sensing image.

[0109] Specifically, in soil carbon research, according to scientific principles, the minimum value of soil carbon should theoretically not be negative. However, in certain specific regions, the estimated soil carbon storage may be negative. This typically occurs in river areas, snow-covered areas, or when there are missing values ​​(usually denoted as na) in the original image data. These missing values ​​are generally filled with -9999 for the na values. It's also possible that during data processing, some invalid values ​​were marked, and these na values ​​may, in some cases, lead to negative calculation results. In short, this does not reflect the real-world situation, because in nature, soil carbon values ​​are always non-negative. Therefore, once negative values ​​are found during data collection or analysis, measures are taken to correct the carbon emission values ​​for these areas to 0 to ensure data accuracy and reliability. In this technical solution, a graph optimization algorithm is used to globally adjust all data blocks, thereby reducing overall errors during stitching and ensuring spatial consistency.

[0110] To ensure the consistency of the stitched data across the entire system when integrating the sub-remote sensing image information corresponding to each sub-region, please refer to the following:

[0111] First, conduct a consistency assessment and check the splicing quality:

[0112] Mean Squared Error (MSE) is used to assess the degree of difference. Specifically, it evaluates the accuracy of the stitching by calculating the difference between each pixel at the stitching point of two adjacent blocks, squared these differences, and then averaging them. The smaller the value, the smaller the pixel difference at the stitching point, and the higher the consistency of the image.

[0113] The Structural Similarity Index (SSIM) measures the structural similarity between two images. This index comprehensively considers similarity across three dimensions: brightness, contrast, and structure. It focuses not only on pixel-level differences but also delves into the similarity of texture structures. Through this multi-dimensional evaluation, SSIM can more comprehensively reflect the degree of similarity between images.

[0114] Edge Smoothness Metric is used to evaluate the smoothness of image edges, focusing on the visual coherence of the image stitching area and ensuring that there are no abrupt breaks or obvious edge effects at the stitching point. This evaluation ensures that the stitched image appears more natural and fluid.

[0115] The consistency of feature space distribution, such as based on Kullback-Leibler divergence (KL divergence), uses mathematical methods to compare whether the feature distributions of two datasets are consistent. This is suitable for analyzing data with complex structures, such as the spectral characteristics of vegetation in different regions. By calculating the KL divergence, we can quantitatively assess whether the distributions of two datasets match in the feature space, thereby determining whether they have similar characteristics.

[0116] Taking the concatenation region Ω of any two adjacent data blocks A and B as an example, the following metrics can be defined to quantify the concatenation effect:

[0117]

[0118] Where Ω is the splicing region of adjacent blocks A and B (i.e. the overlapping or adjacent boundary region that needs to be evaluated), |Ω| is the total number of pixels in the splicing region Ω, and A(x) and B(x) represent the pixel values ​​(which can be grayscale values, spectral values, etc.) of blocks A and B at position x, respectively.

[0119]

[0120] Where, μ A μ B σ represents the average pixel values ​​of block A and block B within the stitching region Ω, reflecting the overall brightness or signal strength of the region; A σ B σ represents the standard deviation of pixels in block A and block B within the stitching region Ω, reflecting the degree of fluctuation (contrast) in pixel values ​​within the region; AB C1 represents the covariance of block A and block B within the stitching region Ω, measuring the correlation of their pixel values ​​(i.e., whether their structures change synchronously); C2 and C1 are both stabilization factors (usually very small positive numbers, such as 10). -4 (), used to prevent the denominator from being zero or the value from being unstable.

[0121] MSE Ω Focusing on absolute differences in pixel values, SSIM is suitable for detecting brightness or numerical tortuosity at stitching points. SSIM, on the other hand, focuses on the consistency of structural information, making it suitable for evaluating the coherence of high-level features such as texture and edges. Combining both allows for a more comprehensive evaluation of the global consistency of stitched remote sensing data.

[0122] Secondly, after completing the consistency assessment, a feedback parameter tuning mechanism based on the consistency index is introduced. The core idea is to use the consistency assessment results of the stitched region to constrain the model's prediction output, as follows:

[0123] The prediction error for each block in the stitching area is: The overall loss function is then:

[0124]

[0125] in: This represents the total loss of the i-th block. λ represents the original task loss, and λ represents the boundary loss weight (empirical value 0.1 to 0.5). This indicates the error in the splicing area. Where i represents the index of the current block (e.g., the i-th sub-block after the remote sensing image is divided into blocks), j represents the index of the block adjacent to the current block i, and the value of j is determined by N(i), where N(i) represents the set of adjacent blocks, and Ω represents the set of indices of all blocks adjacent to the i-th block (e.g., adjacent blocks above, below, left, and right). ij This represents the area where block i and block j are joined (i.e., the overlapping or adjacent boundary area of ​​the two blocks).

[0126] This indicates that block i and block j are in the splicing region Ω. ij The mean squared error within the range, where i and j can replace A and B in the above text.

[0127] The sum of the stitching boundary errors between the i-th block and all its neighboring blocks is calculated; a smaller value indicates higher stitching consistency between the block and its surrounding blocks. By accumulating the mean squared error (MSE) between neighboring blocks, pixel-level differences in the stitching region are quantified, providing a clear optimization objective for model adjustment (such as driving model parameters to reduce boundary differences). The overall loss function includes both the original task score and the error score at the stitching point; by adjusting the weight λ, the model is made to prioritize stitching quality.

[0128] Next, let's introduce the specific ways to enhance consistency:

[0129] First, the weighted fusion method:

[0130] P fused (x)=w A (x)P A (x)+w B (x)P B (x),

[0131] Among them, w A (x)=d B (x) / (d A (x)+d B (x)), w B (x)=1-w A (x), P fused (x) represents the final value of pixel x after fusion, obtained by weighted averaging of pixel values ​​from two adjacent blocks, P A (x) represents the original value (such as spectral value, gray value) of a pixel in block A, P B(x) represents the original value of pixel x in block B (another block adjacent to block A), w A (x), w B (x) are the fusion weights of block A and block B at pixel x, respectively, satisfying w A (x)+w B (x) = 1. d A (x) represents the distance from pixel x to the center of block A (distance can be measured using Euclidean distance, etc.), d B (x) represents the distance from pixel x to the center of block B.

[0132] The closer a pixel is to the center of block A (d) A (The smaller x) is, the greater the weight w of block A. A The larger (x) is, the closer the fusion value at that point is to P. A (x); conversely, the closer to the center of block B (d) B (The smaller x) is, the greater the weight w of block B. B The larger (x) is, the closer the fusion value is to P. B (x).

[0133] This formula achieves smooth fusion of adjacent blocks through distance weighting. The core parameter is the "distance from the pixel to the center of the block". By dynamically adjusting the weights, the stitching result maintains the internal features of each block while the boundary areas transition naturally, thus improving the overall consistency of remote sensing data stitching.

[0134] Second, feature alignment method:

[0135] Adding a convolutional residual module before the model output corrects the feature distribution differences between different blocks and improves the consistency of boundary regions. Specifically, before the machine learning model (such as a convolutional neural network) generates the final prediction result, a convolutional residual module is inserted to specifically correct the feature differences between different blocks, thereby making the splicing boundaries of adjacent blocks more natural and coherent.

[0136] In the process of the model segmenting remote sensing images and generating prediction results, an optimization module is inserted before the final output. During remote sensing image segmentation, different blocks may exhibit inconsistent feature distributions due to factors such as location and lighting (e.g., block A is brighter, block B is darker, or textures of the same scene appear differently in the two blocks). This module, through convolution and residual connections, forces the feature distributions (such as mean, variance, and texture pattern) of different blocks to converge, preventing color banding and texture breaks at the boundaries of adjacent blocks (e.g., the intersection of block A and block B), making the overall image appear as a "complete piece," rather than a patchwork. In other words, this module is a feature calibration mechanism that automatically repairs feature inconsistencies caused by segmentation through a deep learning structure. It does not require changing the core model architecture; simply adding a correction step before the output effectively improves the quality of remote sensing image stitching.

[0137] Understandably, remote sensing big data has the following characteristics: First, massive data volume. The scale of remote sensing data increases dramatically with the improvement of spatial resolution and the expansion of observation areas, especially for long-term series data at global or regional scales, where the data volume is enormous, typically reaching TB or even PB levels. Second, multiple dimensions. Remote sensing data usually contains multiple band information (spectral dimension) and time series information (temporal dimension), and processing this multidimensional data requires more complex algorithms and models. Third, long time span. The time series of remote sensing data covers a long time span, which can be used to analyze long-term environmental changes and trends, but this also increases the difficulty of data processing. Therefore, processing is often inefficient, resource-intensive, and difficult to meet the needs of large-scale data processing.

[0138] Although parallel computing technology has been widely used in remote sensing data processing, existing parallelization strategies often have limited efficiency when dealing with complex machine learning tasks. In particular, when processing computationally complex models such as random forests, insufficient optimization of task scheduling and resource utilization leads to high consumption of computing resources, long processing times, and difficulty in achieving efficient remote sensing data processing.

[0139] To address the aforementioned shortcomings, in some implementation schemes, the method for processing remote sensing big data further includes:

[0140] Set up receiving and summarizing nodes;

[0141] The receiving node receives remote sensing big data;

[0142] Based on remote sensing big data, the total amount of data in the sub-region is determined to be M, and the total amount of computing clusters required is N, where M≤N;

[0143] Data from M sub-regions is distributed to N computing clusters for parallel computation. Once all N computing clusters have completed their computations, the results from the N computing clusters are aggregated through the aggregation node.

[0144] In this scheme, after receiving remote sensing big data, the control node divides the remote sensing big data into M sub-region data according to a pre-set total amount of sub-region data M and a total number of computing clusters N, and according to preset rules. It is important to note that the number of blocks can be pre-set based on the size of the remote sensing big data before block processing to ensure a reasonable total amount of sub-region data. The total number of computing clusters must be greater than or equal to the total amount of sub-region data to ensure that each sub-region data is processed independently, avoiding computational congestion. The M sub-region data are distributed to N computing clusters for parallel computation, meaning the M sub-region data are input in parallel into a pre-constructed remote sensing image prediction model. This remote sensing image prediction model is a local modeling model; each computing cluster's remote sensing image prediction model is used to process the sub-region data corresponding to that computing cluster. This significantly shortens the time required for remote sensing image prediction and improves local accuracy. The results of the N computing clusters can be understood as M sub-remote sensing image information corresponding to the M sub-region data. Given the characteristics of parallel computing, task allocation is based on a cyclic dataset. For example, the number of threads in the computer is 24. Therefore, the first loop task will be assigned to the first 24 thread blocks. When any of these 24 threads completes its task, the next task (the 25th task) will be scheduled to that idle thread, and so on. After all N computing clusters have completed their computations, the M sub-remote sensing image information will be integrated to obtain the remote sensing image.

[0145] Preferably, the computing cluster here can be replaced with a multi-core processor or a distributed computing system to process data from M sub-regions simultaneously. For example, the computing cluster can be replaced with a multi-core CPU, GPU (Graphics Processing Unit), or FPGA (Field Programmable Gate Array). This high-performance computing architecture can utilize a distributed computing framework on a cloud computing platform. Through reasonable task scheduling and resource utilization, this solution significantly shortens computation time and improves the processing efficiency of large-scale remote sensing data when processing large-scale data, making it particularly suitable for high-resolution, multispectral image data analysis scenarios.

[0146] Furthermore, in addition to using a single multi-core CPU or GPU architecture, the block processing and remote sensing image prediction processes can be distributed and executed in a distributed computing environment. Through distributed computing, when processing extremely large-scale data, the computing resources of multiple machines in the network can be fully utilized, further improving processing efficiency.

[0147] In specific carbon storage simulation methods, remote sensing big data processing methods are used to simulate carbon storage, wherein the remote sensing image prediction model has at least the function of carbon storage prediction.

[0148] When the entire image is processed without being divided into blocks, the estimation model used is a general prediction model for the study area without distinguishing between vegetation types. When the image is divided into blocks according to vegetation type, the estimation model used is a prediction model for each vegetation type. When the image is divided into blocks according to vegetation type, it is further refined to different growth conditions of vegetation under each vegetation type. The estimation model used is a prediction model for different growth conditions under each vegetation type.

[0149] like Figure 3 The figure shows the goodness of fit between the predicted SOC density and the reserved validation SOC density under three different processing methods: block-to-major-class, block-to-minor-class, and no block-based processing. In the case of no block-based processing, the slope is 0.242, and the coefficient of determination (R²) is... 2 The slope is 0.312, indicating a significant difference between the predicted and validated values. When segmented into larger classes, the slope is 0.522, and the R-squared is [value missing]. 2 The slope was correspondingly increased to 0.486, resulting in improved prediction accuracy. When further refined to subclasses, the slope became 0.546, and R0... 2 It even reached 0.5. These data clearly demonstrate that the block-based parallel processing method has significant advantages in improving prediction accuracy and result refinement. Compared with existing technologies, this technique has achieved a significant improvement in accuracy, with R... 2 The value increased from 0.3 to 0.5, indicating that the slope between the predicted and validated values ​​is closer to a 1:1 line, thus significantly improving the model's fit. R 2 The value reflects the degree to which the model fits the observed data. The closer the value is to 1, the better the model fits and the more accurate the prediction results.

[0150] Compared to non-block processing, block-parallel processing technology can effectively handle high-resolution multi-band imagery and improve the accuracy of capturing terrain and vegetation details.

[0151] Take, for example, an environmental remote sensing image dataset covering the Eurasian continent with a spatial resolution of 250 meters. Figure 4 The figure shows the computation time for block-based processing and non-block-based processing at different resolutions. It is clear from the figure that with block-based processing, when the spatial resolution reaches 100 meters, the processing time is only about 5 hours. With non-block-based processing, even a long period of 96 hours is not possible to complete the task. x indicates that the processing was not completed within 96 hours. Similarly, as... Figure 5 As shown in the figure, this graph illustrates the computation time for block processing and overall non-block processing at different numbers of bands. It can be observed from the graph that with block processing, when the number of bands reaches 300, the processing time is approximately 6 hours. However, with non-block processing, the task cannot be completed even within a long period of 96 hours. x represents processing that was not completed within 96 hours.

[0152] In summary, in terms of processing time, block parallel processing, through simultaneous operation of multiple nodes / cores, reduces processing time to less than one day, far faster than existing technologies, and even more so than some other technologies, supporting batch and iterative applications. Block parallel processing ensures data processing consistency and accuracy, reduces overall errors, and avoids seams and inconsistencies during image stitching. When processing petabyte-scale remote sensing data, this technology avoids frequent batch loading and unloading, improving computational efficiency and iteration speed, and providing more timely and accurate carbon storage change data.

[0153] In summary, the remote sensing big data processing method proposed in this embodiment organically integrates block processing, parallel computing, and local machine learning modeling, forming a complete remote sensing big data processing chain. This comprehensive integration allows the various technologies to complement and enhance each other, significantly improving overall processing efficiency and data analysis accuracy, and overcoming the limitations of single-technology approaches in existing technologies. Block processing enables parallel computing and local modeling. Applying machine learning models to local modeling of block data improves the prediction accuracy of local areas when processing complex remote sensing data, especially showing a significant advantage in the accurate simulation of carbon storage. Parallel computing, through reasonable resource scheduling and task allocation, ensures efficient processing of each block of data, significantly improving computational efficiency while reducing resource consumption, providing a more cost-effective solution for the practical application of large-scale remote sensing data. Finally, the high-precision spatial positioning and stitching integration technology ensures seamless spatial connection of block data, eliminates spatial location errors, achieves seamless data integration, and generates image data with high spatial accuracy, providing a reliable foundation for subsequent analysis.

[0154] Please refer to Figure 6This embodiment also provides a remote sensing big data processing system 100 based on block-based and parallel machine learning, used to implement the remote sensing big data processing method, specifically including:

[0155] Acquisition module 11 is used to acquire remote sensing big data;

[0156] The segmentation module 12 is used to divide the remote sensing big data into blocks based on preset rules to obtain multiple sub-region data;

[0157] The remote sensing image prediction module 13 is used to input multiple sub-region data into a pre-constructed remote sensing image prediction model in parallel to obtain each sub-remote sensing image information corresponding to each sub-region data; wherein, the remote sensing image prediction model includes multiple sub-models, and each sub-region data selects one sub-model from the multiple sub-models for independent training and prediction based on a preset filtering rule;

[0158] The remote sensing image integration module 14 is used to integrate the remote sensing image information corresponding to each sub-region data to obtain the target remote sensing image.

[0159] This embodiment also provides a computer-readable storage medium storing a remote sensing big data processing program, which can be executed by at least one processor to cause the at least one processor to perform the steps of the remote sensing big data processing method.

[0160] It should be understood that the above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A remote sensing big data processing method based on block-based and parallel machine learning, characterized in that, The specific steps include: Acquiring remote sensing big data; Based on preset rules, the remote sensing big data is divided into blocks to obtain multiple sub-region data; Multiple sub-region data are input in parallel into a pre-constructed remote sensing image prediction model to obtain each sub-remote sensing image information corresponding to each sub-region data. The remote sensing image prediction model includes multiple sub-models. Each sub-region data selects one sub-model from the multiple sub-models for independent training and prediction based on a preset selection rule. Selecting one sub-model from the multiple sub-models specifically includes: traversing each pixel in each sub-region data and selecting the sub-model corresponding to the sub-region data based on land type, vegetation growth information and / or landform information. The target remote sensing image is obtained by integrating the data of each sub-region and the corresponding sub-remote sensing image. Specifically, the remote sensing big data is divided into blocks based on preset rules to obtain multiple sub-region data, including: preset spectral features, normalized vegetation index, spatial resolution threshold, and geographic range threshold; the remote sensing big data is divided into first-level segments based on spectral features to obtain images of various land types; the land type images after the first-level segmentation are divided into second-level segments based on the normalized vegetation index to obtain environmental remote sensing datasets under different growth conditions or different topographic features of each land type image; based on the spatial resolution threshold and geographic range threshold, the environmental remote sensing datasets under different growth conditions or different topographic features of the same land type image are uniformly divided into blocks to obtain multiple sub-region data. Specifically, the process of integrating the remote sensing image information corresponding to each sub-region data to obtain the target remote sensing image includes: scanning the image data file corresponding to each sub-remote sensing image information and reading the geographical location information in the header file; integrating all geographical location information to construct a virtual image file as a directory index; and reading the image data of the image data file corresponding to each sub-remote sensing image information according to the directory index and integrating them to obtain the remote sensing image. In the process of integrating the sub-remote sensing image information corresponding to each sub-region data: a consistency assessment is performed to check the stitching quality; after the consistency assessment is completed, the consistency assessment results of the stitched regions are used to constrain the model's prediction output, and the overall loss function is: ； in, This represents the total loss of the i-th block. Indicates the loss of the original task. This indicates the error in the splicing area. This represents the boundary loss weights, and i represents the index of the current block. Adjustable.

2. The remote sensing big data processing method based on block-based and parallel machine learning according to claim 1, characterized in that, The remote sensing big data is preprocessed, and the preprocessing includes at least reprojection, resampling, filling invalid values, and image overlay of the remote sensing big data.

3. The remote sensing big data processing method based on block-based and parallel machine learning according to claim 1, characterized in that, The training method for the pre-built remote sensing image prediction model includes: Define the random forest model; Acquire remote sensing image training data and remote sensing big data source. The remote sensing image training data includes environmental remote sensing training datasets for multiple land types. The remote sensing big data source includes at least meteorological, vegetation, topographic, and soil environmental information related to the information of the elements to be simulated. The information of the elements to be simulated includes latitude and longitude that correspond to the location information of the remote sensing images and the density of the elements to be simulated that quantifies the number of elements. The random forest model is trained using remote sensing image training data, remote sensing big data sources, and preset hyperparameters through a model tuning wrapper function. The hyperparameters include at least the number of features, sample size, sampling with replacement, node size, number of decision trees, and node splitting rules. Cross-validation was used to determine the remote sensing image prediction model, which includes multiple sub-models, each of which corresponds one-to-one with an environmental remote sensing training dataset for multiple land types.

4. The remote sensing big data processing method based on block-based and parallel machine learning according to claim 3, characterized in that, Multiple sub-region data are input in parallel into a pre-constructed remote sensing image prediction model to obtain each sub-remote sensing image information corresponding to each sub-region data. The specific steps include: For each sub-region of data, each pixel is traversed, and the sub-model corresponding to the sub-region of data is selected based on land type, vegetation growth information and / or landform information. Input a sub-region of data, and use an if statement to filter out the sub-models corresponding to the sub-region of data; The sub-remote sensing image information corresponding to the sub-region data is obtained through sub-model calculation.

5. The remote sensing big data processing method based on block-based and parallel machine learning according to claim 1, characterized in that, Also includes: Set up receiving and summarizing nodes; The receiving node receives remote sensing big data; Based on remote sensing big data, the total amount of data in the sub-region is determined to be M, and the total amount of computing clusters required is N, where M≤N; Data from M sub-regions is distributed to N computing clusters for parallel computation. Once all N computing clusters have completed their computations, the results from the N computing clusters are aggregated through the aggregation node.

6. The remote sensing big data processing method based on block-based and parallel machine learning according to claim 1, characterized in that, After integrating the remote sensing image information corresponding to each of the sub-region data to obtain the remote sensing image, the method further includes: Based on image optimization algorithms, outliers are eliminated according to the actual situation of remote sensing images.

7. A remote sensing big data processing system based on block-based and parallel machine learning, characterized in that, Specifically, it includes: The acquisition module is used to acquire remote sensing big data; The segmentation module is used to divide the remote sensing big data into blocks based on preset rules to obtain multiple sub-region data. The remote sensing image prediction module is used to input multiple sub-region data into a pre-constructed remote sensing image prediction model in parallel to obtain each sub-remote sensing image information corresponding to each sub-region data. The remote sensing image prediction model includes multiple sub-models. For each sub-region data, based on preset filtering rules, one sub-model is selected from the multiple sub-models for independent training and prediction. Selecting one sub-model from the multiple sub-models specifically includes: traversing each pixel in each sub-region data, and selecting the sub-model corresponding to the sub-region data based on land type, vegetation growth information, and / or landform information. The remote sensing image integration module is used to integrate the remote sensing image information corresponding to each of the sub-region data to obtain the target remote sensing image; The segmentation module divides the remote sensing big data into blocks based on preset rules to obtain multiple sub-region data, including: preset spectral features, normalized vegetation index, spatial resolution threshold, and geographic range threshold; it performs a first-level segmentation of the remote sensing big data based on spectral features to obtain images of various land cover types; it performs a second-level segmentation of the land cover images after the first-level segmentation based on the normalized vegetation index to obtain environmental remote sensing datasets under different growth conditions or different topographic features of each land cover image; and it uniformly segments the environmental remote sensing datasets under different growth conditions or different topographic features under the same land cover image based on the spatial resolution threshold and geographic range threshold to obtain multiple sub-region data. The remote sensing image integration module integrates the remote sensing image information corresponding to each sub-region data to obtain the target remote sensing image. Specifically, this includes: scanning the image data file corresponding to each sub-remote sensing image information and reading the geographical location information in the header file; integrating all geographical location information to construct a virtual image file as a directory index; and reading the image data of the image data file corresponding to each sub-remote sensing image information according to the directory index and integrating it to obtain the remote sensing image. The remote sensing image integration module, in the process of integrating the sub-remote sensing image information corresponding to each sub-region data, performs a consistency assessment to check the stitching quality; after completing the consistency assessment, it uses the consistency assessment results of the stitched region to constrain the model's prediction output, and the overall loss function is: ； in, This represents the total loss of the i-th block. Indicates the loss of the original task. This indicates the error in the splicing area. This represents the boundary loss weights, and i represents the index of the current block. Adjustable.

Images