A deep learning-based ecological environment restoration assessment method

Abstract

The present invention provides an ecological environment restoration assessment method based on deep learning, which relates to the field of intelligent ecological environment restoration assessment technology. The method comprises: obtaining multi-source data as a time series sequence, including remote sensing image data, environmental quality data, and biodiversity data; constructing an ecological environment knowledge graph, and reconstructing entity relationships and performing time series prediction on the knowledge graph through a graph neural network; performing multi-class segmentation on the remote sensing image data using a multi-class segmentation model to obtain a semantic category response graph; and obtaining classification results based on the semantic category response graph, the differences in the semantic category response graphs at multiple time points, the output of the final layer of the graph neural network, and the differences between regional environmental quality data and biodiversity data at multiple time points. Through deep learning technology, the present invention achieves the fusion and analysis of multi-source data, predicts ecological environment change trends, assesses the degree of ecological environment restoration, improves the scientific nature and reliability of ecological assessments, and provides intelligent support for ecological protection and restoration.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent ecological environment restoration assessment, and specifically relates to an ecological environment restoration assessment method based on deep learning. Background Art

[0002] As ecological and environmental issues become increasingly complex, efficient and accurate ecological restoration assessment technologies have become a core requirement for environmental management and protection. Existing methods primarily rely on ground-based sampling and remote sensing image analysis. These methods involve manually collecting air, water, and soil samples for laboratory testing, combined with remote sensing imagery to interpret land feature types and land use changes. Although such methods can obtain local environmental parameters, they face significant limitations: First, ecological and environmental data cover multi-source heterogeneous information such as meteorological indicators, biodiversity, and pollutant concentrations. The existing system has difficulties in data integration due to differences in data formats (such as raster images, time-series sensor data, and structured tables) and inconsistent equipment calibration standards; second, ecological and environmental evolution has dynamic spatiotemporal correlation characteristics. For example, pollutant diffusion requires continuous monitoring at the minute level, while traditional ground stations have long sampling intervals and slow remote sensing image update cycles, making it difficult to capture sudden events or continuous evolution processes; third, existing assessment models mostly rely on single indicators (such as the vegetation index NDVI and the air quality index AQI), and lack collaborative modeling of multidimensional factors such as biodiversity, climate change, and land use. In addition, traditional statistical methods (such as linear regression) cannot represent the nonlinear coupling effects between ecological elements, resulting in one-sided assessment results.

[0003] Current research has attempted to improve the accuracy of ecological restoration assessments through machine learning, but technical bottlenecks remain. While traditional neural network-based methods (such as LSTM) can process time-series data, they struggle to model spatial heterogeneity (e.g., interregional ecological connections). Models based on single-modal data (e.g., classification networks that rely solely on remote sensing imagery) are unable to integrate multi-source information (e.g., meteorological data, ground-based observations). Furthermore, existing methods lack the ability to represent dynamic knowledge, making them difficult to adapt to the ecological characteristics of different regions. Therefore, a monitoring and assessment method that integrates multimodal data, accounts for spatiotemporal dynamics, and can quantify complex ecological connections is urgently needed to address the core issues of data fragmentation, model incompleteness, and insufficient nonlinear modeling. Summary of the Invention

[0004] To address the issues raised in the background art, the present invention provides a deep learning-based ecological environment restoration assessment method. This method utilizes a multimodal data fusion model, combined with knowledge graph construction, remote sensing image analysis, and time-series data modeling, to achieve comprehensive assessment and monitoring of ecological environment restoration. This method first processes collected multi-source data to construct an ecological environment knowledge graph. A graph neural network (GNN) is then used to mine potential relationships within the graph. By reconstructing entity relationships, missing ecological connections are filled, while simultaneously optimizing the implicit connection between environmental quality and biodiversity. Furthermore, the GNN is used to accurately predict future environmental quality and biodiversity trends at nodes in different regions. Secondly, a pre-trained Swin-Unet model is used to analyze remote sensing image data. Combining the global and local feature extraction capabilities of the Swin Transformer with the powerful segmentation capabilities of the U-Net, this model achieves accurate classification and semantic segmentation of different land feature categories. The segmentation results are then used to estimate the actual area of ​​each category by counting pixels. Subsequently, a multi-layer perceptron (MLP) constructs an embedded representation of the area and change estimates of remote sensing image categories. This is combined with a GNN embedding to represent the spatial information of environmental quality and biodiversity, and an MLP embedding to represent the temporal information of environmental quality and biodiversity. A convolutional network is used for multimodal data fusion. Finally, a linear transformation and a softmax classifier are used to output an ecological restoration assessment grade. This invention is achieved through the following technical solutions:

[0005] A deep learning-based ecological environment restoration assessment method includes the following steps:

[0006] S100. Acquire multi-source data within a historical time period as a time series and preprocess the data, wherein the multi-source data includes remote sensing image data, environmental quality data, and biodiversity data, wherein the environmental quality data and biodiversity data are both tabular data;

[0007] S200. Perform time series alignment, standardization, and missing value filling on environmental quality data and biodiversity data respectively;

[0008] S300. Construct an ecological environment knowledge graph based on the environmental quality data and biodiversity data at the most recent moment in the historical time period;

[0009] S400. Take the constructed knowledge graph as input, use the graph neural network model to learn node representations, and use the graph neural network to reconstruct entity relationships and perform time series prediction on the knowledge graph;

[0010] S500. Perform multi-class segmentation on the remote sensing image data using a multi-class segmentation model to obtain a semantic category response map;

[0011] S600. Obtain feature representation F based on the semantic category response graph at the latest moment in the historical time period a , the feature representation F is obtained based on the differences in the semantic category response maps at multiple time points in the time series c ; Use the final layer output of the graph neural network as the region embedding representation F b ; The characteristic representation F is obtained based on the differences between regional environmental quality data and biodiversity data at multiple time points in the time series d ;

[0012] S700.F a and F b Combined into a static multimodal embedding representation F1, F c and F d The dynamic multimodal embedding representation F2 is combined, and F1 and F2 are input into the convolution classification module to perform classification prediction of the ecological restoration level to obtain the classification results, which include four levels: excellent, good, qualified and unqualified.

[0013] Specifically, the preprocessing in S100 includes: image enhancement of remote sensing impact data, and the image enhancement operation includes one or more of image denoising, image cropping, contrast enhancement and geometric correction; the time series alignment in S200 is to align the time series of different source table data, including timestamp unification and resampling. Timestamp unification converts the time format of different data sources into a standard format, and resampling resamples the data recorded at different times at fixed time intervals.

[0014] Preferably, the remote sensing image data is image data of the ecological area to be evaluated obtained through satellite remote sensing images;

[0015] The environmental quality data includes regional information within the area to be evaluated and parameter information corresponding to each region, including but not limited to PM2.5 values, AQI values, ozone concentrations, temperature, humidity, water environment quality parameters, NDVI values, and soil quality parameters, including but not limited to water eutrophication index, water pH value, NTU value, and DO value, and soil quality parameters including but not limited to soil pH value and heavy metal concentration;

[0016] The biodiversity data includes regional information and corresponding parameter information, and the parameter information includes but is not limited to vegetation type data, animal and plant population data, and vegetation coverage.

[0017] Furthermore, the S300 includes:

[0018] S301. Use regional information in environmental quality data and biodiversity data as node entities in the knowledge graph;

[0019] S302. Parameter information of environmental quality information and biodiversity information is used as node attributes in the knowledge graph;

[0020] S303. Combine the environmental quality data and biodiversity data of the node entity to form a feature vector , used to express the ecological characteristics of region i;

[0021] S304. Calculate the ecological similarity between different feature vectors, and construct edge connections between nodes using the ecological similarity between regions as weights; the ecological similarity between different feature vectors is calculated using cosine similarity. For any two regions i and j, their ecological feature vectors are X i 、X j , calculate X i 、X j The cosine similarity value of

[0022]

[0023] Among them, x ik and x jk Represent the ecological characteristic vector X i and X j The kth eigenvalue in , where m is the total number of eigenvalues.

[0024] Furthermore, the S400 includes:

[0025] S401. In the graph neural network, let the initial feature of each node be , in the first layer of the graph neural network, the representation of each node is Aggregate and update based on the characteristics of its neighboring nodes. The specific process is as follows:

[0026]

[0027] in, For nodes The neighbor set of , AGGREGATE is the neighbor feature aggregation function, and UPDATE is the nonlinear transformation after combining the neighborhood information;

[0028] In the final layer Layer, get node High-dimensional embedding representation of , denoted as F b , which represents the global representation of the region in the ecological graph structure, including its own attributes and neighborhood ecological information.

[0029] Furthermore, the S600 includes:

[0030] S601. Flatten the semantic category response map obtained in S500 into a one-dimensional vector, denoted as F a , dimension is 1*d1;

[0031] S602. Obtain N*J area change representations by analyzing the category area changes between multiple time points in a historical time period, where N represents the number of categories and J represents the number of time periods. Expand the area change representation into one dimension and input it into a multi-layer perceptron for modeling to obtain a structural dynamic embedding representation F. c , the dimension is 1*d3, which is used to capture the evolution characteristics of spatial morphology;

[0032] S603. Based on the regional environmental quality data and biodiversity data in a historical period, after vectorization, the MLP network is used to perform dimension transformation to obtain the ecological change feature embedding representation F d .

[0033] Furthermore, the S700 includes:

[0034] The S700 includes the following steps:

[0035] S701. Concatenate the two embedding pairs into a whole representation.

[0036]

[0037]

[0038] in, Represents a splicing operation;

[0039] S702. Map the two overall representations to a unified dimension d through a fully connected layer to achieve dimensional alignment between the modalities, which can be expressed as follows:

[0040]

[0041]

[0042] Among them, W1 and W2 represent the weight matrices of linear transformation, b1 and b2 represent bias terms;

[0043] S703. and Concatenate into a joint tensor representation in the channel dimension:

[0044]

[0045] Among them, Concat represents concatenation in the channel dimension, and Conv represents the standard convolution operation;

[0046] S704. The joint feature representation z is sequentially passed through the fully connected layer for dimensionality reduction and feature compression, and finally connected to the Softmax layer for classification prediction of the ecological restoration level. The output categories include "excellent", "good", "qualified" and "unqualified".

[0047] Preferably, the loss function of the graph neural network during training includes a loss function for edge prediction and a loss function for time series prediction;

[0048] For any two nodes and , whether there is an edge label , the GNN output prediction probability is , the binary cross entropy loss function for edge prediction is:

[0049]

[0050] in, is the set of edges that are randomly removed during training;

[0051] The loss function of time series prediction includes MSE error and MAE error. For the true value of time step t, , model prediction value , the loss function of time series prediction is,

[0052]

[0053] in, Control the weight of mean square error loss and mean absolute error loss, is the length of the time series;

[0054] The final GNN loss function is,

[0055]

[0056] in 、 is the weight coefficient.

[0057] Preferably, the multi-class segmentation model is a Swin-Unet model, and the segmentation categories include but are not limited to woodland, water body, cultivated land, and built-up area;

[0058] The loss function of the Swin-UNet model includes cross entropy loss and IoU loss. The cross entropy loss is,

[0059]

[0060] Among them, N is the total number of pixels, C is the number of categories, is the one-hot vector of the true label of pixel i, is the softmax probability of the model output;

[0061] For each class c, the IoU loss is,

[0062]

[0063] in, A small constant to prevent division by zero;

[0064] The final combined loss function is,

[0065]

[0066] in, is the balance factor, and its value range is (0,1).

[0067] Preferably, the loss function of the convolution classification module adopts multi-class cross entropy loss.

[0068]

[0069] The total number of samples is , the number of categories is , for each sample , the true label is the one-hot vector y i , the model predicts .

[0070] In summary, by adopting the above technical solution, the present invention has the following advantages:

[0071] 1. During data processing, a graph neural network (GNN) is used to extract and reconstruct relationships within the ecological and environmental knowledge graph, enabling multimodal fusion of environmental quality and biodiversity data. By completing missing relationships, the associations between ecological elements are optimized, improving the integrity of ecological restoration assessment data. Secondly, the GNN's reasoning capabilities are leveraged to provide spatial information on environmental quality and biodiversity for subsequent modules.

[0072] 2. Combining the global and local feature extraction capabilities of the Swin Transformer with the powerful segmentation capabilities of the U-Net, this model is fine-tuned to adapt to different land feature categories in remote sensing imagery. After segmentation by the U-Net module, the area information of each category is calculated, which provides a certain degree of feedback on the quality of the ecological environment.

[0073] 3. To further utilize the temporal and spatial information of multi-source data, the present invention integrates the multimodal spatiotemporal information of remote sensing images, environmental quality data, and biodiversity data to ensure that the ecological monitoring system can adapt to the contribution of different data modalities, accurately identify key environmental changes, and improve the adaptability and generalization ability of the ecological restoration assessment system. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention or in the description of the prior art. It should be understood that the following drawings are only some embodiments of the present invention and should not be considered as limiting the scope of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0075] Figure 1 It is the overall principle diagram of the present invention; Figure 1 In the knowledge graph, the corresponding data below the lake are also environmental quality and biodiversity. Because they are blocked by arrows, they are replaced by ellipsis.

[0076] Figure 2 is an example of a semantic category response graph in a specific embodiment of the present invention;

[0077] Figure 3 This is an example of a changed semantic category response graph in a specific implementation of the present invention. DETAILED DESCRIPTION

[0078] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0079] like Figure 1 As shown, this embodiment provides an ecological environment restoration assessment method based on deep learning, comprising the following steps:

[0080] S100. Acquire multi-source data from a historical time period as a time series and preprocess it. The multi-source data includes remote sensing image data, environmental quality data, and biodiversity data. The environmental quality data and biodiversity data are both tabular data. It should be understood that the historical time period here refers to a period prior to the current prediction time, rather than a distant historical period, such as three, five, or ten years prior to the current time. The specific time interval is determined based on actual needs. Preprocessing specifically involves performing image enhancement on the remote sensing impact data. Image enhancement operations include one or more of image denoising, image cropping, contrast enhancement, and geometric correction. Image denoising includes mean filtering, median filtering, and BM3D (Block Matching and 3D Filtering). Mean filtering and median filtering can smooth images and remove salt and pepper noise. BM3D is suitable for denoising high-resolution remote sensing images. Image cropping can remove irrelevant areas and highlight areas of interest. Since remote sensing images are typically large, sliding window cropping can be used to extract partial image data. Contrast enhancement uses histogram equalization to improve overall contrast. Geometric correction includes affine transformation, which performs linear geometric correction through operations such as rotation, scaling, and translation.

[0081] S200. Perform time series alignment, standardization, and missing value filling on environmental quality data and biodiversity data, respectively. Time series alignment involves aligning the time series of data from different source tables, including timestamp standardization and resampling. Timestamp standardization involves converting the time formats of different data sources to a standard format. Resampling involves resampling data recorded at different times at fixed time intervals (e.g., hours, days, or months). Standardization synchronizes data from different sources. Z-score standardization is used to eliminate unit and scale differences. The standardization formula is as follows:

[0082]

[0083] in, is the standardized value, is the characteristic mean, is the feature variance. Missing value filling is done by selecting the closest time point to align the data based on nearest neighbor sampling and filling in the missing values ​​of the sample.

[0084] The remote sensing image data is image data of the ecological area to be evaluated obtained through satellite remote sensing images (such as MODIS, Landsat); the satellite remote sensing images include vegetation coverage and biological habitat conditions.

[0085] The environmental quality data includes regional information and parameter information corresponding to each region. The parameter information includes but is not limited to PM2.5 values, AQI values, ozone concentrations, temperature, humidity, water environment quality parameters, NDVI values, and soil quality parameters. Water environment quality parameters include but are not limited to water eutrophication index, water pH value, NTU value, and DO value. Soil quality parameters include but are not limited to soil pH value and heavy metal concentrations (such as cadmium and lead, which must be compared with the limits in the Soil Environmental Quality Standards). These environmental quality data are used for calibration of environmental quality characteristics. This information usually comes from official monitoring agencies such as ground air quality monitoring stations or scientific research and third-party testing agencies, and is collected regularly.

[0086] Biodiversity data includes regional information and its corresponding parameter information, including but not limited to vegetation species data, plant and animal population data, and vegetation coverage. Biodiversity data is used to reflect the species composition and community structure of regional ecosystems and is used for biodiversity verification. Specifically, vegetation species data and plant and animal population data include not only the number of each type of plant and animal, but also their specific classification information, such as species name, species, genus, and family, to comprehensively reflect the diversity of the ecosystem. Generally, different families are classified as different species, but the classification level can be customized based on actual needs, such as species and genus. Plant and animal classification can be derived from regular sampling at fixed biological monitoring sites, such as quarterly or semi-annual monitoring of ground vegetation species and coverage, wildlife population numbers, and activity frequencies. Existing historical data can also be used, such as ecological census reports and nature reserve archives. By structuring this data and converting it into a vector form suitable for model calculation, it can support the subsequent construction of graph neural network embedding representations and feature fusion. Combined with time series data, it can also be used to model and analyze biodiversity trends, providing support for accurate assessment of ecological restoration status.

[0087] S300. Construct an ecological environment knowledge graph based on the most recent environmental quality data and biodiversity data within a historical period; including:

[0088] S301. Use regional information in environmental quality data and biodiversity data as node entities in the knowledge graph;

[0089] S302. Parameter information of environmental quality information and biodiversity information is used as node attributes in the knowledge graph;

[0090] S303. Combine the environmental quality data and biodiversity data of the node entity to form a feature vector , used to express the ecological characteristics of region i;

[0091] S304. Calculate the ecological similarity between different feature vectors, and construct edge connections between nodes using the ecological similarity between regions as a weight. Specifically, if the similarity exceeds a set threshold, add an edge to the graph to indicate the degree of similarity between the ecological environments of the regions. The ecological similarity is selected from one of cosine similarity, Euclidean distance, or KL divergence. In the present invention, a city, river basin, or protected area can be selected as a region, i.e., a node entity, and the environmental quality data and biodiversity data of the city, river basin, or protected area can be used as node entities, i.e., the corresponding region as node attributes.

[0092] The ecological similarity between different feature vectors is calculated by cosine similarity. For any two regions i and j, their ecological feature vectors are X i 、X j , calculate X i 、X j The cosine similarity value of

[0093]

[0094] Among them, x ik and x jk Represent the ecological characteristic vector X i and X j The kth eigenvalue in , where m is the total number of eigenvalues.

[0095] The constructed knowledge graph can be stored in graph databases (Neo4j, RDF), supporting efficient path queries, pollution tracing, or ecological health assessment applications. The knowledge graph constructed by the present invention enables structured expression and association modeling of regional ecological information. In other specific implementations, spatial proximity edges can also be constructed directly based on the geographic distance between regions. The two aforementioned edge types are different types of edges, representing connections between spatial and ecological dimensions, respectively. Both can serve as structural inputs in graph neural networks for subsequent relationship modeling and ecological trend prediction.

[0096] In order to achieve modeling and prediction of ecological relationships between regions, the present invention introduces a graph neural network (GNN) to learn and reason about the above knowledge graph. GNN can effectively integrate the node's own characteristics and neighbor node information to capture complex spatial dependencies.

[0097] S400. Take the constructed knowledge graph as input, adopt the graph neural network model to learn node representation, and reconstruct entity relationships and perform time series prediction on the knowledge graph through the graph neural network; the graph neural network of the present invention adopts a two-layer GNN structure.

[0098] In a knowledge graph, nodes represent different regions, and edges indicate the proximity and ecological similarities between regions. However, due to incomplete data collection, there may be missing edges in the graph. To restore these missing edges, entity relationship reconstruction uses graph neural networks to learn node embeddings and use these embeddings to predict the existence of edges between pairs of nodes, thereby completing the missing ecological connections. Specifically, during the training phase, some real edges are randomly deleted as prediction targets. The model then estimates the probability of an edge between any two nodes by learning node representations. This probability reflects the potential connection between two regions in terms of spatial proximity and ecological similarity.

[0099] S401. In the graph neural network, let the initial feature of each node be , in the first layer of the graph neural network, the representation of each node is Aggregate and update based on the characteristics of its neighboring nodes. The specific process is as follows:

[0100]

[0101] in, For nodes , AGGREGATE is the neighbor feature aggregation function (which can be a weighted average, attention mechanism, or pooling operation), and UPDATE is a nonlinear transformation (such as a multilayer perceptron) that combines neighborhood information. This propagation process can be iterated multiple times to fully capture the ecological impact of multi-order neighbor relationships.

[0102] In the final layer Layer, get node High-dimensional embedding representation of , denoted as F b , which represents the global representation of the region in the ecological graph structure, including its own attributes and neighborhood ecological information.

[0103] The loss function of the graph neural network during training includes the edge prediction loss function and the time series prediction loss function. Specifically, the goal of the entity relationship reconstruction task is to reconstruct some of the edges that were randomly deleted during training. These edges represent the proximity and ecological similarity between regions. Specific loss function design: edge prediction loss (through graph embedding learning, to complete the missing edges (i.e., regional proximity and ecological similarity). This part of the loss function can use binary cross entropy loss to determine whether there is a certain ecological relationship or proximity between two regions. For any two nodes and , whether there is an edge label , the GNN output prediction probability is , then the binary cross entropy loss of edge prediction is:

[0104]

[0105] in is the set of edges that are randomly removed during training;

[0106] Ecosystems have complex dynamic evolution laws, involving multiple regions and multiple ecological indicators, and are suitable for multi-region, multi-indicator ecological time series analysis tasks. The goal of time series prediction is to accurately predict the environmental quality and biodiversity indicators of nodes in different regions in several future time steps, and help reveal the spatiotemporal evolution trends of ecosystems. Since ecological data have obvious time dependence and volatility, these characteristics must be fully considered when designing the model to ensure that both the prediction accuracy can be improved and the evolution trend of indicators can be stably captured. The loss functions of time series prediction include MSE error and MAE error. MSE is sensitive to large errors and can improve the model's ability to fit specific values, but is more sensitive to outliers; MAE is not sensitive to outliers and emphasizes the stability of trends. Combining the advantages of these two, the time series relationship of graph neural networks is enhanced. For the true value of time step t , model prediction value , the loss function of time series prediction is specifically:

[0107]

[0108] in Control the weight of mean square error loss and mean absolute error loss, is the length of the time series;

[0109] The final GNN loss function is

[0110]

[0111] in 、 is the weight coefficient. This combined loss function not only improves prediction accuracy but also enhances the ability to model the dynamic evolution of ecosystems. It is suitable for multi-region, multi-indicator ecological time series analysis tasks.

[0112] Ecosystems have complex dynamic evolution laws, involving multiple regions and multiple ecological indicators, and are suitable for multi-region, multi-indicator ecological time series analysis tasks. The goal of time series prediction is to accurately predict the environmental quality and biodiversity indicators of nodes in different regions in several future time steps, and help reveal the spatiotemporal evolution trends of ecosystems. Since ecological data have obvious time dependence and volatility, these characteristics must be fully considered when designing the model to ensure that both the prediction accuracy can be improved and the evolution trend of indicators can be stably captured. The loss functions of time series prediction include MSE error and MAE error. MSE is sensitive to large errors and can improve the model's ability to fit specific values, but is more sensitive to outliers; MAE is not sensitive to outliers and emphasizes the stability of trends. Combining the advantages of these two, the time series relationship of graph neural networks is enhanced. For the true value of time step t , model prediction value , the loss function of time series prediction is specifically:

[0113]

[0114] in Control the weight of mean square error loss and mean absolute error loss, is the length of the time series;

[0115] The total loss of the final graph neural network is + ;

[0116] S500. Multi-class segmentation of the remote sensing image is performed using a multi-class segmentation model, a Swin-Unet model, to generate a semantic category response map. The multi-class segmentation model is a Swin-Unet model, which segments categories including, but not limited to, forest land, water bodies, cultivated land, and built-up areas. The area of ​​each segmented category module is calculated based on actual reference objects, and the area change of each category is estimated based on historical data. The Swin-Unet model combines the global and local feature extraction capabilities of the Swin Transformer with the powerful segmentation capabilities of the U-Net. The Swin-Unet model can utilize a pre-trained model and fine-tune it to adapt to different feature categories in remote sensing imagery (e.g., forest, lake, cultivated land, housing, etc.). This process is performed using existing remote sensing data, leveraging transfer learning to enhance the model's performance in specific domains. Fine-tuning the trained Swin-Unet model achieves multi-class semantic segmentation, performing pixel-level classification of remote sensing images and accurately identifying different feature categories (e.g., forest, lake, cultivated land, housing, etc.). Each category is independently extracted using the segmentation network, ensuring that the model assigns the correct label to each pixel. For the segmentation results, the actual area of ​​each category is estimated by counting pixels. Given the ground area represented by each pixel (e.g., 1 pixel corresponds to a certain ground unit area), the total area of ​​each category is calculated.

[0117] To achieve effective fusion of remote sensing image data with multimodal data such as environmental quality data and biodiversity data, and to perform classification prediction of ecological status based on multimodal data.

[0118] The Swin-UNet loss function in this invention includes cross-entropy loss (Cross-Entropy Loss) and IoU loss (Intersection over Union Loss); this combination can not only optimize the accuracy of image classification, but also improve the accuracy of segmentation, especially when the target area and background area are unbalanced. Among them, the cross-entropy loss is,

[0119]

[0120] Among them, N is the total number of pixels, C is the number of categories, is the one-hot vector of the true label of pixel i, is the softmax probability of the model output;

[0121] For each class c, the IoU loss is,

[0122]

[0123] in A small constant to prevent division by zero;

[0124] The final combined loss function is,

[0125]

[0126] in is the balance factor, and its value range is (0,1).

[0127] S600. Obtain feature representation F based on the semantic category response graph at the latest moment in the historical time period a , the feature representation F is obtained based on the differences in the semantic category response maps at multiple time points in the time series c ; Use the final layer output of the graph neural network as the region embedding representation F b ; The characteristic representation F is obtained based on the differences between regional environmental quality data and biodiversity data at multiple time points in the time series d ;

[0128] S601. Flatten the semantic category response map obtained in S500 into a one-dimensional vector, denoted as F a , dimension is 1*d1; used to characterize regional landscape pattern and land use structure. aThe data expression comes from the segmentation result of the remote sensing image at time point T, and its dimension is N*M, where N represents the number of categories and M is the feature dimension. In order to more significantly express the area information of the remote sensing image, the changing trend of the area of ​​each category in the time series is calculated based on the remote sensing segmentation result. Assume that the semantic category response map obtained after the remote sensing data passes through Swin-Unet is at time point T, and the area of ​​each category of forest, lake, cultivated land and housing is (a1, b1, c1, d1), as shown in the following example: Figure 2 As shown, the area of ​​the four categories is (4,4,4,4). Of course, the actual number of categories and area size are much larger than Figure 2 complex.

[0129] S602. Obtain N*J area change representations by analyzing the category area changes between multiple time points in a historical time period, where N represents the number of categories and J represents the number of time periods. Expand the area change representation into one dimension and input it into a multi-layer perceptron for modeling to obtain a structural dynamic embedding representation F. c , with a dimension of 1*d3, is used to capture the evolution characteristics of spatial morphology. Specifically, at time point T+1, the area of ​​each category is (a2, b2, c2, d2), as shown Figure 3 As shown in the figure, the areas of the four categories of forests, lakes, cultivated land and housing are (2, 6, 4, 4), and the area changes between time point T+1 and time point T are (a2-a1, b2-b1, c2-c1, d2-d1). Similarly, the area changes of the categories between any two time points at different time points T+2, T+3, …, T+J can be obtained.

[0130] S603. Based on the regional environmental quality data and biodiversity data in a historical period, after vectorization, the MLP network is used to perform dimension changes to obtain the ecological change feature embedding F d , F d Used to reflect the evolutionary trends of ecological attributes;

[0131] In this specific implementation, regional environmental quality data and biodiversity data constitute the original input in the form of time series. For example, a time series record table is constructed for the monitoring data of the past five years: each row corresponds to a time point, and each column is PM2.5 value, AQI value, ozone concentration, air temperature, air humidity, NDVI value, water eutrophication index, water pH value, NTU value, DO value, soil pH value, cadmium concentration value, and lead concentration value. The missing time period data in the sample is processed according to the linear interpolation method. The time series table can be formally represented as a matrix , where J represents the number of time periods and K represents the number of monitoring indicators. For unified processing and modeling, the matrix is ​​first flattened into a one-dimensional vector and then input into the multi-layer perceptron network to obtain the ecological change embedding feature F d, with a dimension of 1*d4. This embedding is used to characterize the historical evolution trend of the regional ecological status.

[0132] S700.F a and F b Combined into a static multimodal embedding representation F1, F c and F d The dynamic multimodal embedding representation F2 is combined and fed into the convolutional classification module to predict the ecological restoration grade. The resulting classification results include four grades: excellent, good, qualified, and unqualified. Specifically, the grades can be divided into three, two, or five levels based on actual needs.

[0133] The S700 includes the following steps:

[0134] S701. Concatenate the two embedding pairs into a whole representation.

[0135]

[0136]

[0137] in, Represents a splicing operation;

[0138] S702. Map the two overall representations to a unified dimension d through a fully connected layer to achieve dimensional alignment between the modalities, which can be expressed as follows:

[0139]

[0140]

[0141] Among them, W1 and W2 represent the weight matrices of linear transformation, b1 and b2 represent bias terms;

[0142] S703. In order to further integrate the static and dynamic modal features, and Concatenate into a joint tensor representation in the channel dimension:

[0143]

[0144] Among them, Concat represents splicing in the channel dimension, and Conv represents the standard convolution operation; this convolution operation can effectively extract coupling information between channels and space, and realize feature fusion between multi-modality and multi-time series.

[0145] S704. The joint feature representation Z is sequentially passed through the fully connected layer for dimensionality reduction and feature compression, and finally connected to the Softmax layer for classification prediction of ecological restoration level. The output categories include "excellent", "good", "qualified" and "unqualified". The loss function of the convolution classification module adopts multiclass cross-entropy loss.

[0146]

[0147] The total number of samples is , the number of categories is , for each sample , the true label is the one-hot vector y i , the model predicts .

[0148] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for evaluating ecological environment restoration based on deep learning, characterized in that: The following steps are involved: S100. Acquire multi-source data within a historical time period as a time series and preprocess the data, wherein the multi-source data includes remote sensing image data, environmental quality data, and biodiversity data, wherein the environmental quality data and biodiversity data are both tabular data; S200. Perform time series alignment, standardization, and missing value filling on environmental quality data and biodiversity data respectively; S300. Construct an ecological environment knowledge graph based on the environmental quality data and biodiversity data at the most recent moment in the historical time period; S400. Take the constructed knowledge graph as input, use the graph neural network model to learn node representations, and use the graph neural network to reconstruct entity relationships and perform time series prediction on the knowledge graph; S500. Perform multi-class segmentation on the remote sensing image data using a multi-class segmentation model to obtain a semantic category response map; S600. Obtain feature representation F based on the semantic category response graph at the latest moment in the historical time period a , the feature representation F is obtained based on the differences in the semantic category response maps at multiple time points in the time series c ; Use the final layer output of the graph neural network as the region embedding representation F b ; The characteristic representation F is obtained based on the differences between regional environmental quality data and biodiversity data at multiple time points in the time series d ; S700.F a and F b Combined into a static multimodal embedding representation F1, F c and F d The dynamic multimodal embedding representation F2 is combined, and F1 and F2 are input into the convolution classification module to perform classification prediction of the ecological restoration level to obtain the classification results, which include four levels: excellent, good, qualified and unqualified.

2. The ecological environment restoration assessment method based on deep learning according to claim 1 is characterized in that: The preprocessing in S100 includes: image enhancement of remote sensing impact data, and the image enhancement operation includes one or more of image denoising, image cropping, contrast enhancement and geometric correction; the time series alignment in S200 is to align the time series of different source table data, including timestamp unification and resampling. Timestamp unification converts the time format of different data sources into a standard format, and resampling resamples the data recorded at different times at fixed time intervals.

3. The method for evaluating ecological environment restoration based on deep learning according to claim 1, characterized in that: The remote sensing image data is image data of the ecological area to be evaluated obtained through satellite remote sensing images; The environmental quality data includes regional information within the area to be evaluated and parameter information corresponding to each region, including but not limited to PM2.5 values, AQI values, ozone concentrations, temperature, humidity, water environment quality parameters, NDVI values, and soil quality parameters, including but not limited to water eutrophication index, water pH value, NTU value, and DO value, and soil quality parameters including but not limited to soil pH value and heavy metal concentration; The biodiversity data includes regional information and corresponding parameter information, and the parameter information includes but is not limited to vegetation type data, animal and plant population data, and vegetation coverage.

4. The method for evaluating ecological environment restoration based on deep learning according to claim 1, characterized in that: The S300 includes: S301. Use regional information in environmental quality data and biodiversity data as node entities in the knowledge graph; S302. Parameter information of environmental quality information and biodiversity information is used as node attributes in the knowledge graph; S303. Combine the environmental quality data and biodiversity data of the node entity to form a feature vector , used to express the ecological characteristics of region i; S304. Calculate the ecological similarity between different feature vectors, and construct edge connections between nodes using the ecological similarity between regions as weights; the ecological similarity between different feature vectors is calculated using cosine similarity. For any two regions i and j, their ecological feature vectors are X i 、X j , calculate X i 、X j The cosine similarity value of ； Among them, x ik and x jk Represent the ecological characteristic vector X i and X j The kth eigenvalue in , where m is the total number of eigenvalues.

5. The ecological environment restoration assessment method based on deep learning according to claim 1 is characterized in that: The S400 includes: S401. In the graph neural network, let the initial feature of each node be , in the first layer of the graph neural network, the representation of each node is Aggregate and update based on the characteristics of its neighboring nodes. The specific process is as follows: ； in, For nodes The neighbor set of , AGGREGATE is the neighbor feature aggregation function, and UPDATE is the nonlinear transformation after combining the neighborhood information; In the final layer Layer, get node High-dimensional embedding representation of , denoted as F b , which represents the global representation of the region in the ecological graph structure, including its own attributes and neighborhood ecological information.

6. The method for evaluating ecological environment restoration based on deep learning according to claim 1, characterized in that: The S600 includes: S601. Flatten the semantic category response map obtained in S500 into a one-dimensional vector, denoted as F a , dimension is 1*d1; S602. Obtain N*J area change representations by analyzing the category area changes between multiple time points in a historical time period, where N represents the number of categories and J represents the number of time periods. Expand the area change representation into one dimension and input it into a multi-layer perceptron for modeling to obtain a structural dynamic embedding representation F. c , the dimension is 1*d3, which is used to capture the evolution characteristics of spatial morphology; S603. Based on the regional environmental quality data and biodiversity data in a historical period, after vectorization, the MLP network is used to perform dimension transformation to obtain the ecological change feature embedding representation F d .

7. The method for evaluating ecological environment restoration based on deep learning according to claim 1, characterized in that: The S700 includes: The S700 includes the following steps: S701. Concatenate the two embedding pairs into a whole representation. ； ； in, Represents a splicing operation; S702. Map the two overall representations to a unified dimension d through a fully connected layer to achieve dimensional alignment between the modalities, which can be expressed as follows: ； ； Among them, W1, W2 represent the weight matrix of linear transformation, b1, b2 represent the bias term; S703. and Concatenate into a joint tensor representation in the channel dimension: ； Among them, Concat represents concatenation in the channel dimension, and Conv represents the standard convolution operation; S704. The joint feature representation z is sequentially passed through the fully connected layer for dimensionality reduction and feature compression, and finally connected to the Softmax layer for classification prediction of the ecological restoration level. The output categories include "excellent", "good", "qualified", and "unqualified".

8. The method for evaluating ecological environment restoration based on deep learning according to claim 1, characterized in that: The loss function of the graph neural network during training includes the loss function of edge prediction and the loss function of time series prediction; For any two nodes and , whether there is an edge label , the GNN output prediction probability is , the binary cross entropy loss function for edge prediction is: ； in, is the set of edges that are randomly removed during training; The loss function of time series prediction includes MSE error and MAE error. For the true value of time step t, , model prediction value , the loss function of time series prediction is, ； in, Control the weight of mean square error loss and mean absolute error loss, is the length of the time series; The final GNN loss function is, ； in 、 is the weight coefficient.

9. The method for evaluating ecological environment restoration based on deep learning according to claim 1, characterized in that: The multi-class segmentation model is a Swin-Unet model, and the segmentation categories include but are not limited to woodland, water, cultivated land, and built-up areas; The loss function of the Swin-UNet model includes cross entropy loss and IoU loss. The cross entropy loss is, ； Among them, N is the total number of pixels, C is the number of categories, is the one-hot vector of the true label of pixel i, is the softmax probability of the model output; For each class c, the IoU loss is, ； in, A small constant to prevent division by zero; The final combined loss function is, ； in, is the balance factor, and its value range is (0,1).

10. The method for evaluating ecological environment restoration based on deep learning according to claim 1, characterized in that: The loss function of the convolution classification module adopts multi-class cross entropy loss. ； The total number of samples is , the number of categories is , for each sample , the true label is the one-hot vector y i , the model predicts .

Images