Natural resource dynamic monitoring method and system based on multi-source data fusion

Abstract

The invention provides a natural resource dynamic monitoring method and system based on multi-source data fusion, and the method comprises the steps: obtaining multi-source monitoring data of natural resources of a to-be-monitored region, extracting multi-source spatial-temporal features from the multi-source monitoring data, and carrying out the splicing of the multi-source spatial-temporal features to obtain preliminary fusion features; constructing a feature graph structure based on the preliminary fusion features, performing aggregation transformation on nodes in the feature graph structure by using a graph convolutional network to obtain high-order features of the preliminary fusion features, and performing principal component analysis on the high-order features to obtain target fusion features; inputting the target fusion feature into a dynamic monitoring network for analysis to obtain a resource state analysis result and a change trend prediction result; a graph structure can capture a complex dependency relationship between feature dimensions, and a high-order relationship between the feature dimensions is extracted through a graph convolutional network, so that the discrimination of features is enhanced, and the accuracy of state classification and prediction tasks is improved; dimensionality reduction is carried out on high-order features through principal component analysis, feature redundancy is reduced, and the calculation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural resource monitoring, and specifically relates to a method and system for dynamic monitoring of natural resources based on multi-source data fusion. Background Art

[0002] With the growth of the population and the development of the economy, the demand for natural resources has been increasing continuously, leading to the over-exploitation of resources and the deterioration of the environment. By monitoring natural resources, the health status of the ecosystem can be understood in a timely manner, effective protection measures can be formulated, and sustainable development can be promoted.

[0003] Under the current technical background, natural resource monitoring has developed from traditional single data sources (such as ground surveys) to a new stage of multi-source data fusion. The natural resources in the area to be monitored are analyzed based on multi-source data. With the development of satellite remote sensing and Internet of Things technologies, it has become more feasible to obtain high-quality natural resource monitoring data.

[0004] Although satellite remote sensing and unmanned aerial vehicles can quickly cover large areas, due to the huge amount of data collected, there may be a large amount of redundant information, resulting in a large amount of calculation and difficulty in meeting the real-time nature of natural resource monitoring; moreover, multi-source data has different data structures and characteristics, and there may be problems with the integrity and accuracy after simple splicing and fusion, making it difficult to ensure the accuracy of monitoring, and thus it is impossible to detect and process sudden disaster areas in a timely manner. Summary of the Invention

[0005] To overcome the deficiencies of the above-mentioned prior art, the present invention proposes a method for dynamic monitoring of natural resources based on multi-source data fusion, including:

[0006] Obtain multi-source monitoring data of natural resources in the area to be monitored, and extract multi-source spatio-temporal features from the multi-source monitoring data and splice them to obtain a preliminary fusion feature;

[0007] Based on the linear correlation between each feature dimension in the preliminary fusion feature, construct a feature map structure; use a graph convolutional network to perform an aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion feature, and perform principal component analysis on the high-order features to obtain the target fusion feature of the multi-source monitoring data;

[0008] Input the target fusion feature into a pre-constructed dynamic monitoring network for resource state analysis and change trend prediction, and obtain the resource state analysis result and change trend prediction result of the natural resources in the area to be detected.

[0009] Optionally, the constructing a feature map structure based on the linear correlation between each feature dimension in the preliminary fusion feature includes:

[0010] By calculating the Pearson correlation coefficient between every two of the feature dimensions, the linear correlation between every two of the feature dimensions is obtained;

[0011] Based on the linear correlation between every two of the feature dimensions and a correlation threshold, an adjacency matrix of the preliminary fusion features is constructed;

[0012] Using each feature dimension in the preliminary fusion features as nodes and the connection relationship between every two of the feature dimensions in the adjacency matrix as edges, a feature graph structure of the preliminary fusion features is constructed.

[0013] Optionally, the obtaining of the high-order features of the preliminary fusion features by aggregating and transforming the nodes in the feature graph structure using a graph convolutional network includes:

[0014] Adding an identity matrix to the adjacency matrix of the preliminary fusion features to obtain an adjacency matrix incorporating node self-connections, and normalizing the adjacency matrix incorporating node self-connections to obtain a normalized adjacency matrix;

[0015] In the first convolutional layer of the graph convolutional network, multiplying the normalized adjacency matrix by the element matrix of the preliminary fusion features to obtain the aggregated features after aggregation of all nodes in the feature graph structure and their adjacent nodes; transforming the aggregated features through a learnable weight matrix and a non-linear activation function, and outputting new preliminary fusion features;

[0016] Taking the new preliminary fusion features as the input of the next convolutional layer, and repeating the operations of the first convolutional layer using the next convolutional layer until after passing through multiple convolutional layers, outputting the high-order features of the initial fusion features.

[0017] Optionally, the multi-source monitoring data includes: remote sensing image data, unmanned aerial vehicle image data, and ground sensor data of the area to be detected;

[0018] The extracting of multi-source spatio-temporal features from the multi-source monitoring data and stitching them to obtain preliminary fusion features includes:

[0019] Using a deep convolutional neural network to extract the spatial features of the remote sensing image data and the unmanned aerial vehicle image data;

[0020] Using a long short-term memory network to extract the time series features of the ground sensor data;

[0021] Stitching the spatial features of the remote sensing image data and the unmanned aerial vehicle image data and the time series features of the ground sensor data to obtain preliminary fusion features;

[0022] The multi-source spatio-temporal features include the spatial features and the time series features.

[0023] Optionally, the process of constructing the dynamic monitoring network includes:

[0024] Based on the fusion features in the historical multi-source monitoring data, model training is performed on the deep learning models corresponding to different resource status analysis requirements to obtain multiple resource status analysis models;

[0025] Based on the fusion features at different time points in the historical multi-source monitoring data, model training is performed on the long short-term memory network to obtain a change trend prediction model;

[0026] Combining multiple said resource status analysis models and said change trend prediction model to construct a dynamic monitoring network.

[0027] Optionally, the model training of the deep learning models corresponding to different resource status analysis requirements based on the fusion features in the historical multi-source monitoring data to obtain multiple resource status analysis models includes:

[0028] Dividing the dataset of the fusion features extracted from the historical multi-source monitoring data into a training set and a validation set;

[0029] Selecting multiple deep learning models according to different resource status analysis requirements;

[0030] Based on each said deep learning model, tuning the hyperparameters of the deep learning model by combining the grid search method according to the validation set to obtain the initial resource status analysis models corresponding to different resource status analysis requirements;

[0031] Inputting the training set into the initial resource status analysis model for multiple iterative trainings. The initial analysis model adjusts its internal parameters during the iterative training process to minimize the loss function until the iterative termination condition is reached, obtaining the resource status analysis models corresponding to different resource status analysis requirements.

[0032] Optionally, the model training of the long short-term memory network based on the fusion features at different time points in the historical multi-source monitoring data to obtain a change trend prediction model includes:

[0033] Arranging the fusion features at different time points in the historical multi-source monitoring data in chronological order and labeling the corresponding change category labels for the fusion features at different time points to construct a time series dataset;

[0034] Using the fusion features in the time series dataset as the input, the change categories corresponding to the fusion features at different time points as the output, and the cross-entropy loss as the loss function for the classification task to train the long short-term memory network to obtain a change prediction model.

[0035] Optionally, before extracting multi-source spatio-temporal features from the multi-source monitoring data and splicing them to obtain a preliminary fusion feature, it further includes:

[0036] Using an edge computing device to compress and screen the multi-source monitoring data to obtain processed multi-source monitoring data.

[0037] Based on the same inventive concept, the present invention proposes a natural resource dynamic monitoring system based on multi-source data fusion, including:

[0038] A feature extraction module, configured to obtain multi-source monitoring data of natural resources in a to-be-monitored area, and extract multi-source spatio-temporal features from the multi-source monitoring data and splice them to obtain a preliminary fusion feature;

[0039] A feature fusion module, configured to construct a feature map structure based on the linear correlation between each feature dimension in the preliminary fusion feature; use a graph convolutional network to perform aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion feature, and perform principal component analysis on the high-order features to obtain the target fusion feature of the multi-source monitoring data;

[0040] A data analysis module, configured to input the target fusion feature into a pre-constructed dynamic monitoring network for resource state analysis and change trend prediction, and obtain a resource state analysis result and a change trend prediction result of the natural resources in the to-be-detected area.

[0041] Optionally, the feature fusion module is specifically configured to:

[0042] Calculate the Pearson correlation coefficient between every two feature dimensions to obtain the linear correlation between every two feature dimensions;

[0043] Construct an adjacency matrix of the preliminary fusion feature based on the linear correlation between every two feature dimensions and a correlation threshold;

[0044] Construct a feature map structure of the preliminary fusion feature with each feature dimension in the preliminary fusion feature as a node and the connection relationship between every two feature dimensions in the adjacency matrix as an edge.

[0045] Optionally, the feature fusion module is specifically configured to:

[0046] Add an identity matrix to the adjacency matrix of the preliminary fusion feature to obtain an adjacency matrix incorporating node self-connections, and normalize the adjacency matrix incorporating node self-connections to obtain a normalized adjacency matrix;

[0047] In the first convolutional layer of the graph convolutional network, multiply the normalized adjacency matrix by the element matrix of the preliminary fusion features to obtain the aggregated features after aggregation of all nodes in the feature map structure and their adjacent nodes; transform the aggregated features through a learnable weight matrix and a non-linear activation function, and output new preliminary fusion features;

[0048] Use the new preliminary fusion features as the input of the next convolutional layer, and repeat the operations of the first convolutional layer using the next convolutional layer until after passing through multiple convolutional layers, output the high-order features of the initial fusion features.

[0049] Optionally, the feature extraction module is specifically used for:

[0050] The extraction of multi-source spatio-temporal features from the multi-source monitoring data and stitching them to obtain preliminary fusion features includes:

[0051] Use a deep convolutional neural network to extract the spatial features of the remote sensing image data and the UAV image data;

[0052] Use a long short-term memory network to extract the time series features of the ground sensor data;

[0053] Stitch the spatial features of the remote sensing image data and the UAV image data and the time series features of the ground sensor data to obtain preliminary fusion features;

[0054] The multi-source spatio-temporal features include the spatial features and the time series features.

[0055] Optionally, the data analysis module is specifically used for:

[0056] Based on the fusion features in the historical multi-source monitoring data, train deep learning models corresponding to different resource state analysis requirements to obtain multiple resource state analysis models;

[0057] Based on the fusion features at different time points in the historical multi-source monitoring data, train a long short-term memory network to obtain a change trend prediction model;

[0058] Combine multiple resource state analysis models and the change trend prediction model to construct a dynamic monitoring network.

[0059] Optionally, the data analysis module is specifically used for:

[0060] Divide the dataset of the fusion features extracted from the historical multi-source monitoring data into a training set and a validation set;

[0061] Select multiple deep learning models according to different resource state analysis requirements;

[0062] Based on each of the deep learning models, according to the validation set, combined with the grid search method, tune the hyperparameters of the deep learning model to obtain an initial resource status analysis model corresponding to different resource status analysis requirements;

[0063] Input the training set into the initial resource status analysis model for multiple iterative trainings. During the iterative training process, the initial analysis model adjusts its internal parameters to minimize the loss function until the iterative termination condition is reached, obtaining a resource status analysis model corresponding to different resource status analysis requirements.

[0064] Optionally, the data analysis module is specifically configured to:

[0065] Arrange the fusion features at different time points in the historical multi-source monitoring data in chronological order, and label the corresponding change category labels for the fusion features at different time points to construct a time series data set;

[0066] Use the fusion features in the time series data set as input, the change categories corresponding to the fusion features at different time points as output, and the cross-entropy loss as the loss function for the classification task to train the long short-term memory network to obtain a change prediction model.

[0067] Optionally, the feature extraction module is further configured to:

[0068] Use an edge computing device to compress and screen the multi-source monitoring data to obtain processed multi-source monitoring data.

[0069] On the other hand, the present application also provides an electronic device, including: at least one processor and a memory; the memory and the processor are connected by a bus;

[0070] The memory is used to store one or more programs;

[0071] When the one or more programs are executed by the at least one processor, implement a method for dynamically monitoring natural resources based on multi-source data fusion as described above.

[0072] On the other hand, the present application also provides a computer-readable storage medium, on which an execution program is stored. When the execution program is executed, implement a method for dynamically monitoring natural resources based on multi-source data fusion as described above.

[0073] Compared with the closest prior art, the beneficial effects of the present invention are as follows:

[0074] A natural resource dynamic monitoring method and system based on multi-source data fusion provided by the present invention include: obtaining multi-source monitoring data of natural resources in the area to be monitored, and extracting multi-source spatio-temporal features from the multi-source monitoring data and splicing them to obtain preliminary fusion features; constructing a feature map structure based on the linear correlation between each feature dimension in the preliminary fusion features; using a graph convolutional network to perform aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion features, and performing principal component analysis on the high-order features to obtain target fusion features of the multi-source monitoring data; inputting the target fusion features into a pre-constructed dynamic monitoring network for resource state analysis and change trend prediction to obtain a resource state analysis result and a change trend prediction result of the natural resources in the area to be detected; in the present invention, through feature splicing, the features of multi-source data can be fused into a unified feature space, retaining the diversity information of multi-source data. However, simple feature splicing can only retain the shallow information of features and cannot capture the high-order relationships between features. The graph structure can explicitly capture the complex dependencies between global feature dimensions and provide input for the subsequent graph convolutional network. Through the graph convolutional network, the high-order (between multi-level adjacent nodes) relationships between feature dimensions can be extracted, enhancing the discriminability of features, thereby improving the accuracy of natural area state classification and prediction tasks.

[0075] High-order features require more storage space and computing resources. Especially when dealing with large-scale data, the computing cost will increase significantly; using principal component analysis to map high-order features to a low-dimensional space through linear transformation, retaining the main information in the data and removing redundant features at the same time, significantly reducing the computational complexity and improving the computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a schematic flowchart of a natural resource dynamic monitoring method based on multi-source data fusion provided by the present invention;

[0077] Figure 2 is a schematic structural diagram of a natural resource dynamic monitoring system based on multi-source data fusion provided by the present invention;

[0078] Figure 3 is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0079] The following further elaborates on the specific implementation manners of the present invention with reference to the accompanying drawings.

[0080] Example 1

[0081] A natural resource dynamic monitoring method based on multi-source data fusion provided by the present invention, as Figure 1 shown, includes:

[0082] S1. Obtain multi-source monitoring data of natural resources in the area to be monitored, and extract multi-source spatio-temporal features from the multi-source monitoring data and splice them to obtain a preliminary fusion feature;

[0083] S2. Based on the linear correlation between each feature dimension in the preliminary fusion feature, construct a feature map structure; use a graph convolutional network to perform an aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion feature, and perform principal component analysis on the high-order features to obtain the target fusion feature of the multi-source monitoring data;

[0084] S3. Input the target fusion feature into a pre-constructed dynamic monitoring network for resource status analysis and change trend prediction, and obtain the resource status analysis result and change trend prediction result of the natural resources in the area to be detected.

[0085] In step S1, the multi-source monitoring data includes: remote sensing image data, unmanned aerial vehicle (UAV) image data, and ground sensor data of the area to be detected.

[0086] S11. Use satellite remote sensing equipment and UAV aerial survey equipment to obtain remote sensing image data and UAV image data of the area to be monitored respectively;

[0087] Use ground monitoring equipment to obtain ground monitoring data such as soil moisture, water quality, and meteorology of the area to be monitored;

[0088] Combine the remote sensing image data, UAV image data, and the ground monitoring data to obtain the multi-source monitoring data of the natural resources in the area to be monitored.

[0089] Among them, the remote sensing image data includes: high-resolution image data for analyzing land use, vegetation cover, water resource distribution, etc. Deploy remote sensing satellites in appropriate satellite orbits to ensure that they can cover the target monitoring area; configure the sensor parameters of the remote sensing satellites, such as spectral bands, resolution, etc., to meet different types of monitoring requirements. Using satellite remote sensing technology can obtain large-scale macroscopic data, which can provide a global perspective and help identify the basic distribution and change trends of natural resources.

[0090] When the UAV aerial survey equipment obtains UAV image data, it is necessary to formulate a flight plan for the UAV aerial survey equipment in advance based on monitoring requirements, including flight path, flight altitude, shooting frequency, etc.; conduct regular aerial surveys in key areas or areas where high-resolution data is required to obtain high-precision terrain, landform, and land use data.

[0091] The ground sensor data includes various environmental parameters such as air quality, soil humidity, water quality, and meteorology. Installation points are pre-selected based on the monitoring requirements of the natural environment for installing devices such as soil humidity sensors, air quality monitors, water quality monitors, and weather stations; and the sampling frequency is configured to ensure real-time collection of environmental data. These data can provide real-time environmental condition information and are an important basis for dynamic monitoring.

[0092] For each monitoring device (including remote sensing satellites, UAV aerial survey devices, and ground monitoring devices), the monitoring device can autonomously adjust the sampling frequency according to the environment and requirements. When a significant change in environmental parameters in a certain area is detected, the corresponding sensor device automatically increases the sampling frequency and sends an alarm to the data center.

[0093] In the case of relatively stable environment, the preset sampling frequency is relatively low, which can avoid redundant sampling data and reduce the data processing pressure; when environmental parameters are abnormal, the adaptive adjustment of the sampling frequency can ensure the acquisition of real-time information, and then ensure the ability to achieve accurate and timely emergency response.

[0094] S12. After obtaining multi-source monitoring data of natural resources in the area to be monitored by using multiple monitoring devices: Use the edge computing device to perform compression and screening processing on the multi-source monitoring data to obtain the processed multi-source monitoring data.

[0095] Each edge computing device corresponds to a monitoring device, which is used to receive the monitoring data of the monitoring device and complete preliminary data processing tasks locally, only retaining key information for subsequent feature extraction, reducing the subsequent data processing volume, and improving the real-time performance of data processing; and the edge computing device provides a distributed computing architecture, enabling each edge computing device node to work independently and cooperate with other nodes, improving the flexibility and response speed of the system.

[0096] S13. Extract multi-source spatio-temporal features from the multi-source monitoring data and splice them to obtain preliminary fusion features. The multi-source spatio-temporal features include the spatial features and the time series features.

[0097] S13-1. Use a deep convolutional neural network to extract the spatial features of the remote sensing image data and the UAV image data.

[0098] Specifically, the remote sensing image (as well as the UAV image) is divided into multiple image blocks and normalized; the multiple normalized image blocks are input into a CNN (Convolutional Neural Network) model for spatial feature extraction, and the output of the convolutional layer is extracted as the spatial feature. CNN can automatically learn the spatial features in the image and extract feature vectors that can effectively represent ground object information.

[0099] The spatial features include: edge features, which extract the boundary information of ground objects, such as the demarcation line between forests and farmlands, the contour of rivers, etc.; texture features, which describe the surface texture of ground objects, such as the canopy texture of forests, the uniformity of grasslands, etc.; shape features, which extract the geometric shapes of ground objects, such as the shape of lakes, the contour of buildings, etc.

[0100] Spatial features can capture the geometric structure and spatial distribution of ground objects and are applicable to tasks such as land use classification and ground object boundary detection.

[0101] Extracting features using a deep convolutional neural network can capture the deep features of data and improve the accuracy of target recognition and classification.

[0102] S13-2. Extract the time series features of the ground sensor data using a long short-term memory network.

[0103] The memory function of the long short-term memory network can capture the long-term dependencies in the data and obtain the time series feature vector. The time series features are a multi-dimensional vector, including statistical features, trend features, periodic features, domain features, time domain features, etc. These features can comprehensively reflect the dynamic changes and trends of environmental parameters and provide important time dimension information for multi-source data fusion.

[0104] S13-3. Stitch the spatial features of the remote sensing image data and the UAV image data and the time series features of the ground sensor data to obtain the preliminary fusion features.

[0105] Specifically, stitch the spatial feature F of the remote sensing image data, r the spatial feature F of the UAV image data, d and the time series feature F of the ground sensor data s to obtain the preliminary fusion feature F concat = [F r ; F d ; F s , to ensure that the multi-source monitoring data reflects the state of the same geographical area at the same moment and integrate the features of different data sources into a comprehensive feature vector. Both the spatial features and the time series features are features of multi-dimensional vectors.

[0106] Feature stitching can fuse the features of multi-source data into a unified feature space, retain the diversity information of multi-source data, and provide a basis for subsequent graph structure construction and feature fusion.

[0107] In step S2, based on the linear correlation between each feature dimension in the preliminary fusion features, a feature graph structure is constructed; the nodes in the feature graph structure are aggregated and transformed by using a graph convolutional network to obtain the high-order features of the preliminary fusion features, and principal component analysis is performed on the high-order features to obtain the target fusion features of the multi-source monitoring data.

[0108] Specifically:

[0109] S21. Based on the linear correlation between each feature dimension in the preliminary fusion features, constructing a feature graph structure, including:

[0110] By calculating the Pearson correlation coefficient between every two feature dimensions, the linear correlation between every two feature dimensions is obtained; the calculation formula for the Pearson correlation coefficient between two feature dimensions is:

[0111]

[0112] where F i , F j respectively represent the i-th and j-th dimensions in the feature vector, ρ ij represents the Pearson correlation coefficient between the i-th and j-th feature dimensions, that is, the linear correlation between two feature dimensions; cov(F i , F j ) represents the covariance between two feature dimensions, and respectively represent the standard deviations of the two feature dimensions F i and F j .

[0113] Based on the linear correlation between every two feature dimensions and the correlation threshold, an adjacency matrix of the preliminary fusion features is constructed;

[0114] If the linear correlation is greater than the correlation threshold, the element in the adjacency matrix is 1, otherwise it is 0; or alternatively, the linear correlation between every two feature dimensions is directly used as the element in the adjacency matrix;

[0115] Taking each feature dimension in the preliminary fusion features as a node and the connection relationship (the element in the adjacency matrix) between every two feature dimensions in the adjacency matrix as an edge, a feature graph structure of the preliminary fusion features is constructed.

[0116] When constructing the adjacency matrix, it is usually necessary to calculate the correlation coefficient between every two feature dimensions to ensure complete capture of the relationship between features. This is because the correlation between features may be global, and ignoring some correlations may lead to information loss.

[0117] S22. Aggregate and transform the nodes in the feature map structure using a graph convolutional network to obtain the high-order features of the preliminary fusion features.

[0118] Add the identity matrix to the adjacency matrix of the preliminary fusion features to obtain an adjacency matrix incorporating node self-connections, and normalize the adjacency matrix incorporating node self-connections to obtain a normalized adjacency matrix; adding self-connections is to ensure that when aggregating features, the feature information of the node itself is also considered, rather than relying solely on the features of neighboring nodes, which improves the accuracy of feature aggregation.

[0119] In the first convolutional layer of the graph convolutional network, multiply the normalized adjacency matrix by the element matrix of the preliminary fusion features to obtain the aggregated features after aggregating all nodes in the feature map structure with their adjacent nodes; transform the aggregated features through a learnable weight matrix and a non-linear activation function, and output new preliminary fusion features; specifically, the convolution operation formula of the preliminary fusion features in each convolutional layer is:

[0120]

[0121] where H (l) represents the node features of the l-th convolutional layer (the feature dimension in the preliminary fusion features), and H (l+1) represents the node features after the convolution operation in the l-th convolutional layer, which is used as the input for the next layer; is the degree matrix of is the adjacency matrix after adding self-connections; W (l) represents the learnable weight matrix of the l-th convolutional layer; σ represents the non-linear activation function.

[0122] Use the new preliminary fusion features as the input for the next convolutional layer, and repeat the operations of the first convolutional layer using the next convolutional layer until after multiple convolutional layers, output the high-order features of the initial fusion features.

[0123] By constructing a graph structure and using the graph structure to extract the high-order relationships between features, the complex dependencies between multi-source features can be captured. These features not only contain the original information of the nodes themselves but also incorporate the information of their adjacent nodes.

[0124] S23. Perform principal component analysis on each of the high-order features to obtain the target fusion features of the multi-source monitoring data.

[0125] Use principal component analysis to reduce the dimension of the high-order features through linear transformation to obtain low-dimensional features, and use the low-dimensional features as the final target fusion features of the multi-source monitoring data.

[0126] Higher-order features require more storage space and computing resources. Especially when dealing with large-scale data, the computational cost will increase significantly. By using principal component analysis, the higher-order features are mapped to a low-dimensional space through linear transformation, retaining the main information in the data while removing redundant features, significantly reducing the computational complexity.

[0127] In step S3, S31. The construction process of the dynamic monitoring network includes:

[0128] S31-1. Based on the fusion features in the historical multi-source monitoring data, train deep learning models corresponding to different resource state analysis requirements to obtain multiple resource state analysis models. Specifically:

[0129] Divide the dataset of the fusion features extracted from the historical multi-source monitoring data into a training set and a validation set;

[0130] Select multiple deep learning models according to different resource state analysis requirements;

[0131] Based on each deep learning model, according to the validation set, combine the grid search method to tune the hyperparameters of the deep learning model to obtain the initial resource state analysis models corresponding to different resource state analysis requirements;

[0132] Input the training set into the initial resource state analysis model for multiple iterative trainings. The initial analysis model adjusts its internal parameters during the iterative training process to minimize the loss function until the iterative termination condition is reached, obtaining the resource state analysis models corresponding to different resource state analysis requirements.

[0133] In this embodiment, taking the resource state analysis requirement of forest degradation monitoring as an example, train the resource state analysis model. Specifically:

[0134] Select the spatio-temporal graph convolutional network as the deep learning model;

[0135] Customize the hyperparameter search space in the spatio-temporal graph convolutional network. The hyperparameters include learning rate, batch size, number of convolutional layers, and loss function weight;

[0136] Based on the validation set, combine the grid search method to tune the hyperparameters in the spatio-temporal graph convolutional network to obtain the best hyperparameter combination;

[0137] Initialize the spatio-temporal graph convolutional network based on the best hyperparameter combination to obtain the initial resource state analysis model for forest degradation monitoring;

[0138] The training set includes fusion features and corresponding label data. The label data is the forest degradation level (0 - healthy, 1 - mild degradation, 2 - severe degradation);

[0139] Input the training set into the initial resource state analysis model for multiple iterative trainings. Select the weighted cross - entropy loss as the loss function, use the analysis model to output the predicted classification results, calculate the loss based on the predicted classification results and the label data, and back - propagate to adjust the internal parameters of the analysis model to minimize the loss function until the preset number of iterations is reached, obtaining a trained resource state analysis model for forest degradation monitoring.

[0140] S31 - 2. Based on the fusion features at different time points in the historical multi - source monitoring data, train the long - short - term memory network to obtain a change trend prediction model. Specifically:

[0141] Arrange the fusion features at different time points in the historical multi - source monitoring data in chronological order, and label the corresponding change category labels for the fusion features at each time point to construct a time - series data set;

[0142] Use the fusion features in the time - series data set as the input, the change categories corresponding to different time points as the output, and the cross - entropy loss as the loss function for the classification task to train the long - short - term memory network to obtain a change prediction model.

[0143] S31 - 3. Combine multiple resource state analysis models and change trend prediction models to construct a dynamic monitoring network.

[0144] S32. Input the target fusion features into the pre - constructed dynamic monitoring network for resource state analysis to obtain the resource state analysis results and change trend prediction results of the natural resources in the area to be detected.

[0145] Input the target fusion features into the dynamic monitoring network, which includes multiple resource state analysis models and change trend prediction models.

[0146] The resource state analysis model can output the classification results of current natural resources (such as forest degradation level, water pollution degree, etc.);

[0147] The change trend prediction model can predict the change categories of resource states based on the fusion features in the historical multi - source monitoring data;

[0148] Combine the outputs of the two types of models to generate a comprehensive analysis report (such as "The current degradation level in Area A is 2, and the predicted deterioration probability reaches 65%").

[0149] In this application, multi-source monitoring data can provide information at different scales and of different types, achieving comprehensive coverage of natural resources. For example, satellite remote sensing provides macroscopic data, unmanned aerial vehicle (UAV) aerial survey provides local high-precision data, ground monitoring stations provide real-time environmental data, and portable detection devices provide detailed microscopic data. Through the fusion of these data, a complete natural resource monitoring system can be constructed. After the multi-source data is fused, multi-dimensional analysis can be carried out to extract richer information. The comprehensive utilization of multi-modal data helps to improve the prediction accuracy of the model. When assessing the state of natural resources, by combining image data and environmental parameters recorded by ground sensors, the uncertainty brought by a single data source can be reduced, enabling the model to more accurately judge whether there are abnormalities in the area to be monitored.

[0150] Moreover, through feature splicing, the features of multi-source data can be fused into a unified feature space, retaining the diversity information of multi-source data. However, simple feature splicing can only retain the shallow information of features and cannot capture the high-order relationships between features. Through the multi-layer convolution operation of the graph convolutional network, the information of neighbor nodes can be aggregated, high-order features can be extracted, and irrelevant features can be suppressed. High-order features can better reflect the global structure and complex dependency relationships of the data, are more discriminative, and thus improve the accuracy of natural area state classification and prediction tasks.

[0151] Based on historical multi-source monitoring data to train a change prediction model, the system can predict the change trend of natural resources and can provide early warning prompts based on the predicted trend, which helps to achieve the sustainable management and protection of natural resources and is applicable to the field of natural resource protection and rational utilization.

[0152] Embodiment 2

[0153] Based on the same inventive concept, the present invention also provides a natural resource dynamic monitoring system based on multi-source data fusion, as Figure 2 shown, including:

[0154] A feature extraction module, configured to obtain multi-source monitoring data of natural resources in the area to be monitored, and extract multi-source spatio-temporal features from the multi-source monitoring data for splicing to obtain a preliminary fusion feature;

[0155] A feature fusion module, configured to construct a feature map structure based on the linear correlation between each feature dimension in the preliminary fusion feature; use a graph convolutional network to perform an aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion feature, and perform principal component analysis on the high-order features to obtain the target fusion feature of the multi-source monitoring data;

[0156] The data analysis module is used to input the target fusion features into a pre-constructed dynamic monitoring network for resource status analysis and change trend prediction, and obtain the resource status analysis result and change trend prediction result of the natural resources in the area to be detected.

[0157] In a possible implementation manner, the above feature fusion module is specifically used for:

[0158] By calculating the Pearson correlation coefficient between every two of the feature dimensions, obtain the linear correlation between every two of the feature dimensions;

[0159] Based on the linear correlation between every two of the feature dimensions and a correlation threshold, construct the adjacency matrix of the preliminary fusion features;

[0160] Using each feature dimension in the preliminary fusion features as nodes and the connection relationship between every two of the feature dimensions in the adjacency matrix as edges, construct the feature map structure of the preliminary fusion features.

[0161] In a possible implementation manner, the above feature fusion module is specifically used for:

[0162] Add the identity matrix to the adjacency matrix of the preliminary fusion features to obtain an adjacency matrix incorporating node self-connections, and normalize the adjacency matrix incorporating node self-connections to obtain a normalized adjacency matrix;

[0163] In the first convolutional layer of the graph convolutional network, multiply the normalized adjacency matrix by the element matrix of the preliminary fusion features to obtain the aggregated features after aggregation of all nodes and their adjacent nodes in the feature map structure; transform the aggregated features through a learnable weight matrix and a non-linear activation function, and output new preliminary fusion features;

[0164] Use the new preliminary fusion features as the input of the next convolutional layer, and repeat the operations of the first convolutional layer using the next convolutional layer until after passing through multiple convolutional layers, output the high-order features of the initial fusion features.

[0165] In a possible implementation manner, the above feature extraction module is specifically used for:

[0166] The extraction of multi-source spatio-temporal features from the multi-source monitoring data and stitching them to obtain preliminary fusion features includes:

[0167] Use a deep convolutional neural network to extract the spatial features of the remote sensing image data and the UAV image data;

[0168] Use a long short-term memory network to extract the time series features of the ground sensor data;

[0169] Stitch the spatial features of the remote sensing image data and the UAV image data and the time series features of the ground sensor data to obtain preliminary fusion features;

[0170] The multi-source spatio-temporal features include the spatial features and the time series features.

[0171] In a possible implementation manner, the above data analysis module is specifically configured to:

[0172] Based on the fusion features in the historical multi-source monitoring data, train deep learning models corresponding to different resource state analysis requirements to obtain multiple resource state analysis models;

[0173] Based on the fusion features at different time points in the historical multi-source monitoring data, train the long short-term memory network to obtain a change trend prediction model;

[0174] Combine multiple resource state analysis models and the change trend prediction model to construct a dynamic monitoring network.

[0175] In a possible implementation manner, the above data analysis module is specifically configured to:

[0176] Divide the dataset of the fusion features extracted from the historical multi-source monitoring data into a training set and a validation set;

[0177] Select multiple deep learning models according to different resource state analysis requirements;

[0178] Based on each deep learning model, according to the validation set, combine the grid search method to tune the hyperparameters of the deep learning model to obtain initial resource state analysis models corresponding to different resource state analysis requirements;

[0179] Input the training set into the initial resource state analysis model for multiple iterative trainings. The initial analysis model adjusts internal parameters during the iterative training process to minimize the loss function until the iterative termination condition is reached, to obtain resource state analysis models corresponding to different resource state analysis requirements.

[0180] In a possible implementation manner, the above data analysis module is specifically configured to:

[0181] Arrange the fusion features at different time points in the historical multi-source monitoring data in chronological order, and label the corresponding change category labels for the fusion features at different time points to construct a time series dataset;

[0182] Taking the fused features in the time series dataset as input, the change categories corresponding to the fused features at different time points as output, and the cross-entropy loss as the loss function for the classification task, the long short-term memory network is trained to obtain a change prediction model.

[0183] In a possible implementation manner, the above feature extraction module is further configured to:

[0184] Use an edge computing device to compress and screen the multi-source monitoring data to obtain processed multi-source monitoring data.

[0185] Embodiment 3

[0186] As Figure 3 shown, the present invention further provides an electronic device, which may be a computer device, a single-chip microcomputer device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected through a bus; the memory can be used to store an execution program, and the exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, and this data can be called and / or modified when the instructions are executed.

[0187] The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of a natural resource dynamic monitoring method based on multi-source data fusion in the above embodiment.

[0188] Embodiment 4

[0189] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device-readable storage medium (Memory). The electronic device-readable storage medium is a memory device in the electronic device and is used to store programs and data. It can be understood that the storage medium here can include both the built-in storage medium in the electronic device and, of course, the extended storage medium supported by the electronic device. The storage medium provides a storage space, and this storage space stores the operating system of the terminal. Moreover, in this storage space, there is also stored one or more instructions suitable for being loaded and executed by the processor. These instructions can be one or more executable programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. By the processor loading and executing one or more instructions stored in the storage medium, the steps of a method for dynamically monitoring natural resources based on multi-source data fusion in the above embodiments can be realized.

[0190] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0191] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the function specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0192] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and this instruction device realizes the function in Figure 1 one flow or multiple flows and / or blocks Figure 1The functions specified in one or more boxes.

[0193] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in Figure 1 one process or more processes and / or boxes Figure 1 the functions specified in one box or more boxes.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the scope of its protection. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent replacements to the specific implementation manners of the application. However, these changes, modifications or equivalent replacements are all within the scope of the protection of the claims pending for approval of the application.

Claims

1. A natural resource dynamic monitoring method based on multi-source data fusion, characterized in that Including: Obtain multi-source monitoring data of natural resources in the area to be monitored, and extract multi-source spatio-temporal features from the multi-source monitoring data and splice them to obtain preliminary fusion features; Based on the linear correlation between each feature dimension in the preliminary fusion features, construct a feature map structure; use a graph convolutional network to perform an aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion features, and perform principal component analysis on the high-order features to obtain the target fusion features of the multi-source monitoring data; Input the target fusion features into a pre-constructed dynamic monitoring network for resource status analysis and change trend prediction to obtain the resource status analysis result and change trend prediction result of the natural resources in the area to be detected.

2. The method according to claim 1, wherein The constructing a feature map structure based on the linear correlation between each feature dimension in the preliminary fusion features includes: Calculate the Pearson correlation coefficient between every two of the feature dimensions to obtain the linear correlation between every two of the feature dimensions; Based on the linear correlation between every two of the feature dimensions and a correlation threshold, construct an adjacency matrix of the preliminary fusion features; Use each feature dimension in the preliminary fusion features as a node and the connection relationship between every two of the feature dimensions in the adjacency matrix as an edge to construct a feature map structure of the preliminary fusion features.

3. The method according to claim 2, wherein The using a graph convolutional network to perform an aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion features includes: Add an identity matrix to the adjacency matrix of the preliminary fusion features to obtain an adjacency matrix incorporating node self-connections, and normalize the adjacency matrix incorporating node self-connections to obtain a normalized adjacency matrix; In the first convolutional layer of the graph convolutional network, multiply the normalized adjacency matrix by the element matrix of the preliminary fusion features to obtain an aggregation feature after aggregating all nodes in the feature map structure with their adjacent nodes; transform the aggregation feature through a learnable weight matrix and a non-linear activation function, and output a new preliminary fusion feature; Use the new preliminary fusion feature as the input of the next convolutional layer, and use the next convolutional layer to repeat the operation of the first convolutional layer until after multiple convolutional layers, output the high-order features of the initial fusion features.

4. The method according to claim 1, wherein The multi-source monitoring data includes: remote sensing image data, unmanned aerial vehicle image data, and ground sensor data of the area to be detected; The extracting multi-source spatio-temporal features from the multi-source monitoring data and splicing them to obtain preliminary fusion features includes: Use a deep convolutional neural network to extract the spatial features of the remote sensing image data and the unmanned aerial vehicle image data; Use a long short-term memory network to extract the time series features of the ground sensor data; Splice the spatial features of the remote sensing image data and the unmanned aerial vehicle image data and the time series features of the ground sensor data to obtain preliminary fusion features; The multi-source spatio-temporal features include the spatial features and the time series features.

5. The method according to claim 1, characterized in that, The construction process of the dynamic monitoring network includes: Based on the fusion features in historical multi-source monitoring data, model training is performed on deep learning models corresponding to different resource status analysis requirements to obtain multiple resource status analysis models; Based on the fusion features at different time points in historical multi-source monitoring data, model training is performed on a long short-term memory network to obtain a change trend prediction model; Combining multiple said resource status analysis models and said change trend prediction model to construct a dynamic monitoring network.

6. The method according to claim 5, wherein The method of performing model training on deep learning models corresponding to different resource status analysis requirements based on the fusion features in historical multi-source monitoring data to obtain multiple resource status analysis models includes: Dividing the dataset of fusion features extracted from historical multi-source monitoring data into a training set and a validation set; Selecting multiple deep learning models according to different resource status analysis requirements; Based on each said deep learning model, tuning the hyperparameters of the deep learning model by combining the grid search method according to the validation set to obtain an initial resource status analysis model corresponding to different resource status analysis requirements; Inputting the training set into the initial resource status analysis model for multiple iterative trainings. The initial analysis model adjusts its internal parameters during the iterative training process to minimize the loss function until the iterative termination condition is reached, obtaining resource status analysis models corresponding to different resource status analysis requirements.

7. The method according to claim 5, characterized in that, The method of performing model training on a long short-term memory network based on the fusion features at different time points in historical multi-source monitoring data to obtain a change trend prediction model includes: Arranging the fusion features at different time points in historical multi-source monitoring data in chronological order and labeling corresponding change category labels for the fusion features at different time points to construct a time series dataset; Using the fusion features in the time series dataset as input, the change categories corresponding to the fusion features at different time points as output, and the cross-entropy loss as the loss function for the classification task to train the long short-term memory network to obtain a change prediction model.

8. The method according to claim 1, wherein Before extracting multi-source spatio-temporal features from the multi-source monitoring data and splicing them to obtain preliminary fusion features, it further includes: Using an edge computing device to compress and screen the multi-source monitoring data to obtain processed multi-source monitoring data.

9. A natural resource dynamic monitoring system based on multi-source data fusion, characterized in that, It includes: A feature extraction module for obtaining multi-source monitoring data of natural resources in the area to be monitored and extracting multi-source spatio-temporal features from the multi-source monitoring data and splicing them to obtain preliminary fusion features; A feature fusion module for constructing a feature map structure based on the linear correlation between each feature dimension in the preliminary fusion features; using a graph convolutional network to perform aggregation transformation on the nodes in the feature map structure to obtain high-order features of the preliminary fusion features, and performing principal component analysis on the high-order features to obtain the target fusion features of the multi-source monitoring data; A data analysis module for inputting the target fusion features into a pre-constructed dynamic monitoring network for resource status analysis and change trend prediction to obtain the resource status analysis result and change trend prediction result of the natural resources in the area to be detected.

10. The system according to claim 9, characterized in that, The feature fusion module is specifically used for: By calculating the Pearson correlation coefficient between every two of the feature dimensions, the linear correlation between every two of the feature dimensions is obtained; Based on the linear correlation between every two of the feature dimensions and the correlation threshold, an adjacency matrix of the preliminary fusion features is constructed; Taking each feature dimension in the preliminary fusion features as a node and the connection relationship between every two of the feature dimensions in the adjacency matrix as an edge, a feature map structure of the preliminary fusion features is constructed.

Images