A land use time series prediction method and related equipment based on multi-source data collaboration

Abstract

The present invention provides a land use time series prediction method and related equipment based on multi-source data collaboration. The method converts historical laser point cloud data of a target area into a digital surface model and then superimposes the model with historical spectral image data to form spectral-geometric mixed data. Feature extraction is performed on the spectral-geometric mixed data to generate multi-channel data that fuses spectral information, spatial information, and geometric information. The multi-channel data is input into a constructed fusion convolutional neural network for fusion to obtain spectral-geometric-temporal-spatial fusion features. The spectral-geometric-temporal-spatial fusion features are input into a trained land use time series prediction network to predict land use changes, thereby obtaining land use prediction results for the target area at both macro and micro levels. This improves the accuracy of land use time series prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of land use prediction, and in particular to a multi-source data collaborative land use time series prediction method and related equipment. Background Art

[0002] With the acceleration of urbanization, land use and land cover change have become a core issue in global environmental research. The spatiotemporal dynamics of land use are crucial for natural resource management, ecological protection, urban planning, and environmental monitoring. Accurately predicting the spatiotemporal evolution of land use, particularly with the support of remote sensing and LiDAR data, is of great practical significance to decision-makers at all levels. However, the field of land use prediction still faces numerous challenges.

[0003] Currently, research methods for land use prediction mainly include spectral-based prediction, geometric-based prediction, and prediction based on the fusion of spectral and geometric information. Spectral-based prediction typically uses satellite remote sensing imagery for land cover classification, using traditional machine learning methods such as support vector machines (SVMs), decision trees, and random forests for analysis. Geometric-based prediction focuses on inferring surface features using digital surface models (DSMs) acquired by LiDAR (Light Detection and Ranking), thereby predicting and classifying land use. Prediction based on the fusion of spectral and geometric information is gaining increasing attention. By combining these two methods, a more comprehensive picture of land use change can be captured. For example, some researchers have fused remote sensing imagery with LiDAR (Light Detection and Ranging) data and employed deep learning methods such as convolutional neural networks (CNNs) to improve prediction accuracy. However, existing fusion methods often lack effective spatial-temporal synergy models when processing multimodal data, making it difficult to fully utilize temporal variation information, resulting in a weak ability to capture temporal dynamics during the prediction process.

[0004] While the aforementioned existing approaches offer preliminary insights for land use prediction, they suffer from significant shortcomings in processing multimodal data, time series data, and the fusion of spatial and temporal features. First, many studies lack effective feature extraction and fusion mechanisms, failing to fully exploit the complementarity between different data sources (such as spectral and geometric data), limiting the model's expressive power. Second, existing prediction methods are mostly confined to a single data processing stage and fail to account for spectral and geometric features, as well as information on spatiotemporal variation. Therefore, constructing a model capable of extracting and fusing features from multiple perspectives and levels is crucial.

[0005] Furthermore, existing convolutional neural network (CNN) methods primarily focus on processing static images and are less capable of processing dynamic time series data, making it difficult to effectively capture the temporal information of land use change. Temporal features play a crucial role in land use prediction, as land use is influenced by multiple factors and exhibits certain regularities over time. Existing methods lack sufficient mechanisms for integrating spatiotemporal features when processing these temporal changes, limiting their ability to predict land use evolution. Summary of the Invention

[0006] The present invention provides a multi-source data collaborative land use time series prediction method and related equipment, the purpose of which is to improve the accuracy of land use time series prediction.

[0007] To achieve the above objectives, the present invention provides a land use time series prediction method based on multi-source data collaboration, comprising:

[0008] Step 1: Acquire historical spectral image data and historical laser point cloud data of the target area;

[0009] Step 2: convert the historical laser point cloud data into a digital surface model, and superimpose the historical spectral image data with the digital surface model to form spectral geometric mixed data;

[0010] Step 3: Extract features from the spectral-geometric mixed data to generate multi-channel data that integrates spectral information, spatial information, and geometric information;

[0011] Step 4: Input the multi-channel data into the constructed fusion convolutional neural network for fusion to obtain the spectral geometric spatiotemporal fusion features;

[0012] Step 5: Input the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes, and obtain the land use prediction results of the target area at both macro and micro levels;

[0013] The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a first convolutional network module for fusing spatial dimension features and temporal dimension features in spectral geometric features output by the first convolutional network module;

[0014] The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

[0015] More specifically, step 1 includes:

[0016] Utilize multiple spectral channels of remote sensing satellites or aerial monitoring platforms to obtain historical spectral image data of the target area. The historical spectral image data includes visible light spectral images and infrared spectral images, which are used to characterize the different spectral characteristics of the target area.

[0017] The historical laser point cloud data of the target area is collected by lidar to represent the surface height information of the target area.

[0018] More specifically, step 2 includes:

[0019] Process historical laser point cloud data to generate a digital surface model to represent the elevation and three-dimensional structure of the surface;

[0020] Performing registration and alignment processing on the digital surface model and the historical spectral image data to obtain the registered historical spectral image data;

[0021] The digital surface model is superimposed on the registered historical spectral image data as a geometric information channel, and corresponds to each time node of the registered historical spectral image data to form spectral geometric mixed data.

[0022] More specifically, step 3 includes:

[0023] The Sobel operator is used to extract the horizontal and vertical edge features of the spectral geometric mixed data to obtain multiple first feature maps;

[0024] The Laplacian operator is used to extract the second-order derivative features of the spectral geometric mixing data to obtain multiple second feature maps;

[0025] Decomposing the spectral geometric mixed data by using discrete wavelet decomposition to obtain multiple third characteristic maps;

[0026] The local texture features of the spectral geometric mixed data are analyzed using local binary patterns to obtain multiple fourth-character feature maps;

[0027] Using edge detection algorithm to extract edge features from spectral geometric mixed data, a plurality of fifth feature maps are obtained;

[0028] The maximum noise separation transform is used to enhance the features of the spectral geometric mixed data to obtain multiple sixth feature maps;

[0029] Multi-channel data integrating spectral information, spatial information and geometric information is generated based on all first feature maps, all second feature maps, all third feature maps, all fourth feature maps, all fifth feature maps and all sixth feature maps.

[0030] More specifically, step 4 includes:

[0031] Input multi-channel data into the constructed fusion convolutional neural network;

[0032] fusing the multi-channel data through the first convolutional layer in the first convolutional network module to obtain multiple first feature fusion results;

[0033] All first feature fusion results are fused through the second convolutional layer in the first convolutional network module to obtain multiple second feature fusion results;

[0034] All the second feature fusion results are fused through the third convolutional layer in the first convolutional network module to obtain the spectral geometric features;

[0035] The spectral geometric features are temporally and spatially fused through the first convolutional layer in the second convolutional network module to obtain multiple temporal and spatial fusion features;

[0036] All the spatiotemporal fusion features are fused through the second convolutional layer in the second convolutional network module to obtain multiple first spatiotemporal feature fusion results;

[0037] All the first spatiotemporal feature fusion results are processed in spatial dimension through the deconvolution layer in the second convolutional network module to obtain the spectral geometric spatiotemporal fusion features.

[0038] Furthermore, the loss function expression of the land use time series prediction network is:

[0039] ;

[0040] in, represents the total loss, Represents the image data dimension, represents the macro cross entropy loss, represents the microscopic cross entropy loss, Indicates the loss of macro-micro consistency.

[0041] Furthermore, the function expression of macro-micro consistency loss is:

[0042] ;

[0043] in, represents relative entropy, Indicates a point The predicted distribution at the macro level, Indicates a point Predicted distribution at the micro level.

[0044] The present invention also provides a land use time series prediction device based on multi-source data collaboration, comprising:

[0045] An acquisition module is used to acquire historical spectral image data and historical laser point cloud data of the target area;

[0046] A processing module is used to convert historical laser point cloud data into a digital surface model, and to superimpose the historical spectral image data with the digital surface model to form spectral-geometry hybrid data;

[0047] The extraction module is used to extract features from spectral and geometric mixed data to generate multi-channel data that integrates spectral information, spatial information, and geometric information;

[0048] The fusion module is used to fuse the multi-channel data input into the constructed fusion convolutional neural network to obtain the spectral geometric spatiotemporal fusion features;

[0049] The prediction module is used to input the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes and obtain the land use prediction results of the target area at both macro and micro levels;

[0050] The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a first convolutional network module for fusing spatial dimension features and temporal dimension features in spectral geometric features output by the first convolutional network module;

[0051] The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

[0052] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, a land use time series prediction method based on multi-source data collaboration is implemented.

[0053] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a land use time series prediction method based on multi-source data collaboration.

[0054] The above solution of the present invention has the following beneficial effects:

[0055] The present invention converts the historical laser point cloud data of the target area into a digital surface model and then superimposes it with the historical spectral image data to form spectral geometric mixed data; extracts features from the spectral geometric mixed data to generate multi-channel data that fuses spectral information, spatial information and geometric information; inputs the multi-channel data into the constructed fusion convolutional neural network for fusion, and obtains spectral geometric spatiotemporal fusion features; inputs the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes, and obtains land use prediction results at both macro and micro levels for the target area; Compared with the prior art, the present invention, by fusing spectral data, can achieve the goal of land use prediction in the target area at both macro and micro levels. The spectral and geometric mixed data containing rich information are generated by combining spectral data and laser point cloud data, which effectively improves the feature expression ability of the model and can comprehensively reflect the spatial changes and three-dimensional characteristics of land use; through the combination of multiple feature extraction methods, the deep information in spectral data and laser point cloud data can be effectively mined; by fusing convolutional neural networks, the temporal changes of land use can be effectively captured; land use change prediction is carried out through the land use time series prediction network, which can not only accurately predict the large-scale changes in land use, but also carefully predict the microscopic changes in land use, making the prediction results more detailed and precise, thereby improving the accuracy of land use time series prediction.

[0056] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention;

[0058] Figure 2 is a network architecture diagram according to an embodiment of the present invention;

[0059] Figure 3 Schematic diagram of the structure of the land use time series prediction device in an embodiment of the present invention;

[0060] Figure 4 Schematic diagram of the structure of the terminal device in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0062] In the description of the present invention, it should be noted that the terms "first", "second" and "third" are only used for descriptive purposes and should not be understood as indicating or implying relative importance.

[0063] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0064] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0065] In response to existing problems, the present invention provides a land use time series prediction method and related equipment based on multi-source data collaboration.

[0066] like Figure 1 、 Figure 2 As shown, an embodiment of the present invention provides a land use time series prediction method based on multi-source data collaboration, including:

[0067] Step 1: Obtain historical spectral image data and historical laser point cloud data of the target area;

[0068] Step 2: convert the historical laser point cloud data into a digital surface model, and superimpose the historical spectral image data with the digital surface model to form spectral geometric mixed data;

[0069] Step 3: extract features from the spectral-geometric mixed data to generate multi-channel data that integrates spectral information, spatial information, and geometric information;

[0070] Step 4: Input the multi-channel data into the constructed fusion convolutional neural network for fusion to obtain the spectral geometric spatiotemporal fusion features;

[0071] Step 5: Input the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes, and obtain the land use prediction results of the target area at both macro and micro levels;

[0072] The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a first convolutional network module for fusing spatial dimension features and temporal dimension features in spectral geometric features output by the first convolutional network module;

[0073] The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

[0074] Specifically, step 1 includes:

[0075] Use multiple spectral channels of remote sensing satellites or aerial monitoring platforms to obtain historical spectral image data of the target area every month. The spectral image data dimension of each spectral channel is ,in and Indicates the number of rows and columns of the image. The historical spectral image data includes visible light spectral images and infrared spectral images, which are used to characterize the different spectral characteristics of the target area;

[0076] The laser point cloud data of the target area is collected by lidar to represent the surface height information of the target area.

[0077] Since spectral image data and laser point cloud data need to be fused to form comprehensive spectral geometric mixed data, in this process, the laser point cloud data must be superimposed with the data of each spectral channel to ensure that each pixel contains spatial information and spectral information.

[0078] Specifically, step 2 includes:

[0079] Process historical laser point cloud data to generate a digital surface model to represent the elevation and three-dimensional structure of the surface;

[0080] Performing registration and alignment processing on the digital surface model and the historical spectral image data to obtain the registered historical spectral image data;

[0081] The digital surface model is superimposed on the registered historical spectral image data as a geometric information channel, and corresponds to each time node of the registered historical spectral image data to form spectral geometric mixed data.

[0082] In an embodiment of the present invention, historical laser point cloud data is first preprocessed, including filtering, denoising, and classification, to extract valid ground and non-ground point cloud data. Based on this point cloud data, an interpolation algorithm is used to construct a digital surface model consistent with the spatial resolution of the spectral image, which is used to characterize the elevation and three-dimensional structure information of the surface of the object.

[0083] Then, the historical spectral image data is preprocessed, including geometric correction, radiometric calibration, and spatial registration, to make it consistent with the generated digital surface model in terms of coordinate system, scale, and resolution, thereby achieving data level unification.

[0084] Subsequently, the multi-channel spectral image data of each time node is fused with the digital surface model corresponding to that time point. The fusion method is channel-level superposition, that is, the digital surface model is used as an additional geometric information channel and superimposed on the original spectral image to form enhanced image data containing multiple spectral channels and one geometric channel.

[0085] Finally, the enhanced image data after fusion of all time series are sorted in chronological order to construct a spectral-geometric mixed data sequence at a continuous time scale. This spectral-geometric mixed data sequence not only carries the spectral response characteristics of the ground objects at different times, but also integrates the information of its terrain geometric structure changes, providing more comprehensive input data support for subsequent multidimensional feature extraction and land use change prediction models.

[0086] Specifically, step 3 includes:

[0087] Since the Sobel operator can be used to extract edge features of images and can identify edge information in the horizontal and vertical directions of images, the Sobel operator is used to extract horizontal and vertical edge features of spectral geometric mixed data, thereby effectively capturing the boundaries of objects in the image and obtaining multiple first feature maps, where each spectral channel generates two first feature maps, and the first feature map is an edge feature map;

[0088] Since the Laplacian operator is mainly used to extract the second-order derivative features of the image to enhance the expression of image details, the Laplacian operator is used to extract the second-order derivative features of the spectral geometric mixed data to enhance the image details in the spectral geometric mixed data, and multiple second feature maps are obtained. Among them, each spectral channel generates a second feature map, and the second feature map is the Laplacian feature map. These feature maps can significantly enhance the details in the image, so that the subsequent network can better identify complex land use types;

[0089] Discrete wavelet decomposition is used to decompose the spectral geometric mixed data. The image in the spectral geometric mixed data is decomposed into low-frequency sub-band and high-frequency sub-band features to obtain multiple third feature maps. Among them, each spectral channel generates 8 third feature maps. The third feature maps are discrete wavelet decomposition feature maps. These feature maps can capture multi-scale information in the image and help improve the sensitivity to land use changes.

[0090] Since local binary patterns are mainly used to extract local texture information of images, especially with high sensitivity to texture transformations, local binary patterns are used to analyze the local texture features of spectral geometric mixed data to obtain multiple fourth feature maps, where each spectral channel generates one fourth feature map. The fourth feature map is a local texture feature map. These feature maps can help the model analyze the structural changes of the image at the local level and identify subtle land use changes.

[0091] Edge detection algorithms are used to extract edge features from spectral and geometric mixed data to obtain multiple fifth feature maps, where one fifth feature map is generated for each spectral channel. The fifth feature map is an edge feature map, which helps to identify the boundaries of land objects and classify different types of land use.

[0092] Since the maximum noise separation transform is mainly used for image noise reduction and signal enhancement, the noise and useful signal can be separated by the maximum noise separation transform to improve the quality of image data. Therefore, the maximum noise separation transform is used to enhance the features of the useful signal in the spectral geometric mixed data to obtain multiple sixth feature maps, where each spectral channel generates one sixth feature map. The sixth feature map is the enhanced feature map, which helps to improve the robustness and accuracy of the model.

[0093] Multi-channel data integrating spectral information, spatial information and geometric information is generated based on all first feature maps, all second feature maps, all third feature maps, all fourth feature maps, all fifth feature maps and all sixth feature maps.

[0094] The embodiment of the present invention generates a total of The multi-channel data that integrates spectral information, spatial information and geometric information significantly enriches the dimension and information volume of feature expression. At this time, the dimension of the time series data of Y months is .

[0095] In the embodiment of the present invention, the edge detection algorithm is a Canny edge detection algorithm, which is mainly used to enhance edge information in an image, thereby extracting edge features in the image.

[0096] More specifically, step 4 includes:

[0097] Input multi-channel data into the constructed fusion convolutional neural network;

[0098] The multi-channel data is fused through the first convolution layer in the first convolutional network module to obtain multiple first feature fusion results. The first convolution layer uses 7 convolution kernels, and the size of each convolution kernel is , the filling method uses Same Padding, the main function is to Multi-channel data are fused to generate 7 first feature fusion results with a dimension of ;

[0099] All the first feature fusion results are fused through the second convolution layer in the first convolutional network module to obtain multiple second feature fusion results. The second convolution layer uses 4 convolution kernels, each of which has a size of 7×3×3 and also uses the same padding. Its main function is to further fuse the 7 first feature fusion results to generate 4 second feature fusion results with dimensions of ;

[0100] All the second feature fusion results are fused through the third convolution layer in the first convolutional network module to obtain the spectral geometric feature. The third convolution layer uses one convolution kernel, each convolution kernel has a size of 4×2×2, and the filling method is Same Padding. Its main function is to fuse the four second feature fusion results again to generate one spectral geometric feature, and the dimension becomes ;

[0101] The spectral geometric features are temporally and spatially fused through the first convolutional layer in the second convolutional network module to obtain multiple temporal and spatial fusion features. The first convolutional layer uses 20 convolution kernels, and the size of each convolution kernel is , the filling method uses Same Padding. This layer performs spatiotemporal fusion of spectral geometric features to generate 20 spatiotemporal fusion features with a dimension of ;

[0102] All spatiotemporal fusion features are fused through the second convolutional layer in the second convolutional network module to obtain multiple first spatiotemporal feature fusion results. The second convolutional layer uses 15 convolution kernels, each with a size of 20×2×2 and no padding. This layer fuses 20 spatiotemporal fusion features to generate 15 first spatiotemporal feature fusion results with a dimension of ;

[0103] The deconvolution layer in the second convolutional network module performs spatial dimension upscaling on all the first spatiotemporal feature fusion results to obtain spectral geometric spatiotemporal fusion features. The deconvolution layer uses 12 deconvolution kernels, each with a size of 15×2×2, a step size of 2, and a padding method of Valid Padding. This layer performs spatial dimension upscaling on the 15 first spatiotemporal feature fusion results to generate 12 spectral geometric spatiotemporal fusion features with a dimension of .

[0104] Based on the output of the fused convolutional neural network, the embodiment of the present invention constructs a land use time series prediction network to predict the macro and micro land use conditions of the next month pixel by pixel. The network includes two parts: macro land use prediction and micro land use prediction. The macro land use prediction module is a fully connected network used to transform the dimension of The spectral geometry and spatiotemporal fusion features of are used as input to predict the macro land use category of each pixel, map it to D macro categories, and use the macro cross entropy loss function to calculate the loss; the micro land use prediction module is also a fully connected network used to transform the dimension The spectral-geometric-spatiotemporal fusion features of are used as input to predict the micro-land use category for each pixel, map them to L micro-categories, and use the micro-cross entropy loss function to calculate the loss.

[0105] In the embodiment of the present invention, although the macro and micro prediction tasks have different objectives, their output categories may have some hierarchical relationship. Therefore, it is necessary to introduce a penalty term to promote the consistency of the prediction results of the macro task and the metoclopramide task at the same position. In particular, when the macro category belongs to a certain large category, the prediction result of the micro category should conform to the large category. Therefore, the relationship between the macro and metoclopramide prediction categories can be strengthened by calculating the soft label consistency or cross entropy loss. The function expression of the macro-micro consistency loss is:

[0106] ;

[0107] in, represents the loss of macro-micro consistency, represents relative entropy, Indicates a point The predicted distribution at the macro level, Indicates a point Predicted distribution at the micro level.

[0108] Specifically, the loss function expression of the land use time series prediction network is:

[0109] ;

[0110] in, represents the total loss, Represents the image data dimension, represents the macro cross entropy loss, represents the microscopic cross entropy loss.

[0111] The land use time series prediction network is trained using the above loss function to optimize the network parameters. The network is trained using the gradient descent algorithm to optimize the loss function L. The network weights are continuously adjusted through multiple data cycles (epochs), and the Adam optimizer is used to accelerate convergence. During the training process, the data is divided into a training set and a validation set to evaluate the generalization ability of the network and avoid overfitting.

[0112] Next, the embodiment of the present invention verifies the provided method with specific data, and the process is as follows:

[0113] Multispectral remote sensing data and LiDAR laser point cloud data were selected and used. The multispectral remote sensing data used was from the Sentinel-2 satellite, which provides 13 spectral bands including blue, green, red, and near-infrared, with a spatial resolution of 10 meters. LiDAR laser point cloud data was acquired through a laser point cloud scanner to generate a digital surface model (DSM). The data has a spatial resolution of 10 meters and a size of 256×256 pixels.

[0114] The acquired multispectral smoke sensor data is processed by geometric correction, radiation correction, etc. to ensure data quality;

[0115] In order to improve the prediction accuracy, the multispectral remote sensing data and digital surface model of each month are superimposed to form a Specifically, the 13 spectral bands of multispectral remote sensing data are superimposed with the ground elevation information of DSM to generate 14 channel data;

[0116] For the 14 channels of data for each month, the following feature extraction methods were used:

[0117] Sobel operator: extracts horizontal and vertical edge information of the image and obtains 2 edge feature maps for each channel; Laplacian operator: extracts second-order derivative features, enhances image details, and obtains 1 Laplacian feature map for each channel; Discrete Wavelet Transformation (DWT): decomposes the image into low-frequency and high-frequency sub-band features, generating 8 DWT feature maps for each channel; Local Binary Patterns (LBP): used to analyze the local texture features of the image, generating 1 LBP feature map for each channel; Canny edge detection: extracts edges from the image, generating 1 Canny edge feature map for each channel; Maximum Noise Fraction (MNF) transform: used to reduce noise and enhance useful signals, generating 1 MNF feature map for each channel; Through these methods, the final image is generated. feature maps, totaling 182 channels, providing rich input data for the subsequent fusion convolutional neural network;

[0118] The multi-channel data is input into the constructed fusion convolutional neural network to fuse the spatial and temporal information to obtain the spectral geometric spatiotemporal fusion features;

[0119] The spectral geometric spatiotemporal fusion features are input into the trained land use time series prediction network to predict land use changes. The obtained macro categories include 6 major categories, each of which has 3 micro categories, totaling 18 micro categories.

[0120] The experimental results are as follows: by inputting the training data of the first 10 months, the land use time series prediction network successfully predicted the land use situation in the 11th month.

[0121] The performance comparison results of the present invention and the comparative method in time series prediction are shown in Table 1 below:

[0122] Table 1 Performance comparison table

[0123] ;

[0124] As can be seen from the above table, the method provided by the embodiment of the present invention achieved high accuracy in the prediction task, with a final overall accuracy rate of 90.4%. This result proves that this method can effectively improve the accuracy and reliability of land use prediction, especially in complex urban environments and changing land use scenarios, and has broad application prospects.

[0125] The embodiment of the present invention converts the historical laser point cloud data of the target area into a digital surface model and then superimposes it with the historical spectral image data to form spectral geometric mixed data; extracts features from the spectral geometric mixed data to generate multi-channel data that integrates spectral information, spatial information and geometric information; inputs the multi-channel data into the constructed fusion convolutional neural network for fusion to obtain spectral geometric spatiotemporal fusion features; inputs the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes, and obtains land use prediction results of the target area at both macro and micro levels; Compared with the prior art, the embodiment of the present invention obtains land use prediction results of the target area at both macro and micro levels by fusion Spectral data and laser point cloud data generate spectral geometric mixed data containing rich information, which effectively improves the feature expression ability of the model and can comprehensively reflect the spatial changes and three-dimensional characteristics of land use; through the combination of multiple feature extraction methods, it can effectively mine the deep information in spectral data and laser point cloud data; through the fusion of convolutional neural networks, it can effectively capture the temporal changes of land use; through the land use time series prediction network, land use change prediction can not only accurately predict large-scale changes in land use, but also carefully predict the microscopic changes in land use, making the prediction results more detailed and precise, thereby improving the accuracy of land use time series prediction.

[0126] Corresponding to the multi-source data collaborative land use time series prediction method described in the above embodiment, Figure 3 As shown, an embodiment of the present invention further provides a land use time series prediction device 100 based on multi-source data collaboration, and the land use time series prediction device 100 includes:

[0127] An acquisition module 101 is used to acquire historical spectral image data and historical laser point cloud data of a target area;

[0128] The processing module 102 is used to convert the historical laser point cloud data into a digital surface model, and to superimpose the historical spectral image data with the digital surface model to form spectral-geometric hybrid data;

[0129] Extraction module 103, used to extract features from spectral-geometric mixed data to generate multi-channel data that integrates spectral information, spatial information, and geometric information;

[0130] The fusion module 104 is used to fuse the multi-channel data into the constructed fusion convolutional neural network to obtain the spectral geometric spatiotemporal fusion features;

[0131] The prediction module 105 is used to input the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes and obtain land use prediction results at both macro and micro levels in the target area;

[0132] The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a first convolutional network module for fusing spatial dimension features and temporal dimension features in spectral geometric features output by the first convolutional network module;

[0133] The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

[0134] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0136] The embodiment of the present invention further provides a terminal device, such as Figure 4 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 4 Only one processor is shown), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the above-mentioned multi-source data collaborative land use time series prediction method is implemented.

[0137] The terminal device D10 can be a computing device such as a desktop computer, a notebook, a PDA, a server, a server cluster, a cloud server, etc. The terminal device may include, but is not limited to, a processor D100 and a memory D101. It will be understood by those skilled in the art that Figure 4This is merely an example of the terminal device D10 and does not constitute a limitation on the terminal device D10 . The terminal device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device D10 may also include input and output devices, network access devices, etc.

[0138] The processor D100 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0139] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device D10. Furthermore, the memory D101 may include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory D101 may also be used to temporarily store data that has been output or is about to be output.

[0140] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0142] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a land use time series prediction method based on multi-source data collaboration.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a construction device / terminal device, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Examples include a USB flash drive, a removable hard drive, a magnetic disk, or an optical disk.

[0144] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A land use time series prediction method based on multi-source data collaboration, characterized by: include: Step 1: Acquire historical spectral image data and historical laser point cloud data of the target area; Step 2: converting the historical laser point cloud data into a digital surface model, and superimposing the historical spectral image data with the digital surface model to form spectral geometric mixed data; Step 3, performing feature extraction on the spectral-geometric mixed data to generate multi-channel data that integrates spectral information, spatial information, and geometric information; Step 4, inputting the multi-channel data into the constructed fusion convolutional neural network for fusion to obtain spectral geometric spatiotemporal fusion features; Step 5: Inputting the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes, and obtaining land use prediction results of the target area at both macro and micro levels; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a second convolutional network module for fusing spatial dimension features and temporal dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macro land use prediction module for predicting land use categories at a macro level and a micro land use prediction module for predicting land use categories at a micro level.

2. The multi-source data collaborative land use time series prediction method according to claim 1 is characterized in that: The step 1 comprises: Acquire historical spectral image data of a target area using multiple spectral channels of a remote sensing satellite or an aerial monitoring platform, wherein the historical spectral image data includes visible light spectral images and infrared spectral images, and is used to characterize different spectral characteristics of the target area; The historical laser point cloud data of the target area is collected by lidar to represent the surface height information of the target area.

3. The multi-source data collaborative land use time series prediction method according to claim 2 is characterized in that: The step 2 includes: Processing the historical laser point cloud data to generate a digital surface model for representing elevation and three-dimensional structural information of the surface; Performing registration and alignment processing on the digital surface model and the historical spectral image data to obtain registered historical spectral image data; The digital surface model is superimposed on the registered historical spectral image data as a geometric information channel, and spectral geometric mixed data is formed corresponding to each time node of the registered historical spectral image data.

4. The multi-source data collaborative land use time series prediction method according to claim 3 is characterized in that: The step 3 includes: Extracting horizontal and vertical edge features of the spectral geometric mixed data using a Sobel operator to obtain a plurality of first feature maps; Extracting second-order derivative features of the spectral geometric mixing data using a Laplacian operator to obtain a plurality of second feature maps; Decomposing the spectral geometric mixed data by using discrete wavelet decomposition to obtain a plurality of third characteristic graphs; Analyzing local texture features of the spectral geometric mixed data using local binary patterns to obtain a plurality of fourth feature maps; extracting edge features from the spectral geometric mixed data using an edge detection algorithm to obtain a plurality of fifth feature maps; Performing feature enhancement on the spectral geometric mixed data using a maximum noise separation transform to obtain a plurality of sixth feature maps; Multi-channel data integrating spectral information, spatial information and geometric information is generated based on all first feature maps, all second feature maps, all third feature maps, all fourth feature maps, all fifth feature maps and all sixth feature maps.

5. The multi-source data collaborative land use time series prediction method according to claim 4 is characterized in that: The step 4 comprises: Inputting the multi-channel data into the constructed fusion convolutional neural network; fusing the multi-channel data through the first convolutional layer in the first convolutional network module to obtain a plurality of first feature fusion results; Fusing all first feature fusion results through the second convolutional layer in the first convolutional network module to obtain multiple second feature fusion results; All second feature fusion results are fused through the third convolutional layer in the first convolutional network module to obtain spectral geometric features; Performing spatiotemporal fusion on the spectral geometric features through the first convolutional layer in the second convolutional network module to obtain a plurality of spatiotemporal fusion features; Fusing all the spatiotemporal fusion features through the second convolutional layer in the second convolutional network module to obtain multiple first spatiotemporal feature fusion results; All first spatiotemporal feature fusion results are processed by spatial dimension upgrading through the deconvolution layer in the second convolutional network module to obtain spectral geometric spatiotemporal fusion features.

6. The multi-source data collaborative land use time series prediction method according to claim 1, characterized in that: The loss function expression of the land use time series prediction network is: in, represents the total loss, Represents the image data dimension, represents the macro cross entropy loss, represents the microscopic cross entropy loss, Indicates the loss of macro-micro consistency.

7. The multi-source data collaborative land use time series prediction method according to claim 6, characterized in that: The functional expression of the macro-micro consistency loss is: in, represents relative entropy, Indicates a point The predicted distribution at the macro level, Indicates a point Predicted distribution at the micro level.

8. A land use time series prediction device based on multi-source data collaboration, characterized in that: include: An acquisition module is used to acquire historical spectral image data and historical laser point cloud data of the target area; a processing module, configured to convert the historical laser point cloud data into a digital surface model, and superimpose the historical spectral image data with the digital surface model to form spectral-geometry hybrid data; An extraction module is used to extract features from the spectral-geometric mixed data to generate multi-channel data that integrates spectral information, spatial information, and geometric information; A fusion module is used to fuse the multi-channel data into the constructed fusion convolutional neural network to obtain spectral geometric spatiotemporal fusion features; A prediction module is used to input the spectral geometric spatiotemporal fusion features into the trained land use time series prediction network to predict land use changes, and obtain land use prediction results of the target area at both macro and micro levels; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a second convolutional network module for fusing spatial dimension features and temporal dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macro land use prediction module for predicting land use categories at a macro level and a micro land use prediction module for predicting land use categories at a micro level.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the multi-source data collaborative land use time series prediction method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multi-source data collaborative land use time series prediction method according to any one of claims 1 to 7 is implemented.

Images