Multi-source data collaborative land utilization time sequence prediction method and related equipment

Abstract

The invention provides a multi-source data collaborative land utilization time series prediction method and related equipment, and the method comprises the steps: converting historical laser point cloud data of a target region into a digital surface model, and superposing the digital surface model with historical spectral image data, thereby forming spectral geometric mixed data; performing feature extraction on the spectral geometric mixed data to generate multi-channel data fused with spectral information, spatial information and geometric information; inputting multi-channel data into the constructed fusion convolutional neural network for fusion to obtain spectral geometric space-time fusion features; inputting the spectral geometric space-time fusion features into the trained land utilization time sequence prediction network to carry out land utilization change prediction, and obtaining land utilization prediction results of the target area on a macroscopic level and a microcosmic level; and the precision of land utilization time sequence prediction is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of land use prediction, and particularly to a land use time series prediction method and related devices that collaborate with multi-source data. Background Art

[0002] With the acceleration of the urbanization process, land use and land cover change have become one of the core issues in global environmental research. The spatio-temporal dynamic changes of land use are crucial for aspects such as natural resource management, ecological protection, urban planning, and environmental monitoring. Especially with the support of remote sensing data and lidar data, accurately predicting the spatio-temporal evolution of land use has important practical significance for decision-makers at all levels. However, there are still various challenges in the field of land use prediction currently.

[0003] Currently, the research methods for land use prediction mainly include prediction based on spectral information, prediction based on geometric information, and prediction based on the fusion of spectral and geometric information. Prediction based on spectral information usually uses satellite remote sensing images for land cover classification, such as analyzing through traditional machine learning methods such as support vector machine (SVM), decision tree, and random forest; prediction based on geometric information focuses on inferring surface features through the digital surface model (DSM) obtained by lidar, and then conducting land use prediction and classification; prediction based on the fusion of spectral and geometric information has gradually received attention. By combining the two, the characteristics of land use change can be obtained more comprehensively. For example, some researchers fuse remote sensing images with lidar (LIDAR) data and use deep learning methods such as convolutional neural network (CNN) for prediction in order to improve the prediction accuracy. However, when existing fusion methods process multi-modal data, they often do not have an effective spatio-temporal collaboration model and are difficult to fully utilize the time series change information, resulting in weak ability to capture time series dynamics during the prediction process.

[0004] Although the above existing solutions provide preliminary ideas for land use prediction, they have obvious deficiencies in dealing with multi-modal data, time series data, and spatio-temporal feature fusion. First, many studies lack effective feature extraction and fusion mechanisms, unable to fully utilize the complementarity between different data sources (such as spectral data and geometric data), resulting in limited expressive power of the model. Second, most existing prediction methods are limited to a single data processing stage and cannot take into account spectral features, geometric features, and spatio-temporal change information. Therefore, it is particularly crucial to construct a model that can extract and fuse features from multiple perspectives and levels.

[0005] In addition, existing Convolutional Neural Network (CNN) methods mainly focus on the processing of static images and have weak processing capabilities for dynamic time series data, making it difficult to effectively capture the time series information of land use changes. Temporal features play a crucial role in land use prediction because land use is affected by various factors and shows certain regularity over time. Existing methods lack sufficient spatio-temporal feature fusion mechanisms when dealing with these temporal changes, restricting their prediction capabilities for land use evolution. Summary of the Invention

[0006] The present invention provides a method and related device for land use time series prediction with multi-source data collaboration, aiming to improve the accuracy of land use time series prediction.

[0007] To achieve the above objective, the present invention provides a method for land use time series prediction with multi-source data collaboration, including: Step 1, obtaining historical spectral image data and historical lidar point cloud data of the target area; Step 2, converting the historical lidar point cloud data into a digital surface model, and superimposing the historical spectral image data with the digital surface model to form spectral geometric hybrid data; Step 3, extracting features from the spectral geometric hybrid data to generate multi-channel data fused with spectral information, spatial information, and geometric information; Step 4, inputting the multi-channel data into the constructed fusion convolutional neural network for fusion to obtain spectral geometric spatio-temporal fusion features; Step 5, inputting the spectral geometric spatio-temporal fusion features into the trained land use time series prediction network for land use change prediction to obtain land use prediction results at both the macroscopic and microscopic levels of the target area; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in the multi-channel data, and a first convolutional network module for fusing spatial dimension features and time dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

[0008] Furthermore, step 1 includes: Obtain historical spectral image data of the target area using multiple spectral channels of remote sensing satellites or aerial monitoring platforms. The historical spectral image data includes visible light spectral images and infrared spectral images, which are used to characterize different spectral characteristics of the target area; Collect historical lidar point cloud data of the target area, which is used to characterize the surface height information of the target area.

[0009] Furthermore, step 2 includes: Process the historical lidar point cloud data to generate a digital surface model, which is used to characterize the elevation and three-dimensional structure information of the ground surface; Perform registration and alignment processing on the digital surface model and the historical spectral image data to obtain the registered historical spectral image data; Superimpose the digital surface model as a geometric information channel onto the registered historical spectral image data and correspond to each time node of the registered historical spectral image data to form spectral geometric mixed data.

[0010] Furthermore, step 3 includes: Use the Sobel operator to extract the horizontal and vertical edge features of the spectral geometric mixed data to obtain multiple first feature maps; Use the Laplacian operator to extract the second derivative features of the spectral geometric mixed data to obtain multiple second feature maps; Use discrete wavelet decomposition to decompose the spectral geometric mixed data to obtain multiple third feature maps; Use local binary pattern to analyze the local texture features of the spectral geometric mixed data to obtain multiple fourth feature maps; Use edge detection algorithms to extract edge features in the spectral geometric mixed data to obtain multiple fifth feature maps; Use maximum noise separation transform to enhance the features of the spectral geometric mixed data to obtain multiple sixth feature maps; Generate multi-channel data that fuses spectral information, spatial information, and geometric information based on all the first feature maps, all the second feature maps, all the third feature maps, all the fourth feature maps, all the fifth feature maps, and all the sixth feature maps.

[0011] Furthermore, step 4 includes: Input the multi-channel data into the constructed fusion convolutional neural network; Fuse multi-channel data through the first convolutional layer in the first convolutional network module to obtain multiple first feature fusion results; Fuse all the first feature fusion results through the second convolutional layer in the first convolutional network module to obtain multiple second feature fusion results; Fuse all the second feature fusion results through the third convolutional layer in the first convolutional network module to obtain spectral geometric features; Perform spatio-temporal fusion on the spectral geometric features through the first convolutional layer in the second convolutional network module to obtain multiple spatio-temporal fusion features; Fuse all the spatio-temporal fusion features through the second convolutional layer in the second convolutional network module to obtain multiple first spatio-temporal feature fusion results; Perform upsampling in the spatial dimension on all the first spatio-temporal feature fusion results through the deconvolutional layer in the second convolutional network module to obtain spectral geometric spatio-temporal fusion features.

[0012] Furthermore, the expression of the loss function of the land use time series prediction network is: ; Wherein, represents the total loss, represents the dimension of the image data, represents the macro cross-entropy loss, represents the micro cross-entropy loss, represents the macro-micro consistency loss.

[0013] Furthermore, the function expression of the macro-micro consistency loss is: ; Wherein, represents the relative entropy, represents the point the predicted distribution at the macro level, represents the point the predicted distribution at the micro level.

[0014] The present invention also provides a land use time series prediction device for multi-source data collaboration, including: An acquisition module for acquiring historical spectral image data and historical laser point cloud data of a target area; A processing module for converting the historical laser point cloud data into a digital surface model and superimposing the historical spectral image data with the digital surface model to form spectral geometric mixed data; An extraction module for extracting features from the spectral geometric mixed data to generate multi-channel data fused with spectral information, spatial information, and geometric information; A fusion module, which is used to fuse multi-channel data by inputting it into a constructed fusion convolutional neural network to obtain spectral geometric spatio-temporal fusion features; A prediction module, which is used to input the spectral geometric spatio-temporal fusion features into a trained land use time series prediction network for land use change prediction to obtain land use prediction results at the macro and micro levels in the target area; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a first convolutional network module for fusing spatial dimension features and time dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

[0015] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a land use time series prediction method with multi-source data collaboration is implemented.

[0016] The present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a land use time series prediction method with multi-source data collaboration is implemented.

[0017] The above solution of the present invention has the following beneficial effects: In the present invention, the historical laser point cloud data of the target area is converted into a digital surface model and then superimposed with the historical spectral image data to form spectral geometric hybrid data; feature extraction is performed on the spectral geometric hybrid data to generate multi-channel data fused with spectral information, spatial information, and geometric information; the multi-channel data is input into the constructed fusion convolutional neural network for fusion to obtain spectral geometric spatio-temporal fusion features; the spectral geometric spatio-temporal fusion features are input into the trained land use time series prediction network for land use change prediction to obtain the land use prediction results at the macroscopic and microscopic levels of the target area; compared with the prior art, by fusing spectral data and laser point cloud data, the present invention generates spectral geometric hybrid data containing rich information, effectively improving the feature expression ability of the model and being able to comprehensively reflect the spatial changes and three-dimensional features of land use; through the combination of various feature extraction methods, the deep information in spectral data and laser point cloud data can be effectively mined; through the fusion convolutional neural network, the temporal changes of land use can be effectively captured; through the land use time series prediction network for land use change prediction, not only the large-scale changes of land use can be accurately predicted, but also the microscopic changes of land use can be predicted in detail, making the prediction results more detailed and accurate, thereby improving the accuracy of land use temporal prediction.

[0018] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic flowchart of an embodiment of the present invention; Figure 2 is a network architecture diagram in an embodiment of the present invention; Figure 3 is a schematic structural diagram of a land use time series prediction device in an embodiment of the present invention; Figure 4 is a schematic structural diagram of a terminal device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] In order to make the technical problems, technical solutions, and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] In the description of the present invention, it should be noted that the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0022] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0023] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] The present invention provides a multi-source data collaborative land use time series prediction method and related devices for existing problems.

[0025] As Figure 1 、 Figure 2 shown, an embodiment of the present invention provides a multi-source data collaborative land use time series prediction method, including: Step 1, obtaining historical spectral image data and historical laser point cloud data of a target area; Step 2, converting the historical laser point cloud data into a digital surface model, and superimposing the historical spectral image data and the digital surface model to form spectral geometric hybrid data; Step 3, performing feature extraction on the spectral geometric hybrid data to generate multi-channel data fused with spectral information, spatial information, and geometric information; Step 4, inputting the multi-channel data into a constructed fusion convolutional neural network for fusion to obtain spectral geometric spatio-temporal fusion features; Step 5, inputting the spectral geometric spatio-temporal fusion features into a trained land use time series prediction network for land use change prediction to obtain land use prediction results of the target area at both the macroscopic and microscopic levels; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in the multi-channel data, and a first convolutional network module for fusing spatial dimension features and time dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macroscopic land use prediction module for predicting land use categories at the macroscopic level and a microscopic land use prediction module for predicting land use categories at the microscopic level.

[0026] Specifically, step 1 includes: Using multiple spectral channels of a remote sensing satellite or an aerial monitoring platform to obtain historical spectral image data of the target area every month, and the dimension of the spectral image data of each spectral channel is , where and represent the number of rows and columns of the image. The historical spectral image data includes visible light spectral images and infrared spectral images, which are used to characterize the different spectral characteristics of the target area; The lidar is used to collect the lidar point cloud data of the target area, which is used to characterize the surface height information of the target area.

[0027] Since the spectral image data and the lidar point cloud data need to be fused to form comprehensive spectral geometric hybrid data, in this process, the lidar point cloud data needs to be superimposed on the data of each spectral channel to ensure that each pixel contains spatial information and spectral information.

[0028] Specifically, step 2 includes: Processing the historical lidar point cloud data to generate a digital surface model, which is used to characterize the elevation and three-dimensional structure information of the ground surface; Registering and aligning the digital surface model with the historical spectral image data to obtain the registered historical spectral image data; Superimposing the digital surface model as a geometric information channel onto the registered historical spectral image data and corresponding to each time node of the registered historical spectral image data to form spectral geometric hybrid data.

[0029] In the embodiment of the present invention, first, preprocessing operations including filtering, denoising, and classification are performed on the historical lidar point cloud data to extract effective ground and non-ground point cloud data. Based on these point cloud data, an interpolation algorithm is used to construct a digital surface model with the same spatial resolution as the spectral image, which is used to characterize the elevation undulation and three-dimensional structure information of the ground object surface; Next, preprocessing operations including geometric correction, radiometric calibration, and spatial registration are performed on the historical spectral image data to make it consistent with the generated digital surface model in the coordinate system, scale, and resolution, so as to achieve the unification of the data level; Subsequently, the multi-channel spectral image data of each time node is fused with the digital surface model corresponding to this time point. The fusion method is channel-level superposition, that is, the digital surface model is used as an additional geometric information channel and superimposed on the original spectral image to form enhanced image data containing multiple spectral channels and one geometric channel; Finally, the enhanced image data fused at all time series is sorted in chronological order to construct a spectral geometric hybrid data sequence under a continuous time scale. This spectral geometric hybrid data sequence not only carries the spectral response characteristics of the ground object at different times but also fuses its terrain geometric structure change information, providing more comprehensive input data support for subsequent multi-dimensional feature extraction and land use change prediction models.

[0030] Specifically, step 3 includes: Since the Sobel operator can be used to extract the edge features of an image and can identify the edge information in the horizontal and vertical directions of the image, the Sobel operator is used to extract the horizontal and vertical edge features of the spectral geometric mixed data, so as to effectively capture the object boundaries in the image and obtain multiple first feature maps. Among them, 2 first feature maps are generated for each spectral channel, and the first feature map is an edge feature map; Since the Laplacian operator is mainly used to extract the second derivative features of an image to enhance the performance of image details, the Laplacian operator is used to extract the second derivative features of the spectral geometric mixed data to enhance the image details in the spectral geometric mixed data and obtain multiple second feature maps. Among them, 1 second feature map is generated for each spectral channel, and the second feature map is a Laplacian feature map. These feature maps can significantly enhance the details in the image, enabling the subsequent network to better identify complex land use types; The spectral geometric mixed data is decomposed by using discrete wavelet decomposition, and the image in the spectral geometric mixed data is decomposed into low-frequency subband and high-frequency subband features to obtain multiple third feature maps. Among them, 8 third feature maps are generated for each spectral channel, and the third feature map is a discrete wavelet decomposition feature map. These feature maps can capture the multi-scale information in the image and help improve the sensitivity to land use changes; Since the local binary pattern is mainly used to extract the local texture information of an image, especially being highly sensitive to the texture transformation of the image, the local binary pattern is used to analyze the local texture features of the spectral geometric mixed data to obtain multiple fourth feature maps. Among them, 1 fourth feature map is generated for each spectral channel, and the fourth feature map is a local texture feature map. These feature maps can help the model analyze the structural changes in the image at the local level and identify subtle land use changes; The edge detection algorithm is used to extract edge features in the spectral geometric mixed data to obtain multiple fifth feature maps. Among them, 1 fifth feature map is generated for each spectral channel, and the fifth feature map is an edge feature map. These features help identify the boundaries of ground objects and classify different types of land use; Since the maximum noise separation transform is mainly used for image noise reduction and signal enhancement, and through the maximum noise separation transform, noise and useful signals can be separated to improve the quality of image data, the maximum noise separation transform is used to enhance the features of the useful signals in the spectral geometric mixed data to obtain multiple sixth feature maps. Among them, 1 sixth feature map is generated for each spectral channel, and the sixth feature map is an enhanced feature map. These feature maps help improve the robustness and accuracy of the model; Generate multi-channel data that fuses spectral information, spatial information, and geometric information based on all the first feature maps, all the second feature maps, all the third feature maps, all the fourth feature maps, all the fifth feature maps, and all the sixth feature maps.

[0031] In the embodiments of the present invention, through the above-mentioned multiple feature extraction methods, a total of multi-channel data that fuses spectral information, spatial information, and geometric information are generated, significantly enriching the dimension and information volume of feature expression. At this time, the dimension of the time series data for Y months is .

[0032] In the embodiments of the present invention, the edge detection algorithm is the Canny edge detection algorithm, which is mainly used to enhance the edge information in the image, so as to extract the edge features in the image.

[0033] Furthermore, step 4 includes: Input the multi-channel data into the constructed fusion convolutional neural network; Fuse the multi-channel data through the first convolutional layer in the first convolutional network module to obtain multiple first feature fusion results. This first convolutional layer uses 7 convolutional kernels, and the size of each convolutional kernel is , and the padding method adopts Same Padding. Its main function is to fuse multi-channel data to generate 7 first feature fusion results, with a dimension of ; Fuse all the first feature fusion results through the second convolutional layer in the first convolutional network module to obtain multiple second feature fusion results. This second convolutional layer uses 4 convolutional kernels, and the size of each convolutional kernel is 7×3×3. It also adopts Same Padding. Its main function is to further fuse the 7 first feature fusion results to generate 4 second feature fusion results, with a dimension of ; Fuse all the second feature fusion results through the third convolutional layer in the first convolutional network module to obtain the spectral geometric feature. This third convolutional layer uses 1 convolutional kernel, and the size of each convolutional kernel is 4×2×2. The padding method is Same Padding. Its main function is to fuse the 4 second feature fusion results again to generate 1 spectral geometric feature, and the dimension becomes ; Perform spatio-temporal fusion on the spectral geometric feature through the first convolutional layer in the second convolutional network module to obtain multiple spatio-temporal fusion features. This first convolutional layer uses 20 convolutional kernels, and the size of each convolutional kernel is , and the padding method adopts Same Padding. This layer performs spatio-temporal fusion on the spectral geometric feature to generate 20 spatio-temporal fusion features, with a dimension of ; Fuse all spatio-temporal fusion features through the second convolutional layer in the second convolutional network module to obtain multiple first spatio-temporal feature fusion results. The second convolutional layer uses 15 convolutional kernels, each with a size of 20×2×2, and the padding method is no padding. This layer fuses 20 spatio-temporal fusion features to generate 15 first spatio-temporal feature fusion results, with a dimension of ; Perform upsampling on all first spatio-temporal feature fusion results in the spatial dimension through the deconvolution layer in the second convolutional network module to obtain spectral geometric spatio-temporal fusion features. The deconvolution layer uses 12 deconvolutional kernels, each with a size of 15×2×2, a stride of 2, and the padding method is Valid Padding. This layer upsamples 15 first spatio-temporal feature fusion results in the spatial dimension to generate 12 spectral geometric spatio-temporal fusion features, with a dimension of .

[0034] Based on the output of the fusion convolutional neural network in the embodiments of the present invention, a land use time series prediction network is constructed to predict the macro and micro land use conditions of the next month pixel by pixel. The network includes two parts: macro land use prediction and micro land use prediction; among them, the macro land use prediction module is a fully connected network, which is used to use the spectral geometric spatio-temporal fusion features with a dimension of as input to predict the macro land use category for each pixel, map it to D macro categories, and use the macro cross-entropy loss function to calculate the loss; the micro land use prediction module is also a fully connected network, which is used to use the spectral geometric spatio-temporal fusion features with a dimension of as input to predict the micro land use category for each pixel, map it to L micro categories, and use the micro cross-entropy loss function to calculate the loss.

[0035] In the embodiments of the present invention, although the macro and micro prediction tasks have different objectives, their output categories may have certain hierarchical relationships. Therefore, it is necessary to introduce a penalty term to make the prediction results of the macro task and the metoclopramide task as consistent as possible at the same position. Especially when the macro category belongs to a certain major category, the prediction result of the micro category should conform to this major category. Therefore, the relationship between them can be strengthened by calculating the soft label consistency or cross-entropy loss of the macro and metoclopramide prediction categories. The function expression of the macro-micro consistency loss is: ; Among them, represents the macro-micro consistency loss, represents the relative entropy, represents the point prediction distribution at the macro level, Representation point The predicted distribution at the micro level.

[0036] Specifically, the loss function expression of the land use time series prediction network is: ; Among them, represents the total loss, represents the dimension of the image data, represents the macro cross-entropy loss, represents the micro cross-entropy loss.

[0037] The land use time series prediction network is trained through the above loss function to optimize the network parameters. The network is trained using the gradient descent algorithm to optimize the loss function L. The network weights are continuously adjusted through multiple data cycles (epochs), and the Adam optimizer is used to accelerate convergence. During the training process, the data is divided into a training set and a validation set to evaluate the generalization ability of the network and avoid overfitting.

[0038] Next, the embodiments of the present invention verify the provided method in combination with specific data, and the process is as follows: Multispectral remote sensing data and LiDAR laser point cloud data are selected. The multispectral remote sensing data used comes from the Sentinel-2 satellite, providing 13 spectral bands including blue, green, red, near-infrared, etc., with a spatial resolution of 10 meters; the LiDAR laser point cloud data is obtained through a laser point cloud scanner to generate a digital surface model (DSM), with a spatial resolution of 10 meters and a size of 256×256 pixels; The obtained multispectral smoke sensor data is processed such as geometric correction and radiometric correction to ensure data quality; To improve the prediction accuracy, the multispectral remote sensing data and the digital surface model of each month are superimposed to form a spectral geometric mixed data with channels. Specifically, the 13 spectral bands of the multispectral remote sensing data are superimposed with the ground elevation information of the DSM to generate 14-channel data; For the 14-channel data of each month, the following feature extraction method is adopted: Sobel operator: Extract horizontal and vertical edge information of the image to obtain 2 edge feature maps for each channel; Laplacian operator: Extract second derivative features to enhance image details and obtain 1 Laplacian feature map for each channel; Discrete Wavelet Transformation (DWT): Decompose the image into low-frequency and high-frequency sub-band features and generate 8 DWT feature maps for each channel; Local Binary Patterns (LBP): Used to analyze local texture features of the image and generate 1 LBP feature map for each channel; Canny edge detection: Extract edges from the image and generate 1 Canny edge feature map for each channel; Minimum Noise Fraction (MNF) transformation: Used to reduce noise and enhance useful signals and generate 1 MNF feature map for each channel; Through these methods, a total of feature maps are finally generated, with a total of 182 channels, providing rich input data for the subsequent fusion convolutional neural network; Input the multi-channel data into the constructed fusion convolutional neural network for spatial and temporal information fusion to obtain spectral geometric spatio-temporal fusion features; Input the spectral geometric spatio-temporal fusion features into the trained land use time series prediction network for land use change prediction. The obtained macro categories include 6 major categories, with 3 micro-categories under each major category, totaling 18 micro-categories.

[0039] The experimental results are as follows: After training with the data of the first 10 months as input, the land use time series prediction network successfully predicted the land use situation in the 11th month.

[0040] The performance comparison results between the present invention and the comparative method in time series prediction are shown in Table 1 below: Table 1 Performance comparison table ; As can be seen from the above table, the method provided by the embodiment of the present invention has achieved high accuracy in the prediction task. The final overall correct rate is 90.4%. This result proves that the method can effectively improve the accuracy and reliability of land use prediction, especially in complex urban environments and changing land use scenarios, and has broad application prospects.

[0041] In the embodiment of the present invention, the historical lidar point cloud data of the target area is converted into a digital surface model and then superimposed with the historical spectral image data to form spectral geometric hybrid data; feature extraction is performed on the spectral geometric hybrid data to generate multi-channel data fused with spectral information, spatial information, and geometric information; the multi-channel data is input into the constructed fusion convolutional neural network for fusion to obtain spectral geometric spatio-temporal fusion features; the spectral geometric spatio-temporal fusion features are input into the trained land use time series prediction network for land use change prediction, and the land use prediction results at both the macroscopic and microscopic levels of the target area are obtained; compared with the prior art, in the embodiment of the present invention, by fusing spectral data and lidar point cloud data, spectral geometric hybrid data containing rich information is generated, effectively improving the feature expression ability of the model and being able to comprehensively reflect the spatial changes and three-dimensional features of land use; through the combination of various feature extraction methods, the deep information in spectral data and lidar point cloud data can be effectively mined; the temporal changes of land use are effectively captured through the fusion convolutional neural network; by using the land use time series prediction network for land use change prediction, not only the large-scale changes in land use can be accurately predicted, but also the microscopic changes in land use can be predicted in detail, making the prediction results more detailed and accurate, thereby improving the accuracy of land use time series prediction.

[0042] Corresponding to the multi-source data collaborative land use time series prediction method described in the above embodiment, as Figure 3 shown, the embodiment of the present invention further provides a multi-source data collaborative land use time series prediction device 100, and the land use time series prediction device 100 includes: An acquisition module 101, configured to acquire the historical spectral image data and historical lidar point cloud data of the target area; A processing module 102, configured to convert the historical lidar point cloud data into a digital surface model and superimpose the historical spectral image data with the digital surface model to form spectral geometric hybrid data; An extraction module 103, configured to perform feature extraction on the spectral geometric hybrid data to generate multi-channel data fused with spectral information, spatial information, and geometric information; A fusion module 104, configured to input the multi-channel data into the constructed fusion convolutional neural network for fusion to obtain spectral geometric spatio-temporal fusion features; A prediction module 105, configured to input the spectral geometric spatio-temporal fusion features into the trained land use time series prediction network for land use change prediction to obtain the land use prediction results at both the macroscopic and microscopic levels of the target area; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a first convolutional network module for fusing spatial dimension features and temporal dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

[0043] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units, due to being based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details will not be elaborated here.

[0044] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment, and details will not be elaborated here.

[0045] The embodiment of the present invention also provides a terminal device, such as Figure 4 shown. The terminal device D10 in this embodiment includes: at least one processor D100 ( Figure 4 only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the above-mentioned multi-source data collaborative land use time series prediction method is implemented.

[0046] The terminal device D10 can be a computing device such as a desktop computer, a notebook, a palm computer, a server, a server cluster, and a cloud server. The terminal device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art can understand, Figure 4This is only an example of the terminal device D10, which does not constitute a limitation on the terminal device D10. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0047] The so-called processor D100 may be a central processing unit (CPU, Central Processing Unit). This processor D100 may also be other general-purpose processors, digital signal processors (DSP, Digital Signal Processor), application specific integrated circuits (ASIC, Application Specific Integrated Circuit), field-programmable gate arrays (FPGA, Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0048] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as the hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart media card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both the internal storage unit and the external storage device of the terminal device D10. The memory D101 is used to store an operating system, application programs, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or will be output.

[0049] It should be noted that for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, please refer to the method embodiment part specifically, and details will not be repeated here.

[0050] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0051] The present invention also provides a computer-readable storage medium storing a computer program, which when executed by a processor implements a multi-source data collaborative land use time series prediction method.

[0052] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of the present application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the construction device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc.

[0053] The above is the preferred implementation manner of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle described in the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for predicting land use time series with multi-source data collaboration, characterized in that, Including: Step 1: Obtain historical spectral image data and historical lidar point cloud data of the target area; Step 2: Convert the historical lidar point cloud data into a digital surface model, and superimpose the historical spectral image data on the digital surface model to form spectral geometric hybrid data; Step 3: Extract features from the spectral geometric hybrid data to generate multi-channel data fused with spectral information, spatial information, and geometric information; Step 4: Input the multi-channel data into a constructed fusion convolutional neural network for fusion to obtain spectral geometric spatio-temporal fusion features; Step 5: Input the spectral geometric spatio-temporal fusion features into a trained land use time series prediction network for land use change prediction to obtain land use prediction results at both the macroscopic and microscopic levels of the target area; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a first convolutional network module for fusing spatial dimension features and temporal dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macroscopic land use prediction module for predicting land use categories at the macroscopic level and a microscopic land use prediction module for predicting land use categories at the microscopic level.

2. The method for predicting the land use time series with multi-source data collaboration according to claim 1, wherein The said Step 1 includes: Obtain historical spectral image data of the target area using multiple spectral channels of a remote sensing satellite or an aerial monitoring platform. The historical spectral image data includes visible light spectral images and infrared spectral images, which are used to characterize different spectral characteristics of the target area; Collect historical lidar point cloud data of the target area through lidar, which is used to characterize the surface height information of the target area.

3. The method for predicting the land use time series with multi-source data collaboration according to claim 2, wherein The said Step 2 includes: Process the historical lidar point cloud data to generate a digital surface model, which is used to characterize the elevation and three-dimensional structure information of the ground surface; Perform registration and alignment processing on the digital surface model and the historical spectral image data to obtain registered historical spectral image data; Superimpose the digital surface model as a geometric information channel onto the registered historical spectral image data, and form spectral geometric hybrid data corresponding to each time node of the registered historical spectral image data.

4. The method for predicting the land use time series with multi-source data collaboration according to claim 3, wherein The said Step 3 includes: Use the Sobel operator to extract the horizontal and vertical edge features of the spectral geometric hybrid data to obtain multiple first feature maps; Use the Laplacian operator to extract the second derivative features of the spectral geometric hybrid data to obtain multiple second feature maps; Use discrete wavelet decomposition to decompose the spectral geometric hybrid data to obtain multiple third feature maps; Use local binary pattern to analyze the local texture features of the spectral geometric hybrid data to obtain multiple fourth feature maps; Use an edge detection algorithm to extract edge features in the spectral geometric hybrid data to obtain multiple fifth feature maps; Use the maximum noise fraction transform to enhance the features of the spectral geometric hybrid data to obtain multiple sixth feature maps; Generate multi-channel data that fuses spectral information, spatial information, and geometric information based on all the first feature maps, all the second feature maps, all the third feature maps, all the fourth feature maps, all the fifth feature maps, and all the sixth feature maps.

5. The method for predicting the land use time series with multi-source data collaboration according to claim 4, characterized in that Step 4 includes: Input the multi-channel data into the constructed fusion convolutional neural network; Fuse the multi-channel data through the first convolutional layer in the first convolutional network module to obtain multiple first feature fusion results; Fuse all the first feature fusion results through the second convolutional layer in the first convolutional network module to obtain multiple second feature fusion results; Fuse all the second feature fusion results through the third convolutional layer in the first convolutional network module to obtain spectral geometric features; Perform spatio-temporal fusion on the spectral geometric features through the first convolutional layer in the second convolutional network module to obtain multiple spatio-temporal fusion features; Fuse all the spatio-temporal fusion features through the second convolutional layer in the second convolutional network module to obtain multiple first spatio-temporal feature fusion results; Perform upsampling in the spatial dimension on all the first spatio-temporal feature fusion results through the deconvolutional layer in the second convolutional network module to obtain spectral geometric spatio-temporal fusion features.

6. The method for predicting the land use time series with multi-source data collaboration according to claim 1, characterized in that The expression of the loss function of the land use time series prediction network is: Among them, represents the total loss, represents the dimension of the image data, represents the macro cross-entropy loss, represents the micro cross-entropy loss, represents the macro-micro consistency loss.

7. The method for predicting the land use time series with multi-source data collaboration according to claim 6, wherein The function expression of the macro-micro consistency loss is: Among them, represents the relative entropy, represents the point of the predicted distribution at the macroscopic level, represents the point of the predicted distribution at the microscopic level.

8. A land use time series prediction device for multi-source data collaboration, characterized in that, It includes: An acquisition module for acquiring historical spectral image data and historical lidar point cloud data of a target area; A processing module for converting the historical lidar point cloud data into a digital surface model and superimposing the historical spectral image data with the digital surface model to form spectral geometric mixed data; An extraction module for extracting features from the spectral geometric mixed data to generate multi-channel data that fuses spectral information, spatial information, and geometric information; A fusion module for inputting the multi-channel data into the constructed fusion convolutional neural network for fusion to obtain spectral geometric spatio-temporal fusion features; A prediction module for inputting the spectral geometric spatio-temporal fusion features into the trained land use time series prediction network for land use change prediction to obtain the land use prediction results of the target area at both the macro and micro levels; The fusion convolutional neural network includes a first convolutional network module for fusing spectral information and geometric information in multi-channel data, and a second convolutional network module for fusing spatial dimension features and time dimension features in the spectral geometric features output by the first convolutional network module; The land use time series prediction network includes a macro land use prediction module for predicting land use categories at the macro level and a micro land use prediction module for predicting land use categories at the micro level.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multi-source data collaborative land use time series prediction method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-source data collaborative land use time series prediction method according to any one of claims 1 to 7.

Images