Land use classification method based on intelligent fusion of multi-source data

Abstract

This invention discloses a land use classification method based on intelligent fusion of multi-source data. It acquires multi-source geospatial data of the study area, including remote sensing imagery, street view imagery, building vector data, road network vector data, points of interest (POIs), mobile phone signaling data, and Weibo check-in data, forming a land use classification experimental dataset. It constructs remote sensing imagery features, street view imagery features, built environment features, POIs, and population distribution features for each study unit, forming a feature set for the land use classification task. The constructed features are paired to generate feature pairs composed of different modalities. A contrastive learning method is used to align the features of different modalities, and the aligned features are added to the feature set for the land use classification task. Land use classification is performed based on an XGBoost ensemble learning model, and five-fold cross-validation is used to verify the model's effectiveness. This invention improves the accuracy of land use classification.

Description

Technical Field

[0001] This invention relates to multi-source data fusion analysis and application, specifically to a land use classification method based on intelligent fusion of multi-source data. Background Technology

[0002] Urban land use classification information reflects the spatial distribution and interaction patterns of different urban functions, providing an important basis for studying how human activities and ecological factors shape the spatial structure of cities. It plays a crucial role in urban management, government decision-making, and monitoring of human activities. With the acceleration of urbanization and the rapid expansion of urban scale, a series of urban problems, such as traffic congestion and housing shortages, have emerged, resulting in diverse and complex urban functional zones. Efficient and accurate identification of land use types is essential for effective urban management and planning.

[0003] Traditional land use classification relies primarily on land use planning maps and questionnaires. However, survey-based methods are expensive and slow; besides the costs of manpower and time, the reliability of the results is also significantly affected by subjective factors such as time, location, and the investigator's personal experience. With the continuous advancement of remote sensing technology and the advent of the big data era, high-resolution remote sensing imagery and various types of socially-sensory big data are becoming increasingly readily available. Data from different sources carry different information and can fully characterize the physical and socio-economic features of geospatial space. Integrating geospatial data from various sources can yield more comprehensive and complete information, thereby enhancing the reliability and accuracy of land use classification results.

[0004] Currently, many researchers from various fields are dedicated to studying data fusion techniques, aiming to utilize complementary information from multiple data sources to obtain an accurate overall description of the data, thereby more effectively serving tasks such as decision-making, prediction, and classification. Related research shows that methods combining remote sensing imagery and socially perceived data for urban land use classification can significantly improve performance compared to classification methods based on a single data source. However, existing research typically focuses on pixel-level features of remote sensing images and textual features of points of interest (POIs), without fully considering the correlations between different regions of the remote sensing image and the spatial distribution characteristics of POIs. Furthermore, the extracted multimodal features are not aligned before fusion, failing to fully reflect the information interaction between different modal features and ignoring the correlations and patterns between different data sources. Summary of the Invention

[0005] The purpose of this invention is to propose a land use classification method based on intelligent fusion of multi-source data, in order to solve the problems that existing land use classification methods do not fully extract the global spatial correlation features of remote sensing image data and the spatial distribution features of point of interest data, and do not perform feature alignment for multimodal data.

[0006] The technical solution for achieving the present invention is: a land use classification method based on intelligent fusion of multi-source data, comprising the following steps:

[0007] S1: Obtain the road network vector data of the study area, and divide the study area into several block plots by performing line-to-surface conversion on the road network vector data, which serve as study units;

[0008] S2: Acquire multi-source geospatial data of the study area, including remote sensing image data, street view image data, building vector data, road network vector data, point of interest data, mobile phone signaling data, and Weibo check-in data, to form a land use classification experimental dataset, which is used to construct different characteristics of the study units;

[0009] S3: Based on the multi-source geospatial data in the land use classification experimental dataset, construct the remote sensing image features, street view image features, built environment features, points of interest features, and population distribution features for each research unit to form the feature set for the land use classification task.

[0010] S4: Combine the various features constructed in S3 into pairs to generate feature pairs composed of features of different modalities. Align the features of different modalities using a contrastive learning method, and add the aligned features to the feature set of the land use classification task.

[0011] S5: Input all features in the land use classification task feature set into the XGBoost ensemble learning model for land use classification, and use five-fold cross-validation to test the effectiveness of the model.

[0012] Furthermore, S3: Based on the multi-source geospatial data in the land use classification experimental dataset, remote sensing image features, street view image features, built environment features, points of interest features, and population distribution features are constructed for each research unit, including:

[0013] Remote sensing image features include pixel-level features and global correlation features;

[0014] Built environment characteristics include building attribute characteristics, building distribution characteristics, and transportation facility distribution characteristics;

[0015] Interest point features include interest point category features and distribution features;

[0016] Population distribution characteristics include the distribution characteristics of the existing population and the distribution characteristics of the active population.

[0017] Furthermore, the remote sensing image features of each research unit are constructed using the following method:

[0018] 1) Extract pixel-level features from remote sensing images using a semantic segmentation model;

[0019] DeeplabV3+ was chosen as the semantic segmentation model to extract pixel-level features from remote sensing images. DeeplabV3+ employs an encoder-decoder structure. The encoder consists of two modules: a deep convolutional neural network (DCNN) and a dilated spatial convolutional pooling pyramid (ASPP). The DCNN module uses the Xception_65 backbone network to extract features, while the ASPP module processes the features output by the DCNN module by applying dilated convolutions with different dilation rates to detect convolutional features at multiple scales. After the encoder, the decoder first performs channel dimensionality reduction on the encoder features using 1×1 convolutions, followed by bilinear upsampling with a sampling coefficient of 4. The sampled result is then concatenated with low-level features from the encoder backbone network that have undergone 1×1 convolutions and have the same spatial resolution. Finally, a 3×3 convolution is applied to the concatenated features, and the convolution result is upsampled with a sampling coefficient of 4, resulting in a segmentation result of the same size as the input image, which serves as the pixel-level feature of the remote sensing image.

[0020] 2) Construct global correlation features of remote sensing images using a mask autoencoder model;

[0021] The MAE model is selected to extract global association features from remote sensing images. The MAE model consists of an encoder and a decoder. The input image is first segmented into regular, non-overlapping blocks, and a certain proportion of the blocks are randomly occluded. The encoder only encodes the uncovered blocks. After the encoder, occlusion markers are placed on the occluded blocks and sent to the decoder along with the uncovered blocks for processing. The decoder reconstructs the pixels of the original image. Based on the remote sensing image data of the study area, the pre-trained MAE model is fine-tuned in a classification task. The image features extracted by the encoder will be used as global association features of the remote sensing images.

[0022] Furthermore, the street view image features for each research unit are constructed using the following method:

[0023] Street view image data of the study area were input into a ResNet-50 network pre-trained on the Places2 dataset for scene classification. The proportion of street view images of different scene categories was counted for each block, and the street view image features of the blocks were constructed.

[0024] Furthermore, the built environment characteristics of each research unit are constructed, including building attribute characteristics, building distribution characteristics, and transportation facility distribution characteristics. The specific method is as follows:

[0025] Based on the building vector data of the study area, the average and standard deviation of the building perimeter, area, number of floors, number of control points, and year of construction of each block are calculated, and these are used as the building attribute characteristics of the block.

[0026] Calculate the plot ratio and the nearest neighbor distance of buildings within the block to serve as the building distribution characteristics of the block;

[0027]

[0028] Where R represents the plot ratio characteristic of the block, N represents the total number of buildings within the block, and a i f represents the footprint of the i-th building. i Let A represent the number of floors in the i-th building, and let A represent the area of ​​the block.

[0029]

[0030] Where D represents the nearest neighbor distance characteristic of buildings within a block, N represents the total number of buildings within a block, and d i This represents the distance from the i-th building to the nearest building;

[0031] The road network density near the plots is calculated based on the road network vector data of the study area, which serves as a characteristic of the distribution of transportation facilities in the block plots;

[0032]

[0033] Where T represents the road network density characteristics of the block, L represents the total length of roads within the block and its buffer zone, and A represents the total area of ​​the block and its buffer zone.

[0034] Furthermore, interest feature characteristics are constructed for each research unit, including interest category features and distribution features. The specific method is as follows:

[0035] First, the interest point data of the study area is classified using an interest point classification system. The density and proportion of each type of interest point within each block are calculated to construct the interest point category features of the block. Then, based on the location information of the interest points, a Delaunay triangulation network is constructed, in which an explicit random walk is performed to capture the category co-occurrence sequence of interest points. Subsequently, a category encoder is trained to obtain the category embedding vector of each interest point. Next, the Delaunay triangulation network with the interest point category embedding vector of each block is input into a multilayer perceptron (MLP) to obtain a memory vector. The memory vector is then repeatedly input into a long short-term memory neural network (LSTM) to update the memory state, thereby using an LSTM pointer network to obtain the memory vector with the best order. Finally, the memory vector containing the semantic information of the spatial distribution of interest points is output as the interest point distribution feature within the block.

[0036] Furthermore, the population distribution characteristics of each research unit are constructed using the following method:

[0037] First, based on mobile signaling data, the distribution characteristics of the existing population are constructed. The number of mobile signaling samples per hour in a day for each block is calculated to generate a time spectrum curve of the existing population. Then, a one-dimensional convolutional neural network is used to extract the deep features of the time spectrum curve of the existing population and output the corresponding feature vector. The time spectrum curve of the existing population and its corresponding feature vector are used as the distribution characteristics of the existing population.

[0038] Secondly, based on the Weibo check-in data with geographic tags, we construct the distribution characteristics of the active population. We calculate the number of Weibo check-ins for each block within one day in hours and within one week in days to generate two active population time spectrum curves. We then use a one-dimensional convolutional neural network to extract the deep features of the active population time spectrum curves in hours and output the corresponding feature vectors. The two active population time spectrum curves and the feature vectors corresponding to the active population time spectrum curves in hours are used as the distribution characteristics of the active population.

[0039] Furthermore, S4: The constructed features are combined in pairs to generate feature pairs composed of features from different modalities. A contrastive learning method is used to align the features from different modalities, and the aligned features are then added to the feature set for the land use classification task. The specific method is as follows:

[0040] For each feature pair, first select several research units as a group, construct multiple positive and negative sample pairs, take the data features of different modalities from the same research unit as positive sample pairs, and take the data features of different modalities from different research units as negative sample pairs;

[0041] Subsequently, two multilayer perceptrons (MLPs) are constructed to project features from two different modalities onto the same dimension. After projection, the cosine similarity between features is calculated to generate a similarity matrix. The optimal parameters of the two MLPs are learned using gradient descent. The optimization objective is to maximize the cosine similarity between positive sample pairs and minimize the cosine similarity between negative sample pairs.

[0042] Finally, the two optimized MLPs will output aligned features of the same dimension. Due to the potential correlation between features, the effect of feature alignment will be weakened. Therefore, based on the optimized similarity matrix, the aligned features of feature pairs with similarity below the threshold are selected and added to the feature set of the land use classification task.

[0043] A land use classification system based on intelligent fusion of multi-source data is provided, which implements the land use classification method based on intelligent fusion of multi-source data to achieve land use classification based on intelligent fusion of multi-source data.

[0044] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the land use classification method based on intelligent fusion of multi-source data, thereby realizing land use classification based on intelligent fusion of multi-source data.

[0045] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the land use classification method based on intelligent fusion of multi-source data is implemented to achieve land use classification based on intelligent fusion of multi-source data.

[0046] Compared with existing technologies, the significant advantages of this invention are as follows: It employs a Mask Autoencoder (MAE) model to extract global dependencies between different image patches in remote sensing images. Simultaneously, based on a Delaunay triangulation network for points of interest, it uses a random walk strategy and a Long Short-Term Memory (LSTM) neural network to obtain the distribution characteristics of these points, enhancing the feature representation of both remote sensing image data and point-of-interest data, thus effectively improving the accuracy of land use classification. Furthermore, it uses contrastive learning for multimodal feature alignment before fusion, utilizing the similarity or difference between different samples to learn feature representations. This not only improves the quality of single-modal representations but also enhances the classification effect after multimodal fusion. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the present invention;

[0048] Figure 2 This is a diagram showing the comparative analysis results of land use classification accuracy in the example. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] A land use classification method based on intelligent fusion of multi-source data includes the following steps:

[0051] S1: Obtain the road network vector data of the study area, and divide the study area into several block plots by performing line-to-surface conversion on the road network vector data, which serve as study units;

[0052] Road networks naturally divide urban space, forming relatively independent land parcel units. These units often correspond to different urban functions, and the land use types and intensities within the same parcel are usually quite consistent, facilitating land use classification and analysis. Therefore, this invention uses street blocks as the research unit.

[0053] S2: Acquire multi-source geospatial data of the study area, including remote sensing image data, street view image data, building vector data, road network vector data, point of interest data, mobile phone signaling data, Weibo check-in data, etc., to form a land use classification experimental dataset, which is used to construct different characteristics of the study unit;

[0054] S3: Based on multi-source geospatial data in the land use classification experimental dataset, five types of features are constructed for each research unit: remote sensing image features, street view image features, built environment features, points of interest features, and population distribution features, forming a feature set for the land use classification task. Among these, remote sensing image features include pixel-level features and global correlation features; built environment features include building attribute features, building distribution features, and transportation facility distribution features; points of interest features include point of interest category features and distribution features; and population distribution features include existing population distribution features and active population distribution features.

[0055] S31: Construct remote sensing image features for research units based on remote sensing image data, including pixel-level features and global correlation features of remote sensing images;

[0056] Remote sensing image data of the study area is input into the DeepLabV3+ semantic segmentation model for pixel-level land cover classification, and pixel-level features of the remote sensing images are extracted. Remote sensing image data of more areas are obtained to pre-train the mask autoencoder (MAE) model. Then, the model is fine-tuned in the classification task based on the remote sensing image data of the study area to learn the correlation between image patches in different areas of the remote sensing images. Finally, the image features of the encoder are used as global association features of the remote sensing images.

[0057] S311: Extract pixel-level features from remote sensing images using a semantic segmentation model;

[0058] This invention selects DeeplabV3+ as the semantic segmentation model to extract pixel-level features from remote sensing images. It follows a classic encoder-decoder structure, which is widely used in semantic segmentation tasks. These encoder-decoder networks consist of an encoder module that gradually reduces the feature map and captures higher-dimensional semantic information, and a decoder module that gradually recovers spatial information. Semantic segmentation tasks can effectively identify image features of pixels and their neighborhoods, making it suitable as a proxy task for local feature extraction from remote sensing images.

[0059] The DeeplabV3 encoder consists of two modules: a Deep Convolutional Neural Network (DCNN) and a Dilated Spatial Convolutional Pooling Pyramid (ASPP). In this invention, the DCNN module uses the Xception_65 backbone network to extract features. The Xception_65 backbone network replaces all max-pooling operations in the original Xception with depthwise separable convolutions to obtain more detailed local features. Furthermore, to accelerate network convergence and avoid overfitting, batch normalization and ReLU activation are added after every 3×3 depthwise convolutions. The ASPP module processes the features output by the DCNN module, probing convolutional features at multiple scales by applying dilated convolutions with different dilation rates.

[0060] Following the encoder, the decoder first performs channel dimensionality reduction on the encoder features using 1×1 convolution, followed by bilinear upsampling with a sampling coefficient of 4. The sampled result is then concatenated with low-level features from the encoder backbone network that have undergone 1×1 convolution and have the same spatial resolution. Finally, a 3×3 convolution is applied to the concatenated features, and the convolution result is upsampled with a sampling coefficient of 4, resulting in a segmentation result of the same size as the input image, which serves as the pixel-level feature of the remote sensing image.

[0061] S312: Construct global correlation features of remote sensing images using a mask autoencoder model;

[0062] This invention selects the Masking Effect (MAE) model to extract global correlation features from remote sensing images. The MAE model consists of an encoder and a decoder, and its core idea is mask learning, which involves randomly masking certain parts of the input image and attempting to reconstruct the masked pixels. The input image is first segmented into regular, non-overlapping blocks, and a certain proportion of these blocks are randomly masked. The encoder only encodes the uncovered blocks. The encoder model structure adapted to this process is the Vision Transformer (ViT), whose core idea is to segment the image into a series of fixed-size blocks and treat them as elements in a sequence. The Transformer model is used to process these sequences, thereby capturing the global dependencies between image blocks. After the encoder, masking markers are placed on the masked blocks, and together with the uncovered blocks, they are sent to a lightweight decoder for processing. The decoder attempts to reconstruct the pixels of the original image. In this process, the model can learn the correlations between blocks at different locations, which is an effective way to represent global spatial correlations. The features extracted by the encoder can serve as global correlation features of the image. Therefore, this invention fine-tunes the pre-trained MAE model in a classification task based on remote sensing image data of the study area, and uses the image features extracted by the encoder in this process as global association features of the remote sensing image.

[0063] S32: Constructing street view image features for the research unit based on street view image data;

[0064] Street view imagery, as an important supplement to remote sensing imagery, can enrich the physical characteristics of geospatial space. However, limited by acquisition equipment, the visible range of street view imagery is limited, and it contains a lot of noise irrelevant to land use classification tasks. Therefore, this invention considers scene category identification as a proxy task to extract image subject features and filter noise. Street view imagery data of the study area is input into a ResNet-50 network pre-trained on the Places2 dataset for scene classification. The proportion of street view images of different scene categories is counted for each block, and the street view imagery features of the blocks are constructed.

[0065] S33: Construct the built environment characteristics of the research unit based on building vector data and road network vector data, including building attribute characteristics, building distribution characteristics, and transportation facility distribution characteristics;

[0066] Based on the building vector data of the study area, the average and standard deviation of attributes such as building perimeter, area, number of floors, number of control points, and year of construction within each block are calculated and used as the building attribute characteristics of the block. The plot ratio and the nearest neighbor distance of buildings within the block are calculated according to the following formulas to represent the building distribution characteristics of the block:

[0067]

[0068] Where R represents the plot ratio characteristic, N represents the total number of buildings within the plot, and a i f represents the footprint of the i-th building. i Let A represent the number of floors in the i-th building, and let A represent the area of ​​the plot.

[0069]

[0070] Where D represents the nearest neighbor distance characteristic of buildings within the plot, N represents the total number of buildings within the plot, and d i This represents the distance from the i-th building to the nearest building.

[0071] The road network density near the plot is calculated based on the road network vector data of the study area, serving as a characteristic of the plot's transportation facility distribution.

[0072]

[0073] Where T represents the road network density characteristics of the land parcel, L represents the total length of roads within the land parcel and its buffer zone, and A represents the total area of ​​the land parcel and its buffer zone.

[0074] S34: Construct interest point features for research units based on interest point data, including interest point category features and distribution features;

[0075] Since the density and proportion of various points of interest in different types of land parcels often vary, reflecting the differences in the types and intensity of human activities carried by the land parcels, this invention classifies the point of interest data of the study area according to a common point of interest classification system, and constructs the point of interest category characteristics of the land parcels by calculating the density of various types of points of interest and the proportion of different types of points of interest in each land parcel.

[0076] To extract the distribution features of points of interest (POIs) within land parcels, the following steps are first taken: First, the category information corresponding to each PPI needs to be obtained, and a Delaunay triangulation network is constructed to delineate the PPIs. Explicit random walks are then performed within this network to capture the co-occurrence sequences of PPI categories. A category encoder is then trained to obtain the category embedding vector for each PPI. Next, the Delaunay triangulation network containing the PPI category embedding vectors for each land parcel is input into a multilayer perceptron (MLP) to obtain a memory vector. This memory vector is then repeatedly fed into a long short-term memory (LSTM) neural network to update the memory state. An LSTM pointer network is then used to obtain memory vectors with the optimal order. Finally, the memory vector containing the semantic information of the spatial distribution of PPIs is output as the distribution features of PPIs within the land parcel.

[0077] S35: Construct the population distribution characteristics of the research unit based on mobile phone signaling data and Weibo check-in data, including the distribution characteristics of the existing population and the distribution characteristics of the active population;

[0078] Among them, mobile phone signaling data is used to extract the distribution characteristics of the existing population. The specific steps are as follows: First, calculate the number of mobile phone signaling samples per hour for each block within a day to generate a time spectrum curve of the existing population; Second, use a one-dimensional convolutional neural network to extract the deep features of the time spectrum curve of the existing population and output the corresponding feature vector; Finally, add the time spectrum curve of the existing population and its corresponding feature vector as the distribution characteristics of the existing population to the feature set of the land use classification task.

[0079] Geographically tagged Weibo check-in data is used to extract active population distribution features. The specific steps are as follows: First, calculate the number of Weibo check-ins for each block within one day (hourly) and within one week (daily) to generate two active population time-spectrum curves. Second, use a one-dimensional convolutional neural network to extract the deep features of the active population time-spectrum curves within one hour and output the corresponding feature vectors. Finally, add the two active population time-spectrum curves and the feature vectors corresponding to the active population time-spectrum curves within one hour as active population distribution features to the feature set of the land use classification task.

[0080] S4: Combine the various features constructed in S3 in pairs to generate feature pairs composed of features from different modalities. Align the features of different modalities using a contrastive learning method, and add the aligned features to the feature set for the land use classification task;

[0081] For each feature pair, firstly, several research units are selected as a group to construct multiple positive and negative sample pairs. Data features from different modalities within the same research unit are used as positive sample pairs, and data features from different modalities within different research units are used as negative sample pairs. Then, two multilayer perceptrons (MLPs) are constructed to project features from two different modalities onto the same dimension (the projection method is determined by the parameters of the MLPs). After projection, the cosine similarity between features is calculated to generate a similarity matrix. The optimal parameters of the two MLPs are learned using gradient descent, with the optimization objective being to maximize the cosine similarity between positive sample pairs and minimize the cosine similarity between negative sample pairs. Finally, the two optimized MLPs will output aligned features of the same dimension. Due to the potential correlation between features, the effect of feature alignment is weakened. Therefore, based on the optimized similarity matrix, it can be selected whether to add the aligned features of the current feature pair to the feature set of the land use classification task.

[0082] S5: Input all features in the land use classification task feature set into the XGBoost ensemble learning model for land use classification, and use five-fold cross-validation to test the effectiveness of the model.

[0083] This invention also proposes a land use classification system based on intelligent fusion of multi-source data, implementing the aforementioned land use classification method based on intelligent fusion of multi-source data to achieve land use classification based on intelligent fusion of multi-source data.

[0084] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the land use classification method based on intelligent fusion of multi-source data, thereby realizing land use classification based on intelligent fusion of multi-source data.

[0085] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the land use classification method based on intelligent fusion of multi-source data is implemented to achieve land use classification based on intelligent fusion of multi-source data.

[0086] Example

[0087] To verify the effectiveness of the present invention, the following experiment was conducted.

[0088] Figure 1 The flowchart of the present invention is presented, and the specific implementation steps are as follows:

[0089] S1: Obtain the road network vector data of the study area, and divide the study area into several block plots by performing line-to-surface conversion on the road network vector data, which serve as study units.

[0090] In this embodiment, the vector data of the road network within the Fifth Ring Road of Beijing was processed by ArcGIS software to convert lines into polygons, resulting in a study area composed of street blocks, which contains a total of 12,197 street blocks.

[0091] S2: Acquire multi-source geospatial data of the study area, including remote sensing image data, street view image data, building vector data, road network vector data, point of interest data, mobile phone signaling data, Weibo check-in data, etc., to form a land use classification experimental dataset, which is used to construct different characteristics of the study units.

[0092] In this embodiment, the remote sensing image data is sourced from historical Google Earth imagery, with a resolution of 1.2 meters, dated 2019; the street view data is sourced from Tencent Street View, dated 2018, totaling 660,000 images; the building vector data records the geometry of 161,000 buildings within Beijing's Fifth Ring Road, dated 2018, including attributes such as perimeter, area, number of floors, number of control points, and year of construction; the road network vector data is sourced from OpenStreetMap, dated 2019, describing the geometry and spatial location of the road network in the form of vector lines; the point of interest (POI) data is sourced from Baidu Maps, dated 2019, containing coordinates, categories, names, and other information for 284,000 POIs within Beijing's Fifth Ring Road; the Weibo check-in data records the coordinates of 5.82 million check-in locations within Beijing's Fifth Ring Road from 2012 to 2017; and the mobile signaling data records the sampling quantity per hour for each grid within Beijing's Fifth Ring Road after it was divided into 250-meter grids, with a sampling time of 2018.

[0093] S3: Based on multi-source geospatial data in the land use classification experimental dataset, five types of features are constructed for each research unit: remote sensing image features, street view image features, built environment features, points of interest features, and population distribution features. The remote sensing image features include pixel-level features and global correlation features; the built environment features include building attribute features, building distribution features, and transportation facility distribution features; the points of interest features include point of interest category features and distribution features; and the population distribution features include existing population distribution features and active population distribution features.

[0094] S31: Construct remote sensing image features for the research unit based on remote sensing image data, including pixel-level features and global correlation features of the remote sensing image.

[0095] Remote sensing image data of the study area is input into the DeepLabV3+ semantic segmentation model for pixel-level land cover classification, and pixel-level features of the remote sensing images are extracted. Remote sensing image data of more areas are obtained to pre-train the mask autoencoder (MAE) model. Then, the model is fine-tuned in the classification task based on the remote sensing image data of the study area to learn the correlation between image patches in different areas of the remote sensing images. Finally, the image features of the encoder are used as global association features of the remote sensing images.

[0096] It should be noted that step S31 specifically includes:

[0097] S311: Extract pixel-level features from remote sensing images using a semantic segmentation model.

[0098] DeepLabV3+ was chosen as the semantic segmentation model to extract pixel-level features from remote sensing images. It follows the classic encoder-decoder structure, which is widely used in semantic segmentation tasks. During the training phase of this example, nine land cover categories were defined, including impermeable surfaces, buildings, low vegetation, trees, vehicles, debris / background, roads, water, and soil. The parameters of the Xception network pre-trained on the ImageNet-1k dataset were used as the initial parameters for the DeepLabV3+ backbone network. A stochastic gradient descent optimizer with a momentum of 0.9 was used during training, employing a poly learning rate strategy with a base learning rate of 0.0001 and a power of 0.9, for a total of 400k iterations. Finally, the land cover type percentage for each study unit was calculated as the local feature of that unit.

[0099] S312: Constructing global correlation features of remote sensing images using a mask autoencoder model.

[0100] The MAE model was chosen to extract global association features from remote sensing images. In the pre-training phase of this example, remote sensing images of built-up areas in Beijing, Shanghai, Guangzhou, and Shenzhen were selected, and random locations were cropped to construct a pre-training image dataset. In generating pre-training samples, considering adaptation to subsequent fine-tuning tasks, the empirical cumulative distribution function (ECDF) of the shape (length, aspect ratio) of the study units in Beijing was calculated. New samples were generated through inverse ECDF sampling to ensure that the shape of these samples followed the distribution of the original data. The encoder structure was chosen as vit_base_patch16, the mask rate was set to 0.75, the optimizer was Adam, and optimization strategies such as learning rate warmup were employed. In the fine-tuning phase of this example, since the task type changed from image completion to land use classification, the decoder component in MAE was discarded, and the encoder ViT architecture was used to classify the remote sensing images of the study area. Simultaneously, to ensure that the training and test sets were not confused, a classification model was fine-tuned for each of the five splits in step S5's five-fold cross-validation. After training, the image features extracted by the encoder in the classification task were used as the global association features of the remote sensing images.

[0101] S32: Constructing street view image features for the research unit based on street view image data

[0102] In this embodiment, a ResNet-50 network pre-trained on the Places2 dataset is selected as the classification network to identify the 3D scene of street view images. Applying this network yields a 365-dimensional output, representing the probability that a given input street view image belongs to one of 365 predefined scene types. The scene type with the highest probability is defined as the scene type of the street view image, and the proportion of street views of different scene types is statistically analyzed for each research unit as the street view feature of the plot.

[0103] S33: Construct the built environment characteristics of the research unit based on building vector data and road network vector data, including building attribute characteristics, building distribution characteristics, and transportation facility distribution characteristics.

[0104] Based on the building vector data of the study area, the average value and standard deviation of the building perimeter, area, number of floors, number of control points, and year of construction of each block are calculated and used as the building attribute characteristics of the block. Based on the geometric information of the block and the building vector data, the plot ratio and the nearest neighbor distance of the buildings within the block are calculated as the building distribution characteristics of the block. Based on the road network vector data of the study area, the road network density near the block is calculated as the transportation facility distribution characteristics of the block.

[0105] S34: Construct interest point features for research units based on interest point data, including interest point category features and distribution features;

[0106] The interest point category features of each plot are constructed by calculating the density and proportion of various interest points within each plot. In this embodiment, referencing Baidu Maps' classification system for interest points, a category standard for interest points was established, including 23 categories such as residential areas, companies and enterprises, education and training, office buildings, medical facilities, parking lots, shopping, restaurants, life services, universities, government agencies, car services, hotels, leisure and entertainment, beauty and wellness, sports, finance, culture and media, tourist attractions, transportation facilities, kindergartens, primary and secondary schools, research institutions, and others. After calculating the density and proportion of each type of interest point in each plot, a 46-dimensional feature vector is output as the interest point category feature of the plot.

[0107] To extract the distribution features of points of interest (POIs) in a land parcel, it is first necessary to obtain the category information corresponding to each PPO and construct a Delaunay triangulation network for the PPOs. Explicit random walks are then performed within this network to capture the category co-occurrence sequences of PPOs. Subsequently, a category encoder is trained to obtain the category embedding vector for each PPO. In this embodiment, ten random walks are performed on each PPO to obtain ten PPO sequences. An PPO category encoder is then constructed based on these sequences. The SkipGram model is used to train the category encoder to obtain the category embedding vector for each PPO. This vector contains both the spatial location relationships and category information of the PPOs.

[0108] Next, a Delaunay triangulation network containing interest point category embedding vectors for each plot is input into a multilayer perceptron (MLP) to obtain memory vectors. These memory vectors are then repeatedly fed into a long short-term memory neural network (LSTM) to update the memory state, thereby using an LSTM pointer network to obtain memory vectors with the optimal order. Finally, the output memory vectors containing semantic information about the spatial distribution of interest points serve as the interest point distribution features within the plots.

[0109] The corresponding feature vectors are output; finally, the two active population time spectrum curves and the hourly active population data are combined to construct the population distribution characteristics of the research unit, including the distribution characteristics of the existing population and the distribution characteristics of the active population.

[0110] In this embodiment, mobile phone signaling data is used to extract the distribution characteristics of the existing population, and Weibo check-in data with geographic tags is used to extract the distribution characteristics of the active population.

[0111] Specifically, the method for extracting the distribution characteristics of the existing population is as follows:

[0112] First, the number of mobile phone signaling samples taken hourly within a day for each block is calculated to generate a time spectrum curve of the existing population. Second, a one-dimensional convolutional neural network is used to extract the deep features of the time spectrum curve of the existing population and output the corresponding feature vector. Finally, the time spectrum curve of the existing population and its corresponding feature vector are added to the feature set of the land use classification task as the distribution features of the existing population.

[0113] The method for extracting the distribution characteristics of the active population is as follows:

[0114] First, the number of Weibo check-ins for each block is calculated in hours within a day and in days within a week to generate two active population time spectrum curves. Second, a one-dimensional convolutional neural network is used to extract the feature vectors corresponding to the active population time spectrum curves in hours, which are then added to the feature set of the land use classification task as active population distribution features.

[0115] S4: Combine the various features constructed in S3 in pairs to generate feature pairs composed of features from different modalities. Align the features of different modalities using a contrastive learning method, and add the aligned features to the feature set for the land use classification task;

[0116] In this embodiment, firstly, the various features output from S3 are paired to form 10 feature pairs. Secondly, each group of 10 research units is divided into multiple positive and negative sample pairs. Data features from different modalities within the same research unit are used as positive sample pairs, and data features from different modalities within different research units are used as negative sample pairs. Subsequently, two multilayer perceptrons (MLPs) are constructed to project features from two different modalities onto the same dimension (the projection method is determined by the MLP parameters). After projection, the cosine similarity between features is calculated to generate a similarity matrix. The optimal parameters of the two MLPs are learned using gradient descent, with the optimization objective being to maximize the cosine similarity between positive sample pairs and minimize the cosine similarity between negative sample pairs. Finally, the optimized two MLPs output aligned features of the same dimension. Due to the potential correlation between features, which weakens the effect of feature alignment, this embodiment selects aligned features of remote sensing image features and point of interest features with low similarity based on the optimized similarity matrix and adds them to the feature set of the land use classification task.

[0117] S5: Input all features in the land use classification task feature set into the XGBoost ensemble learning model for land use classification, and use five-fold cross-validation to test the effectiveness of the model.

[0118] In this embodiment, referring to the predefined land use classification standard in the "Classification of Current Land Use" (GB / T 21010-2017), eight land use types were defined for the selected study area: commercial and financial, public facilities, industrial and mining storage, transportation facilities, natural land, education and research, residential land, and agricultural land. Based on the predefined classification standard, the land use type of each plot was semi-automatically labeled, and training and testing sets were constructed. The labeling process was based on the land use planning map provided by the Beijing Municipal Commission of Planning and Natural Resources. First, the planning map was mapped according to the granularity of block plots, and the area proportion of each land use type was calculated. If the largest proportion exceeded 60%, or the largest proportion exceeded 40% and the second largest proportion was less than 20%, then a label was assigned based on the dominant land use type; otherwise, labels were manually assigned based on remote sensing imagery and points of interest.

[0119] Based on the characteristics of the task, the XGBoost ensemble learning model was chosen for land use classification. Due to the uneven distribution of data across land use categories, XGBoost employs a boosting sampling method, allowing the model to pay more attention to misclassified samples in categories with fewer samples, thus maximizing the accuracy of each category. Furthermore, XGBoost performs well with sparse data, making it suitable for scenarios with sparse, fine-grained plot data. In the training and testing phases of this example, all constructed features were input into the XGBoost classifier, and five-fold cross-validation was used to verify the model's performance. After multiple parameter tunings, the final parameter values ​​for the XGBoost model were set as follows: maximum depth of the classification tree (max_depth) = 7; learning rate (eta) = 0.1; and number of iterations (num_round) = 1000. In this example, the overall classification accuracy for the eight predefined land use types was 84.46%.

[0120] To further evaluate the beneficial effects of this invention, the proposed XGboost-based land use classification method is compared with three other methods widely used in land use classification: Multilayer Perceptron (MLP), Support Vector Machine (SVM), and Random Forest (RF). The land use classification accuracy is also compared under three conditions: considering only socio-economic features, considering only physical features, and integrating both socio-economic and physical features (all-data). Figure 2 The results of a comparative analysis of the accuracy of land use classification are presented.

[0121] Table 1. Confusion Matrix of Land Use Classification Results in Examples

[0122]

[0123] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A land use classification method based on intelligent fusion of multi-source data, characterized in that, Includes the following steps: S1: Obtain the road network vector data of the study area, and divide the study area into several block plots by performing line-to-surface conversion on the road network vector data, which serve as study units; S2: Acquire multi-source geospatial data of the study area, including remote sensing image data, street view image data, building vector data, road network vector data, point of interest data, mobile phone signaling data, and Weibo check-in data, to form a land use classification experimental dataset, which is used to construct different characteristics of the study units; S3: Based on the multi-source geospatial data in the land use classification experimental dataset, construct the remote sensing image features, street view image features, built environment features, points of interest features, and population distribution features for each research unit to form the feature set for the land use classification task. S4: Combine the various features constructed in S3 into pairs to generate feature pairs composed of features from different modalities. Align the features of different modalities using a contrastive learning method, and add the aligned features to the feature set for the land use classification task. The specific method is as follows: For each feature pair, first select several research units as a group, construct multiple positive and negative sample pairs, take the data features of different modalities from the same research unit as positive sample pairs, and take the data features of different modalities from different research units as negative sample pairs; Subsequently, two multilayer perceptrons (MLPs) were constructed to project features from two different modalities onto the same dimension. After projection, the cosine similarity between features was calculated to generate a similarity matrix. The optimal parameters of the two MLPs were learned using gradient descent. The optimization objective was to maximize the cosine similarity between positive sample pairs and minimize the cosine similarity between negative sample pairs. Finally, the two optimized MLPs will output alignment features of the same dimension. Due to the potential correlation between features, the effect of feature alignment will be weakened. Therefore, based on the optimized similarity matrix, the alignment features of feature pairs with similarity below the threshold are selected and added to the feature set of the land use classification task. S5: Input all features in the land use classification task feature set into the XGBoost ensemble learning model for land use classification, and use five-fold cross-validation to test the effectiveness of the model.

2. The land use classification method based on intelligent fusion of multi-source data according to claim 1, characterized in that, S3: Based on multi-source geospatial data from the land use classification experimental dataset, remote sensing image features, street view image features, built environment features, points of interest features, and population distribution features are constructed for each research unit, including: Remote sensing image features include pixel-level features and global correlation features; Built environment characteristics include building attribute characteristics, building distribution characteristics, and transportation facility distribution characteristics; Interest point features include interest point category features and distribution features; Population distribution characteristics include the distribution characteristics of the existing population and the distribution characteristics of the active population.

3. The land use classification method based on intelligent fusion of multi-source data according to claim 2, characterized in that, The remote sensing image features for each research unit are constructed using the following method: 1) Extract pixel-level features from remote sensing images using a semantic segmentation model; DeeplabV3+ was chosen as the semantic segmentation model to extract pixel-level features from remote sensing images. DeeplabV3+ employs an encoder-decoder structure. The encoder consists of two modules: a deep convolutional neural network (DCNN) and a dilated spatial convolutional pooling pyramid (ASPP). The DCNN module uses the Xception_65 backbone network to extract features, while the ASPP module processes the features output by the DCNN module by applying dilated convolutions with different dilation rates to detect convolutional features at multiple scales. After the encoder, the decoder first performs channel dimensionality reduction on the encoder features using 1 × 1 convolutions, followed by bilinear upsampling with a sampling coefficient of 4. The sampled results are then concatenated with low-level features from the encoder backbone network that have undergone 1 × 1 convolutions and have the same spatial resolution. Finally, a 3 × 3 convolution is applied to the concatenated features, and the convolution result is upsampled with a sampling coefficient of 4 to obtain a segmentation result of the same size as the input image, which is used as the pixel-level feature of the remote sensing image. 2) Construct global correlation features of remote sensing images using a mask autoencoder model; The MAE model is selected to extract global association features from remote sensing images. The MAE model consists of an encoder and a decoder. The input image is first segmented into regular, non-overlapping blocks, and a certain proportion of the blocks are randomly occluded. The encoder only encodes the uncovered blocks. After the encoder, occlusion markers are placed on the occluded blocks and sent to the decoder along with the uncovered blocks for processing. The decoder reconstructs the pixels of the original image. Based on the remote sensing image data of the study area, the pre-trained MAE model is fine-tuned in a classification task. The image features extracted by the encoder will be used as global association features of the remote sensing images.

4. The land use classification method based on intelligent fusion of multi-source data according to claim 2, characterized in that, The street view image features for each research unit are constructed using the following method: Street view image data of the study area were input into a ResNet-50 network pre-trained on the Places2 dataset for scene classification. The proportion of street view images of different scene categories was counted for each block, and the street view image features of the blocks were constructed.

5. The land use classification method based on intelligent fusion of multi-source data according to claim 2, characterized in that, The built environment characteristics of each research unit are constructed, including building attribute characteristics, building distribution characteristics, and transportation facility distribution characteristics. The specific method is as follows: Based on the building vector data of the study area, the average and standard deviation of the building perimeter, area, number of floors, number of control points, and year of construction of each block are calculated, and these are used as the building attribute characteristics of the block. Calculate the plot ratio and the nearest neighbor distance of buildings within the block to serve as the building distribution characteristics of the block; ； Where R represents the plot ratio of the block, and N represents the total number of buildings within the block. Let represent the floor area of ​​the i-th building. Let A represent the number of floors in the i-th building, and let A represent the area of ​​the block. ； Where D represents the nearest neighbor distance characteristic of buildings within a block, and N represents the total number of buildings within a block. This represents the distance from the i-th building to the nearest building; The road network density near the plots is calculated based on the road network vector data of the study area, which serves as a characteristic of the distribution of transportation facilities in the block plots; ； Where T represents the road network density characteristics of the block, L represents the total length of roads within the block and its buffer zone, and A represents the total area of ​​the block and its buffer zone.

6. The land use classification method based on intelligent fusion of multi-source data according to claim 2, characterized in that, Construct the interest point features for each research unit, including interest point category features and distribution features. The specific method is as follows: First, the interest point data of the study area is classified and processed by combining the interest point classification system. The density and proportion of various interest points in each block are calculated to construct the interest point category characteristics of the block. Then, based on the location information of the points of interest, a Delaunay triangulation network is constructed to perform explicit random walks in the network to capture the co-occurrence sequence of the categories of the points of interest. Subsequently, a category encoder is trained to obtain the category embedding vector of each point of interest. Next, the Delaunay triangulation network with the category embedding vector of the points of interest for each plot is input into a multilayer perceptron (MLP) to obtain a memory vector. The memory vector is then repeatedly input into a long short-term memory (LSTM) neural network to update the memory state, thereby using an LSTM pointer network to obtain memory vectors with the best order. Finally, the memory vector containing the semantic information of the spatial distribution of the points of interest is output as the distribution feature of the points of interest within the plot.

7. The land use classification method based on intelligent fusion of multi-source data according to claim 2, characterized in that, The population distribution characteristics of each research unit are constructed using the following method: First, based on mobile signaling data, the distribution characteristics of the existing population are constructed. The number of mobile signaling samples per hour in a day for each block is calculated to generate a time spectrum curve of the existing population. Then, a one-dimensional convolutional neural network is used to extract the deep features of the time spectrum curve of the existing population and output the corresponding feature vector. The time spectrum curve of the existing population and its corresponding feature vector are used as the distribution characteristics of the existing population. Secondly, based on the Weibo check-in data with geographic tags, we construct the distribution characteristics of the active population. We calculate the number of Weibo check-ins for each block within one day in hours and within one week in days to generate two active population time spectrum curves. We then use a one-dimensional convolutional neural network to extract the deep features of the active population time spectrum curves in hours and output the corresponding feature vectors. The two active population time spectrum curves and the feature vectors corresponding to the active population time spectrum curves in hours are used as the distribution characteristics of the active population.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the land use classification method based on intelligent fusion of multi-source data as described in any one of claims 1-7, thereby realizing land use classification based on intelligent fusion of multi-source data.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the land use classification method based on intelligent fusion of multi-source data as described in any one of claims 1-7, thereby realizing land use classification based on intelligent fusion of multi-source data.

Images