Multi-source remote sensing data-based urban functional area fine classification method and system

Abstract

The invention discloses an urban functional area fine classification method and system based on multi-source remote sensing data. The method comprises the following steps: acquiring a remote sensing data set of an urban functional area; preprocessing the remote sensing data set of the urban functional area to obtain a preprocessed remote sensing data set; performing feature extraction and feature splicing on the preprocessed remote sensing data set to obtain shallow layer splicing data features, middle layer splicing data features and deep layer splicing data features; performing data fusion on the shallow-layer splicing data features, the middle-layer splicing data features and the deep-layer splicing data features by using a multi-dimensional feature fusion model to obtain shallow-layer fusion features, middle-layer fusion features and deep-layer fusion features; and classifying the shallow splicing data features, the shallow fusion features, the middle fusion features and the deep fusion features by using a multi-scale feature recombination classification model to obtain a functional region classification result. The functional area division method is high in accuracy and high in functional area updating speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing data processing, and more particularly, to a method and system for fine classification of urban functional areas based on multi-source remote sensing data. BACKGROUND

[0002] The division of urban functional areas has become increasingly important with the development of low-altitude economy. Only by fully considering the distribution and attributes of road network, green network, water network, residential area, industrial area, commercial area, and other urban functional areas can the planning of low-altitude air routes avoid various potential risk factors and maximize the use of limited air route resources.

[0003] Traditional methods of identifying urban functional areas rely on manual surveys and other methods, so there is often a delay of several months from data collection to completion of functional area division, and the accuracy of the functional area division result quickly decays with the progress of urban renewal. Although the use of a single type of remote sensing data can also achieve rapid updating of urban functional area division, because there are many factors affecting urban functional areas, the use of a single type of remote sensing data cannot accurately divide urban functional areas.

[0004] The prior art discloses a method for identifying urban functional areas based on remote sensing image feature classification, which specifically comprises the following steps: S1: selecting a data source and obtaining remote sensing image data of a selected area; S2: preprocessing the remote sensing image data obtained in step S1, including image rectification, image fusion, image color uniformity, and image inlay; S3: using the preprocessed remote sensing image in step S2 as a data source, training a deep learning semantic segmentation network, and extracting a feature classification image; S4: using open operation and close operation in mathematical morphology on the feature classification image extracted in step S3; S5: constructing an urban functional area identification regression model; and S6: obtaining an urban functional area identification result. The method uses relatively single remote sensing data, so the accuracy of the urban functional areas divided by the method is low. SUMMARY

[0005] The present application provides a method for fine classification of urban functional areas based on multi-source remote sensing data to overcome the defects of low accuracy and slow updating speed of functional area division in the prior art.

[0006] A method for fine classification of urban functional areas based on multi-source remote sensing data, comprising: S1: obtaining a set of remote sensing data of urban functional areas; S2: preprocessing the set of remote sensing data of urban functional areas to obtain a set of preprocessed remote sensing data; S3: feature extraction and feature splicing are performed on the preprocessed remote sensing data set to obtain shallow spliced data features, middle spliced data features and deep spliced data features; S4: a multi-dimensional feature fusion model is used to perform data fusion on the shallow spliced data features, the middle spliced data features and the deep spliced data features to obtain shallow fusion features, middle fusion features and deep fusion features; S5: a multi-scale feature reorganization classification model is used to classify the shallow spliced data features, the shallow fusion features, the middle fusion features and the deep fusion features to obtain a city functional area classification result.

[0007] Further, the remote sensing data set of the city functional area includes multi-modal remote sensing data, and in step S2, the preprocessing includes geometric correction, projection unification, missing value interpolation and normalization processing.

[0008] Further, in step S3, the feature extraction and feature splicing on the preprocessed remote sensing data set include: S301: selecting one type of remote sensing data in the preprocessed remote sensing data set as first remote sensing data; S302: inputting the first remote sensing data into a first feature extraction model to obtain first shallow data features, first middle data features and first deep data features; S303: inputting the first shallow data features into a second feature extraction model to obtain second shallow data features; inputting the first middle data features into the second feature extraction model to obtain second middle data features; and inputting the first deep data features into the second feature extraction model to obtain second deep data features; S304: placing the second shallow data features into a shallow data feature group; placing the second middle data features into a middle data feature group; and placing the second deep data features into a deep data feature group; S305: selecting another type of remote sensing data in the preprocessed remote sensing data set as new first remote sensing data, and repeatedly performing steps S302-S304 until all remote sensing data in the preprocessed remote sensing data set are selected; S306: splicing the data features in the shallow data feature group to obtain shallow spliced data features; splicing the data features in the middle data feature group to obtain middle spliced data features; and splicing the data features in the deep data feature group to obtain deep spliced data features.

[0009] Further, the second feature extraction model includes a channel adaptation module, a basic sampling network module, an offset prediction convolution module, a deformable sampler, a first convolution layer and a first addition point. The output end of the channel adaptation module is connected with the input end of the offset prediction convolution module and the input end of the deformable sampler, the output end of the basic sampling network module and the output end of the offset prediction convolution module are connected with the input end of the first adding point, the output end of the first adding point is connected with the input end of the deformable sampler, and the output end of the deformable sampler is connected with the input end of the first convolution layer.

[0010] Further, the calculation formula of the deformable sampler is as follows:

[0011] the coordinates of the convolution center position of the deformable sampler, the output feature of the deformable sampler, the input feature of the deformable sampler, N represents the total number of features, and n represents the feature serial number, the convolution kernel weight parameter of the feature n, the convolution kernel neighborhood position coordinates of the feature n, the convolution kernel spatial offset of the feature n.

[0012] Further, the multi-dimensional feature fusion model comprises a channel gating module, a spatial gating module, a separable convolution module, a first multiplying point, a second adding point and a feature weight module. The output end of the channel gating module is connected with the input end of the separable convolution module, the output end of the separable convolution module is connected with the input end of the feature weight module, the output end of the feature weight module and the output end of the spatial gating module are connected with the input end of the first multiplying point, and the output end of the first multiplying point is connected with the input end of the second adding point.

[0013] Further, the multi-scale feature reorganization classification model comprises a shallow feature reorganization module, a middle feature reorganization module, a deep feature reorganization module and a classification module. The shallow layer spliced data feature and the shallow layer fusion feature are input into the input end of the shallow feature reorganization module, the middle layer fusion feature is input into the input end of the middle feature reorganization module, and the deep layer fusion feature is input into the input end of the deep feature reorganization module. The output end of the shallow feature reorganization module, the output end of the middle feature reorganization module and the output end of the deep feature reorganization module are connected with the input end of the classification module, and the classification module outputs a city function area classification result.

[0014] Further, the shallow feature reorganization module comprises a second convolution layer, a third adding point, a third convolution layer, a batch normalization layer and a first activation layer. The shallow layer spliced data feature inputs an input end of a second convolutional layer, the shallow layer fusion feature inputs an input end of a third convolutional layer, an output end of the third convolutional layer is connected with an input end of a batch normalization layer, an output end of the batch normalization layer is connected with an input end of a first activation layer, an output end of the first activation layer and an output end of the second convolutional layer are connected with an input end of a third adding point, and the third adding point outputs a shallow layer reorganization feature; The middle layer feature reorganization module comprises a spatial mapping layer, a second activation layer, an attention weighting layer, a second multiplication point and a fourth adding point. The middle layer fusion feature respectively inputs an input end of the spatial mapping layer and an input end of the fourth adding point, an output end of the spatial mapping layer is connected with an input end of the second activation layer and an input end of the second multiplication point respectively, an output end of the second activation layer is connected with an input end of the attention weighting layer, an output end of the attention weighting layer is connected with an input end of the second multiplication point, an output end of the second multiplication point is connected with an input end of the fourth adding point, and an output end of the fourth adding point outputs a middle layer reorganization feature. The deep layer feature reorganization module comprises a third multiplication point, an average pooling layer, a first linear layer, a third activation layer, a second linear layer and a weight fusion layer. The deep layer fusion feature respectively inputs an input end of the third multiplication point and an input end of the average pooling layer, an output end of the average pooling layer is connected with an input end of the first linear layer, an output end of the first linear layer is connected with an input end of the third activation layer, an output end of the third activation layer is connected with an input end of the second linear layer, an output end of the second linear layer is connected with an input end of the weight fusion layer and an input end of the third multiplication point, an output end of the third multiplication point is connected with an input end of the weight fusion layer, and an output end of the weight fusion layer outputs a deep layer reorganization feature.

[0015] Further, the classification module comprises a feature splicing layer, an up-sampling layer and a fourth activation layer. The shallow layer reorganization feature, the middle layer reorganization feature and the deep layer reorganization feature input an input end of the feature splicing layer, an output end of the feature splicing layer is connected with an input end of the fourth activation layer, and an output end of the fourth activation layer outputs a city functional area classification result.

[0016] A city functional area fine classification system based on multi-source remote sensing data comprises: A data acquisition module acquires a remote sensing data set of a city functional area; A preprocessing module pre-processes the remote sensing data set of the city functional area to obtain a pre-processed remote sensing data set; A feature extraction and feature splicing module extracts and splices features of the pre-processed remote sensing data set to obtain shallow layer spliced data features, middle layer spliced data features and deep layer spliced data features; The data fusion module: a multi-dimensional feature fusion model is used to perform data fusion on the shallow layer spliced data features, the middle layer spliced data features and the deep layer spliced data features, so as to obtain shallow layer fusion features, middle layer fusion features and deep layer fusion features. The classification module: a multi-scale feature reorganization classification model is used to classify the shallow layer spliced data features, the shallow layer fusion features, the middle layer fusion features and the deep layer fusion features, so as to obtain a city functional area classification result.

[0017] Compared with the prior art, the beneficial effects of the present application are: The present application uses remote sensing data from multiple sources, so that the factors considered in the division of the city functional area are more comprehensive. Feature extraction and feature splicing are performed on the preprocessed remote sensing data set, and then feature matching and alignment can be adaptively performed for different regional scale differences, so as to realize fine scale conversion between multi-source data. At the same time, the present application uses a multi-dimensional feature fusion model to dynamically learn the weight coefficients in the three-dimensional feature space, so as to realize accurate cross-modal feature adaptive fusion. The present application designs a multi-scale feature reorganization classification model to effectively integrate and classify the shallow layer spliced data features, the shallow layer fusion features, the middle layer fusion features and the deep layer fusion features, so as to further enhance the semantic expression and spatial discrimination ability of the features and improve the classification accuracy.

[0018] In summary, the city functional area division method of the present application has high accuracy of the divided functional area and fast updating speed of the functional area. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A flowchart of a city functional area fine classification method based on multi-source remote sensing data provided for embodiment 1.

[0020] Figure 2 A flowchart of feature extraction and feature splicing on a preprocessed remote sensing data set provided for embodiment 1.

[0021] Figure 3 A principle schematic diagram of a city functional area fine classification method based on multi-source remote sensing data provided for embodiment 1.

[0022] Figure 4 A structure diagram of a second feature extraction model provided for embodiment 1.

[0023] Figure 5 A structure schematic diagram of a second feature extraction model provided for embodiment 1.

[0024] Figure 6 A structure diagram of a multi-dimensional feature fusion model provided for embodiment 1.

[0025] Figure 7This is a schematic diagram of the structure of the multi-dimensional feature fusion model provided in Example 1.

[0026] Figure 8 This is a structural diagram of the multi-scale feature recombination classification model provided in Example 1.

[0027] Figure 9 This is a schematic diagram of the structure of the multi-scale feature recombination classification model provided in Example 1.

[0028] Figure 10 This is a structural diagram of the shallow feature reconstruction module provided in Example 1.

[0029] Figure 11 This is a structural diagram of the middle-level feature reconstruction module provided in Example 1.

[0030] Figure 12 This is a structural diagram of the deep feature reconstruction module provided in Example 1.

[0031] Figure 13 This is a structural diagram of the classification module provided in Example 1.

[0032] Figure 14 This is the commercial district classification result diagram provided in Example 1.

[0033] Figure 15 This is the residential area classification result diagram provided in Example 1.

[0034] Figure 16 This is the industrial zone classification result diagram provided in Example 1. DETAILED DESCRIPTION

[0035] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent; In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size; It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0036] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0037] Example 1 like Figure 1 As shown in FIG, a fine classification method of urban functional areas based on multi-source remote sensing data includes: S1: Obtain remote sensing data sets of urban functional areas; S2: preprocessing the remote sensing data group of the urban functional area to obtain a preprocessed remote sensing data group; S3: feature extraction and feature splicing are performed on the preprocessed remote sensing data set to obtain shallow spliced data features, middle spliced data features, and deep spliced data features; S4: a multi-dimensional feature fusion model is used to perform data fusion on the shallow spliced data features, the middle spliced data features, and the deep spliced data features to obtain shallow fusion features, middle fusion features, and deep fusion features; S5: a multi-scale feature reorganization classification model is used to classify the shallow spliced data features, the shallow fusion features, the middle fusion features, and the deep fusion features to obtain a city functional area classification result.

[0038] Further, the remote sensing data set of the city functional area includes multi-modal remote sensing data, and in step S2, the preprocessing includes geometric correction, projection unification, missing value interpolation, and normalization processing.

[0039] It should be noted that the sliding cropping processing divides the image into pixels) of a certain size.

[0040] Further, as shown in Figure 2 , in step S3, the feature extraction and feature splicing are performed on the preprocessed remote sensing data set, including: S301: selecting one type of remote sensing data in the preprocessed remote sensing data set as first remote sensing data; S302: inputting the first remote sensing data into a first feature extraction model to obtain first shallow data features, first middle data features, and first deep data features; S303: inputting the first shallow data features into a second feature extraction model to obtain second shallow data features; inputting the first middle data features into the second feature extraction model to obtain second middle data features; and inputting the first deep data features into the second feature extraction model to obtain second deep data features; S304: placing the second shallow data features into a shallow data feature group; placing the second middle data features into a middle data feature group; and placing the second deep data features into a deep data feature group; S305: selecting another type of remote sensing data in the preprocessed remote sensing data set as new first remote sensing data, and repeating steps S302-S304 until all remote sensing data in the preprocessed remote sensing data set are selected; S306: splicing the data features in the shallow data feature group to obtain shallow spliced data features; splicing the data features in the middle data feature group to obtain middle spliced data features; and splicing the data features in the deep data feature group to obtain deep spliced data features.

[0041] In a specific embodiment, Figure 3 As shown, the first feature extraction model is the Resnet34 network, where the residual block Output the first shallow data feature, residual block Output the first middle layer data features, residual block Output the first deep data feature.

[0042] Further, if Figure 4 、 Figure 5 As shown, the second feature extraction model includes: a channel adaptation module, a basic sampling network module, an offset prediction convolution module, a deformable sampler, a first convolution layer, and a first addition point; The output end of the channel adaptation module is connected to the input end of the offset prediction convolution module and the input end of the deformable sampler, the output end of the basic sampling network module and the output end of the offset prediction convolution module are connected to the input end of the first addition point, the output end of the first addition point is connected to the input end of the deformable sampler, and the output end of the deformable sampling point is connected to the input end of the first convolution layer.

[0043] It should be noted that Figure 5 The solid line in the middle represents control information, such as the offset and coordinates of the feature; the dotted line represents the original feature content being sampled.

[0044] In a specific embodiment, the convolution kernel spatial offset of feature n (e.g. Figure 5 The formula 1) is as follows:

[0045] represents the offset generation network, represents the input features of the deformable sampler, express A collection of parameters for the Science Department.

[0046] In a specific embodiment, the formula of the basic sampling network module (such as Figure 5 Formula 2) is as follows:

[0047] h represents the preset height of the remote sensing data after sampling, w represents the preset width of the remote sensing data after sampling, x represents the horizontal coordinate of the remote sensing data after sampling, y represents the vertical coordinate of the remote sensing data after sampling, W represents the maximum value of the preset width of the remote sensing data after sampling, and H represents the maximum value of the preset height of the remote sensing data after sampling.

[0048] Furthermore, the calculation formula of the deformable sampler (such as Figure 5 The formula 3) is as follows:

[0049] denotes the coordinate of the convolution center position of the deformable sampler, denotes the output feature of the deformable sampler, denotes the input feature of the deformable sampler, N denotes the total number of features, and n denotes the feature serial number, denotes the convolution kernel weight parameter of the feature n, denotes the convolution kernel neighborhood position coordinate of the feature n, denotes the convolution kernel spatial offset of the feature n.

[0050] It should be noted that the deformable sampler enables the present application to adaptively perform feature matching and alignment for different remote sensing data regional scale differences, thereby realizing fine scale conversion between multi-source data.

[0051] Further, as shown in Figure 6 、 Figure 7 , the multi-dimensional feature fusion model comprises a channel gating module, a spatial gating module, a separable convolution module, a first multiplication point, a second addition point, and a feature weight module. The output end of the channel gating module is connected with the input end of the separable convolution module, and the output end of the separable convolution module is connected with the input end of the feature weight module; the output end of the feature weight module and the output end of the spatial gating module are connected with the input end of the first multiplication point, and the output end of the first multiplication point is connected with the input end of the second addition point.

[0052] It should be noted that the channel gating module and the spatial gating module dynamically learn the weight coefficients in the three-dimensional feature space of remote sensing data, thereby realizing accurate cross-modal feature adaptive fusion.

[0053] The formula of the first multiplication point (such as formula 4 in Figure 7 ) is as follows:

[0054] denotes a convolution operation, denotes spliced data features, denotes a convolution kernel weight matrix, denotes a convolution kernel bias term, denotes a nonlinear sigmoid activation function.

[0055] The formula of the second addition point (such as formula 5 in Figure 7 ) is as follows:

[0056] The spliced data features are represented, The output result of the first multiplication point is represented.

[0057] It should be noted that the residual connection mechanism of the second addition point avoids the gradient disappearance problem that may exist in a deep network. The dynamic weight control is performed, and the original input features are superimposed to ensure that the feature information can be effectively reserved when the inhibition of the gating unit on the feature weight is strong.

[0058] Further, as shown in Figure 8 , Figure 9 The multi-scale feature reorganization classification model comprises a shallow feature reorganization module, a middle feature reorganization module, a deep feature reorganization module, and a classification module. The shallow spliced data features and the shallow fused features are input into the input end of the shallow feature reorganization module, the middle fused features are input into the input end of the middle feature reorganization module, and the deep fused features are input into the input end of the deep feature reorganization module. The output end of the shallow feature reorganization module, the output end of the middle feature reorganization module, the output end of the deep feature reorganization module, and the input end of the classification module are connected, and the classification module outputs a city functional area classification result.

[0059] It should be noted that the multi-scale feature reorganization classification model effectively integrates the shallow spliced data features, the shallow fused features, the middle fused features, and the deep fused features, further enhances the semantic expression and spatial discrimination ability of the features, and improves the classification accuracy.

[0060] Further, as shown in Figure 10 The shallow feature reorganization module comprises a second convolutional layer, a third addition point, a third convolutional layer, a batch normalization layer, and a first activation layer. The shallow spliced data features are input into the input end of the second convolutional layer, and the shallow fused features are input into the input end of the third convolutional layer. The output end of the third convolutional layer is connected to the input end of the batch normalization layer, the output end of the batch normalization layer is connected to the input end of the first activation layer, and the output end of the first activation layer and the output end of the second convolutional layer are connected to the input end of the third addition point. The third addition point outputs shallow reorganization features. In one specific embodiment, the formula of the second convolutional layer (as shown in formula 6 in Figure 9 ) is as follows:

[0061] The shallow spliced data features are represented, The bias term is represented, The convolution operation is represented, The convolution kernel weight matrix is represented, denotes a residual feature.

[0062] The formula of the third addition point (e.g., formula 7 in Figure 9 is as follows:

[0063] denotes a recombined feature, denotes a residual feature.

[0064] It should be noted that the second convolutional layer functions to mine the correlation of detailed features in the spatial neighborhood, and the local residual connection mechanism of the third addition point prevents the loss of detailed information of shallow features in the fusion.

[0065] As shown in Figure 11 , the middle-layer feature recombination module includes a spatial mapping layer, a second activation layer, an attention weighting layer, a second multiplication point, and a fourth addition point. The middle-layer fusion features are input to the input end of the spatial mapping layer and the input end of the fourth addition point, the output end of the spatial mapping layer is connected to the input end of the second activation layer and the input end of the second multiplication point, the output end of the second activation layer is connected to the input end of the attention weighting layer, the output end of the attention weighting layer is connected to the input end of the second multiplication point, the output end of the second multiplication point is connected to the input end of the fourth addition point, and the output end of the fourth addition point outputs the middle-layer recombined feature. It should be noted that the feature space resolution at the middle-layer scale is moderate, and has more rich semantic information.

[0066] The formula of the spatial mapping layer (e.g., formula 8 in Figure 9 is as follows:

[0067]

[0068]

[0069] denotes a spatial q mapping weight matrix, denotes a spatial k mapping weight matrix, denotes a spatial v mapping weight matrix, denotes a convolution operation, denotes a middle-layer fusion feature.

[0070] The formula of the attention weighting layer (e.g., formula 9 in Figure 9 is as follows:

[0071] denotes the number of channels of spatial k, a spatial q mapping feature, a spatial k mapping feature.

[0072] The formula of the fourth addition point (e.g., formula 10 in Figure 9 ) is as follows:

[0073] a middle layer fusion feature, an output of the attention weighting layer, a spatial v mapping result.

[0074] As shown in Figure 12 , the deep feature reorganization module includes a third multiplication point, an average pooling layer, a first linear layer, a third activation layer, a second linear layer, and a weight fusion layer. The deep fusion features are respectively input to the input end of the third multiplication point and the input end of the average pooling layer. The output end of the average pooling layer is connected to the input end of the first linear layer. The output end of the first linear layer is connected to the input end of the third activation layer. The output end of the third activation layer is connected to the input end of the second linear layer. The output end of the second linear layer is connected to the input end of the weight fusion layer and the input end of the third multiplication point. The output end of the third multiplication point is connected to the input end of the weight fusion layer. The output end of the weight fusion layer outputs a deep reorganization feature.

[0075] It should be noted that the deep scale feature pays more attention to the overall structure and pattern at the regional level. To better capture such macro patterns, a deep scale reorganization mechanism based on global context awareness is designed.

[0076] The formula of the weight fusion layer (e.g., formula 12 in Figure 9 ) is as follows:

[0077] a deep reorganization feature, , a learnable feature enhancement factor, an output of the third multiplication point, an output of the second linear layer.

[0078] Further, as shown in Figure 13 , the classification module includes a feature concatenation layer, an up-sampling layer, and a fourth activation layer. The shallow reorganization feature, the middle reorganization feature, and the deep reorganization feature are input to the input end of the feature concatenation layer. The output end of the feature concatenation layer is connected to the input end of the fourth activation layer. The output end of the fourth activation layer outputs a city functional area classification result.

[0079] A city function area fine classification system based on multi-source remote sensing data, comprising: A data acquisition module: acquiring a remote sensing data set of a city function area; A preprocessing module: preprocessing the remote sensing data set of the city function area to obtain a preprocessed remote sensing data set; A feature extraction and feature splicing module: performing feature extraction and feature splicing on the preprocessed remote sensing data set to obtain shallow spliced data features, middle spliced data features and deep spliced data features; A data fusion module: using a multi-dimensional feature fusion model to perform data fusion on the shallow spliced data features, the middle spliced data features and the deep spliced data features respectively to obtain shallow fusion features, middle fusion features and deep fusion features; A classification module: using a multi-scale feature reorganization classification model to classify the shallow spliced data features, the shallow fusion features, the middle fusion features and the deep fusion features to obtain a city function area classification result.

[0080] The method of the application takes Haizhu District of Guangzhou City as a training area, and the obtained result is as shown in Figures 14~16 The performance indicators of the method obtained by training on the test set are as follows: mIoU: 0.853, F1-score: 0.894, overall accuracy (OA): 0.934.

[0081] The same or similar reference signs correspond to the same or similar components; The terms describing the positional relationship in the drawings are only used for illustrative description, and should not be understood as a limitation on the patent; Obviously, the above embodiments of the application are only examples for clearly illustrating the application, and are not intended to limit the implementation modes of the application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the implementation modes are not required or can not be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principle of the application should be included in the protection scope of the claims of the application.

Claims

1. A fine classification method for urban functional areas based on multi-source remote sensing data, characterized by: include: S1: Obtain remote sensing data sets of urban functional areas; S2: preprocessing the remote sensing data group of the urban functional area to obtain a preprocessed remote sensing data group; S3: performing feature extraction and feature splicing on the pre-processed remote sensing data group to obtain shallow splicing data features, middle splicing data features, and deep splicing data features; S4: using a multi-dimensional feature fusion model to perform data fusion on the shallow splicing data features, the middle splicing data features, and the deep splicing data features, respectively, to obtain shallow fusion features, middle fusion features, and deep fusion features; S5: Using a multi-scale feature recombination classification model, classify the shallow splicing data features, the shallow fusion features, the middle fusion features, and the deep fusion features to obtain a classification result of an urban functional area.

2. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 1, characterized in that: The remote sensing data set of the urban functional area includes multimodal remote sensing data. In step S2, the preprocessing includes geometric correction, projection unification, missing value interpolation and normalization processing.

3. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 1, characterized in that: In step S3, feature extraction and feature splicing are performed on the pre-processed remote sensing data set, including: S301: selecting a type of remote sensing data from the pre-processed remote sensing data group as first remote sensing data; S302: Inputting the first remote sensing data into a first feature extraction model to obtain a first shallow layer data feature, a first middle layer data feature, and a first deep layer data feature; S303: Inputting the first shallow data feature into a second feature extraction model to obtain a second shallow data feature; inputting the first middle data feature into a second feature extraction model to obtain a second middle data feature; inputting the first deep data feature into a second feature extraction model to obtain a second deep data feature; S304: putting the second shallow data feature into a shallow data feature group; putting the second middle data feature into a middle data feature group; putting the second deep data feature into a deep data feature group; S305: selecting another type of remote sensing data from the pre-processed remote sensing data set as new first remote sensing data, and repeating steps S302 to S304 until all remote sensing data from the pre-processed remote sensing data set are selected; S306: splicing the data features in the shallow data feature group to obtain shallow spliced ​​data features; splicing the data features in the middle data feature group to obtain middle spliced ​​data features; splicing the data features in the deep data feature group to obtain deep spliced ​​data features.

4. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 3 is characterized in that: The second feature extraction model includes: a channel adaptation module, a basic sampling network module, an offset prediction convolution module, a deformable sampler, a first convolution layer, and a first addition point; The output end of the channel adaptation module is connected to the input end of the offset prediction convolution module and the input end of the deformable sampler, the output end of the basic sampling network module and the output end of the offset prediction convolution module are connected to the input end of the first addition point, the output end of the first addition point is connected to the input end of the deformable sampler, and the output end of the deformable sampling point is connected to the input end of the first convolution layer.

5. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 4 is characterized in that: The calculation formula of the deformable sampler is as follows: Represents the convolution center position coordinates of the deformable sampler, represents the output features of the deformable sampler, Represents the input features of the deformable sampler, N represents the total number of features, n represents the feature number, Represents the convolution kernel weight parameter of feature n, Represents the convolution kernel neighborhood position coordinates of feature n, Represents the spatial offset of the convolution kernel for feature n.

6. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 1, characterized in that: The multidimensional feature fusion model includes: a channel gating module, a spatial gating module, a separable convolution module, a first multiplication point, a second addition point, and a feature weight module; The output end of the channel gating module is connected to the input end of the separable convolution module, and the output end of the separable convolution module is connected to the input end of the feature weight module; the output end of the feature weight module and the output end of the spatial gating module are connected to the input end of the first multiplication point, and the output end of the first multiplication point is connected to the input end of the second addition point.

7. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 1, characterized in that: The multi-scale feature recombination classification model includes: a shallow feature recombination module, a mid-level feature recombination module, a deep feature recombination module, and a classification module; The shallow splicing data features and the shallow fusion features are input to the input end of the shallow feature reconstruction module; the middle-layer fusion features are input to the input end of the middle-layer feature reconstruction module, and the deep fusion features are input to the input end of the deep feature reconstruction module; The output end of the shallow feature reorganization module, the output end of the middle feature reorganization module, and the output end of the deep feature reorganization module are connected to the input end of the classification module, and the classification module outputs the classification result of the urban functional area.

8. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 7, characterized in that: The shallow feature reconstruction module includes: a second convolutional layer, a third addition point, a third convolutional layer, a batch normalization layer, and a first activation layer; The shallow splicing data feature is input into the input end of the second convolutional layer, the shallow fusion feature is input into the input end of the third convolutional layer, the output end of the third convolutional layer is connected to the input end of the batch normalization layer, the output end of the batch normalization layer is connected to the input end of the first activation layer, the output end of the first activation layer and the output end of the second convolutional layer are connected to the input end of the third addition point, and the third addition point outputs the shallow reorganization feature; The middle-layer feature reorganization module includes: a spatial mapping layer, a second activation layer, an attention weighting layer, a second multiplication point, and a fourth addition point; The middle-layer fusion features are respectively input to the input end of the spatial mapping layer and the input end of the fourth addition point, the output end of the spatial mapping layer is respectively connected to the input end of the second activation layer and the input end of the second multiplication point, the output end of the second activation layer is connected to the input end of the attention weighted layer, the output end of the attention weighted layer is connected to the input end of the second multiplication point, the output end of the second multiplication point is connected to the input end of the fourth addition point, and the output end of the fourth addition point outputs the middle-layer reorganized features; The deep feature reconstruction module includes: a third multiplication point, an average pooling layer, a first linear layer, a third activation layer, a second linear layer, and a weight fusion layer; The deep fusion features are respectively input into the input end of the third multiplication point and the input end of the average pooling layer. The output end of the average pooling layer is connected to the input end of the first linear layer. The output end of the first linear layer is connected to the input end of the third activation layer. The output end of the third activation layer is connected to the input end of the second linear layer. The output end of the second linear layer is connected to the input end of the weight fusion layer and the input end of the third multiplication point. The output end of the third multiplication point is connected to the input end of the weight fusion layer. The output end of the weight fusion layer outputs the deep reorganization features.

9. The method for fine classification of urban functional areas based on multi-source remote sensing data according to claim 8, characterized in that: The classification module includes: a feature splicing layer, an upsampling layer, and a fourth activation layer; The shallow reorganization features, the middle reorganization features, and the deep reorganization features are input into the input end of the feature splicing layer, the output end of the feature splicing layer is connected to the input end of the fourth activation layer, and the output end of the fourth activation layer outputs the urban functional area classification result.

10. A fine classification system for urban functional areas based on multi-source remote sensing data, applied to the classification method according to any one of claims 1 to 9, characterized in that: include: Data acquisition module: obtain remote sensing data sets of urban functional areas; Preprocessing module: preprocessing the remote sensing data group of the urban functional area to obtain a preprocessed remote sensing data group; Feature extraction and feature splicing module: performs feature extraction and feature splicing on the pre-processed remote sensing data group to obtain shallow splicing data features, middle splicing data features, and deep splicing data features; Data fusion module: using a multi-dimensional feature fusion model to fuse the shallow splicing data features, the middle splicing data features, and the deep splicing data features, to obtain shallow fusion features, middle fusion features, and deep fusion features; Classification module: using a multi-scale feature recombinant classification model to classify the shallow splicing data features, the shallow fusion features, the middle fusion features, and the deep fusion features to obtain the urban functional area classification results.

Images