A method and system for fine classification of urban functional areas based on multi-source remote sensing data

Abstract

The application discloses a kind of based on multi-source remote sensing data's urban function area fine classification method and system.The method includes the following steps: obtaining the remote sensing data group of urban function area;The remote sensing data group of urban function area is preprocessed, and the remote sensing data group after being preprocessed is obtained;Feature extraction and feature splicing are carried out to the remote sensing data group after being preprocessed, and shallow splicing data feature, middle splicing data feature, deep splicing data feature are obtained;Data fusion is carried out to shallow splicing data feature, middle splicing data feature, deep splicing data feature respectively using multi-dimensional feature fusion model, and shallow fusion feature, middle fusion feature, deep fusion feature are obtained;Shallow splicing data feature, shallow fusion feature, middle fusion feature, deep fusion feature are classified using multi-scale feature reorganization classification model, and function area classification result is obtained.The function area division method of the application is high in accuracy, and function area update speed is fast.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing data processing, and more particularly, to a method and system for fine classification of urban functional areas based on multi-source remote sensing data. BACKGROUND

[0002] The division of urban functional areas has become increasingly important with the development of low-altitude economy. Only by fully considering the distribution and attributes of road networks, green networks, water networks, residential areas, industrial areas, commercial areas, and other urban functional areas can the planning of low-altitude air routes enable low-altitude flying objects to avoid various potential risk factors and maximize the use of limited air route resources.

[0003] Traditional methods of identifying urban functional areas rely on manual surveys and other methods, so there is often a delay of several months from data collection to the completion of functional area division, and the accuracy of the functional area division results quickly deteriorates with the progress of urban renewal. While using a single type of remote sensing data can achieve rapid updating of urban functional area division, it is difficult to accurately divide urban functional areas using a single type of remote sensing data because there are many factors affecting urban functional areas.

[0004] The prior art discloses a method for identifying urban functional areas based on remote sensing image feature classification, which specifically includes the following steps: S1: selecting a data source and obtaining remote sensing image data for a selected area; S2: preprocessing the remote sensing image data obtained in step S1, including image rectification, image fusion, image color uniformity, and image inlay; S3: using the preprocessed remote sensing image in step S2 as a data source, training a deep learning semantic segmentation network, and extracting a feature classification image; S4: using open operation and close operation in mathematical morphology on the feature classification image extracted in step S3; S5: constructing an urban functional area identification regression model; and S6: obtaining an urban functional area identification result. The method uses relatively single remote sensing data, so the accuracy of the urban functional areas obtained by the method is low. SUMMARY

[0005] The present application provides a method for fine classification of urban functional areas based on multi-source remote sensing data to overcome the defects of low accuracy and slow updating speed of functional area division in the prior art.

[0006] A method for fine classification of urban functional areas based on multi-source remote sensing data, comprising:

[0007] S1: obtaining a set of remote sensing data for urban functional areas;

[0008] S2: preprocessing the set of remote sensing data for urban functional areas to obtain a set of preprocessed remote sensing data;

[0009] S3: performing feature extraction and feature splicing on the preprocessed remote sensing data set to obtain shallow spliced data features, middle spliced data features, and deep spliced data features;

[0010] S4: performing data fusion on the shallow spliced data features, the middle spliced data features, and the deep spliced data features respectively by using a multi-dimensional feature fusion model to obtain shallow fusion features, middle fusion features, and deep fusion features;

[0011] S5: performing classification on the shallow spliced data features, the shallow fusion features, the middle fusion features, and the deep fusion features by using a multi-scale feature reorganization classification model to obtain a classification result of the urban functional area.

[0012] Further, the remote sensing data set of the urban functional area includes multi-modal remote sensing data, and in step S2, the preprocessing includes geometric correction, projection unification, missing value interpolation, and normalization processing.

[0013] Further, in step S3, performing feature extraction and feature splicing on the preprocessed remote sensing data set includes:

[0014] S301: selecting one type of remote sensing data in the preprocessed remote sensing data set as first remote sensing data;

[0015] S302: inputting the first remote sensing data into a first feature extraction model to obtain first shallow data features, first middle data features, and first deep data features;

[0016] S303: inputting the first shallow data features into a second feature extraction model to obtain second shallow data features; inputting the first middle data features into the second feature extraction model to obtain second middle data features; and inputting the first deep data features into the second feature extraction model to obtain second deep data features;

[0017] S304: placing the second shallow data features into a shallow data feature group; placing the second middle data features into a middle data feature group; and placing the second deep data features into a deep data feature group;

[0018] S305: selecting another type of remote sensing data in the preprocessed remote sensing data set as new first remote sensing data, and repeatedly performing steps S302-S304 until all remote sensing data in the preprocessed remote sensing data set are selected;

[0019] S306: splicing the data features in the shallow layer data feature group to obtain shallow layer spliced data features; splicing the data features in the middle layer data feature group to obtain middle layer spliced data features; and splicing the data features in the deep layer data feature group to obtain deep layer spliced data features.

[0020] Further, the second feature extraction model comprises a channel adaptation module, a basic sampling network module, an offset prediction convolution module, a deformable sampler, a first convolution layer, and a first addition point.

[0021] The output end of the channel adaptation module is connected with the input end of the offset prediction convolution module and the input end of the deformable sampler, the output end of the basic sampling network module and the output end of the offset prediction convolution module are connected with the input end of the first addition point, the output end of the first addition point is connected with the input end of the deformable sampler, and the output end of the deformable sampler is connected with the input end of the first convolution layer.

[0022] Further, the calculation formula of the deformable sampler is as follows:

[0023]

[0024] denotes the convolution center position coordinates of the deformable sampler, denotes the output feature of the deformable sampler, denotes the input feature of the deformable sampler, N denotes the total number of features, and n denotes the feature serial number, denotes the convolution kernel weight parameter of the feature n, denotes the convolution kernel neighborhood position coordinates of the feature n, denotes the convolution kernel spatial offset of the feature n.

[0025] Further, the multi-dimensional feature fusion model comprises a channel gating module, a spatial gating module, a separable convolution module, a first multiplication point, a second addition point, and a feature weight module.

[0026] The output end of the channel gating module is connected with the input end of the separable convolution module, the output end of the separable convolution module is connected with the input end of the feature weight module, the output end of the feature weight module and the output end of the spatial gating module are connected with the input end of the first multiplication point, and the output end of the first multiplication point is connected with the input end of the second addition point.

[0027] Further, the multi-scale feature reorganization classification model comprises a shallow layer feature reorganization module, a middle layer feature reorganization module, a deep layer feature reorganization module, and a classification module.

[0028] The shallow layer spliced data feature and the shallow layer fusion feature are input into an input end of a shallow layer feature reorganization module; the middle layer fusion feature is input into an input end of a middle layer feature reorganization module, and the deep layer fusion feature is input into an input end of a deep layer feature reorganization module;

[0029] The output end of the shallow layer feature reorganization module, the output end of the middle layer feature reorganization module, the output end of the deep layer feature reorganization module and an input end of a classification module are connected, and the classification module outputs a city function area classification result.

[0030] Further, the shallow layer feature reorganization module comprises a second convolutional layer, a third addition point, a third convolutional layer, a batch normalization layer and a first activation layer.

[0031] The shallow layer spliced data feature is input into an input end of the second convolutional layer, and the shallow layer fusion feature is input into an input end of the third convolutional layer. The output end of the third convolutional layer is connected with the input end of the batch normalization layer. The output end of the batch normalization layer is connected with the input end of the first activation layer. The output end of the first activation layer and the output end of the second convolutional layer are connected with the input end of the third addition point. The third addition point outputs a shallow layer reorganization feature.

[0032] The middle layer feature reorganization module comprises a spatial mapping layer, a second activation layer, an attention weighting layer, a second multiplication point and a fourth addition point.

[0033] The middle layer fusion feature is input into the input end of the spatial mapping layer and the input end of the fourth addition point respectively. The output end of the spatial mapping layer is connected with the input end of the second activation layer and the input end of the second multiplication point respectively. The output end of the second activation layer is connected with the input end of the attention weighting layer. The output end of the attention weighting layer is connected with the input end of the second multiplication point. The output end of the second multiplication point is connected with the input end of the fourth addition point. The output end of the fourth addition point outputs a middle layer reorganization feature.

[0034] The deep layer feature reorganization module comprises a third multiplication point, an average pooling layer, a first linear layer, a third activation layer, a second linear layer and a weight fusion layer.

[0035] The deep layer fusion feature is input into the input end of the third multiplication point and the input end of the average pooling layer respectively. The output end of the average pooling layer is connected with the input end of the first linear layer. The output end of the first linear layer is connected with the input end of the third activation layer. The output end of the third activation layer is connected with the input end of the second linear layer. The output end of the second linear layer is connected with the input end of the weight fusion layer and the input end of the third multiplication point. The output end of the third multiplication point is connected with the input end of the weight fusion layer. The output end of the weight fusion layer outputs a deep layer reorganization feature.

[0036] Further, the classification module comprises a feature splicing layer, an up-sampling layer and a fourth activation layer.

[0037] The shallow recombination feature, the middle layer recombination feature and the deep layer recombination feature are input into an input end of a feature splicing layer, an output end of the feature splicing layer is connected with an input end of a fourth activation layer, and an output end of the fourth activation layer outputs a city function area classification result.

[0038] A city function area fine classification system based on multi-source remote sensing data comprises:

[0039] A data acquisition module acquires a remote sensing data set of the city function area.

[0040] A preprocessing module pre-processes the remote sensing data set of the city function area to obtain a pre-processed remote sensing data set.

[0041] A feature extraction and feature splicing module extracts and splices features from the pre-processed remote sensing data set to obtain shallow spliced data features, middle layer spliced data features and deep layer spliced data features.

[0042] A data fusion module fuses the shallow spliced data features, the middle layer spliced data features and the deep layer spliced data features by using a multi-dimensional feature fusion model to obtain shallow fusion features, middle layer fusion features and deep layer fusion features.

[0043] A classification module classifies the shallow spliced data features, the shallow fusion features, the middle layer fusion features and the deep layer fusion features by using a multi-scale feature recombination classification model to obtain a city function area classification result.

[0044] Compared with the prior art, the city function area division method has the following beneficial effects:

[0045] The city function area division method uses remote sensing data from multiple sources, so that the factors considered in the division of the city function area are more comprehensive. The feature extraction and feature splicing of the pre-processed remote sensing data set can adaptively match and align features for different regional scale differences, thereby realizing fine scale conversion between multi-source data. The multi-dimensional feature fusion model is used to dynamically learn weight coefficients in a three-dimensional feature space, thereby realizing accurate cross-modal feature adaptive fusion. The multi-scale feature recombination classification model is designed to effectively integrate and classify the shallow spliced data features, the shallow fusion features, the middle layer fusion features and the deep layer fusion features, thereby further enhancing the semantic expression and spatial discrimination ability of the features and improving the classification accuracy.

[0046] In summary, the city function area division method has high accuracy and fast update speed. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1A flow chart of a method for fine classification of urban functional areas based on multi-source remote sensing data provided for embodiment 1.

[0048] Figure 2 A flow chart of feature extraction and feature splicing on pre-processed remote sensing data sets provided for embodiment 1.

[0049] Figure 3 A schematic diagram of the principle of a method for fine classification of urban functional areas based on multi-source remote sensing data provided for embodiment 1.

[0050] Figure 4 A structure diagram of a second feature extraction model provided for embodiment 1.

[0051] Figure 5 A structure diagram of a second feature extraction model provided for embodiment 1.

[0052] Figure 6 A structure diagram of a multi-dimensional feature fusion model provided for embodiment 1.

[0053] Figure 7 A structure diagram of a multi-dimensional feature fusion model provided for embodiment 1.

[0054] Figure 8 A structure diagram of a multi-scale feature reorganization classification model provided for embodiment 1.

[0055] Figure 9 A structure diagram of a multi-scale feature reorganization classification model provided for embodiment 1.

[0056] Figure 10 A structure diagram of a shallow feature reorganization module provided for embodiment 1.

[0057] Figure 11 A structure diagram of a middle feature reorganization module provided for embodiment 1.

[0058] Figure 12 A structure diagram of a deep feature reorganization module provided for embodiment 1.

[0059] Figure 13 A structure diagram of a classification module provided for embodiment 1.

[0060] Figure 14 A business district classification result diagram provided for embodiment 1.

[0061] Figure 15 A residential area classification result diagram provided for embodiment 1.

[0062] Figure 16 An industrial area classification result diagram provided for embodiment 1. DETAILED DESCRIPTION

[0063] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0064] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0065] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0066] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0067] Example 1

[0068] like Figure 1 As shown, a method for fine classification of urban functional zones based on multi-source remote sensing data includes:

[0069] S1: Acquire remote sensing data sets of urban functional areas;

[0070] S2: Preprocess the remote sensing data set of the urban functional area to obtain the preprocessed remote sensing data set;

[0071] S3: Perform feature extraction and feature stitching on the preprocessed remote sensing data set to obtain shallow stitched data features, medium stitched data features, and deep stitched data features;

[0072] S4: Use a multi-dimensional feature fusion model to fuse the shallow spliced ​​data features, the middle spliced ​​data features, and the deep spliced ​​data features respectively to obtain shallow fused features, middle fused features, and deep fused features;

[0073] S5: Use a multi-scale feature recombination classification model to classify the shallow spliced ​​data features, the shallow fusion features, the mid-level fusion features, and the deep fusion features to obtain the classification results of urban functional areas.

[0074] Furthermore, the remote sensing data set of the urban functional area includes multimodal remote sensing data. In step S2, the preprocessing includes geometric correction, projection unification, missing value interpolation, and normalization.

[0075] It should be noted that the sliding cropping process segments the image into... The size of a (pixel).

[0076] Furthermore, such as Figure 2 As shown, in step S3, feature extraction and feature stitching are performed on the preprocessed remote sensing data set, including:

[0077] S301: select a type of remote sensing data in the pre-processed remote sensing data set as first remote sensing data;

[0078] S302: input the first remote sensing data into a first feature extraction model to obtain first shallow data features, first middle data features and first deep data features;

[0079] S303: input the first shallow data features into a second feature extraction model to obtain second shallow data features; input the first middle data features into the second feature extraction model to obtain second middle data features; and input the first deep data features into the second feature extraction model to obtain second deep data features;

[0080] S304: place the second shallow data features into a shallow data feature group, place the second middle data features into a middle data feature group, and place the second deep data features into a deep data feature group;

[0081] S305: select another type of remote sensing data in the pre-processed remote sensing data set as new first remote sensing data, and repeat steps S302-S304 until all remote sensing data in the pre-processed remote sensing data set are selected;

[0082] S306: splice the data features in the shallow data feature group to obtain shallow spliced data features, splice the data features in the middle data feature group to obtain middle spliced data features, and splice the data features in the deep data feature group to obtain deep spliced data features.

[0083] In one embodiment, as shown in Figure 3 , the first feature extraction model is a Resnet34 network, wherein a residual block outputs the first shallow data features, a residual block outputs the first middle data features, and a residual block outputs the first deep data features.

[0084] Further, as shown in Figure 4 , Figure 5 , the second feature extraction model comprises a channel adaptation module, a basic sampling network module, an offset prediction convolution module, a deformable sampler, a first convolution layer and a first addition point.

[0085] The output end of the channel adaptation module is connected with the input end of the offset prediction convolution module and the input end of the deformable sampler, the output end of the basic sampling network module and the output end of the offset prediction convolution module are connected with the input end of the first addition point, the output end of the first addition point is connected with the input end of the deformable sampler, and the output end of the deformable sampler is connected with the input end of the first convolution layer.

[0086] It should be noted that, Figure 5 Solid lines represent control information, such as feature offsets and coordinates; dashed lines represent the original feature content being sampled.

[0087] In one specific embodiment, the convolution kernel spatial offset of feature n (e.g. Figure 5 Equation 1) shown is as follows:

[0088]

[0089] This represents the offset generation network. This represents the input features of the deformable sampler. express The parameter set of the Chinese Academy of Sciences.

[0090] In one specific embodiment, the formula for the basic sampling network module (e.g.) Figure 5 Equation 2) is as follows:

[0091]

[0092] h represents the height of the preset sampled remote sensing data, w represents the width of the preset sampled remote sensing data, x represents the horizontal coordinate of the sampled remote sensing data, y represents the vertical coordinate of the sampled remote sensing data, W represents the maximum value of the width of the preset sampled remote sensing data, and H represents the maximum value of the height of the preset sampled remote sensing data.

[0093] Furthermore, the calculation formula for the deformable sampler (such as...) Figure 5 Equation 3) shown is as follows:

[0094]

[0095] This represents the coordinates of the convolution center position of the deformable sampler. This represents the output characteristics of the deformable sampler. The input features of the deformable sampler are represented by N, where N represents the total number of features and n represents the feature index. The convolution kernel weights represent the parameters of feature n. This represents the coordinates of the convolution kernel neighborhood of feature n. This represents the spatial offset of the convolution kernel for feature n.

[0096] It should be noted that the deformable sampler enables the present invention to adaptively perform feature matching and alignment for different remote sensing data regions with scale differences, thereby achieving fine scale conversion between multi-source data.

[0097] Furthermore, such as Figure 6 ,Figure 7 As shown, the multi-dimensional feature fusion model comprises a channel gating module, a spatial gating module, a separable convolution module, a first multiplication point, a second addition point, and a feature weight module.

[0098] The output end of the channel gating module is connected with the input end of the separable convolution module, and the output end of the separable convolution module is connected with the input end of the feature weight module; the output end of the feature weight module and the output end of the spatial gating module are connected with the input end of the first multiplication point, and the output end of the first multiplication point is connected with the input end of the second addition point.

[0099] It should be noted that the channel gating module and the spatial gating module learn the weight coefficients in the three-dimensional feature space of the remote sensing data through dynamic learning, so as to realize accurate cross-modal feature adaptive fusion.

[0100] The formula of the first multiplication point (such as formula 4 in the specification) is as follows: Figure 7

[0101]

[0102] represents a convolution operation, represents spliced data features, represents a convolution kernel weight matrix, represents a convolution kernel bias term, represents a nonlinear sigmoid activation function.

[0103] The formula of the second addition point (such as formula 5 in the specification) is as follows: Figure 7

[0104]

[0105] represents spliced data features, represents the output result of the first multiplication point.

[0106] It should be noted that the residual connection mechanism of the second addition point avoids the gradient vanishing problem that may exist in a deep network. The dynamic weight control is performed by using , and the original input features are superimposed to ensure that the feature information can still be effectively preserved when the gating unit suppresses the feature weight.

[0107] Further, as shown in Figure 8 , Figure 9 The multi-scale feature reorganization classification model comprises a shallow feature reorganization module, a middle feature reorganization module, a deep feature reorganization module, and a classification module.

[0108] ​​The shallow layer spliced data feature, the shallow layer fusion feature, input an input end of a shallow layer feature reorganization module; the middle layer fusion feature inputs an input end of a middle layer feature reorganization module, and the deep layer fusion feature inputs an input end of a deep layer feature reorganization module;

[0109] The output end of the shallow layer feature reorganization module, the output end of the middle layer feature reorganization module, the output end of the deep layer feature reorganization module, and the input end of the classification module are connected, and the classification module outputs a city function area classification result.

[0110] It should be noted that the multi-scale feature reorganization classification model effectively integrates the shallow layer spliced data feature, the shallow layer fusion feature, the middle layer fusion feature, and the deep layer fusion feature, further enhances the semantic expression and spatial discrimination ability of the feature, and improves the classification precision.

[0111] Further, as shown in Figure 10 The shallow layer feature reorganization module includes a second convolutional layer, a third addition point, a third convolutional layer, a batch normalization layer, and a first activation layer.

[0112] The shallow layer spliced data feature inputs an input end of the second convolutional layer, and the shallow layer fusion feature inputs an input end of the third convolutional layer. The output end of the third convolutional layer is connected with the input end of the batch normalization layer. The output end of the batch normalization layer is connected with the input end of the first activation layer. The output end of the first activation layer and the output end of the second convolutional layer are connected with the input end of the third addition point. The third addition point outputs a shallow layer reorganization feature.

[0113] In one specific embodiment, the formula (such as formula 6 in Figure 9 ) of the second convolutional layer is as follows:

[0114]

[0115] represents the shallow layer spliced data feature, represents a bias term, represents a convolution operation, represents a convolution kernel weight matrix, represents a residual feature.

[0116] The formula (such as formula 7 in Figure 9 ) of the third addition point is as follows:

[0117]

[0118] represents a reorganization feature, represents a residual feature.

[0119] It should be noted that the role of the second convolutional layer is to mine the correlation of the detailed features in the spatial neighborhood, and the local residual connection mechanism of the third addition point prevents the loss of detailed information of shallow features in the fusion.

[0120] As shown in Figure 11 , the middle layer feature reorganization module comprises a spatial mapping layer, a second activation layer, an attention weighting layer, a second multiplication point and a fourth addition point.

[0121] The middle layer fusion features are input into the input end of the spatial mapping layer and the input end of the fourth addition point, the output end of the spatial mapping layer is connected with the input end of the second activation layer and the input end of the second multiplication point, the output end of the second activation layer is connected with the input end of the attention weighting layer, the output end of the attention weighting layer is connected with the input end of the second multiplication point, the output end of the second multiplication point is connected with the input end of the fourth addition point, and the output end of the fourth addition point outputs the middle layer reorganized features.

[0122] It should be noted that the feature space resolution at the middle layer scale is moderate, and has more rich semantic information.

[0123] The formula of the spatial mapping layer (such as formula 8 in Figure 9 ) is as follows:

[0124]

[0125]

[0126]

[0127] q denotes a spatial q mapping weight matrix, k denotes a spatial k mapping weight matrix, v denotes a spatial v mapping weight matrix, denotes a convolution operation, denotes a middle layer fusion feature.

[0128] The formula of the attention weighting layer (such as formula 9 in Figure 9 ) is as follows:

[0129]

[0130] k denotes the number of channels of the spatial k, q denotes a spatial q mapping feature, k denotes a spatial k mapping feature.

[0131] The formula of the fourth addition point (such as formula 10 in Figure 9 ) is as follows:

[0132]

[0133] representing the middle layer fusion feature, representing the output of the attention weighting layer, representing the spatial v mapping result.

[0134] As shown in Figure 12 , the deep feature reorganization module includes a third multiplication point, an average pooling layer, a first linear layer, a third activation layer, a second linear layer, and a weight fusion layer.

[0135] The deep fusion features are respectively input into the input end of the third multiplication point and the input end of the average pooling layer. The output end of the average pooling layer is connected with the input end of the first linear layer. The output end of the first linear layer is connected with the input end of the third activation layer. The output end of the third activation layer is connected with the input end of the second linear layer. The output end of the second linear layer is connected with the input end of the weight fusion layer and the input end of the third multiplication point. The output end of the third multiplication point is connected with the input end of the weight fusion layer. The output end of the weight fusion layer outputs the deep reorganization feature.

[0136] It should be noted that the deep scale feature pays more attention to the overall structure and pattern at the regional level. In order to better capture such macro patterns, a deep scale reorganization mechanism based on global context perception is designed.

[0137] The formula of the weight fusion layer (such as formula 12 in Figure 9 ) is as follows:

[0138]

[0139] representing the deep reorganization feature, , representing the learnable feature enhancement factor, representing the output of the third multiplication point, representing the output of the second linear layer.

[0140] Further, as shown in Figure 13 , the classification module includes a feature splicing layer, an up-sampling layer, and a fourth activation layer.

[0141] The shallow reorganization feature, the middle reorganization feature, and the deep reorganization feature are input into the input end of the feature splicing layer. The output end of the feature splicing layer is connected with the input end of the fourth activation layer. The output end of the fourth activation layer outputs the urban functional area classification result.

[0142] A system for fine classification of urban functional areas based on multi-source remote sensing data includes:

[0143] A data acquisition module acquires a set of remote sensing data of urban functional areas.

[0144] a preprocessing module configured to preprocess a remote sensing data set of the urban functional area to obtain a preprocessed remote sensing data set;

[0145] a feature extraction and feature splicing module configured to extract and splice features from the preprocessed remote sensing data set to obtain shallow spliced data features, middle spliced data features and deep spliced data features;

[0146] a data fusion module configured to fuse the shallow spliced data features, the middle spliced data features and the deep spliced data features by using a multi-dimensional feature fusion model to obtain shallow fusion features, middle fusion features and deep fusion features;

[0147] a classification module configured to classify the shallow spliced data features, the shallow fusion features, the middle fusion features and the deep fusion features by using a multi-scale feature reorganization classification model to obtain an urban functional area classification result.

[0148] The method of the present application takes Guangzhou Haizhu District as a training area, and the obtained result is as shown in FIG. 5. Figures 14~16 The performance indicators of the method obtained by training on the test set are as follows: mIoU: 0.853, F1-score: 0.894, overall accuracy (OA): 0.934.

[0149] The same or similar reference signs correspond to the same or similar components;

[0150] The terms used to describe the positional relationship in the drawings are only used for illustrative description, and should not be understood as a limitation on the patent;

[0151] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. For those skilled in the art, other different forms of changes or variations can be made on the basis of the above description. Here, it is not necessary and impossible to enumerate all the embodiments. Any modification, equivalent replacement and improvement made within the spirit and principles of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A method for fine classification of urban functional areas based on multi-source remote sensing data, characterized in that, The method comprises the following steps: S1: obtaining a remote sensing data set of a city functional area; S2: preprocessing the remote sensing data set of the city functional area to obtain a preprocessed remote sensing data set; S3: performing feature extraction and feature splicing on the preprocessed remote sensing data set to obtain shallow spliced data features, middle spliced data features and deep spliced data features; S4: performing data fusion on the shallow spliced data features, the middle spliced data features and the deep spliced data features respectively by using a multi-dimensional feature fusion model to obtain shallow fusion features, middle fusion features and deep fusion features; S5: performing classification on the shallow spliced data features, the shallow fusion features, the middle fusion features and the deep fusion features by using a multi-scale feature reorganization classification model to obtain a city functional area classification result; The multi-scale feature reorganization classification model comprises a shallow feature reorganization module, a middle feature reorganization module, a deep feature reorganization module and a classification module; The shallow spliced data features and the shallow fusion features are input into the input end of the shallow feature reorganization module; the middle fusion features are input into the input end of the middle feature reorganization module; and the deep fusion features are input into the input end of the deep feature reorganization module; The output ends of the shallow feature reorganization module, the middle feature reorganization module and the deep feature reorganization module are connected with the input end of the classification module, and the classification module outputs the city functional area classification result; The shallow feature reorganization module comprises a second convolutional layer, a third addition point, a third convolutional layer, a batch normalization layer and a first activation layer; The shallow spliced data features are input into the input end of the second convolutional layer, and the shallow fusion features are input into the input end of the third convolutional layer; the output end of the third convolutional layer is connected with the input end of the batch normalization layer; the output end of the batch normalization layer is connected with the input end of the first activation layer; the output end of the first activation layer and the output end of the second convolutional layer are connected with the input end of the third addition point; and the third addition point outputs shallow reorganization features; The middle feature reorganization module comprises a spatial mapping layer, a second activation layer, an attention weighting layer, a second multiplication point and a fourth addition point; The middle fusion features are input into the input end of the spatial mapping layer and the input end of the fourth addition point respectively; the output end of the spatial mapping layer is connected with the input end of the second activation layer and the input end of the second multiplication point respectively; the output end of the second activation layer is connected with the input end of the attention weighting layer; the output end of the attention weighting layer is connected with the input end of the second multiplication point; the output end of the second multiplication point is connected with the input end of the fourth addition point; and the output end of the fourth addition point outputs middle reorganization features; The deep feature reorganization module comprises a third multiplication point, an average pooling layer, a first linear layer, a third activation layer, a second linear layer and a weight fusion layer; The deep layer fusion features are respectively input into the input end of the third multiplication point and the input end of the average pooling layer, the output end of the average pooling layer is connected with the input end of the first linear layer, the output end of the first linear layer is connected with the input end of the third activation layer, the output end of the third activation layer is connected with the input end of the second linear layer, the output end of the second linear layer is connected with the input end of the weight fusion layer and the input end of the third multiplication point, the output end of the third multiplication point is connected with the input end of the weight fusion layer, and the output end of the weight fusion layer outputs deep layer reorganization features.

2. The method according to claim 1, wherein, The remote sensing data set of the urban functional area includes multi-modal remote sensing data, and the preprocessing in step S2 includes geometric correction, projection unification, missing value interpolation and normalization processing. 3.The method of claim 1, wherein, In step S3, the preprocessed remote sensing data set is subjected to feature extraction and feature splicing, which includes: S301: selecting one type of remote sensing data in the preprocessed remote sensing data set as first remote sensing data; S302: inputting the first remote sensing data into a first feature extraction model to obtain first shallow layer data features, first middle layer data features and first deep layer data features; S303: inputting the first shallow layer data features into a second feature extraction model to obtain second shallow layer data features, inputting the first middle layer data features into the second feature extraction model to obtain second middle layer data features, and inputting the first deep layer data features into the second feature extraction model to obtain second deep layer data features; S304: placing the second shallow layer data features into a shallow layer data feature group, placing the second middle layer data features into a middle layer data feature group, and placing the second deep layer data features into a deep layer data feature group; S305: selecting another type of remote sensing data in the preprocessed remote sensing data set as new first remote sensing data, and repeatedly executing steps S302-S304 until all remote sensing data in the preprocessed remote sensing data set are selected; S306: splicing the data features in the shallow layer data feature group to obtain shallow layer spliced data features, splicing the data features in the middle layer data feature group to obtain middle layer spliced data features, and splicing the data features in the deep layer data feature group to obtain deep layer spliced data features.

4. The method according to claim 3, wherein, The second feature extraction model includes a channel adaptation module, a basic sampling network module, an offset prediction convolution module, a deformable sampler, a first convolution layer and a first addition point. The output end of the channel adaptation module is connected with the input end of the offset prediction convolution module and the input end of the deformable sampler, the output end of the basic sampling network module and the output end of the offset prediction convolution module are connected with the input end of the first addition point, the output end of the first addition point is connected with the input end of the deformable sampler, and the output end of the deformable sampler is connected with the input end of the first convolution layer.

5. The method according to claim 4, wherein, The calculation formula of the deformable sampler is as follows: a coordinate of a convolution center position of the deformable sampler, an output feature of the deformable sampler, an input feature of the deformable sampler, N represents a total number of features, and n represents a feature serial number, a convolution kernel weight parameter of the feature n, a convolution kernel neighborhood position coordinate of the feature n, a convolution kernel spatial offset of the feature n.

6. The method according to claim 1, wherein, The multi-dimensional feature fusion model includes a channel gating module, a spatial gating module, a separable convolution module, a first multiplication point, a second addition point and a feature weight module. An output end of the channel gating module is connected with an input end of the separable convolution module, an output end of the separable convolution module is connected with an input end of the feature weight module; an output end of the feature weight module and an output end of the spatial gating module are connected with input ends of a first multiplication point, an output end of the first multiplication point is connected with an input end of a second addition point.

7. The method according to claim 1, wherein, The classification module comprises a feature splicing layer, an up-sampling layer and a fourth activation layer. The shallow layer recombined feature, the middle layer recombined feature and the deep layer recombined feature are input into an input end of the feature splicing layer, an output end of the feature splicing layer is connected with an input end of the fourth activation layer, and an output end of the fourth activation layer outputs the urban functional area classification result.

8. A system for fine classification of urban functional areas based on multi-source remote sensing data, applied to the classification method of any one of claims 1-7, characterized in that, It comprises: a data acquisition module configured to acquire a remote sensing data set of an urban functional area; a preprocessing module configured to preprocess the remote sensing data set of the urban functional area to obtain a preprocessed remote sensing data set; a feature extraction and feature splicing module configured to extract and splice features of the preprocessed remote sensing data set to obtain shallow layer spliced data features, middle layer spliced data features and deep layer spliced data features; a data fusion module configured to fuse the shallow layer spliced data features, the middle layer spliced data features and the deep layer spliced data features by using a multi-dimensional feature fusion model to obtain shallow layer fused features, middle layer fused features and deep layer fused features; a classification module configured to classify the shallow layer spliced data features, the shallow layer fused features, the middle layer fused features and the deep layer fused features by using a multi-scale feature recombination classification model to obtain an urban functional area classification result.

Images