Land cover mapping method and system for satellite remote sensing images fused with sparse photos

Abstract

The present invention relates to a method and system for mapping land cover using satellite remote sensing images fused with sparse photos. The method comprises the following steps: step S1: extracting semantic features of sparse photos within the coverage area of ​​satellite remote sensing images using a pre-trained Places-CNN model; step S2: interpolating the semantic features to obtain a sparse photo interpolation feature map; step S3: hierarchically integrating the sparse photo interpolation feature map into the coded features of each level of the satellite remote sensing image using an encoder to obtain a fused feature; decoding the fused feature using a decoder to obtain a land cover map; step S4: visually classifying the objects therein according to their material based on the sparse photos; and using the corresponding satellite remote sensing image pixel material within the interpolation range of the sparse photos as supervision information, performing refined classification of the land cover map outside the interpolation range based on the object material. The method provided by the present invention fuses sparse photos into satellite remote sensing images to achieve more refined land cover classification.

Description

Technical Field

[0001] The present invention relates to the field of geospatial information technology, and in particular to a method and system for mapping land cover using satellite remote sensing images fused with sparse photos. Background Art

[0002] Land cover describes the natural and man-made cover of the Earth's surface, including biota, soils, topography, surface and groundwater, and human structures (Giri and ChandraP, 2012). With the acceleration of economic development and urbanization, the demand for land cover information is increasing, and collecting land cover information has become increasingly important. Existing land cover mapping methods primarily use spectral features from satellite remote sensing imagery. However, for fine-grained classification, the overhead features from satellite remote sensing imagery make it difficult to further distinguish certain land features, such as the tops of various houses appearing as buildings. Therefore, the introduction of side profile features is essential for land cover mapping.

[0003] Remote sensing land cover mapping primarily considers the properties of various materials visible on the Earth's surface. Early solutions classified each pixel based solely on spectral characteristics (Riggan Jr and Weih Jr, 2009), as this information is related to the underlying material. When spectral information is insufficient to distinguish land cover classes, contextual and texture information is often integrated (Myint, 2001), with classification performed by analyzing a fixed-size window around each pixel. Subsequently, unsupervised classification methods have been widely used for image segmentation and object-based classification. Using similar approaches, generating accurate large-scale maps based solely on overhead imagery is challenging, as relying solely on the spectral characteristics of objects is insufficient to identify different land cover types. Recent studies have also explored the use of data from other sources, such as road networks or OpenStreetMap (OSM) vector data, to supplement satellite remote sensing imagery, and have demonstrated their effectiveness through experimental research. At the same time, researchers have also explored the problem of land use mapping from a ground-level perspective, often using other data sources such as ground-based photographs from online repositories (e.g., Flickr, Instagram, and geographic sources) (Leung and Newsam, 2012; Zhuand Newsam, 2015; Tracewski et al., 2017; Zhu et al., 2018). These photographs provide key information about land cover features, such as school entrances, from a ground-level perspective that is typically hidden from overhead views.

[0004] Land cover mapping using geotagged photographs and satellite remote sensing imagery is an emerging field. Currently, most research relies on footprint or label data provided by urban planning departments, but the classification results are relatively crude. Therefore, improving the classification of land features (such as buildings) has become an urgent issue. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for mapping land cover using satellite remote sensing images that fuse sparse photos.

[0006] The technical solution of the present invention is: a method for mapping land cover using satellite remote sensing images fused with sparse photos, comprising:

[0007] Step S1: Obtain a satellite remote sensing image, and use a pre-trained ground scene recognition convolutional neural network Places-CNN model to perform scene recognition on sparse photos of the area corresponding to the satellite remote sensing image to obtain semantic features of the sparse photos;

[0008] Step S2: taking the area covered by the satellite remote sensing image as a study area, setting a predetermined range with the positioning information of the sparse photos with geographical tags as the center, and performing spatial interpolation on the semantic features within the study area within the range to obtain a sparse photo interpolation feature map;

[0009] Step S3: Inputting the sparse photo interpolation feature map and the satellite remote sensing image into an encoder-decoder structure model, using the encoder to perform layered encoding on the sparse photo interpolation feature map and the satellite remote sensing image, respectively, fusing the encoding features corresponding to each layer of the sparse photo interpolation feature map into the encoding features of the satellite remote sensing image of each corresponding layer to obtain fused features; using the decoder to decode the fused features to obtain a land cover map with classification results;

[0010] Step S4: Based on the sparse photos, the features therein are classified according to their materials; using the corresponding satellite remote sensing image pixels and corresponding materials within the interpolation range of the sparse photo interpolation feature map as supervision information, the land cover map with classification results outside the interpolation range is further classified, thereby achieving refined classification of feature types according to their materials.

[0011] Compared with the prior art, the present invention has the following advantages:

[0012] 1. The present invention discloses a method for mapping land cover using satellite remote sensing images by fusing sparse photos, and proposes a theoretical model and method for mapping land cover using remote sensing images by fusing sparse photo information, which can obtain land cover products with better accuracy.

[0013] 2. To address the previous difficulty in quickly and accurately extracting semantic features from sparse photos, this paper comprehensively utilizes convolutional neural network models and large-scale scene recognition datasets to automatically extract attribute information of fine features from sparse photos.

[0014] 3. Because the present invention can distinguish the material of land features based on sparse photo information, the pixels within the sparse photo interpolation area are used as supervisory information in remote sensing images. Compared with previous land cover mapping methods, the present invention can obtain more refined land feature attribute classification results.

[0015] 4. Currently, most existing technologies focus on the integration of urban street scenes and geological information. This invention uses sparse photo information and remote sensing images to integrate them, bringing new ideas for land cover classification in rural areas that lack street scene images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a method for mapping land cover using satellite remote sensing images that fuse sparse photos in an embodiment of the present invention;

[0017] Figure 2 Schematic diagram of the interpolation process of sparse photos with geotags in an embodiment of the present invention;

[0018] Figure 3 Schematic diagram of the encoder-decoder structural model architecture in an embodiment of the present invention;

[0019] Figure 4 Schematic diagram of a process for finely classifying satellite remote sensing images using buildings as an example in an embodiment of the present invention;

[0020] Figure 5 Schematic diagram of the process of finely segmenting buildings in satellite remote sensing images according to an embodiment of the present invention;

[0021] Figure 6 The figure is a structural block diagram of a satellite remote sensing image land cover mapping system that integrates sparse photos in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention provides a satellite remote sensing image land cover mapping method that fuses sparse photos, which fuses sparse photos into satellite remote sensing images to achieve more refined land cover classification.

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementation and in conjunction with the accompanying drawings.

[0024] Example 1

[0025] like Figure 1As shown, an embodiment of the present invention provides a method for mapping land cover using satellite remote sensing images by fusing sparse photos, comprising the following steps:

[0026] Step S1: Obtain satellite remote sensing images, use the pre-trained ground scene recognition convolutional neural network Places-CNN model to perform scene recognition on sparse photos of the corresponding area of ​​the satellite remote sensing image, and obtain the semantic features of the sparse photos;

[0027] Step S2: The area covered by the satellite remote sensing image is used as the study area, and a predetermined range is set with the positioning information of the sparse photos with geotags as the center. The semantic features are spatially interpolated within the range in the study area to obtain a sparse photo interpolation feature map;

[0028] Step S3: Input the sparse photo interpolation feature map and the satellite remote sensing image into the encoder-decoder structure model, use the encoder to perform layered encoding on the sparse photo interpolation feature map and the satellite remote sensing image respectively, fuse the encoding features corresponding to each layer of the sparse photo interpolation feature map into the corresponding encoding feature map of each layer of the satellite remote sensing image to obtain a fused feature map; use the decoder to decode the fused feature map to obtain a land cover map with classification results;

[0029] Step S4: Based on the sparse photos, the features are classified according to their materials; the corresponding satellite remote sensing image pixels and corresponding materials within the interpolation range of the sparse photo interpolation feature map are used as supervision information to further classify the land cover map with classification results outside the interpolation range, thereby achieving refined classification of feature types based on their materials.

[0030] In one embodiment, the above step S1: acquiring a satellite remote sensing image, using a pre-trained ground scene recognition convolutional neural network Places-CNN model to perform scene recognition on sparse photos of the area corresponding to the satellite remote sensing image, and obtaining semantic features of the sparse photos, specifically includes:

[0031] Step S11: Obtain satellite remote sensing images and corresponding sparse photos, pre-train the Places-CNN model using a large database of ground scene photos, input the sparse photos into the Places-CNN model, remove the last fully connected layer in the Places-CNN model, and output a 2048-dimensional semantic feature vector corresponding to each sparse photo;

[0032] This embodiment of the present invention first uses the Places-CNN deep convolutional neural network for ground scene recognition, which is pre-trained on the large-scale ground scene photo database Place365, to identify scenes in sparse photos. The sparse photos are then input into the trained Places-CNN model to extract their image semantic features. The last fully connected layer of the Places-CNN model is removed, and finally a 2048-dimensional semantic feature vector is output for each sparse photo.

[0033] Step S12: Utilize principal component analysis (PCA) to compress each 2048-dimensional semantic feature vector, reducing the feature dimension to 32 dimensions, and obtaining the semantic features of the sparse photo.

[0034] In one embodiment, step S2 above: taking the area covered by the satellite remote sensing image as the study area, setting a predetermined range with the positioning information of the sparse photos with geotags as the center, and performing spatial interpolation of semantic features within the study area within the range to obtain a sparse photo interpolation feature map, specifically includes:

[0035] Step S21: Since the sparse photos can reflect the information of the features within a certain spatial range, but the positioning information of the sparse photos is point information, the semantic features of the sparse photos within the area covered by the satellite remote sensing image are spatially interpolated. The spatial interpolation method is based on the following two assumptions: 1) for a given location, sparse photos that are closer have a greater impact than sparse photos that are farther away; 2) sparse photos can only cover a limited area around the location.

[0036] Step S22: Based on the assumption that the visual coverage of sparse photos is limited, a distance threshold is set to exclude the influence of photos at a longer distance, and a weight is set according to the distance:

[0037]

[0038] Among them, x is the center point, x i is its nearby point, w h (x,x i ) is x i The influence weight on x, d(x,x i ) is the distance between them, and h is used as the cutoff distance threshold;

[0039] Step S23: Based on the location information of the sparse photo, the semantic features obtained in step S1 are interpolated using the Nadaraya-Watson kernel regression method with the location information as the center:

[0040]

[0041] Among them, f(x) is the value of the pixel centered at x, f(x i) is the value of the nearby point, k is the number of nearby points;

[0042] Step S24: Within the coverage of the satellite remote sensing image, pixels that do not obtain any value after executing step S23 are automatically assigned a value of 0, thereby obtaining a sparse photo interpolation feature map.

[0043] like Figure 2 A schematic diagram of the interpolation process of sparse geotagged photos is shown.

[0044] In one embodiment, the above step S3: inputting the sparse photo interpolation feature map and the satellite remote sensing image into the model of the encoder-decoder structure, using the encoder to perform layered encoding on the sparse photo interpolation feature map and the satellite remote sensing image, respectively, fusing the encoding features corresponding to each layer of the sparse photo interpolation feature map into the encoding features of the corresponding satellite remote sensing image layers to obtain fused features; and using the decoder to decode the fused features to obtain a land cover map with classification results, specifically includes:

[0045] Step S31: Inputting the sparse photo interpolation feature map and the satellite remote sensing image into an encoder-decoder structure model, wherein the encoder consists of a satellite remote sensing image branch and a sparse photo branch, each branch has a five-layer structure, and each layer structure contains a predetermined number of CBR blocks and pooling layers, wherein the CBR block consists of convolution, batch normalization, and ReLU;

[0046] like Figure 3 The following diagram shows the encoder-decoder model architecture. The first layer of the satellite remote sensing imagery encoder consists of two CBR blocks and one pooling layer. The first layer of the sparse photo encoder is identical. The features extracted from the second CBR block in the first layer of the sparse photo encoder are fused with the features extracted from the second CBR block in the first layer of the remote sensing image encoder. The fused value is the pixel-by-pixel sum of the two feature layers and added after the two CBR blocks in the remote sensing image encoder. The encoders of both branches then undergo pooling before feeding the second layer. Similar methods are used to integrate the sparse photo features into the remote sensing image feature layer. Dropout is added to layers 3, 4, and 5 to prevent overfitting. In the remote sensing image encoder, layers 1 and 2 each have two CBR blocks, and layers 3, 4, and 5 each have three CBR blocks. Pooling layers follow the CBR blocks in each layer. The architecture of the sparse photo encoder is similar to that of the remote sensing image encoder, except that the fifth layer of the sparse photo encoder lacks pooling and dropout. The input sparse photo interpolation feature map and satellite remote sensing image are processed through a five-layer encoding structure. The dimension is reduced after each layer is processed, and low-dimensional features are finally output.

[0047] Define the encoder parameters as Ω = [ω(1), ω(2), ..., ω(L)], and the convolution operation as G = [g(1), g(2), ..., g(L)], where Represents the kth feature map in the lth layer, and the encoding process of the encoder is:

[0048]

[0049] Among them, σ is the function representation of the encoding, is the bias term of this layer;

[0050] The satellite remote sensing image and the sparse photo interpolation feature map are encoded by the remote sensing image branch and the sparse photo branch of the encoder respectively, and the features of the sparse photo interpolation feature map after each layer of encoding are fused into the encoding layer of the corresponding remote sensing image to obtain the fused feature.

[0051]

[0052] Among them, u and v represent the model parameters of satellite remote sensing image and sparse photo interpolation feature map respectively, a and b represent the values ​​of satellite remote sensing image and sparse photo interpolation feature map respectively, c and d are the bias terms of the two, and

[0053] Step S32: Define the decoder parameters as Δ = [δ(1), δ(2), ..., δ(L)], the deconvolution operation as S = [s(1), s(2), ..., s(L)], and the decoding process of the decoder is:

[0054]

[0055] Therefore, for a given input image X, the encoding-decoding process is obtained by the function ψ(X;Ω):

[0056] ψ(X;Ω)=S(G(X;Ω);Δ)

[0057] Among them, for a given training set X represents the input image with the same size H×W, including satellite remote sensing images and sparse photo interpolation feature maps, N is the number of features of the input image; Y represents the true label, where each pixel represents a true land cover category;

[0058] like Figure 3As shown in the figure, the decoder and encoder structures in this embodiment of the present invention correspond to each other, and each layer in the encoder structure is applied in reverse in the decoder. The pooling parameters recorded in the encoder are applied to the decoder to perform up-pooling of the feature map. CBR blocks are also used in the decoder, and dropout is applied at the same time to prevent overfitting and further improve model performance.

[0059] Step S33: Use the softmax function to obtain the probability distribution of each pixel in the input image belonging to each category c:

[0060]

[0061] Use the cross entropy loss value J to minimize the difference between the predicted class and the true class distribution;

[0062]

[0063] Among them, C is the total number of categories; y c is the true label of category c, y c ∈Y; is the probability of predicting class c.

[0064] In one embodiment, step S4: based on the sparse photos, classifying the features in the sparse photos according to their materials; using the corresponding satellite remote sensing image pixels and corresponding materials within the interpolation range of the sparse photo interpolation feature map as supervision information, further classifying the land cover map with classification results outside the interpolation range, thereby achieving refined classification of the feature types according to their materials, specifically includes:

[0065] Step S41: segmenting the n types of objects in the land cover map with the classification results into vectors according to their categories to obtain vector maps of different types of objects, and dividing the objects into subcategories according to the sparse photos corresponding to the vector maps;

[0066] Some types of land features in the overhead view of satellite remote sensing images have high similarity and are difficult to distinguish, but sparse photos can provide some side information, which is very important for refined classification. Figure 4 The figure shows a flow chart for fine-grained classification of satellite remote sensing images using buildings as an example. Taking the material classification of buildings as an example, the present invention first finds the building features in the land cover map and visually interprets them as reinforced concrete structures (building type 1), old-style blue-red tile structures (building type 2), iron structures (building type 3), resin tile structures (building type 4), and brick-and-earth structure pixels (building type 5) based on the corresponding sparse photos.

[0067] Step S42: Based on the subcategories of each feature in the sparse photos, find the remote sensing image pixels corresponding to their interpolation areas, assign corresponding subcategories, and use the subcategories as supervision information to subcategorize the land cover map with classification results outside the interpolation range, thereby achieving refined classification of feature types based on their materials.

[0068] Figure 5 A schematic diagram showing the process of fine-grained segmentation of buildings in satellite remote sensing images.

[0069] The present invention discloses a method for mapping land cover using satellite remote sensing images that fuse sparse photos, and proposes a theoretical model and method for mapping land cover using remote sensing images that fuse sparse photo information, which can obtain land cover products with better accuracy. In response to the problem that it was difficult to automatically extract semantic features quickly and accurately from sparse photos in the past, the present invention comprehensively utilizes convolutional neural network models and large-scale scene recognition data sets to automatically extract attribute information of fine land objects from sparse photos. Since the present invention can distinguish the material of buildings based on photo information and apply it as supervisory information to remote sensing images, compared with previous land cover mapping methods, the present invention can obtain more refined building category attribute classification results. Currently, most existing technologies focus on the integration of urban street scenes and geological information. The present invention uses sparse photo information and remote sensing images for integration, which brings new ideas to land cover classification in rural areas that lack street scene images.

[0070] Example 2

[0071] like Figure 6 As shown, an embodiment of the present invention provides a satellite remote sensing image land cover mapping system that integrates sparse photos, including the following modules:

[0072] A sparse photo semantic feature generation module 51 uses a pre-trained ground scene recognition convolutional neural network Places-CNN model to perform scene recognition on the sparse photos of the corresponding area of ​​the satellite remote sensing image to generate semantic features of the sparse photos;

[0073] A sparse photo interpolation feature map generating module 52 is configured to use the area covered by the satellite remote sensing image as a study area, set a range of a predetermined distance centered on the location information of the sparse photos with geotags, and perform spatial interpolation of semantic features within the range within the study area to obtain a sparse photo interpolation feature map;

[0074] The module 53 for generating a land cover map with classification results is used to input the sparse photo interpolation feature map and the satellite remote sensing image into an encoder-decoder structure model, use the encoder to perform layer-by-layer encoding on the sparse photo interpolation feature map and the satellite remote sensing image, fuse the encoding features corresponding to each layer of the sparse photo interpolation feature map with the encoding features of the corresponding satellite remote sensing image layers to obtain a fused feature map; and use the decoder to decode the fused feature map to obtain a land cover map with classification results.

[0075] The module 54 for generating a refined classified land cover map is used to classify the land features in the sparse photos according to their materials; the land cover map with classification results outside the interpolation range is further classified using the corresponding satellite remote sensing image pixels and corresponding materials within the interpolation range of the sparse photo interpolation feature map as supervision information, thereby achieving refined classification of land feature types according to their materials.

[0076] The above embodiments are provided for the purpose of describing the present invention only and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the present invention are intended to be within the scope of the present invention.

Claims

1. A method for land cover mapping using satellite remote sensing images fused with sparse photos, characterized in that: include: Step S1: Obtain a satellite remote sensing image, and use a pre-trained ground scene recognition convolutional neural network Places-CNN model to perform scene recognition on sparse photos of the area corresponding to the satellite remote sensing image to obtain semantic features of the sparse photos; Step S2: taking the area covered by the satellite remote sensing image as a study area, setting a predetermined range with the positioning information of the sparse photos with geographical tags as the center, and performing spatial interpolation on the semantic features within the study area within the range to obtain a sparse photo interpolation feature map; Step S3: inputting the sparse photo interpolation feature map and the satellite remote sensing image into an encoder-decoder structure model, using the encoder to perform layered encoding on the sparse photo interpolation feature map and the satellite remote sensing image, respectively, and fusing the encoding features corresponding to each layer of the sparse photo interpolation feature map into the encoding features of the satellite remote sensing image of each corresponding layer to obtain a fused feature; Decoding the fused features using a decoder to obtain a land cover map with classification results; Step S4: Based on the sparse photos, the features therein are classified according to their materials; using the corresponding satellite remote sensing image pixels and corresponding materials within the interpolation range of the sparse photo interpolation feature map as supervision information, the land cover map with classification results outside the interpolation range is further classified, thereby achieving refined classification of feature types according to their materials.

2. The method for land cover mapping using satellite remote sensing images fused with sparse photos according to claim 1, characterized in that: The step S1: obtaining a satellite remote sensing image, and performing scene recognition on sparse photos of the area corresponding to the satellite remote sensing image using a pre-trained ground scene recognition convolutional neural network Places-CNN model to obtain semantic features of the sparse photos, specifically includes: Step S11: Obtain satellite remote sensing images and corresponding sparse photos, pre-train a Places-CNN model using a large database of ground scene photos, input the sparse photos into the Places-CNN model, remove the last fully connected layer in the Places-CNN model, and output a 2048-dimensional semantic feature vector corresponding to each sparse photo; Step S12: Utilizing principal component analysis (PCA) to compress each of the 2048-dimensional semantic feature vectors, reducing the feature dimension to 32 dimensions, and obtaining the semantic features of the sparse photo.

3. The method for land cover mapping using satellite remote sensing images fused with sparse photos according to claim 2, characterized in that: The step S2: taking the area covered by the satellite remote sensing image as a study area, setting a predetermined range with the positioning information of the sparse photos with geographical tags as the center, and performing spatial interpolation on the semantic features within the study area within the range to obtain a sparse photo interpolation feature map, specifically includes: Step S21: Since the sparse photos can reflect the information of the features within a certain spatial range, but the positioning information of the sparse photos is point information, the semantic features of the sparse photos within the area covered by the satellite remote sensing image are spatially interpolated. The spatial interpolation method is based on the following two assumptions: 1) for a given location, sparse photos that are closer have a greater impact than sparse photos that are farther away; 2) sparse photos can only cover a limited area around the location. Step S22: Based on the assumption that the visual coverage of the sparse photos is limited, a distance threshold is set to exclude the influence of photos at a longer distance, and a weight is set according to the distance: in, is the center point, is a nearby point, yes right The influence weight of is the distance between them, Used as a cutoff distance threshold; Step S23: Based on the location information of the sparse photo, the semantic features obtained in step S1 are interpolated using the Nadaraya-Watson kernel regression method with the location as the center: in, Therefore The value of the pixel at the center, is the value of the nearby point, is the number of nearby points; Step S24: Within the coverage of the satellite remote sensing image, pixels that do not obtain any value after executing step S23 are automatically assigned a value of 0, thereby obtaining a sparse photo interpolation feature map.

4. The method for land cover mapping using satellite remote sensing images fused with sparse photos according to claim 3, characterized in that: Step S3: inputting the sparse photo interpolation feature map and the satellite remote sensing image into a model of an encoder-decoder structure, using an encoder to perform layered encoding on the sparse photo interpolation feature map and the satellite remote sensing image, respectively, and fusing the encoding features corresponding to each layer of the sparse photo interpolation feature map into the encoding features of the satellite remote sensing image of each corresponding layer to obtain a fused feature; The fused features are decoded using a decoder to obtain a land cover map with classification results, specifically including: Step S31: inputting the sparse photo interpolation feature map and the satellite remote sensing image into an encoder-decoder structure model, wherein the encoder consists of a satellite remote sensing image branch and a sparse photo branch, each branch has a five-layer structure, and each layer structure includes a predetermined number of CBR blocks and pooling layers, wherein the CBR block consists of convolution, batch normalization and ReLU; The parameters of the encoder are defined as , the convolution operation is ,in Indicates the The first layer feature maps, the encoding process of the encoder is: in, is the encoded function representation, is the bias term of this layer; The satellite remote sensing image and the sparse photo interpolation feature map are encoded by the remote sensing image branch and the sparse photo branch of the encoder respectively, and the features of the sparse photo interpolation feature map after each layer of encoding are fused to the corresponding encoding layer of the remote sensing image to obtain the fused feature. : in, and represent the model parameters of the satellite remote sensing image and the sparse photo interpolation feature map, respectively. and represents the value of the satellite remote sensing image and the sparse photo interpolation feature map, and is the bias term of the two, and , = + ; Step S32: define the decoder parameter as Δ , the deconvolution operation is , the decoding process of the decoder is: Therefore, for a given input image , whose encoding-decoding process is performed by the function get: Among them, for a given training set , Represents the same size The input image includes the satellite remote sensing image and the sparse photo interpolation feature map, is the number of features of the input image; represents the true label, where each pixel represents a true land cover class; Step S33: Use the softmax function to obtain the categories to which each pixel in the input image belongs. The probability distribution of : Using cross entropy loss , to minimize the difference between the predicted class and the true class distribution; ) Where C is the total number of categories; is the true label of category c, ; The predicted category is probability.

5. The method for land cover mapping using satellite remote sensing images fused with sparse photos according to claim 4, characterized in that: The step S4: classifying the features in the sparse photos according to their materials; using the corresponding satellite remote sensing image pixels and corresponding materials within the interpolation range of the sparse photo interpolation feature map as supervision information, further classifying the land cover map with classification results outside the interpolation range, thereby achieving refined classification of the feature types according to their materials, specifically including: Step S41: performing vector segmentation on n types of ground objects in the land cover map with classification results according to their categories to obtain vector diagrams of ground objects of different categories, and dividing the ground objects of each category into subcategories according to the sparse photos corresponding to the vector diagrams; Step S42: according to the subcategories of the objects in the sparse photos, find the satellite remote sensing image pixels corresponding to the interpolation area, assign corresponding subcategories, and use the subcategories as supervision information to subcategorize the land cover map with classification results outside the interpolation range.

6. A land cover mapping system using satellite remote sensing images fused with sparse photos, characterized by: Includes the following modules: A sparse photo semantic feature generation module is used to obtain satellite remote sensing images, use a pre-trained ground scene recognition convolutional neural network Places-CNN model to perform scene recognition on the sparse photos of the area corresponding to the satellite remote sensing image, and generate semantic features of the sparse photos; A sparse photo interpolation feature map generating module is configured to take the area covered by the satellite remote sensing image as a study area, set a range of a predetermined distance centered on the positioning information of the sparse photo with the geographical marker, and perform spatial interpolation of the semantic features within the range within the study area to obtain a sparse photo interpolation feature map; A module for generating a land cover map with classification results is configured to input the sparse photo interpolation feature map and the satellite remote sensing image into an encoder-decoder structure model, perform layered encoding on the sparse photo interpolation feature map and the satellite remote sensing image using an encoder, fuse the encoding features corresponding to each layer of the sparse photo interpolation feature map with the encoding features of the corresponding satellite remote sensing image layers to obtain a fused feature map; and decode the fused feature map using a decoder to obtain a land cover map with classification results. A module for generating a refined classified land cover map is used to classify the land features in the sparse photos according to their materials; using the corresponding satellite remote sensing image pixels and corresponding materials within the interpolation range of the sparse photo interpolation feature map as supervision information, the land cover map with classification results outside the interpolation range is further classified, thereby achieving refined classification of land feature types according to their materials.

Images