A deep learning-based urban open space mapping method and device

Abstract

The application discloses a kind of urban open space mapping method and device based on deep learning. The vegetation area of the image of study area is identified using Unet network, the vegetation area data of the image of study area is object-oriented image segmentation, and the segmentation object is obtained;According to the vegetation sample data selected from the historical image of study area, the sample object is obtained in combination with the segmentation object;The texture feature, spectral feature, index feature and vegetation phenology feature of the study area are used as the characteristic value of sample object;According to the characteristic value of sample object, the random forest classification is carried out on the object to be classified, and the green space classification result is obtained;According to the multi-source data and green space classification result, the urban open space map is obtained.In this process, the sample data is combined with object-oriented segmentation, and the vegetation mixed area is better separated;More image features are obtained by optimizing a variety of features and performing random forest classification based on the optimized features, so as to improve the classification effect.

Description

Technical Field

[0001] This application relates to the field of urban open space mapping technology, and in particular to an urban open space mapping method and apparatus based on deep learning. Background Technology

[0002] With the acceleration of urbanization in my country, studying the spatiotemporal dynamics of urban spatial patterns is of significant practical value for protecting and planning the structure and function of urban green spaces, and improving the urban and living environment. Urban green space refers to land in an urban area that is partially or completely covered by herbaceous plants, trees, shrubs, or other vegetation. In a broader sense, green space is also known as open space, referring to any open, undeveloped land accessible to the public, including parks, community gardens, public green spaces, squares, streets, and water bodies. As an important indicator of urban infrastructure and modernization, urban open space has multiple functions, including regional climate regulation and ecological environmental protection. To meet the needs of urban open space management and planning design, timely and accurate acquisition of its spatial distribution information is crucial.

[0003] Earth observation data acquired remotely (e.g., by satellite), in-situ (e.g., by sensors), or by citizens (e.g., through crowdsourcing) is considered an effective, reliable, timely, and continuous source of information. It can effectively supplement or enhance traditional data sources in urban areas, and is of great significance for urban ecological environment construction and sustainable development. Remote sensing data, in particular, has advantages such as continuous coverage, timeliness, dynamic nature, and low cost, and is widely used in the extraction of urban green spaces.

[0004] Therefore, while artificial intelligence and deep learning technologies are currently used for the automatic extraction of urban open spaces from high-resolution remote sensing images, the accuracy of the resulting urban open space maps is low due to poor classification results when classifying green spaces from remote sensing data. Summary of the Invention

[0005] This application provides a method and apparatus for mapping urban open spaces based on deep learning. By utilizing vegetation models, mixed sample objects, and phenological features, it can achieve fine-type classification of vegetation in remote sensing images, thereby improving classification accuracy.

[0006] Firstly, this application provides a deep learning-based method for mapping urban open spaces, which includes:

[0007] The preprocessed first image of the study area is input into the vegetation model to obtain vegetation area data. The vegetation model is a trained U-net deep neural network model, and the training samples include the vegetation index sample dataset of the first image of the study area.

[0008] Object-oriented image segmentation is performed on vegetation area data to obtain segmented objects;

[0009] Vegetation sample data were selected based on historical images of the study area, and multi-category sample objects were obtained based on the vegetation sample data and segmentation objects.

[0010] The texture features, spectral features, and index features extracted from the first image of the study area, as well as the vegetation phenology features extracted from the preprocessed second image of the study area, are superimposed to obtain a feature set. The mean of each feature in the feature set is used as the feature value of the multi-class sample objects.

[0011] Based on the feature values ​​of multi-class sample objects, random forest classification and reclassification are performed on the objects to be classified in the vegetation area data to obtain the green space classification results.

[0012] Based on the road network area data, park and green space data, water body area data, and green space classification results of the study area, an urban open space map was obtained.

[0013] Optionally, the process of obtaining the vegetation model includes:

[0014] The vegetation index of the first image of the study area was classified, and the optimal threshold was selected based on the classification results.

[0015] Vegetated and non-vegetated areas were extracted from the first image of the study area based on the optimal threshold, and the vegetated and non-vegetated areas were used as labeled data.

[0016] The first image and labeled data of the study area with consistent coordinates were cropped using the regular grid method to obtain a vegetation index sample dataset;

[0017] The vegetation index sample dataset was divided into a training sample set and a test sample set according to a preset ratio using a random sampling method.

[0018] Image enhancement and image transformation are performed on the training sample set to obtain the processed training sample set;

[0019] The U-net deep neural network model is trained using the processed training sample set and tested using the test sample set. After training, the trained U-net deep neural network model is used as the vegetation model.

[0020] Optionally, object-oriented image segmentation is performed on the vegetation area data to obtain segmented objects, including:

[0021] Masking is performed on the vegetation area data to obtain a mask file;

[0022] Based on the spatial, texture, and spectral information in the first image of the study area, the mask file is segmented to obtain the initial segmentation object;

[0023] The initial segmented objects are merged according to their attribute types to obtain the segmented objects.

[0024] Optionally, vegetation sample data are selected based on historical images of the study area, and multi-category sample objects are obtained based on the vegetation sample data and segmented objects, including:

[0025] By comparing historical images of the study area during the vegetation growing season and the vegetation falling season, polygonal feature sample data of various vegetation types were selected.

[0026] Convert polygon feature sample data into point feature sample data, and define the attributes of the point feature sample data;

[0027] The attributes of point feature sample data are assigned to the segmentation object through a one-to-many spatial connection, and the segmentation object containing point feature sample data is used as the sample object.

[0028] Based on the attribute table of the sample object, multiple records of the same sample object are merged to obtain new sample object attributes;

[0029] Assign new sample object attributes to the corresponding sample object to obtain multi-category sample objects.

[0030] Optionally, the texture features, spectral features, and index features extracted from the first image of the study area, along with the vegetation phenological features extracted from the preprocessed second image of the study area, are superimposed in bands to obtain a feature set, including:

[0031] Texture features are obtained by calculating the second-order matrix texture features of the first image of the study area using the gray-level co-occurrence matrix;

[0032] The mean values ​​of the blue, green, red, and near-infrared bands in the first image of the study area were selected as the spectral characteristics of each object in the first image of the study area.

[0033] By using band calculations, multiple indices of the first image of the study area are obtained as index features;

[0034] The monthly maximum values ​​of the annual vegetation phenological characteristics of the second image of the study area were synthesized. After resampling and band overlay, the images were registered with the first image of the study area to obtain the vegetation phenological characteristics.

[0035] The feature set is obtained by overlaying texture features, spectral features, index features and vegetation phenology features in bands.

[0036] Optionally, random forest classification and reclassification are performed on the objects to be classified in the vegetation area data based on the feature values ​​of multi-class sample objects to obtain green space classification results, including:

[0037] The importance sequence is obtained by sorting the eigenvalues ​​from largest to smallest using the Gini importance index.

[0038] The feature with a preset number of importance sequences is used as the classification feature of multi-class sample objects, and the objects to be classified are classified by random forest according to the classification feature to obtain the initial classification result;

[0039] The initial classification results are merged and reclassified to obtain the green space classification results.

[0040] Optionally, the method further includes:

[0041] The accuracy of green space classification results is evaluated using overall classification accuracy and Kappa coefficient.

[0042] Optionally, based on the road network area data, park and green space data, water body area data, and green space classification results of the study area, an urban open space map is obtained, including:

[0043] Based on the OSM road network data of the study area, buffer zones are created for roads of different levels to generate road network surface data;

[0044] Based on the OSM road network data of the study area, urban functional plots were obtained and overlaid with vegetation area data to extract urban functional green spaces.

[0045] The POI data of the study area, which is a park, are assigned the function of urban green space, and park green space data is extracted.

[0046] Based on the first image of the study area, the improved normalized differential water index was calculated.

[0047] The water body extraction threshold is determined based on the normalized differential water body index and threshold segmentation method, and the water body area data in the first image of the study area is extracted using the water body threshold.

[0048] A map of urban open space is obtained by overlaying road network area data, park and green space data, water area data, and green space classification results.

[0049] Optionally, the method further includes:

[0050] By overlaying the evergreen forest data, deciduous forest data, evergreen-deciduous mixed forest data, and grassland data from the green classification results with road network area data, park green space data, and water body area data, a summer urban open space map is obtained.

[0051] By overlaying the evergreen forest data and evergreen-deciduous mixed forest data from the green classification results with the road network area data, park green space data, and water body area data, a winter urban open space map is obtained.

[0052] By overlaying the summer and winter urban open space maps, non-overlapping areas are obtained, and these non-overlapping areas are taken as the areas of change in urban open space.

[0053] Secondly, this application provides a deep learning-based urban open space mapping device, which includes:

[0054] The input unit is used to input the preprocessed first image of the study area into the vegetation model to obtain vegetation area data. The vegetation model is a trained U-net deep neural network model, and the training samples include a vegetation index sample dataset of the first image of the study area.

[0055] The segmentation unit is used to perform object-oriented image segmentation on vegetation area data to obtain segmented objects.

[0056] The selection unit is used to select vegetation sample data based on historical images of the study area, and to obtain multi-category sample objects based on the vegetation sample data and segmentation objects;

[0057] The overlay unit is used to overlay the texture features, spectral features, and index features extracted from the first image of the study area and the vegetation phenology features extracted from the preprocessed second image of the study area to obtain a feature set, and the mean of each feature in the feature set is used as the feature value of the multi-class sample object.

[0058] The classification unit is used to perform random forest classification and reclassification of objects to be classified in vegetation area data based on the feature values ​​of multi-class sample objects, and to obtain green space classification results.

[0059] The acquisition unit is used to obtain an urban open space map based on the road network area data, park and green space data, water area data, and green space classification results of the study area.

[0060] Optionally, the device also includes a training unit, specifically for:

[0061] The vegetation index of the first image of the study area was classified, and the optimal threshold was selected based on the classification results.

[0062] Vegetated and non-vegetated areas were extracted from the first image of the study area based on the optimal threshold, and the vegetated and non-vegetated areas were used as labeled data.

[0063] The first image and labeled data of the study area with consistent coordinates were cropped using the regular grid method to obtain a vegetation index sample dataset;

[0064] The vegetation index sample dataset was divided into a training sample set and a test sample set according to a preset ratio using a random sampling method.

[0065] Image enhancement and image transformation are performed on the training sample set to obtain the processed training sample set;

[0066] The U-net deep neural network model is trained using the processed training sample set and tested using the test sample set. After training, the trained U-net deep neural network model is used as the vegetation model.

[0067] Optionally, the segmentation unit is specifically used for:

[0068] Masking is performed on the vegetation area data to obtain a mask file;

[0069] Based on the spatial, texture, and spectral information in the first image of the study area, the mask file is segmented to obtain the initial segmentation object;

[0070] The initial segmented objects are merged according to their attribute types to obtain the segmented objects.

[0071] Optionally, the selected unit is specifically used for:

[0072] By comparing historical images of the study area during the vegetation growing season and the vegetation falling season, polygonal feature sample data of various vegetation types were selected.

[0073] Convert polygon feature sample data into point feature sample data, and define the attributes of the point feature sample data;

[0074] The attributes of point feature sample data are assigned to the segmentation object through a one-to-many spatial connection, and the segmentation object containing point feature sample data is used as the sample object.

[0075] Based on the attribute table of the sample object, multiple records of the same sample object are merged to obtain new sample object attributes;

[0076] Assign new sample object attributes to the corresponding sample object to obtain multi-category sample objects.

[0077] Optionally, the stacking unit is specifically used for:

[0078] Texture features are obtained by calculating the second-order matrix texture features of the first image of the study area using the gray-level co-occurrence matrix;

[0079] The mean values ​​of the blue, green, red, and near-infrared bands in the first image of the study area were selected as the spectral characteristics of each object in the first image of the study area.

[0080] By using band calculations, multiple indices of the first image of the study area are obtained as index features;

[0081] The monthly maximum values ​​of the annual vegetation phenological characteristics of the second image of the study area were synthesized. After resampling and band overlay, the images were registered with the first image of the study area to obtain the vegetation phenological characteristics.

[0082] The feature set is obtained by overlaying texture features, spectral features, index features and vegetation phenology features in bands.

[0083] Optionally, the classification unit is specifically used for:

[0084] The importance sequence is obtained by sorting the eigenvalues ​​from largest to smallest using the Gini importance index.

[0085] The feature with a preset number of importance sequences is used as the classification feature of multi-class sample objects, and the objects to be classified are classified by random forest according to the classification feature to obtain the initial classification result;

[0086] The initial classification results are merged and reclassified to obtain the green space classification results.

[0087] Optionally, the device further includes:

[0088] The evaluation unit is used to evaluate the accuracy of green space classification results using the overall classification accuracy and the Kappa coefficient.

[0089] Optionally, the obtained unit is specifically used for:

[0090] Based on the OSM road network data of the study area, buffer zones are created for roads of different levels to generate road network surface data;

[0091] Based on the OSM road network data of the study area, urban functional plots were obtained and overlaid with vegetation area data to extract urban functional green spaces.

[0092] The POI data of the study area, which is a park, are assigned the function of urban green space, and park green space data is extracted.

[0093] Based on the first image of the study area, the improved normalized differential water index was calculated.

[0094] The water body extraction threshold was determined based on the normalized differential water body index and threshold segmentation method, and the water body area data in the first image of the study area was extracted using the water body threshold.

[0095] A map of urban open space is obtained by overlaying road network area data, park and green space data, water area data, and green space classification results.

[0096] Optionally, the obtaining unit is also used for:

[0097] By overlaying the evergreen forest data, deciduous forest data, evergreen-deciduous mixed forest data, and grassland data from the green classification results with road network area data, park green space data, and water body area data, a summer urban open space map is obtained.

[0098] By overlaying the evergreen forest data and evergreen-deciduous mixed forest data from the green classification results with the road network area data, park green space data, and water body area data, a winter urban open space map is obtained.

[0099] By overlaying the summer and winter urban open space maps, non-overlapping areas are obtained, and these non-overlapping areas are taken as the areas of change in urban open space.

[0100] Thirdly, this application provides an electronic device including a memory and a processor:

[0101] Memory is used to store computer programs;

[0102] The processor is used to execute the method provided in the first aspect above according to the computer program.

[0103] Fourthly, this application also provides a computer-readable storage medium for storing a computer program for performing the method provided in the first aspect above.

[0104] Therefore, this application has the following beneficial effects:

[0105] This application provides a deep learning-based method for mapping urban open spaces. First, a pre-processed first image of the study area is input into a vegetation model to obtain vegetation region data. Object-oriented image segmentation is then performed on the vegetation region data to obtain segmented objects. Vegetation sample data is selected based on historical images of the study area, and multi-class sample objects are obtained based on the vegetation sample data and the segmented objects. Texture features, spectral features, and index features extracted from the first image of the study area, along with vegetation phenological features extracted from the pre-processed second image of the study area, are overlaid to obtain a feature set. The mean value of each feature in the feature set is used as the feature value of the multi-class sample objects. Then, random forest classification and reclassification are performed on the objects to be classified in the vegetation region data based on the feature values ​​of the multi-class sample objects to obtain green space classification results. Finally, an urban open space map is obtained based on the road network area data, park green space data, water body area data, and green space classification results of the study area. In this process, selected sample points or surfaces can be combined with object-oriented segmentation to select samples from mixed land cover areas. This allows for better classification of mixed vegetation areas. Then, optimal features are obtained through spectral features, texture features, vegetation indices, and vegetation phenological features. Finally, random forest classification is performed based on these optimal features, enabling the random forest model to acquire more image features and thus improve classification performance. Furthermore, using vegetation models and random forests for green space extraction and automatic classification allows for large-scale automatic vegetation extraction, improving classification efficiency. Attached Figure Description

[0106] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0107] Figure 1 This is a flowchart illustrating a deep learning-based urban open space mapping method in an embodiment of this application.

[0108] Figure 2 This is a schematic diagram of an object-oriented image segmentation and classification process in an embodiment of this application;

[0109] Figure 3 This is a flowchart illustrating an embodiment of a deep learning-based urban open space mapping method in this application.

[0110] Figure 4 This is a flowchart illustrating an embodiment of a deep learning-based urban open space mapping method in this application.

[0111] Figure 5This is a schematic diagram of the vegetation area extraction results in the study area in this application embodiment;

[0112] Figure 6 This is a schematic diagram showing the segmentation results of the vegetation area in the study area in the embodiments of this application;

[0113] Figure 7 This is a schematic diagram illustrating the redefinition of sample object attributes in an embodiment of this application;

[0114] Figure 8 This is a schematic diagram of the multi-category sample objects in the study area in an embodiment of this application;

[0115] Figure 9 This is a schematic diagram illustrating the calculation formula and description of texture features in the embodiments of this application;

[0116] Figure 10 This is a schematic diagram of the index features selected in the embodiments of this application;

[0117] Figure 11 This is a schematic diagram showing the ranking of the feature importance of vegetation areas in the study area in this application embodiment;

[0118] Figure 12 This is a schematic diagram of the reclassification results of green space in the study area in this embodiment of the application;

[0119] Figure 13 This is a schematic diagram of the green space classification results in the study area in this application embodiment;

[0120] Figure 14 This is a schematic diagram of the summer urban open space in the study area in the embodiments of this application;

[0121] Figure 15 This is a schematic diagram of the urban open space in the study area during winter, as described in the embodiments of this application.

[0122] Figure 16 This is a schematic diagram showing the changing areas of urban open space in the study area in this application embodiment;

[0123] Figure 17 This is a schematic diagram of the structure of a deep learning-based urban open space mapping device in an embodiment of this application;

[0124] Figure 18 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0125] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0126] In the embodiments of this application, the word "first" in the name "first image of the study area" is only used for name identification and does not represent the first in order. The same rule applies to "second", "third", etc.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0128] First, the technical terms involved in the embodiments of this application are explained as follows:

[0129] (1) GF-1: Gaofen-1 is a group of satellite sensors that provide high-resolution panchromatic and multispectral remote sensing data for monitoring and analyzing changes in the Earth's surface and environmental characteristics.

[0130] (2) Sentinel-2: Sentinel-2 is a group of satellite sensors that provide high-resolution, multispectral time-series remote sensing data for monitoring and analyzing changes in the Earth's surface and environmental features.

[0131] (3) POI Data: POI (Point of Interest) data typically refers to a series of geographic location information, such as the coordinates of shops, restaurants, attractions, etc. In computer technology, POI data can be used for Geographic Information System (GIS) analysis or to display these locations in map software. POI density reflects the concentration of specific points of interest within a certain area, generally including educational institutions, commercial areas, residential buildings, parks, companies, etc. A high density of restaurants or shopping centers may indicate more active economic activity in that area.

[0132] (4) OSM Road Network Data: OpenStreetMap (OSM) is an open, collaborative geographic information database that provides free geographic data worldwide, including road network data. As a collaborative network mapping project, OSM provides users with free and open-source global vector data.

[0133] (5) Texture features: Image texture refers to the detailed structure of an image, which is the frequency of change of image hue as a hierarchical function in space. It is a combination of single, small features. Different land features have different texture features. Therefore, objects with similar spectral features are often identified by their texture differences.

[0134] (6) Feature Selection: Feature selection refers to choosing the subset of features with the best classification or regression capabilities from the original data to improve the performance and efficiency of the machine learning model. It helps to reduce the dimensionality of the feature space, reduce the risk of overfitting, and improve the interpretability of the model.

[0135] (7) Random Forest: Random Forest is an ensemble learning algorithm used for classification and regression problems. It consists of multiple decision trees, each of which is trained independently. Random Forest determines the final classification or regression result by voting or averaging the predictions of each decision tree.

[0136] (8) Overall classification accuracy: This is an indicator that evaluates the prediction accuracy of a classification model across all categories.

[0137] (9) Kappa coefficient: It is a metric used to evaluate the consistency of classifier performance in multi-class or binary classification problems. It takes into account both the accuracy of the classifier and the randomness of the data itself.

[0138] (10) Confusion matrix: also known as error matrix, is a standard format for representing accuracy evaluation, expressed as an n x n matrix. Specific evaluation indicators include overall accuracy, cartographic accuracy, and user accuracy.

[0139] Currently, artificial intelligence and deep learning technologies are commonly used when acquiring urban open space data. However, when classifying green areas from remote sensing data, poor classification results lead to low accuracy in the subsequently generated urban open space maps. This is specifically due to the following four aspects:

[0140] (1) In terms of classification system, the current classification system of green space based on remote sensing data is not refined enough and cannot accurately classify different types of vegetation, water bodies and urban parks.

[0141] (2) In terms of mixed pixel classification, when selecting green space samples, both object-oriented classification and pixel-based classification will have areas with multiple mixed land features. Traditional sample selection can only select a single land feature and it is difficult to select multiple mixed land feature units, so it is also difficult to distinguish mixed land feature areas.

[0142] (3) In terms of the selection of classification features, land use classification mostly uses only the spectral information of land features for feature extraction, thus ignoring other spatial features of the image, such as spatial structure and texture.

[0143] (4) In terms of classification efficiency, existing studies are mostly based on NDVI and FVC images, using threshold calculation and segmentation to extract vegetation range. This method has poor adaptability to different regions, requires multiple threshold screenings for different regions, involves a lot of manual intervention, and has high time costs, resulting in low classification efficiency and the inability to quickly extract vegetation range in different regions.

[0144] In response, this application addresses these four aspects in its embodiments:

[0145] (1) In terms of classification system, the embodiments of this application combine multi-source remote sensing (e.g., GF-1 image and Sentinel-2 image) and geographic information data (POI point data and OSM road network data) to realize the automated and fine classification of urban green space, and the extraction of water bodies and urban parks.

[0146] (2) In terms of mixed pixel classification, the embodiments of this application combine the selection of sample points or sample surfaces with object-oriented segmentation, and select samples of mixed land features, such as evergreen-deciduous mixed forest areas, which can better classify and separate mixed vegetation areas.

[0147] (3) In terms of the selection of classification features, the embodiments of this application combine spectral features, texture features, vegetation index and vegetation phenology features to perform feature optimization, and finally classify the object-oriented random forest according to the optimized features, so that the random forest model can obtain more image features and improve the classification accuracy.

[0148] (4) In terms of classification efficiency, the embodiments of this application use the U-Net deep neural network model (vegetation model) and random forest to realize green space extraction and automatic classification, which can realize large-scale automatic vegetation extraction and improve classification efficiency.

[0149] In specific implementation, the method may include, for example, the following steps: First, input the preprocessed first image of the study area into the vegetation model to obtain vegetation area data; perform object-oriented image segmentation on the vegetation area data to obtain segmented objects; select vegetation sample data based on historical images of the study area, and obtain multi-class sample objects based on the vegetation sample data and segmented objects; superimpose the texture features, spectral features, and index features extracted from the first image of the study area and the vegetation phenological features extracted from the preprocessed second image of the study area to obtain a feature set, and use the mean of each feature in the feature set as the feature value of the multi-class sample objects; then perform random forest classification and reclassification on the objects to be classified in the vegetation area data based on the feature values ​​of the multi-class sample objects to obtain green space classification results; finally, obtain an urban open space map based on the road network area data, park green space data, water area data, and green space classification results of the study area.

[0150] As can be seen, the method provided in this application combines the selected sample points or sample surfaces with object-oriented segmentation to select samples of mixed land features, which can better classify mixed vegetation areas. Then, it obtains preferred features through features such as spectral features, texture features, vegetation index and vegetation phenology features. Finally, it performs random forest classification based on the preferred features, so that the random forest model can obtain more image features, thereby improving the classification effect.

[0151] To facilitate understanding of the specific implementation of the deep learning-based urban open space mapping method provided in the embodiments of this application, the following description will be provided in conjunction with the accompanying drawings.

[0152] It should be noted that the entity implementing this deep learning-based urban open space mapping method can be the deep learning-based urban open space mapping device provided in the embodiments of this application. This deep learning-based urban open space mapping device can be carried in an electronic device or a functional module of an electronic device. The electronic device in the embodiments of this application can be any device capable of implementing the deep learning-based urban open space mapping method in the embodiments of this application, such as an Internet of Things (IoT) device.

[0153] Figure 1 This is a flowchart illustrating a deep learning-based urban open space mapping method provided in an embodiment of this application. This method can be applied to a deep learning-based urban open space mapping device, which may be, for example, a... Figure 17 The deep learning-based urban open space mapping device 1700 shown, or the deep learning-based urban open space mapping device can also be integrated into... Figure 18 Functional modules in the electronic device 1800 shown.

[0154] like Figure 1 As shown, the method includes the following steps S101 to S106:

[0155] S101: Input the preprocessed first image of the study area into the vegetation model to obtain vegetation area data. The vegetation model is a trained U-net deep neural network model, and the training samples include the vegetation index sample dataset of the first image of the study area.

[0156] To obtain a highly accurate urban open space map, the first preprocessed image of the study area is input into a vegetation model to obtain vegetation region data. Object-oriented image segmentation is then performed on the vegetation region data to obtain segmented objects. Vegetation sample data is selected based on historical images of the study area, and multi-category sample objects are obtained based on the vegetation sample data and the segmented objects. Texture features, spectral features, and index features extracted from the first image of the study area, along with vegetation phenological features extracted from the preprocessed second image of the study area, are overlaid to obtain a feature set. The mean value of each feature in the feature set is used as the feature value of the multi-category sample objects. Then, random forest classification and reclassification are performed on the objects to be classified in the vegetation region data based on the feature values ​​of the multi-category sample objects to obtain green space classification results. Finally, an urban open space map is obtained based on the road network area data, park green space data, water body area data, and green space classification results of the study area. Therefore, this embodiment obtains the vegetation region through S101, which serves as a preparatory step for obtaining the segmented objects.

[0157] Before performing S101, this embodiment of the application further includes: preparing data and data preprocessing, wherein the prepared data includes the first image of the study area (GF-1 image), the second image of the study area (Sentinel-2 image), the POI point data of the study area, and the OSM road network data of the study area.

[0158] The above data preprocessing includes: (1) performing radiometric calibration, atmospheric correction, orthorectification, registration, image fusion, and geometric fine correction on the multispectral and panchromatic images of the first image (GF-1 image) of the study area. (2) performing cloud cover removal, radiometric calibration, atmospheric correction, image fusion, and geometric registration on the second image (Sentinel-2 image) of the study area. (3) performing operations such as deleting duplicate information, processing missing values, checking topological relationships, such as eliminating overlaps, intersections, self-intersections, and self-overlaps in the road network, boundary checks, and attribute data cleaning on the traffic network data of the study area to obtain OSM road network data of the study area.

[0159] As an example, the process of obtaining the vegetation model in S101 may include: (1) Creating a vegetation index sample dataset: classifying the vegetation index of the first image of the study area and selecting the best threshold according to the classification results; then extracting the vegetation area and non-vegetation area from the first image of the study area according to the best threshold, and using the vegetation area and non-vegetation area as labeled data; finally, using the regular grid method to crop the first image of the study area and the labeled data with consistent coordinates to obtain a vegetation index sample dataset; (2) Dividing the vegetation index sample dataset: using the random sampling method, dividing the vegetation index sample dataset into a training sample set and a test sample set according to a preset ratio; (3) Processing the training sample set: performing image enhancement and image transformation on the training sample set to obtain the processed training sample set; (4) Training the model: using the processed training sample set to train the U-net deep neural network model, and using the test sample set to test the U-net deep neural network model, and using the completed U-net deep neural network model as the vegetation model after training.

[0160] In this process, the U-net deep neural network model is used to segment and extract vegetation areas, which can realize the automatic extraction of large-scale vegetation.

[0161] S102: Perform object-oriented image segmentation on the vegetation area data to obtain the segmented objects.

[0162] As an example, S102 may include: S1021, performing masking processing on the vegetation area data to obtain a mask file; S1022, segmenting the mask file based on spatial, texture, and spectral information in the first image of the study area to obtain initial segmented objects, wherein the preferred segmentation scale is 25; S1023, merging the initial segmented objects according to attribute type to obtain segmented objects, wherein the preferred merging scale is 90. Selecting the preferred segmentation scale and preferred merging scale allows for the segmentation of objects with different textures, brightness, and saturation, achieving better segmentation results.

[0163] In this process, the ESP algorithm can also be used for multi-scale segmentation. The optimal segmentation scale can be found based on the maximum value of the variance change rate to obtain the segmented objects.

[0164] The above-mentioned image segmentation operation using object-oriented methods can be, for example... Figure 2 The diagram illustrates the segmentation process in the object-oriented image segmentation and classification workflow.

[0165] S103: Select vegetation sample data based on historical images of the study area, and obtain multi-category sample objects based on the vegetation sample data and segmentation objects.

[0166] As an example, S103 may include: S1031, by comparing historical images of the study area during the vegetation growing season and historical images of the study area during the vegetation falling season, selecting polygonal feature sample data of multiple vegetation types, where the historical images of the study area can be, for example, historical remote sensing images from Google Earth Pro, and the polygonal feature sample data of multiple vegetation types can include: evergreen forest, deciduous forest, grassland, etc.; S1032, converting the polygonal feature sample data into point feature sample data, and defining the attributes of the point feature sample data, for example, defining the attribute of evergreen forest as 1, the attribute of deciduous forest as 2, and the attribute of grassland as 3; S1033, assigning the attributes of the point feature sample data to the segmentation object through a one-to-many spatial connection, and using the segmentation object containing the point feature sample data as the sample object; S1034, merging multiple records of the same sample object according to the attribute table of the sample object to obtain new sample object attributes; S1035, assigning the new sample object attributes to the corresponding sample object to obtain multi-category sample objects.

[0167] In S1031, point feature sample data of multiple vegetation types can also be directly selected from historical images of the study area.

[0168] In this process, by combining vegetation sample data of point features and area features with segmentation objects, multiple vegetation mixed area samples can be selected, such as evergreen-deciduous mixed forest areas, so as to better classify and separate vegetation mixed areas in the future.

[0169] S104: The texture features, spectral features, and index features extracted from the first image of the study area, as well as the vegetation phenology features extracted from the preprocessed second image of the study area, are superimposed to obtain a feature set, and the mean of each feature in the feature set is used as the feature value of the multi-class sample objects.

[0170] As an example, S104 may include: S1041, calculating the texture features of the second-order matrix of the first image of the study area using the gray-level co-occurrence matrix to obtain texture features; S1042, selecting the mean values ​​of the blue, green, red, and near-infrared bands in the first image of the study area as the spectral features of each object in the first image of the study area; S1043, using band operations to obtain multiple indices of the first image of the study area as index features; S1044, synthesizing the monthly maximum values ​​of the annual vegetation phenological features of the second image of the study area, and after resampling and band overlay, performing image registration with the first image of the study area to obtain vegetation phenological features; S1045, overlaying the texture features, spectral features, index features, and vegetation phenological features by band overlay to obtain a feature set.

[0171] In S1044, the winter index features and summer index features of the second image of the study area can be used as vegetation phenological features to achieve fine classification of vegetation. However, the effect of extracting grassland may not be as good as using the vegetation phenological features of the whole year as vegetation phenological features.

[0172] In this process, the phenological, textural, and spectral characteristics of vegetation can be used as feature values ​​for multi-category sample objects to facilitate the subsequent extraction of mixed regions of multi-type vegetation.

[0173] S105: Based on the feature values ​​of multi-category sample objects, perform random forest classification and reclassification on the objects to be classified in the vegetation area data to obtain the green space classification results.

[0174] As an example, S105 may include: S1051, sorting the feature values ​​from largest to smallest importance using Gini importance to obtain an importance sequence; S1052, using the features with a preset number of positions in the importance sequence as classification features for multi-class sample objects, and performing random forest classification on the objects to be classified according to the classification features to obtain an initial classification result, wherein the preset number of positions can be the first 12; S1053, merging and reclassifying the initial classification results to obtain a green space classification result.

[0175] In S1052, other machine learning algorithms such as support vector machine (SVM), maximum likelihood method, principal component analysis, K nearest neighbor algorithm, etc. can also be used for pixel-based or object-oriented classification. However, random forest performs better in handling high-dimensional data and feature selection, and has good robustness to missing values ​​and outliers, making it suitable for handling large-scale datasets.

[0176] In this process, feature optimization is performed on the feature values ​​of multi-category sample objects, and random forest is used to classify the objects to be classified in the vegetation area data, so as to realize automatic classification of green space, improve classification efficiency, and achieve an accuracy of over 80%.

[0177] In addition, after obtaining the green space classification results, it is necessary to evaluate the accuracy of the green space classification results, specifically including: evaluating the accuracy of the green space classification results using the overall classification accuracy and the Kappa coefficient. The formula for calculating the overall classification accuracy is shown in formula (1):

[0178]

[0179] Among them, TP i FN represents the number of samples predicted to be of class i and whose true label is also class i. i This represents the number of samples whose true label is class i but are predicted to be other classes, and N represents the total number of classes.

[0180] The formulas for calculating the Kappa coefficient are shown in formulas (2), (3), and (4):

[0181]

[0182] Where Po represents the classifier's accuracy (Observed Accuracy), which is the sum of the number of correctly classified samples in each class divided by the total number of samples, i.e., the overall classification accuracy, and Pe represents the accuracy of random prediction (Expected Accuracy).

[0183]

[0184] Among them, TP i FP represents the number of samples predicted to be of class i and whose true label is also class i. i This represents the number of samples whose true label is class i but are predicted to be other classes, and N represents the total number of classes.

[0185]

[0186] Where A1, A2, ..., AN represent the number of samples whose actual label is class i, B1, B2, ..., BN represent the number of samples whose predicted label is class i, and N represents the total number of classes.

[0187] It should be noted that the Kappa coefficient ranges from -1 to 1. Generally, a value closer to 1 indicates better classifier performance, a value closer to 0 indicates performance similar to random prediction, and a value closer to -1 indicates very poor classifier performance. In this embodiment, the overall classification accuracy and the Kappa coefficient are used to obtain a confusion matrix, which can intuitively represent the accuracy evaluation results.

[0188] S106: Based on the road network area data, park and green space data, water area data, and green space classification results of the study area, obtain the urban open space map.

[0189] As an example, S106 may include: S1061, buffering roads of different levels based on OSM road network data of the study area to generate road network areal data; S1062, obtaining urban functional plots based on OSM road network data of the study area and overlaying them with vegetation area data to extract urban functional green space; S1063, assigning POI data with the attribute of park in the study area to urban functional green space to extract park green space data; S1064, calculating an improved normalized difference water body index based on the first image of the study area; S1065, determining the water body extraction threshold based on the normalized difference water body index and threshold segmentation method, and using the water body threshold to extract water body area data in the first image of the study area; S1066, overlaying the road network areal data, park green space data, water body area data, and green space classification results to obtain an urban open space map.

[0190] In S1063, POIs with the attribute of scenic spots in the first-level classification can be used to extract urban open spaces, or POIs with the attribute of botanical gardens and zoos in the third-level classification can be used to further subdivide urban parks, such as specialized parks.

[0191] In S1065, the water body threshold is used to extract preliminary water body area data. Based on this preliminary data, further filtering is performed to accurately extract the water body area data for the study area. Threshold segmentation or supervised classification methods can also be used, but the water body boundaries are often difficult to maintain a coherent and complete manner, resulting in extraction performance that is not as good as that achieved by selecting specific objects.

[0192] Furthermore, after obtaining the urban open space map, it is also possible to extract summer and winter urban open space maps, specifically including:

[0193] (1) Overlay the evergreen forest data, deciduous forest data, evergreen-deciduous mixed forest data and grassland data in the green classification results with the road network area data, park green space data and water body area data to obtain a summer urban open space map;

[0194] (2) Overlay the evergreen forest data and evergreen-deciduous mixed forest data in the green classification results with the road network area data, park green space data and water body area data to obtain the winter urban open space map.

[0195] This process allows for a more detailed depiction of the seasonal distribution of urban open spaces, which is beneficial for improving the urban ecological environment. Furthermore, by analyzing the distribution patterns of urban open spaces in winter, different types of vegetation combinations can be optimized, providing the government with a basis for decision-making to better achieve green space equity.

[0196] In addition, to rationally plan the spatiotemporal distribution of urban green spaces and open spaces during winter, summer and winter urban open space maps can be overlaid to obtain non-overlapping areas, which can then be considered as areas of change in urban open spaces. By understanding these changing areas, the spatiotemporal dynamics of urban open space patterns can be grasped, providing data support for urban construction and planning, optimizing the pattern of urban open spaces, and possessing significant application value for protecting and planning the structure and function of urban green spaces and improving the urban and living environment.

[0197] As can be seen, the embodiments of this application combine selected sample points or sample surfaces with object-oriented segmentation to select samples of mixed land features, thereby better classifying mixed vegetation areas. Then, by using spectral features, texture features, vegetation indices and vegetation phenological features, etc., preferred features are obtained. Finally, random forest classification is performed based on the preferred features, so that the random forest model can obtain more image features, thereby improving the classification effect.

[0198] To make the methods provided in the embodiments of this application clearer and easier to understand, the following is combined with... Figure 3 A specific example of this method will be used to illustrate the concept.

[0199] like Figure 3 As shown, this implementation may include, for example:

[0200] S301: Acquire study area image 1, study area image 2, POI data of the study area, and OSM road network data of the study area.

[0201] like Figure 4 As shown in the flowchart, the study area image 1, study area image 2, POI data, and OSM road network data are first acquired. Study area image 1 can be the Gaofen-1 (GF-1) image data of Beijing on August 4, 2020. Study area image 2 can be the Sentinel-2 image data of Beijing in 2020. The POI data of the study area can be the 2020 Beijing POI dataset obtained using the Gaode Application Programming Interface (API). The OSM road network data of the study area can be the 2020 Beijing traffic road network data obtained from the OSM project.

[0202] It should be noted that the study area image 1 can be, for example, the first study area image in the above embodiments, and the study area image 2 can be, for example, the second study area image in the above embodiments.

[0203] S302: Preprocess the study area image 1, study area image 2 and OSM road network data.

[0204] like Figure 4 As shown in the flowchart, the obtained data needs to be preprocessed. Preprocessing of image 1 in the study area includes: radiometric calibration of the multispectral and panchromatic images in the GF-1 imagery, aiming to make the satellite-observed data more accurately reflect actual physical quantities. Secondly, atmospheric correction is used, setting parameter values ​​based on multispectral image pixel data to eliminate the influence of atmospheric and illumination factors on ground reflection and obtain the actual surface reflectance. Then, orthorectification is performed on the panchromatic and multispectral images to correct image tilt deviations and errors generated during projection. The orthorectified multispectral data is then registered with the panchromatic panoramic data, using the panchromatic image as a reference for multispectral image registration. Finally, the registered images are fused to achieve very high resolution accuracy. To eliminate non-systematic geometric deformations, ground control points can be used for geometric fine correction of the images, achieving a geometric registration accuracy within 0.5 pixels.

[0205] The preprocessing of the study area image 2 includes: cloud cover removal, radiometric calibration, atmospheric correction, and image fusion of the Sentinel-2 image in GEE to generate multispectral remote sensing images.

[0206] The preprocessing of OSM road network data includes: removing duplicate information, handling missing values, checking topological relationships such as eliminating overlaps, intersections, self-intersections and self-overlaps in the road network, boundary checks, and attribute data cleaning to obtain OSM road network data.

[0207] During this process, the data is preprocessed to improve the accuracy of the subsequently generated urban open space map.

[0208] S303: Input the study area image 1 into the vegetation model to obtain vegetation area data. The vegetation model is a trained U-net deep neural network model, and the training samples include the vegetation index sample dataset of the study area image 1.

[0209] like Figure 4 As shown in the flowchart, since the vegetation model is a trained U-net deep neural network model, the study area image 1 can be directly input into the vegetation model to predict the vegetation areas of the study area, thereby extracting the vegetation areas from the study area image 1 and obtaining the vegetation area data. The specific vegetation area extraction results for Beijing are shown below. Figure 5 As shown.

[0210] It should be noted that the training process of the vegetation model is as described in the above embodiments, and will not be repeated here.

[0211] In this process, the U-net deep neural network model is used to segment and extract vegetation areas, which can realize the automatic extraction of large-scale vegetation.

[0212] S304: Perform object-oriented image segmentation on vegetation area data to obtain segmented objects.

[0213] like Figure 4 As shown in the flowchart, this application embodiment selects an object-oriented segmentation method to obtain segmented objects of different types in the green space of the study area. The specific process includes: first, masking the vegetation area data to obtain a mask file; then, based on the spatial, texture, and spectral information in image 1 of the study area, segmenting the mask file at a segmentation scale of 25 to obtain initial segmented objects; finally, merging the initial segmented objects according to attribute type at a merging scale of 90 to obtain segmented objects. The specific segmentation results of the vegetation area in Beijing are shown below. Figure 6 As shown.

[0214] S305: Based on historical images of the study area, obtain point feature sample data and define the attributes of the point feature sample data.

[0215] like Figure 4 As shown in the flowchart, historical images of the study area are first obtained, such as remote sensing images of Beijing in 2020 from Google Earth Pro. Then, by comparing remote sensing images from the vegetation growing season and the vegetation shedding season, polygonal feature sample data of three vegetation types are manually selected: evergreen forest, deciduous forest, and grassland. After obtaining the polygonal feature sample data, the polygonal feature sample data is converted into point feature sample data, and the attributes of the point feature sample data are defined. The definitions include: evergreen forest is defined as 1, deciduous forest as 2, and grassland as 3.

[0216] Among these requirements, the selection of samples should be accurate. All samples representing the actual land cover category should be consistent with that category and representative. The statistical characteristics of the samples should be close to the overall statistical characteristics of that category. There should be a sufficient number of samples that can well represent the characteristics of different land cover features.

[0217] S306: Assign the attributes of the point feature sample data to the segmentation object, and use the segmentation object containing the point feature sample data as the sample object.

[0218] By using a one-to-many spatial join, the attributes of point feature sample data are assigned to segmentation objects. Segmentation objects without any sample points are then removed through filtering, resulting in segmentation objects containing sample points, i.e., sample objects. Each sample object has multiple records (one or more attributes), and each record corresponds to a sample point for one attribute.

[0219] S307: Based on the attribute table of the sample object, merge multiple records of the same sample object to obtain new sample object attributes, and assign the new sample object attributes to the corresponding sample object to obtain multi-category sample objects.

[0220] Since the same sample object has multiple records, it is necessary to redefine the sample object attributes, such as... Figure 7 As shown, multiple records with the same OBJECTED attribute (the same sample object) in the attribute table of the sample object are merged into one record. The unique value of CLASS is taken and arranged in ascending order to form a new CLASS, which is the new attribute of the sample object. The new CLASS value is then assigned to the sample object. The final multi-category sample objects include 7 types: 1 represents evergreen forest, 2 represents deciduous forest, 3 represents grassland, 12 represents evergreen-deciduous mixed forest, 13 represents evergreen forest-grassland, 23 represents deciduous forest-grassland, and 123 represents evergreen-deciduous-grassland. The obtained multi-category sample objects are as follows: Figure 8 As shown.

[0221] In this process, by combining vegetation sample data of point features and area features with segmentation objects, multiple vegetation mixed area samples can be selected, such as evergreen-deciduous mixed forest areas, so as to better classify and separate vegetation mixed areas in the future.

[0222] S308: The texture features, spectral features, and index features extracted from image 1 of the study area, as well as the vegetation phenological features extracted from image 2 of the study area, are superimposed to obtain a feature set.

[0223] like Figure 4 As shown in the flowchart, a feature set is obtained based on texture features, spectral features, index features, and vegetation phenological features. The specific process includes:

[0224] (1) Texture Features: The texture features of the second-order matrix of image 1 in the study area were calculated using the gray-level co-occurrence matrix. The window size was 3*3, the X and Y variation values ​​were set to the default 1:1, and the gray-level quantization level was 64. After calculation, 38 texture features were extracted. The calculation formula and description of each texture are as follows: Figure 9 As shown.

[0225] (2) Spectral characteristics: The mean values ​​of the blue band, green band, red band and near-infrared band in the study area image 1 were selected as the spectral characteristics of each object in the study area image 1.

[0226] (3) Index characteristics: Select Figure 10 The 19 indices shown are used as index features, which are specifically calculated using band operations.

[0227] (4) Vegetation phenological characteristics: First, the vegetation phenological characteristics of the study area from January to December 2020 were obtained from image 2 (Sentinel-2 image) at a spatial resolution of 2m. Then, the monthly maximum values ​​of all vegetation phenological characteristics from January to December were synthesized, resampled to a spatial resolution of 2m, and band-stacked. This was then image-registered with the fused study area image 1 to obtain the monthly maximum values ​​of vegetation phenological characteristics of the study area in 2020.

[0228] (5) Feature set: The texture features, spectral features, index features and vegetation phenology features are superimposed by bands to obtain the feature set.

[0229] In this process, the phenological, textural, and spectral characteristics of vegetation can be used as feature values ​​for multi-category sample objects to facilitate the subsequent extraction of mixed regions of multi-type vegetation.

[0230] S309: Use Gini importance to sort feature values ​​from largest to smallest to obtain an importance sequence, and use the features with a preset number of positions in the importance sequence as classification features for multi-class sample objects.

[0231] like Figure 4 As shown in the flowchart, feature optimization is performed, specifically including: sorting the feature values ​​from largest to smallest importance based on Gini importance, as shown in the feature importance ranking diagram. Figure 11 As shown. The top 12 features in terms of importance, from largest to smallest, are: March Normalized Difference Vegetation Index (NDVI), February NDVI, mean green band reflectance, brightness index (BI), mean red band reflectance, April NDVI, redness index (RI), January NDVI, brightness index (BI2), mean near-infrared band reflectance, color rendering index (CI), and mean blue band reflectance.

[0232] S310: Perform random forest classification on the objects to be classified in the vegetation area data based on classification characteristics to obtain initial classification results.

[0233] like Figure 4 As shown in the flowchart, random forest classification is performed on the objects to be classified in the vegetation area data according to the classification characteristics to obtain the initial classification results, which include 7 categories of objects: evergreen forest, deciduous forest, grassland, evergreen-deciduous mixed forest, evergreen forest-grassland, deciduous forest-grassland and evergreen-deciduous-grassland.

[0234] In this process, feature optimization is performed on the feature values ​​of multi-category sample objects, and random forest is used to classify the objects to be classified in the vegetation area data, so as to realize automatic classification of green space, improve classification efficiency, and achieve an accuracy of over 80%.

[0235] S311: Merge and reclassify the initial classification results to obtain the green space classification results.

[0236] The initial classification results are merged and reclassified to obtain the green space classification results as follows: Figure 12 The results show four classifications: evergreen forest, deciduous forest, evergreen-deciduous mixed forest, and grassland. The final green space classification result for the study area is shown in the image below. Figure 13 As shown.

[0237] S312: Evaluate the accuracy of green space classification results using overall classification accuracy and Kappa coefficient.

[0238] like Figure 4 As shown in the flowchart, the accuracy of the green space classification results needs to be evaluated, specifically using the overall classification accuracy and Kappa coefficient. The obtained overall classification accuracy is 88.4513%, and the Kappa coefficient is 0.8240, indicating that the green space classification results have good accuracy. Based on the overall classification accuracy and Kappa coefficient, the confusion matrix is ​​obtained, as shown in Table 1 below:

[0239] Table 1. Confusion Matrix of Green Space Classification Results

[0240]

[0241] It should be noted that the calculation process for the overall classification accuracy and Kappa coefficient is as described in the above embodiments, and will not be repeated here.

[0242] S313: Obtain park green space data based on OSM road network data, POI point data and vegetation area data of the study area.

[0243] like Figure 4 The flowchart shown illustrates the process of obtaining park green space data. The specific steps include: first, segmenting the study area based on the OSM road network data to obtain urban functional plots; then, overlapping the urban functional plots with the vegetation area data to extract urban functional green spaces; next, filtering out POI data with the attribute of "park" based on the POI data; and finally, assigning the POI data with the attribute of "park" to urban functional green spaces to extract park green space data.

[0244] S314: Determine the water body extraction threshold based on the study area image 1 and the threshold segmentation method, and use the water body threshold to extract the water body area data in the first image of the study area.

[0245] like Figure 4The flowchart shown illustrates the process of obtaining water body area data. The specific steps include: calculating the improved normalized difference water index based on the study area image 1; determining the water body extraction threshold based on the normalized difference water index and the threshold segmentation method; initially extracting water body area data based on the water body extraction threshold and the study area image 1; and finally, manually filtering the objects based on the initially extracted water body area data to accurately extract the water body area data of the study area.

[0246] S315: Based on the OSM road network data of the study area, buffer zones are created for roads of different levels to generate road network surface data.

[0247] like Figure 4 The flowchart shown illustrates the process of obtaining road network surface data. The specific steps include: based on OSM road network data, and according to the minimum recommended width of urban road right-of-way in the Beijing local standard "Code for Spatial Planning and Design of Urban Roads," buffer zones are created for different road levels to generate road network surface data. Specific road levels are shown in Table 2.

[0248] Table 2 Road Widths of Different Grades

[0249]

[0250] S316: Overlay the evergreen forest data, deciduous forest data, evergreen-deciduous mixed forest data, and grassland data from the green classification results with the road network area data, park green space data, and water body area data to obtain a summer urban open space map.

[0251] like Figure 4 The flowchart shown illustrates how to obtain an urban open space map. Specifically, a summer urban open space map is obtained by overlaying evergreen forest, deciduous forest, evergreen-deciduous mixed forest, and grassland data from the green classification results with road network area data, park green space data, and water body area data. The detailed summer urban open space map is shown below. Figure 14 As shown.

[0252] S317: Overlay the evergreen forest data and evergreen-deciduous mixed forest data from the green classification results with the road network area data, park green space data, and water body area data to obtain a winter urban open space map.

[0253] like Figure 4 The flowchart shown illustrates how to obtain an urban open space map, specifically a winter urban open space map. This map is generated by overlaying evergreen forest and evergreen-deciduous mixed forest data from the green classification results with road network data, park green space data, and water body area data. The detailed winter urban open space map is shown below. Figure 15 As shown.

[0254] This process allows for a more detailed depiction of the seasonal distribution of urban open spaces, which is beneficial for improving the urban ecological environment. Furthermore, by analyzing the distribution patterns of urban open spaces in winter, different types of vegetation combinations can be optimized, providing the government with a basis for decision-making to better achieve green space equity.

[0255] S318: Overlay the summer urban open space map with the winter urban open space map to obtain the non-overlapping area, and take the non-overlapping area as the area of ​​change of urban open space.

[0256] Specific areas of change in urban open space, such as Figure 16 As shown, after obtaining the changes in urban open space, we can grasp the spatiotemporal dynamics of the urban open space pattern based on these changes, providing data support for urban construction and planning, optimizing the pattern of urban open space, and having important application value for protecting and planning the structure and function of urban green space and improving the urban and living environment.

[0257] This embodiment provides a deep learning-based method for mapping urban open spaces. It combines selected sample points or surfaces with object-oriented segmentation to select mixed land cover areas, enabling better classification of vegetation-mixed regions. Optimal features are then obtained through spectral features, texture features, vegetation indices, and vegetation phenological characteristics. Finally, random forest classification is performed based on these optimized features, allowing the random forest model to acquire more image features and thus improve classification accuracy. Furthermore, the use of vegetation models and random forests for green space extraction and automatic classification enables large-scale automatic vegetation extraction, improving classification efficiency.

[0258] Furthermore, the changing areas of urban open space in this application embodiment allow for a macroscopic understanding of the spatiotemporal dynamics of urban open space patterns, providing data support for urban construction and planning, optimizing the pattern of urban open spaces, and having significant application value for protecting and planning the structure and function of urban green spaces and improving the urban and living environment.

[0259] See Figure 17 This application provides a deep learning-based urban open space mapping device 1700, which includes:

[0260] Input unit 1701 is used to input the preprocessed first image of the study area into the vegetation model to obtain vegetation area data. The vegetation model is a trained U-net deep neural network model, and the training samples include the vegetation index sample dataset of the first image of the study area.

[0261] Segmentation unit 1702 is used to perform object-oriented image segmentation on vegetation area data to obtain segmentation objects;

[0262] Unit 1703 is used to select vegetation sample data based on historical images of the study area, and to obtain multi-category sample objects based on the vegetation sample data and segmentation objects.

[0263] The overlay unit 1704 is used to overlay the texture features, spectral features and index features extracted from the first image of the study area and the vegetation phenology features extracted from the preprocessed second image of the study area to obtain a feature set, and to use the mean of each feature in the feature set as the feature value of the multi-class sample object.

[0264] Classification unit 1705 is used to perform random forest classification and reclassification of objects to be classified in vegetation area data based on the feature values ​​of multi-class sample objects, and to obtain green space classification results.

[0265] Unit 1706 is used to obtain an urban open space map based on the road network area data, park and green space data, water area data, and green space classification results of the study area.

[0266] Optionally, the device 1700 also includes a training unit, specifically used for:

[0267] The vegetation index of the first image of the study area was classified, and the optimal threshold was selected based on the classification results.

[0268] Vegetated and non-vegetated areas were extracted from the first image of the study area based on the optimal threshold, and the vegetated and non-vegetated areas were used as labeled data.

[0269] The first image and labeled data of the study area with consistent coordinates were cropped using the regular grid method to obtain a vegetation index sample dataset;

[0270] The vegetation index sample dataset was divided into a training sample set and a test sample set according to a preset ratio using a random sampling method.

[0271] Image enhancement and image transformation are performed on the training sample set to obtain the processed training sample set;

[0272] The U-net deep neural network model is trained using the processed training sample set and tested using the test sample set. After training, the trained U-net deep neural network model is used as the vegetation model.

[0273] Optionally, the segmentation unit 1702 is specifically used for:

[0274] Masking is performed on the vegetation area data to obtain a mask file;

[0275] Based on the spatial, texture, and spectral information in the first image of the study area, the mask file is segmented to obtain the initial segmentation object;

[0276] The initial segmented objects are merged according to their attribute types to obtain the segmented objects.

[0277] Optionally, unit 1703 is specifically used for:

[0278] By comparing historical images of the study area during the vegetation growing season and the vegetation falling season, polygonal feature sample data of various vegetation types were selected.

[0279] Convert polygon feature sample data into point feature sample data, and define the attributes of the point feature sample data;

[0280] The attributes of point feature sample data are assigned to the segmentation object through a one-to-many spatial connection, and the segmentation object containing point feature sample data is used as the sample object.

[0281] Based on the attribute table of the sample object, multiple records of the same sample object are merged to obtain new sample object attributes;

[0282] Assign new sample object attributes to the corresponding sample object to obtain multi-category sample objects.

[0283] Optionally, the stacking unit 1704 is specifically used for:

[0284] Texture features are obtained by calculating the second-order matrix texture features of the first image of the study area using the gray-level co-occurrence matrix;

[0285] The mean values ​​of the blue, green, red, and near-infrared bands in the first image of the study area were selected as the spectral characteristics of each object in the first image of the study area.

[0286] By using band calculations, multiple indices of the first image of the study area are obtained as index features;

[0287] The monthly maximum values ​​of the annual vegetation phenological characteristics of the second image of the study area were synthesized. After resampling and band overlay, the images were registered with the first image of the study area to obtain the vegetation phenological characteristics.

[0288] The feature set is obtained by overlaying texture features, spectral features, index features and vegetation phenology features in bands.

[0289] Optionally, classification unit 1705 is specifically used for:

[0290] The importance sequence is obtained by sorting the eigenvalues ​​from largest to smallest using the Gini importance index.

[0291] The feature with a preset number of importance sequences is used as the classification feature of multi-class sample objects, and the objects to be classified are classified by random forest according to the classification feature to obtain the initial classification result;

[0292] The initial classification results are merged and reclassified to obtain the green space classification results.

[0293] Optionally, the device 1700 further includes:

[0294] The evaluation unit is used to evaluate the accuracy of green space classification results using the overall classification accuracy and the Kappa coefficient.

[0295] Optionally, obtaining unit 1706 is specifically used for:

[0296] Based on the OSM road network data of the study area, buffer zones are created for roads of different levels to generate road network surface data;

[0297] Based on the OSM road network data of the study area, urban functional plots were obtained and overlaid with vegetation area data to extract urban functional green spaces.

[0298] The POI data of the study area, which is a park, are assigned the function of urban green space, and park green space data is extracted.

[0299] Based on the first image of the study area, the improved normalized differential water index was calculated.

[0300] The water body extraction threshold was determined based on the normalized differential water body index and threshold segmentation method, and the water body area data in the first image of the study area was extracted using the water body threshold.

[0301] A map of urban open space is obtained by overlaying road network area data, park and green space data, water area data, and green space classification results.

[0302] Optionally, the obtaining unit 1706 is also used for:

[0303] By overlaying the evergreen forest data, deciduous forest data, evergreen-deciduous mixed forest data, and grassland data from the green classification results with road network area data, park green space data, and water body area data, a summer urban open space map is obtained.

[0304] By overlaying the evergreen forest data and evergreen-deciduous mixed forest data from the green classification results with the road network area data, park green space data, and water body area data, a winter urban open space map is obtained.

[0305] By overlaying the summer and winter urban open space maps, non-overlapping areas are obtained, and these non-overlapping areas are taken as the areas of change in urban open space.

[0306] It should be noted that the specific implementation method and achieved effect of the deep learning-based urban open space mapping device 1700 can be found in the above. Figure 1 or Figure 3 The relevant descriptions in the provided methods will not be repeated here.

[0307] This application also provides an electronic device 1800, such as... Figure 18 As shown, the device 1800 includes a memory 1801 and a processor 1802:

[0308] Memory 1801 is used to store computer programs;

[0309] Processor 1802 is used to execute the above according to the computer program. Figure 1 or Figure 3 The methods provided.

[0310] In addition, this application also provides a computer-readable storage medium for storing a computer program, the computer program being executed. Figure 1 or Figure 3 The method of supply.

[0311] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0312] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate. Components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the objectives of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0313] The above description is merely an exemplary implementation of this application and is not intended to limit the scope of protection of this application.

Claims

1. A deep learning-based method for mapping urban open spaces, characterized in that, include: The preprocessed first image of the study area is input into the vegetation model to obtain vegetation area data. The vegetation model is a trained U-net deep neural network model, and the training samples include the vegetation index sample dataset of the first image of the study area. The vegetation area data is subjected to object-oriented image segmentation to obtain segmentation objects; By comparing historical images of the study area during the vegetation growing season and the vegetation falling season, polygonal feature sample data of multiple vegetation types were selected. The polygon feature sample data is converted into point feature sample data, and the attributes of the point feature sample data are defined. The attributes of the point feature sample data are assigned to the segmentation object through a one-to-many spatial connection, and the segmentation object containing the point feature sample data is used as the sample object. Based on the attribute table of the sample object, multiple records of the same sample object are merged to obtain new sample object attributes; The new sample object attributes are assigned to the corresponding sample object to obtain multi-category sample objects; The texture features, spectral features, and index features extracted from the first image of the study area, and the vegetation phenology features extracted from the preprocessed second image of the study area are superimposed by bands to obtain a feature set. The mean value of each feature in the feature set is used as the feature value of the multi-category sample object. The vegetation phenology features are obtained by synthesizing the annual data of the second image of the study area through monthly maximum values, resampling, and band superposition, and then image registration with the first image. Based on the feature values ​​of the multi-category sample objects, random forest classification and reclassification are performed on the objects to be classified in the vegetation area data to obtain the green space classification results. Based on the road network area data, park and green space data, water body area data, and the green space classification results of the study area, an urban open space map is obtained. The evergreen forest data, deciduous forest data, evergreen-deciduous mixed forest data, and grassland data in the green space classification results are overlaid with the road network area data, the park green space data, and the water body area data to obtain a summer urban open space map; The evergreen forest data and evergreen-deciduous mixed forest data in the green space classification results are overlaid with the road network area data, the park green space data, and the water body area data to obtain a winter urban open space map; The summer urban open space map is overlaid with the winter urban open space map to obtain non-overlapping areas, which are then used as the areas of change in urban open space.

2. The method according to claim 1, characterized in that, The process of obtaining the vegetation model includes: The vegetation index of the first image of the study area is classified, and the optimal threshold is selected based on the classification results. Based on the optimal threshold, vegetation areas and non-vegetation areas are extracted from the first image of the study area, and the vegetation areas and non-vegetation areas are used as annotation data. The first image of the study area with consistent coordinates and the labeled data are cropped using a regular grid method to obtain the vegetation index sample dataset; The vegetation index sample dataset is divided into a training sample set and a test sample set according to a preset ratio using a random sampling method. The training sample set is subjected to image enhancement and image transformation to obtain the processed training sample set; The U-net deep neural network model is trained using the processed training sample set, and the U-net deep neural network model is tested using the test sample set. After training, the completed U-net deep neural network model is used as the vegetation model.

3. The method according to claim 1, characterized in that, The step of performing object-oriented image segmentation on the vegetation area data to obtain segmentation objects includes: The vegetation area data is masked to obtain a mask file; The mask file is segmented based on the spatial, texture, and spectral information in the first image of the study area to obtain the initial segmentation object; The initial segmented objects are merged according to their attribute types to obtain the segmented objects.

4. The method according to claim 1, characterized in that, The feature set is obtained by band overlaying the texture features, spectral features, and index features extracted from the first image of the study area and the vegetation phenological features extracted from the preprocessed second image of the study area, including: The texture features are obtained by calculating the second-order matrix texture features of the first image of the study area using the gray-level co-occurrence matrix; The average values ​​of the blue, green, red, and near-infrared bands in the first image of the study area are selected as the spectral features of each object in the first image of the study area. By using band operations, multiple indices are obtained from the first image of the study area, which serve as the index features; The annual vegetation phenological characteristics of the second image of the study area are synthesized by monthly maximum values. After resampling and band overlay, the images are registered with the first image of the study area to obtain the vegetation phenological characteristics. The feature set is obtained by superimposing the texture features, spectral features, exponential features, and vegetation phenological features into bands.

5. The method according to claim 1, characterized in that, The step of performing random forest classification and reclassification of objects to be classified in vegetation area data based on the feature values ​​of the multi-category sample objects to obtain green space classification results includes: The eigenvalues ​​are sorted by importance from largest to smallest using the Gini importance ranking to obtain an importance sequence; The feature with a preset number of bits in the importance sequence is used as the classification feature of the multi-class sample object, and the object to be classified is classified by random forest according to the classification feature to obtain the initial classification result; The initial classification results are merged and reclassified to obtain the green space classification results.

6. The method according to claim 1, characterized in that, The method further includes: The accuracy of the green space classification results is evaluated using the overall classification accuracy and the Kappa coefficient.

7. The method according to claim 1, characterized in that, The process of obtaining an urban open space map based on the road network area data, park and green space data, water body area data, and the green space classification results of the study area includes: Based on the OSM road network data of the study area, buffer zones are created for roads of different levels to generate the road network surface data. Based on the OSM road network data of the study area, urban functional land parcels are obtained and overlaid with the vegetation area data to extract urban functional green spaces. Assign the POI data of the study area with the attribute of park to the urban functional green space, and extract the park green space data; Based on the first image of the study area, the improved normalized differential water index was calculated; The water body threshold is determined based on the normalized differential water body index and threshold segmentation method, and the water body area data in the first image of the study area is extracted using the water body threshold. The urban open space map is obtained by overlaying the road network area data, the park and green space data, the water body area data, and the green space classification results.

8. A deep learning-based urban open space mapping device, characterized in that, The urban open space mapping device is used to perform the method of any one of claims 1 to 7, comprising: The input unit is used to input the preprocessed first image of the study area into the vegetation model to obtain vegetation area data. The vegetation model is a trained U-net deep neural network model, and the training samples include the vegetation index sample dataset of the first image of the study area. The segmentation unit is used to perform object-oriented image segmentation on the vegetation area data to obtain segmented objects; The selection unit is used to select polygonal feature sample data of multiple vegetation types by comparing historical images of the study area during the vegetation growing season and the vegetation falling season; convert the polygonal feature sample data into point feature sample data and define the attributes of the point feature sample data; assign the attributes of the point feature sample data to the segmentation object through a one-to-many spatial join, and use the segmentation object containing the point feature sample data as the sample object; merge multiple records of the same sample object according to the attribute table of the sample object to obtain new sample object attributes; assign the new sample object attributes to the corresponding sample object to obtain multi-category sample objects. The overlay unit is used to overlay the texture features, spectral features, and index features extracted from the first image of the study area and the vegetation phenology features extracted from the preprocessed second image of the study area to obtain a feature set. The mean value of each feature in the feature set is used as the feature value of the multi-category sample object. The vegetation phenology features are obtained by synthesizing the annual maximum value of the second image of the study area through monthly maximum value, resampling, and overlaying, and then performing image registration with the first image. The classification unit is used to perform random forest classification and reclassification of the objects to be classified in the vegetation area data based on the feature values ​​of the multi-category sample objects, and to obtain the green space classification result. The acquisition unit is used to obtain an urban open space map based on the road network area data, park and green space data, water body area data, and the green space classification results of the study area. The obtaining unit is further configured to overlay the evergreen forest data, deciduous forest data, evergreen-deciduous mixed forest data, and grassland data from the green space classification results with the road network area data, the park green space data, and the water body area data to obtain a summer urban open space map; overlay the evergreen forest data and evergreen-deciduous mixed forest data from the green space classification results with the road network area data, the park green space data, and the water body area data to obtain a winter urban open space map; and overlap the summer urban open space map with the winter urban open space map to obtain non-overlapping areas, which are then used as the changing areas of urban open space.

Images