Remote sensing image recognition method based on large-scale pre-trained neural network

Abstract

The present invention is a remote sensing image recognition method based on a large-scale pre-trained neural network, which relates to the field of image processing technology. First, large-scale satellite remote sensing images are sampled and divided into blocks at equal intervals, and the CLIP image encoder is used to process local image blocks to obtain high-dimensional feature vectors. Secondly, cosine distance, multidimensional scaling, Gaussian mixture model and other technologies are started to understand the similarity relationship between image blocks. Finally, high-value areas are manually screened, and a kernel density predictor is used to generate a heat map of the target area. The designed method is applied to the land space planning practice of Qingjian County, Shaanxi Province, and old urban communities are identified through high-resolution remote sensing images. The calculation results show that the present invention has high recognition accuracy and low resource consumption, and accurately delineates the scope of renovation of old urban communities.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a remote sensing image recognition method based on a large-scale pre-trained neural network. Background Art

[0002] Urban renewal is an important part of national land space planning, and the renovation of old residential areas is the specific practice and implementation of national land space planning at the level of urban renewal. At present, the scope of renovation of old residential areas is usually based on the comprehensive assessment of existing residential areas in urban built-up areas, assessing the building age, building structure, facilities and equipment, green environment, residents' opinions and other aspects of the residential areas to clarify the scope and objects that need to be renovated. However, in actual work, a lot of manpower, material resources and time are consumed. In particular, in the case of insufficient human resources in county towns, the progress of the demarcation of the scope of renovation of old residential areas is relatively inefficient, mostly remaining at the subjective qualitative level, and highly dependent on the professional ability of the evaluator. With the continuous advancement of national land space planning and urban physical examination, urban geographic information data, such as satellite remote sensing images, land survey data, and aerial images, continue to improve. In recent years, satellite remote sensing images have become an important data source for clarifying urban surface information and understanding the development status of urban space. By analyzing satellite remote sensing images, spatial characteristics such as buildings, roads, green spaces, and waters can be obtained, and urban housing quality, living environment, transportation, and supporting facilities can be analyzed, providing a reliable data basis and evaluation basis for the demarcation of the scope of renovation of urban old residential areas.

[0003] With the development of basic theories and computing platforms of artificial intelligence, machine learning and deep learning have made great progress in the field of image recognition and are widely used in tasks such as residential area renewal, road greening rate measurement, urban building extraction, and urban landscape analysis. Image analysis technology with convolutional neural network (CNN) as the core has become a hot topic for researchers. However, with the continuous deepening of research and application, convolutional neural networks have the following shortcomings: first, the accuracy of convolutional neural networks in extracting image features is still limited, especially the analysis ability of long-distance connected structures such as road networks and rivers is insufficient; second, the current convolutional neural network usually includes two steps, namely transfer learning and model training, in application deployment. Researchers need to collect enough samples for specific examples and set target labels, which consumes a lot of time and computing resources. That is, there are some problems in the existing technology, such as insufficient accuracy of image extraction and large resource consumption in order to collect enough samples. Summary of the invention

[0004] In order to overcome the shortcomings of the above-mentioned prior art in terms of insufficient accuracy in partial image extraction and large resource consumption in order to collect sufficient samples, the main purpose of the present invention is to provide a method for urban area division based on a large-scale pre-trained neural network.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The remote sensing image recognition method based on large-scale pre-trained neural network mainly includes the following four basic links: image preprocessing, image feature extraction, image analysis and image range delineation. The image preprocessing module extracts a large number of image blocks from the entire satellite remote sensing image through an equal-interval sampling method to form an experimental sample set.

[0007] In the image feature extraction link, this application uses the CLIP (Contrastive Language Image Pretraining) model to finely extract visual feature vectors from satellite images, not only focusing on natural features such as mountains, water, forests, and fields, but also simultaneously analyzing non-natural features such as towns and rural areas. The CLIP neural network takes contrastive learning as its core idea, and realizes self-supervised learning of neural networks by constantly comparing the similarity relationship between images and language samples. After training, the CLIP image encoder is downloaded to the local computer for analyzing satellite remote sensing images. In order to improve image analysis capabilities, the CLIP neural network uses the ViT (Vision Transformer) architecture to replace convolutional neural networks and feedforward neural networks. This network structure has a strong ability to extract image features, effectively analyzes long-distance connected spaces in images, and is embedded in emerging frameworks such as Stable Diffusion and Segment Anything to further solve complex problems such as image generation and semantic segmentation.

[0008] The image analysis module is dedicated to understanding the similarity relationship between each image block and realizing the adaptive organization of image blocks. For example, although they are all urban features, the image features of different land use categories such as schools, commercial areas, and residential areas are also different, and even the images of commercial housing, affordable housing, new rural areas, and old communities that belong to the same residential land also have differences. It is necessary to let the computer understand the relationship between various image blocks. The image analysis module includes the following specific steps: (1) Use cosine distance to calculate the similarity between high-dimensional feature vectors. (2) Based on the distance matrix, use manifold learning technology to visualize the morphological similarity of image blocks. As a classic manifold learning technology, multidimensional scaling is dedicated to projecting data points in high-dimensional space into low-dimensional space and maintaining the similarity between samples. In the feature space, each point represents an image block, and two image blocks with high similarity have a small distance between them. (3) Start the Gaussian mixture model, perform clustering operations on data points, and organize image blocks reasonably. Compared with methods such as K-means clustering and spectral clustering, the Gaussian mixture model can not only determine the category of each image block, but also give the membership probability, and understand the relationship between image blocks more finely.

[0009] Based on the understanding of the similarities and differences of various types of images, the computer outputs the centroid image block of each cluster based on the clustering results of the Gaussian mixture model. This application defines the centroid of the cluster as the sample closest to the geometric center in each set. In the image blocks sampled by image preprocessing, high-value image blocks in the target area are manually selected according to the remote sensing image features of the old community, such as building layout, volume, quality, density, etc., as well as the surrounding green environment, road facilities, etc. In order to more accurately characterize the spatial distribution state of the target area, the kernel density estimation method is used, and the membership probability of the Gaussian mixture model is used as the input training data to visualize the specific location and spatial distribution of high-value targets in the remote sensing image, and draw a heat map of the entire satellite image area. The heat map is superimposed and analyzed with the residential land image in the area, and classified according to the threshold of the kernel density. Finally, the high threshold image area is manually checked to obtain the urban area division result, and then the scope of the old community that needs to be transformed is obtained.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention avoids the neural network training step in the prior art, saves a lot of computing time and hardware resources, has a faster computing efficiency, accurately identifies and locates old urban areas from high-resolution remote sensing images, and defines the scope of urban renewal and transformation. Considering the computing efficiency and recognition effect, the present application has the potential to be applied in a wider range of scenarios such as land feature classification and road network recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the structure of the present invention;

[0012] Figure 2 is the high-value target sample and probability distribution of the present invention;

[0013] Figure 3 This is a schematic diagram of the scope of renovation of old residential areas in the case of the present invention;

[0014] Figure 4 is the program calculation time of the present invention;

[0015] Figure 5 It is a schematic diagram of the process structure of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0017] Example 1

[0018] See also Figure 1-5 ,The remote sensing image recognition method based on large-scale pre-trained neural network can be applied to confirm the ,area of ​​old urban communities, and then to confirm the ,renovation scope of old communities. The division method mainly includes the ,following four basic links: image preprocessing, image feature extraction, image analysis and image ,range delineation. The image preprocessing module extracts a large number of image blocks through ,equally spaced sampling method for the entire satellite remote sensing image to ,constitute the experimental sample set.

[0019] In the image feature extraction stage, this application uses the CLIP-Contrastive Language Image Pretraining contrast language image pretraining model to finely extract visual features in satellite images, not only focusing on natural features such as mountains, water, forests, and fields, but also simultaneously analyzing non-natural features such as towns and rural areas. The CLIP neural network takes contrastive learning as its core idea, and realizes self-supervised learning of neural networks by constantly comparing the similarity relationship between images and language samples. After training, the CLIP image encoder is downloaded to the local computer for analyzing satellite remote sensing images. In order to improve image analysis capabilities, the CLIP neural network uses the ViT architecture to replace convolutional neural networks and feedforward neural networks. This network structure has strong image feature extraction capabilities, effectively analyzes long-distance connected spaces in images, and is embedded in emerging frameworks such as Stable Diffusion and Segment Anything to further solve complex problems such as image generation and semantic segmentation.

[0020] The image analysis module is dedicated to understanding the similarity relationship between each image block and realizing the adaptive organization of image blocks. For example, although they are all urban features, the image features of different land use categories such as schools, commercial areas, and residential areas are also different, and even the images of commercial housing, affordable housing, new rural areas, and old communities that belong to the same residential land also have differences. It is necessary to let the computer understand the relationship between various types of images. The image analysis module includes the following specific steps: (1) Use cosine distance to calculate the similarity between high-dimensional visual feature vectors. (2) Based on the distance matrix, use manifold learning technology to visualize the morphological similarity of image blocks. As a classic manifold learning technology, multidimensional scaling is dedicated to projecting data points in high-dimensional space into low-dimensional space and maintaining the similarity between samples. In the feature space, each point represents an image block, and two image blocks with high similarity have a small distance between them. (3) Start the Gaussian mixture model, perform clustering operations on data points, and organize image blocks reasonably. Compared with methods such as K-means clustering and spectral clustering, the Gaussian mixture model can not only determine the category of each image block, but also give the membership probability, and understand the relationship between image blocks more finely.

[0021] Based on the understanding of the similarities and differences of various types of images, the centroid image block of each cluster is output based on the clustering results of the Gaussian mixture model. This application defines the centroid of the cluster as the sample closest to the geometric center in each set. In the image blocks sampled by image preprocessing, high-value image blocks in the target area are manually selected based on the image characteristics of the old residential area, such as building layout, volume, quality, density, etc., as well as the surrounding green environment, road facilities, etc. In order to more accurately characterize the spatial distribution state of the target area, the kernel density estimation method is used, and the membership probability of the Gaussian mixture model is used as training data to draw a heat map of the entire satellite image area. The heat map is superimposed and analyzed with the residential land image in the area, and classified according to the threshold of the kernel density. Finally, the high threshold image area is manually checked to obtain the urban area division result, and then the scope of the old residential area that needs to be transformed is obtained.

[0022] Embodiment 2:

[0023] Based on Example 1, in this example, Qingjian County is selected as the target object. Qingjian County is located at the southernmost end of Yulin City, Shaanxi Province, where the Yellow River and Wuding River meet. The total area is 1,881 square kilometers, and the terrain slopes from southeast to northwest, belonging to the hilly gully area of ​​the Loess Plateau. The case studied in this application is the old city of Qingjian County, located in Wenchang Community of Kuanzhou Town, which belongs to the core area of ​​the central city.

[0024] In the analysis of the current status of land use in the national territory, the current residential land in the old city is 42.21 hectares, accounting for 75.52% of the total land use, far exceeding the relevant standard indicators. Most of the residential land in the urban area is self-built houses, cave dwellings and traditional dwellings in rural areas, with poor construction quality, high building density, insufficient supporting facilities, and generally low land use efficiency. On the basis of meeting the total housing supply in the old city, the plan reasonably promotes the efficient and intensive use of residential land resources, optimizes and improves the quality of the residential environment, focuses on preserving the traditional dwellings with local characteristics in Qingjian, improves the construction of supporting service facilities in the residential group, and orderly promotes the renovation of old communities in the old city area.

[0025] This application uses large-scale pre-trained neural networks as its core technology. Its advantage is that researchers do not need to build various complex training models, or even specially build data sets, or label the data. All of the above work is completed by the OpenAI open platform, which greatly reduces the workload of researchers' early manual operations and improves work efficiency. Therefore, the data images used in this application only include remote sensing images to be identified, and the research scope of the old city is delineated in combination with the latest urban development boundary scope and the current construction foundation.

[0026] For the satellite remote sensing images of Qingjian County, this application samples the remote sensing images at equal intervals with a step size of 125 pixels. The scale of each image block is 500×500 pixels, covering an area of ​​11,025 square meters, and a total of 3,136 image blocks are obtained. In the image feature extraction module, this application uses the CLIP image encoder and ViT-B / 32 as the backbone network to extract the visual features of each image block. In order to effectively analyze the spatial relationship between various elements in the image, the CLIP image encoder uses a 512-dimensional vector to represent the visual features of an image block, and finally forms a 3136×512 feature matrix. Subsequently, the cosine distance program is run to calculate the similarity between any two image blocks to form a distance matrix of size 3136×3136. Based on the distance matrix, multidimensional scaling technology is used to achieve data dimensionality reduction and visualization. In this case, we set the dimension of the feature space to 2, and all image blocks form a two-dimensional scatter plot. Finally, the Gaussian mixture model is started to divide all image blocks into several groups. In order to adaptively calculate the number of clusters, we refer to the maximum value of the silhouette coefficient curve and divide all image blocks into 31 categories. Each group of images contains the same or similar feature elements.

[0027] Since the selected case Qingjian County is a typical small city with obvious natural environment characteristics, it can be generally classified into a natural environment characteristic group dominated by landscape elements, an urban environment characteristic group dominated by building roads, and a rural environment characteristic group mixed with nature and cities. For example, in urban environment characteristic group 2, the elements in the group are mostly single-family high-rise or plate-type multi-story residential buildings. After comparing with the data of the third land survey, it can be found that the elements in this group are basically urban residential land in terms of land use category, and the building characteristics are more accurately identified. It is worth noting that different elements have different proportions in satellite remote sensing images, and the number of images and the number of classification groups are also different. The more groups are set, the higher the similarity of each group of images and the smaller the differentiation, but it will increase the calculation time and invalid classification.

[0028] After accurately identifying the mutual affiliation between image blocks, this application selects old residential areas that are in urgent need of renovation as appropriate target areas. Figure 2 As shown in the figure, as the only manual intervention link, this application selects 4 image groups based on the characteristics of poor building quality, messy layout, small building volume, high building density, weak road network connectivity and poor living environment. Then, based on the sampling coordinates of the selected image blocks, a kernel density predictor is used to draw a heat map to clarify the spatial location of the old community. The calculation results of the kernel density predictor are shown in Figure 3 . By superimposing with the original satellite remote sensing image, it can be seen that the old residential areas in the old city that are in urgent need of renovation are mainly distributed on the west side of Qingjian East Street in the middle of the city, on both sides of Qingjian North Street in the northwest, and along some areas along the Qingjian River. After on-site investigation and survey, the old residential areas in this area basically meet the renovation needs. Finally, the threshold of the heat map is divided into three levels. The darker the color, the higher the thermal threshold of the area, the more obvious the characteristics of the old residential area, and the more urgent the renovation needs. The darkest color area can be used as the basis for demarcating the renovation scope of the old residential area, and should be given priority in the implementation stage.

[0029] Finally, the calculation time of each module of this application method is as follows Figure 4 As shown in the figure, the image annotation, network training, network fine-tuning and other steps in the existing methods are avoided. The large-scale pre-trained neural network method takes a total of about 13 minutes, has higher execution efficiency and less manual intervention, and shows good application prospects.

[0030] The basic working principle of the urban area division method based on large-scale pre-trained neural network of the present invention is: first, the large-scale satellite remote sensing images are sampled and divided into blocks at equal intervals, and the CLIP image encoder is used to process the local image blocks to obtain high-dimensional feature vectors. Secondly, the cosine distance, multidimensional scaling, Gaussian mixture model and other technologies are activated to understand the similarity relationship between image blocks. Finally, high-value areas are manually screened, and the kernel density predictor is used to generate a heat map of the target area. The designed method is applied to the practice of land space planning in Qingjian County, Shaanxi Province, and old neighborhoods are identified through high-resolution remote sensing images.

[0031] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0032] The above embodiments are merely examples of the present invention and do not limit the protection scope of the present invention. All designs that are the same or similar to the present invention fall within the protection scope of the present invention.

Claims

1. A remote sensing image recognition method based on a large-scale pre-trained neural network, characterized in that: The following steps are involved: Obtain satellite remote sensing images of the target urban area and perform preprocessing; wherein the preprocessing is to extract a large number of image blocks from the satellite remote sensing images through an equal-interval sampling method to form an experimental sample set; The CLIP neural network model is introduced, and the ViT architecture is used to replace the convolutional neural network and the feedforward neural network to obtain an improved CLIP neural network model. The improved CLIP neural network model extracts high-dimensional visual feature vectors from the experimental sample set; the similarity between any two image blocks is calculated by the cosine distance to obtain the distance matrix of the image blocks; Using multi-dimensional scaling to reduce the dimension of the distance matrix of the image block to obtain a reduced-dimensional distance matrix of the image block; Clustering the reduced-dimensional distance matrix of the image blocks using a Gaussian mixture model to obtain clustering results of the image blocks; Using the kernel density estimation method, the membership probability of the Gaussian mixture model is used as the input training data to draw the heat map of the target town area; The heat map of the target area is overlaid with the residential land image for analysis, and classification is performed according to the threshold of the kernel density. The high-threshold image area is manually checked to obtain the divided high-value target area.

2. The remote sensing image recognition method based on large-scale pre-trained neural network according to claim 1, characterized in that: The image blocks are based on a distance matrix and the morphological similarity between the image blocks is visualized through manifold learning; The image blocks are processed by the Gaussian mixture model, the data points are clustered, and the image blocks are organized reasonably.

3. The remote sensing image recognition method based on large-scale pre-trained neural network according to claim 1, characterized in that: The image blocks are processed by the Gaussian mixture model, and a clustering operation is performed on the data points to obtain the centroid image block of each cluster.

4. The remote sensing image recognition method based on large-scale pre-trained neural network according to claim 1, characterized in that: The high-dimensional visual features include natural features and non-natural features.

Images