A deep learning-based urban gentrification space intelligent identification method

Abstract

The application provides a city gentrification space intelligent identification method based on deep learning, which is based on street view data and realizes intelligent identification of city gentrification space through deep learning. The method emphasizes the use of street view data and deep learning technology to quantitatively evaluate large-scale city space, which is beneficial to the present situation investigation and scheme compilation of city renewal. Meanwhile, the method is based on big data and has the characteristics of strong reusability and high operation efficiency, and can be used to build an intelligent identification platform for city gentrification space, so as to realize instant, continuous and accurate positioning of city space gentrification monitoring and evaluation.

Description

Technical Field

[0001] This invention relates to a technical method for identifying gentrified urban spaces. Background Technology

[0002] Gentrification is a global crisis that refers to the structural changes in residents' occupations, incomes, and lifestyles resulting from the influx of capital into urban centers, the encroachment of high-income groups on existing communities, the elevation of local social classes, the upgrading of the urban landscape, and the direct or indirect replacement of low-income groups. Currently, my country's urbanization rate has exceeded 60%, entering a relatively mature mid-to-late stage of urbanization, with urban development gradually shifting from an incremental model to a stock-renewal model. Gentrification is often accompanied by urban renewal, leading to significant social conflicts, the relocation of impoverished residents to lower-quality areas, exacerbating spatial segregation and social discrimination, and ultimately resulting in social polarization.

[0003] Accurately identifying gentrified spaces will help improve urban renewal strategies, formulate housing policies, guide land redevelopment, ensure more inclusive urban planning policies, and promote social equity and sustainable urban development. Currently, traditional methods for identifying gentrified spaces are based on socioeconomic attributes such as real estate information, income levels, and education levels, judging whether a region has become gentrified by analyzing changes and comparisons with average levels. The limitations of this method in China are mainly due to the slow and inaccurate updating of traditional population data and the fact that real estate transaction data does not cover all community types.

[0004] As street view images become increasingly widespread and higher resolution, this data exhibits advantages such as low acquisition cost, rapid update speed, and objective accuracy. It can provide large-scale urban spatial images in a short time and assist in judging the gentrification of urban spaces. Methods for judging gentrified spaces using street view data involve subjective observation of changes in the built environment across two time-bound street view images to determine whether spatial gentrification has occurred. However, further improvements are needed in the technical methods for intelligently calculating gentrification information from multiple time-bound street view images to form quantitative indicators that objectively reflect whether urban spaces have become gentrified. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and disclose a method for automatically identifying urban gentrification spaces. This method emphasizes the use of street view data and deep learning technology to conduct quantitative evaluations of large-scale urban spaces, which is beneficial for current status surveys and plan development for urban renewal. Furthermore, this method is based on big data, has strong reusability and high operational efficiency, and can be used to build an intelligent identification platform for urban gentrification spaces, achieving real-time, continuous, and accurate monitoring and evaluation of urban gentrification.

[0006] The technical solution of this invention is as follows:

[0007] A deep learning-based intelligent recognition method for urban gentrification spaces includes the following steps:

[0008] Step (1): Obtain time-lapse street view pairs t for all street view sampling points i within the target city space. (i) That is, street view images at two time points, namely the previous street view. and rear street view Among them, the front street view The recording time is earlier than that of the post-street view.

[0009]

[0010] Step (2): Randomly select some sampling points as sample sampling points, and divide the sample sampling points into training set and test set according to the proportion;

[0011] Step (3): Perform manual identification of the gentrification of street view image pairs selected in step (2) for each sample point, and assign a gentrification label y(t) to the time-lapse street view pairs that meet the conditions. (i) );

[0012] Step (4): Use the open-source pre-trained convolutional neural network EfficientNet B0 model as the base model to extract features from the delayed street scene pairs of the sampled points;

[0013] Step (5): Extract the hidden vectors of the time-lapse street view pairs obtained in step (4) above. The linear transformation is converted into a single multidimensional input value, which is then fed into a Sigmoid activation layer to obtain the predicted gentrification probability p(t). (i) ):

[0014] Step (6): Based on the delayed street view of each sample point, assign the gentrification label y(t) in step (3) to the sampled data. (i) ) and the gentrification probability p(t) predicted in step (5) (i) ), calculate the binary cross-entropy loss s(t) (i) ):

[0015] s(t (i) )=y(t (i) )logp(t (i) )+(1-y(t (i) ))log(1-p(t (i) )), formula 4;

[0016] Step (7): The training set of the sample sampling points selected in step (2) is trained in batches in multiple rounds according to the above steps (4), (5) and (6) to obtain the relatively optimal model parameters for street scene image feature extraction.

[0017] Step (8): Based on the optimal street view image feature extraction model parameters recorded in step (7), use the feature extraction, linear transformation and activation functions in steps (4) and (5) to complete the time-delayed street view pairs t for all sampling points i at the city scale. (i) The gentlemanly attribute Y(t) (i) (Prediction);

[0018] Step (9): For each city block, calculate the proportion of sampling points with a gentrification attribute of 1 to all sampling points, and determine whether the block is gentrified.

[0019] Furthermore, in step 3, the specific operation of the gentrification manual identification is as follows:

[0020] For time-lapse street view pairs containing "meaningful changes", the gentrification tag y(t) (i) The label is set to 1; for time-lapse street view pairs with only "meaningless changes", the gentrification label y(t) is set to 1. (i) The symbol is marked as 0, where:

[0021] "Meaningful changes" include: low-rise bungalows with earthen and wooden structures being replaced by high-rise buildings with modern service facilities; renovation of building facades and repair of balconies and windows; transformation of commercial formats from ordinary retail to international brands; an increase in new traffic control signs or structures; an increase in municipal infrastructure; an increase in security facilities; the addition of flower racks outdoors; and lawn maintenance, all signifying gentrification.

[0022] "Meaningless changes" refers to no change or random changes, including: vehicles on the road, weather conditions, and the brightness and shadow of images.

[0023] Furthermore, in step 4, the feature extraction of the time-lapse street view pairs is specifically performed as follows:

[0024] Step 4.1: Extract the hidden vectors for each street view image, including:

[0025] Step 4.1.1: Preprocess each street view image, that is: convert each RGB image into a tensor of size 224x224x3;

[0026] Step 4.1.2: Extract features from each image using the EfficientNet B0 model; as the convolution kernel changes, this process will generate different model parameters, and the model parameters in feature extraction will be recorded simultaneously.

[0027] Step 4.1.3: Extract the features of each image to obtain a feature value of size 1x1x4, denoted as the hidden vector of the image.

[0028] Step 4.2: The hidden vectors of the two images in each pair of time-lapse street views are the hidden vector of the pre-street view and the hidden vector of the post-street view Concatenate the hidden vectors of the two images in each pair of time-lapse street views as the final feature representation of the time-lapse street view pair at each sample sampling point, denoted as the hidden vector

[0029]

[0030] In the formula, represents the hidden vector of the time-lapse street view pair, represents the hidden vector of the pre-street view, represents the hidden vector of the post-street view.

[0031] Furthermore, in step 7, obtaining the model parameters for extracting the relatively optimal street view image features includes:

[0032] Step 7.1: Train the training set of sample sampling points according to the above steps (4), step (5), and step (6), and record the loss s(t (i) ) and the model parameters for feature extraction in step (4);

[0033] Step 7.2: Obtain P for the test set through steps (4) and step (5) i , and compare the obtained P i with the gentrification probability threshold a of the sampling point to predict the gentrification attribute Y(t (i) ), that is: when P i <a, set the value of Y(t (i) ) to 0; when P i ≥a, set the value of Y(t (i) ) to 1, and compare the predicted gentrification attribute Y(t (i) ) on the test set with the gentrification label y(t (i) ), calculate and record the accuracy of the model in each round of training on the test set;

[0034] Step 7.3: When the loss s(t (i) ) decreases and tends to fit and the accuracy on the test set no longer increases significantly, record the model parameters for extracting the street view image features corresponding to this round of training as the optimal model parameters for extracting the street view image features.

[0035] Furthermore, in step 8, for the time-lapse street view pair t at each sampling point (i)Obtain the probability P of gentrification attributes in sequence according to step (4) and step (5). i Compare the obtained probability P of gentrification i with the gentrification probability threshold a of the sampling point to obtain the gentrification attribute of the sampling point. When P i < a, set the value of Y(t (i) ) to 0; when P i ≥ a, set the value of Y(t (i) ) to 1.

[0036] Furthermore, in step 9, for each block in the city, calculate the proportion of sampling points with gentrification attribute of 1 among all sampling points to determine whether the block is gentrified. The specific calculation and judgment formula are as follows:

[0037]

[0038] When P < P0, Y = 0

[0039] When P ≥ P0, Y = 1

[0040] In the formula, P represents the gentrification proportion of each block, Y = 1 represents that the block is gentrified, Y = 0 represents that the block is not gentrified, K represents the number of sampling points in each block, and P0 represents the gentrification proportion threshold for determining whether the block is gentrified.

[0041] The beneficial effects of the present invention are as follows:

[0042] (1) Using street view image data, breaking through the limitations of spatial scale, time, manpower, etc., and expanding the research perspective to the large-scale urban space.

[0043] ]](2) It has strong reusability, and this method can be used to identify whether gentrification has occurred in the blocks of urban spaces in different regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 : Schematic diagram of the process of the present invention;

[0045] Figure 2 : Map of street view sampling points in the central urban area of Shanghai;

[0046] Figure 3 : Street view images of gentrification sampling points in the central urban area of Shanghai;

[0047] Figure 4 : Schematic diagram of extracting image features by the convolutional neural network model;

[0048] Figure 5 : Visualization result map of gentrification sampling points in the central urban area of Shanghai;

[0049] Figure 6 ​A visualization of the gentrification of neighborhoods in central Shanghai. Detailed Implementation

[0050] The technical solutions provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0051] This invention provides a method for intelligent recognition of urban gentrification spaces based on street view data and through deep learning.

[0052] The case studies are selected from the central urban area of ​​Shanghai, as shown in the attached document. Figure 1 As shown, the method steps are as follows:

[0053] Step (1): Based on all street view sampling points in the central urban area of ​​Shanghai (as shown in the attached document) Figure 2 ), obtain the time-lapse street view pair t at sampling point i. (i) That is, street view images at two time points, namely the previous street view. and rear street view Front Street View The recording time is earlier than that of the post-street view.

[0054]

[0055] Step (2): Randomly select 10% of the street view sampling points in the central urban area of ​​Shanghai as sample sampling points; divide the sample sampling points into training set (80%) and test set (20%) according to the proportion.

[0056] Step (3): Perform manual identification of the gentrification of street view image pairs selected in step (2) for each sample point, and assign a gentrification label y(t) to the time-lapse street view pairs that meet the conditions. (i) ). Specific operation: For time-lapse street view pairs containing "meaningful changes", gentrify the label y(t) (i) The label is set to 1; for time-lapse street view pairs with only "meaningless changes", the gentrification label y(t) is set to 1. (i) Marked as 0. Street view images of gentrification sampling points in central Shanghai, i.e., time-lapse street view pairs containing "meaningful changes" as shown in the attached image. Figure 3 As shown. Among them, "meaningful changes" include: the transformation of low-rise bungalows with earthen and wooden structures into high-rise buildings with modern service facilities; the renovation of building facades and the repair of balconies and windows; the transformation of commercial formats from ordinary retail to international brands; an increase in new traffic control signs or structures (such as traffic lights, signs, and no-parking times); an increase in municipal infrastructure such as smart garbage bins; an increase in security facilities such as access control; the addition of flower racks and lawn maintenance outdoors, etc., marking a gentrification;

[0057] "Meaningless changes" refers to changes that are unchanging or random, including: vehicles on the road, weather conditions, and the brightness and shadow of images.

[0058] Step (4): Use the open-source pre-trained convolutional neural network EfficientNet B0 model as the base model to extract features from the delayed street scene pairs of the sampled points. The specific operation is as follows:

[0059] Step (4.1), as follows Figure 4 As shown, the process of extracting the hidden vector for each street view image includes:

[0060] ① Preprocess each street view image, that is, convert each RGB image into a tensor of size 224x224x3.

[0061] ② Features are extracted from each image using the EfficientNet B0 model; the EfficientNet B0 model includes convolutional layers, batch normalization layers, shift-flip bottleneck convolution (MBConv), Swish activation layers, global average pooling layers, dropout layers, fully connected layers, etc. The image feature extraction process is as follows:

[0062] First, the preprocessed street view image is input and subjected to multiple convolutions and shift-flipped bottleneck convolutions (MBConv): The first stage involves convolutions and batch normalization layers, outputting a feature map of dimension 112x112x32; the second stage involves one shift-flipped bottleneck convolution, outputting a feature map of dimension 112x112x16; the third stage involves two shift-flipped bottleneck convolutions, outputting a feature map of dimension 56x56x24; the fourth stage involves two shift-flipped bottleneck convolutions, outputting a feature map of dimension 28x28x40; the fifth stage involves three shift-flipped bottleneck convolutions, outputting a feature map of dimension 14x14x80; the sixth stage involves three shift-flipped bottleneck convolutions, outputting a feature map of dimension 14x14x112; the seventh stage involves three shift-flipped bottleneck convolutions, outputting a feature map of dimension 7x7x192; and the eighth stage involves one shift-flipped bottleneck convolution, outputting a feature map of dimension 7x7x320.

[0063] Then, the image is processed through convolution and Swish activation layers, outputting a feature map with dimensions of 7x7x1280;

[0064] Secondly, the image is passed through a global average pooling layer, which outputs a feature map with a dimension of 1x1x1280;

[0065] Next, the image is passed through a fully connected layer and a dropout layer, outputting a feature map with a dimension of 1x1x1000;

[0066] Finally, the image is passed through a fully connected layer, which outputs a feature map with dimensions of 1x1x4.

[0067] ③ Extract the features of each image to obtain feature values ​​of size 1x1x4, which are denoted as the hidden vector of that image;

[0068] The convolution kernel changes during the above process, which generates different model parameters, while simultaneously recording the model parameters in feature extraction.

[0069] In step (4.2), the hidden vectors of the two images in each time-delayed street view pair are the hidden vectors of the previous street view. Hidden vectors of rear street view The hidden vectors of the two images in each time-lapse street view pair are concatenated to obtain the final feature representation of the time-lapse street view pair for each sample point, denoted as the hidden vector.

[0070]

[0071] In the formula, This represents the hidden vector of a time-lapse street view pair. Represents the hidden vector of the foreground street view. This represents the hidden vector of the rear street view.

[0072] Step (5): Extract the hidden vectors of the time-lapse street view pairs obtained in step (4) above. The linear transformation is converted into a single multidimensional input value, which is then fed into a Sigmoid activation layer to obtain the predicted gentrification probability p(t). (i) ):

[0073]

[0074] In the formula, a T σ represents a linear transformation, and σ represents the activation of the function.

[0075] Step (6): Based on the delayed street view of each sample point, assign the gentrification label y(t) in step (3) to the sampled data. (i) ) and the gentrification probability p(t) predicted in step (5) (i) ), calculate the binary cross-entropy loss s(t) (i) ):

[0076] s(t (i) )=y(t (i) )logp(t (i) )+(1-y(t (i) ))log(1-p(t (i) )), formula 4;

[0077] Step (7): The training set of the sample points obtained in step (2) is trained in batches for multiple rounds according to steps (4), (5), and (6) above to obtain the relatively optimal model parameters for street scene image feature extraction. Specifically:

[0078] ① The training set of sampled points is trained according to steps (4), (5), and (6) above, with a batch size of 4 and a training period of 10. The loss s(t) of each training round is recorded. (i) The model parameters for feature extraction in step (3) and step (4).

[0079] ② Obtain P from the test set through steps (4) and (5). i Using 0.6 as the critical value, the gentlemanly attribute Y(t) is predicted. (i) That is: when P i When <0.6, Y(t) (i) The value of P is 0; when P i When ≥0.6, Y(t) (i) If the value is 1, the predicted gentlemanly attribute Y(t) on the test set will be... (i) ) and the gentlemanly label y(t (i) In comparison, the accuracy of the model on the test set in each training round is calculated and recorded.

[0080] ③ When the loss s(t) (i) When the accuracy on the test set no longer increases significantly after the initial decrease, the model parameters for street view image feature extraction corresponding to that round of training are recorded as the optimal model parameters for street view image feature extraction.

[0081] Step (8): Based on the optimal street view image feature extraction model parameters recorded in step (7), use the feature extraction, linear transformation and activation functions in steps (4) and (5) to complete the time-delayed street view pairs t for all sampling points i in the central urban area of ​​Shanghai. (i) The gentlemanly attribute Y(t) (i) The prediction of the street view at each sampling point, i.e., the prediction of the time-delayed street view for t. (i) By completing steps (4) and (5) in sequence, the probability P of the gentlemanly attribute is obtained. i When P i When <0.6, Y(t) (i) The value of P is 0; when P i When ≥0.6, Y(t) (i) The value is 1. The sampling points in Shanghai's central urban area with a gentrification attribute of 1 are shown in the attached figure. Figure 5 As shown.

[0082] Step (9): For each block in the central urban area of ​​Shanghai, calculate the proportion of sampling points with a gentrification attribute of 1 to all sampling points. Use 30% as the threshold to determine whether a block is gentrified, and obtain the gentrification status of the block. The visualization results of the gentrification status of each block in the central urban area of ​​Shanghai are attached. Figure 6 As shown, the specific calculation formula is as follows:

[0083]

[0084] When P < 30%, Y = 0

[0085] When P ≥ 30%, Y = 1

[0086] In the formula, P represents the gentrification ratio of each block, Y=1 represents a gentrified block, Y=0 represents a non-gentrified block, and K represents the number of sampling points in each block.

[0087] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1.A method for identifying urban gentrification space intelligence based on deep learning, characterized in that, The method comprises the following steps: Step (1): Obtain the time-lapse street view pair t of all street view sampling points i in the target city space (i) , that is, the street view images of two time sections, respectively, the front street view and the rear street view , wherein the recording time of the front street view is earlier than that of the rear street view Step (2): randomly select part of the sampling points as sample sampling points, and divide the sample sampling points into a training set and a test set according to a proportion; Step (3): Gentlemanized manual recognition is performed on the street view image pairs of each sample point selected in step (2), and a gentlemanized label y(t (i) ) is given to the time-lapse street view pairs meeting the conditions Step (4): using an open-source pre-trained convolutional neural network EfficientNet B0 model as a basic model to extract features of the time-lapse street view pair of the sample sampling points; Step (5): The hidden vector of the time-lapse street view pair obtained in the above step (4) is input into the linear transformation layer The linear transformation is a single input value of multiple dimensions, which is input into the Sigmoid function activation layer to obtain the predicted gentleman probability p(t (i) ) Step (6): Calculate the binary cross-entropy loss s(t (i) ) according to the delay street view of each sample point and the gentrification label y(t (i) ) in step (3) and the predicted gentrification probability p(t (i) ) in step (5): s(t (i) ) = -y(t (i) ) log p(t (i) ) - (1 - y(t (i) )) log (1 - p(t (i) )) s(t (i) ) = y(t (i) ) log p(t (i) ) + (1 - y(t (i) )) log (1 - p(t (i) )), Equation 4; Step (7): training the training set in the sample sampling points selected in step (2) according to steps (4), (5) and (6) above in batches to obtain relatively optimal model parameters for extracting features of street view images; Step (8): Based on the optimal model parameters of street view image feature extraction recorded in step (7), using the feature extraction, linear transformation and activation function in step (4) and step (5), complete the prediction of the gentleman attribute Y(t (i) ) of all sampling points i of city scale delay street view pair t (i) ​ Step (9): for each urban block, the proportion of the sampling points with a gentrification attribute of 1 to all the sampling points is calculated to determine whether the block is gentrified. 2.The urban gentrification spatial intelligence identification method based on deep learning according to claim 1, wherein, The step 3, the specific operation of artificial identification of gentrification is: The gentrification label y(t) is marked as 1 for pairs of time-lapse street views containing "meaningful changes"; and 0 for pairs of time-lapse street views with only "meaningless changes", where: (i) y(t) = 1 if |V(t) - V(t - 1)| > 0.1 * V(t - 1) (i) y(t) = 0 otherwise where: "meaningful changes" include: low-rise buildings of civil structures are changed into high-rise buildings with modern service facilities, building facades are renovated and balconies and windows are repaired, commercial formats are changed from ordinary retail to international brands, new signs or structures for traffic control are increased, municipal infrastructure is increased, security facilities are increased, flower racks are added outdoors, lawns are trimmed, and signs of gentrification are marked; "meaningless changes" refer to no changes or random changes, including: vehicles on the road, weather conditions, image light and shadow. 3.The urban gentrification spatial intelligence identification method based on deep learning according to claim 1, wherein, The step 4, the specific operation of feature extraction of the time-lapse street view pair is as follows: Step 4.1: extract the hidden vector of each street view image, including: Step 4.1.1: pre-process each street view image, that is, convert each RGB image into a tensor with a size of 224x 224x3; Step 4.1.2: extract features of each image through the EfficientNet B0 model, and different model parameters are generated due to the change of the convolution kernel, and the model parameters in the feature extraction are recorded; Step 4.1.3: extract the features of each image to obtain a feature value with a size of 1x 1x 4, which is recorded as the hidden vector of the image; Step 4.2: The hidden vector of each pair of time-lapse street view images is the hidden vector of the preceding street view image and the hidden vector of the following street view image The hidden vectors of the two images of each pair of time-lapse street view images are spliced as the final feature representation of each sample sampling point, denoted as the hidden vector wherein represents a hidden vector for a delayed street view pair, represents a hidden vector for a preceding street view, represents a hidden vector for a following street view. 4.The urban gentrification spatial intelligence identification method based on deep learning according to claim 1, wherein, The step 7, obtaining the relatively optimal model parameters for extracting features of street view images comprises: Step 7.1: The training set of sample sampling points is trained according to steps (4), (5), and (6) described above, and the loss s(t (i) ) of each round of training and the model parameters for feature extraction in step (4) are recorded; Step 7.2: P is obtained by step (4), step (5) on the test set i , and the obtained P i is compared with the sampling point gentrification probability threshold a to predict the gentrification attribute Y(t (i) ), that is: when P i <a, Y(t (i) ) is valued as 0; when P i ≥a, Y(t (i) ) is valued as 1, compare the predicted gentrification attribute Y(t (i) ) on the test set with the gentrification label y(t (i) ), calculate the accuracy of the model in each round of training on the test set and record it; Step 7.3: When the loss s(t (i) ) decreases and tends to fit and the accuracy on the test set no longer greatly improves, the model parameters corresponding to the street view image feature extraction of this round of training are recorded as the optimal street view image feature extraction model parameters. 5.The urban gentrification spatial intelligence identification method based on deep learning according to claim 1, wherein, The step 8, the delay street view pair t of each sampling point (i) According to step (4), step (5) in turn, the probability P of the gentrification attribute is obtained i The obtained probability P of the gentrification i The gentrification attribute of the sampling point is obtained by comparing the sampling point gentrification probability threshold a with the probability P, when P i Y(t (i) ) is valued as 0 when P i ≥a, Y(t (i) ) is valued as 1. 6.The urban gentrification spatial intelligence identification method based on deep learning according to claim 1, wherein, The step 9, for each block in the city, the proportion of the sampling points with a gentrification attribute of 1 to all the sampling points is calculated to determine whether the block is gentrified, and the specific calculation and determination formula is as follows: When P < P0, Y = 0 When P ≥ P0, Y = 1 In the formula, P represents the gentrification proportion of each block, Y = 1 represents that the block is gentrified, Y = 0 represents that the block is not gentrified, K represents the number of sampling points in each block, and P0 represents a gentrification proportion threshold for determining whether the block is gentrified.

Images