Historical urban style integrity evaluation method and system based on deep learning

Abstract

The invention relates to a deep learning-based historical urban style integrity evaluation method and system. The method comprises the following steps of: obtaining urban street view image data of a plurality of point locations; extracting a sky ratio and a green vision rate of an urban street scene image according to the urban street scene image data, respectively taking the sky ratio and the green vision rate as a street profile scale spatial feature and a natural element spatial feature, and calculating a signage color number and a color harmony degree of the urban street scene image as building color spatial features; calculating a traditional material proportion and material consistency of the urban street scene image as material mechanism spatial features; for the urban street view image data, obtaining an expert score obtained based on an ELO scoring algorithm; and constructing and training an evaluation model based on an artificial neural network based on the various spatial features and the small-scale expert scoring data set, and realizing historical urban style and appearance integrity evaluation by using the evaluation model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for evaluating the integrity of the appearance of a historical urban area based on deep learning. Background Art

[0002] In the process of urbanization, the phenomenon of "all cities look the same" and the convergence of styles and appearances still exists in large numbers. Urban construction urgently needs to transform from focusing on the "quantity" of urban space supply to improving the "quality" of urban space environment. As a long-term comprehensive accumulation of social culture (style) and material form (appearance), urban style and appearance has become one of the important connotations of high-quality urban development in the future. Especially for historical urban areas, the complete spatial historical characteristics can better convey the cultural connotation and historical heritage of the area to tourists and residents. As an important indicator for evaluating the degree of preservation of urban spatial historical characteristics, the scientific evaluation of the integrity of the style and appearance of historical urban areas is a key step in renewing old cities and enhancing urban characteristics, and is crucial to urban style and appearance management and program evaluation.

[0003] However, the traditional style integrity evaluation uses subjective questionnaire surveys, relies on human subjective judgment, and is easily affected by personal experience and perspective, resulting in large differences in evaluation results and lack of objectivity. In view of the diversity and complexity of the comprehensive consideration factors of the style integrity of historical urban areas, as well as the generally large spatial scale of the evaluation objects, the limitations of existing methods in terms of concept and operation are becoming increasingly prominent: large-scale analysis at the traditional urban scale, such as quantitative evaluation based on morphometric methods, is difficult to take into account the analytical accuracy at the refined scale, and traditional analytical methods at the refined scale are often time-consuming and labor-intensive, and difficult to be used in batches at the urban scale.

[0004] In summary, the current historical urban landscape has the characteristics of a wide research area, diverse elements, and complex relationships between elements. The calculation of its completeness faces the following three technical problems:

[0005] (1) A large amount of data support is required. Traditional manual survey and statistical methods are time-consuming, inefficient, and costly.

[0006] (2) An evaluation system needs to be established. There are many factors that affect urban appearance and they are complex and interrelated. Traditional methods lack a systematic evaluation framework or fail to clearly define the weights of various indicators, resulting in a lack of objectivity in the evaluation process.

[0007] (3) The influencing factors need to be measured objectively and accurately. Traditional methods rely too much on questionnaire surveys and personal experience when measuring influencing factors, which leads to strong subjectivity.

[0008] Therefore, there is an urgent need for a method that can efficiently, quickly and scientifically achieve a refined evaluation of the integrity of the historical urban landscape at a larger scale, so as to facilitate the scientific planning and design analysis of historical urban areas. Summary of the invention

[0009] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method and system for evaluating the integrity of the appearance of a historical urban area based on deep learning, by fully extracting the multi-scale spatial features of urban street view images as the basis for evaluating the integrity of the appearance of a historical urban area.

[0010] The purpose of the present invention can be achieved by the following technical solutions:

[0011] One aspect of the present invention provides a method for evaluating the integrity of historical urban areas based on deep learning, comprising the following steps:

[0012] Obtain city street view image data at multiple points;

[0013] For the urban street view image data, a pre-trained model is used to extract the sky ratio and green view rate of the urban street view images as the streetscape scale spatial characteristics and natural element spatial characteristics, respectively. A color background library is constructed by extracting color information from the urban street view images, and the number of shop sign colors and color harmony in the urban street view images are calculated as architectural color space characteristics. By constructing a sample set of architectural materials in the historical urban area, the proportion of traditional materials and material consistency in the urban street view images are calculated as material mechanism space characteristics.

[0014] As a preferred technical solution, the process of obtaining urban street view image data at multiple points includes:

[0015] For the target city streets, set sampling points at preset intervals and obtain the latitude and longitude coordinate information of the sampling points;

[0016] Based on the latitude and longitude coordinate information of the sampling point, a city street view image parallel to the long axis direction of the street space and in the vertical direction at the sampling point is obtained through road network topology calculation;

[0017] Cleaning processing is performed on the city street scene images showing different seasons to obtain the city street scene image data.

[0018] As a preferred technical solution, the process of acquiring street-scale spatial characteristics includes the following steps:

[0019] Based on the urban street view image data in the long-axis direction, the sky ratio is predicted by using a model based on a convolutional neural network and an encoder-decoder architecture to obtain the streetscape scale spatial features, wherein the encoder is used to extract the features of the input urban street view image, and the decoder is used to reconstruct the sky area.

[0020] As a preferred technical solution, the process of acquiring the spatial characteristics of natural elements includes the following steps:

[0021] Based on the urban street view image data in the long-axis direction, a model based on a convolutional neural network and a U-Net architecture is used to segment green vegetation and predict the green view rate of the street view image as the spatial feature of the natural element.

[0022] As a preferred technical solution, the process of acquiring the architectural color space features includes the following steps:

[0023] Based on the vertical urban street view image data, the color type of the building facade in the image is calibrated. Through shadow detection and color correction, the urban street view image data with weakened lighting effects is obtained. By extracting the image color to build a color background library, the color number information of the store sign is obtained.

[0024] Based on the color background library and the expert ratings of the color harmony of the pictures, a color harmony evaluation model is trained;

[0025] The color harmony evaluation model is used to obtain the predicted color harmony, and the color space characteristics of the building are obtained by combining the color number information of the shop sign.

[0026] As a preferred technical solution, the process of acquiring the spatial characteristics of the material mechanism includes the following steps:

[0027] Based on the urban street view image data in the vertical direction, the architectural element information is extracted through perspective correction and semantic segmentation, the material unit is extracted through adaptive rectangular segmentation, and the architectural material sample set is obtained through calibration;

[0028] Based on the building material sample set, a classification model based on a convolutional neural network is trained to obtain the composition information of the facade materials in the urban street view images, and the proportion of traditional materials and the material consistency score are obtained as the material mechanism spatial characteristics.

[0029] As a preferred technical solution, the traditional materials include brick, wood and stone.

[0030] As a preferred technical solution, it also includes:

[0031] For the urban street view image data, obtaining expert scores based on the ELO scoring algorithm;

[0032] Based on the various spatial features and small-scale expert rating data sets, an evaluation model based on artificial neural networks is constructed and trained, and the evaluation model is used to evaluate the integrity of the historical urban area.

[0033] As a preferred technical solution, the evaluation model training process includes the following steps:

[0034] The evaluation model is trained repeatedly using randomly generated sub-samples, and model selection is achieved using a cross-validation approach.

[0035] As a preferred technical solution, the process of using the evaluation model to evaluate the integrity of the historical urban area includes the following steps:

[0036] The evaluation model is used to calculate the style integrity of each sampling point, a buffer zone is set for the street segment, the average style integrity of the sampling points in the buffer zone is calculated as the style integrity of the street segment, and a visualization signal is output.

[0037] Another aspect of the present invention provides a system for evaluating the integrity of historical urban areas based on deep learning, comprising:

[0038] A data acquisition module, used to acquire urban street view image data at multiple points;

[0039] The feature extraction module is used to extract the sky ratio and green view rate of the urban street view image using a pre-trained convolutional neural network model for the urban street view image data, as the streetscape scale spatial feature and natural element spatial feature respectively, to construct a color background library by extracting color information from the urban street view image, to calculate the number of shop sign colors and color harmony of the urban street view image as the architectural color space feature, and to calculate the traditional material ratio and material consistency of the urban street view image as the material mechanism spatial feature by constructing a sample set of architectural materials in the historical urban area.

[0040] As a preferred technical solution, it also includes:

[0041] An expert scoring module, used for obtaining expert scores based on an ELO scoring algorithm for the urban street view image data;

[0042] A large-scale historical style integrity evaluation module constructs and trains an evaluation model based on artificial neural networks based on the various spatial features and small-scale expert scoring data sets, and uses the evaluation model to achieve historical urban style integrity evaluation.

[0043] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0044] (1) Fully extracting the multi-dimensional spatial features of urban streetscape images: The present invention fully extracts the spatial features of street scale, natural elements, architectural colors and material mechanisms, and combines the above multi-scale spatial features to prepare for the subsequent evaluation of the integrity of the historical urban landscape.

[0045] (2) High efficiency: The present invention utilizes large-scale street view image capture technology to significantly reduce the time for data collection and greatly improve work efficiency. With the help of an intelligent evaluation model based on an artificial neural network, a comprehensive analysis of the target area can be completed in a very short time. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flowchart of a method for evaluating the integrity of historical urban areas based on deep learning in an embodiment;

[0047] Figure 2 This is a schematic diagram of a large-scale urban street view image acquisition process in an embodiment;

[0048] Figure 3 Schematic diagram of the process of capturing a city street view image in the embodiment;

[0049] Figure 4 This is a schematic diagram of the measurement results of "street scale-sky visibility" in the middle city style area of ​​the embodiment;

[0050] Figure 5 A schematic diagram of a process for constructing a building color background library based on street view data and machine learning in an embodiment;

[0051] Figure 6 It is the color background map of the public activity center style area in the embodiment;

[0052] Figure 7 This is a schematic diagram of the measurement results of "architectural color-color harmony" in the middle city style area of ​​the embodiment;

[0053] Figure 8 This is a schematic diagram of the measurement results of "building color - number of shop sign colors" in the middle city style area of ​​the embodiment;

[0054] Fig. 9 A schematic diagram of the process of constructing a building material background library based on street view data and machine learning in an embodiment;

[0055] Fig.10 This is a schematic diagram of the measurement results of "material mechanism-material consistency" in the style area of ​​the middle city in the embodiment;

[0056] Fig.11 This is a schematic diagram of the measurement results of "Material Mechanism - Traditional Material Proportion" in the style area of ​​the middle city in the embodiment;

[0057] Fig.12 This is a schematic diagram of the measurement results of "natural elements-green view rate" in the urban landscape area of ​​the embodiment;

[0058] Fig.13 This is a schematic diagram of the small-scale sample picture scoring process in the embodiment;

[0059] Fig.14 Schematic diagram of the flow of the landscape integrity assessment model based on artificial neural network in the embodiment;

[0060] Fig.15 This is a schematic diagram of the scoring results of the style and appearance integrity of each sampling point in the style and appearance area of ​​the Central City in the embodiment.

[0061] Fig.16 Schematic diagram of a historical urban area style integrity evaluation system based on deep learning in an embodiment. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0063] The following are definitions of some terms involved in the present invention:

[0064] Historical urban appearance: The historical urban appearance refers to a unique external spatial image collection composed of visible material form elements (such as building street scale, architectural color, material texture, natural elements, etc.) in a specific historical urban area.

[0065] Integrity of historical urban features: The integrity of historical urban features is an indicator that evaluates the degree of preservation of the historical characteristics of urban space by comprehensively considering aspects such as street scale, architectural color, material texture, and natural elements within the historical urban area.

[0066] Street view image: Street view image (SVI) refers to the 360-degree visual information of urban streets and their surrounding environment obtained through panoramic photography technology, which can provide a panoramic spatial view of the street that is close to a personal perspective.

[0067] Artificial Neural Network: An artificial neural network (ANN) is a computational model that mimics the structure of neurons in the human brain and is used to handle complex pattern recognition and prediction problems. It consists of a large number of simple processing units (neurons) that optimize performance by adjusting connection weights.

[0068] ArcGIS: A geographic information system platform that is an infrastructure for mapping that can be used to collect, organize, manage, analyze, communicate, and publish geographic information.

[0069] Example 1

[0070] In view of the problems existing in the above-mentioned prior art, this embodiment provides a method for evaluating the integrity of the historical urban landscape based on deep learning. Figure 1 , the method comprises the following steps:

[0071] S1, obtain large-scale urban street view images.

[0072] Large-scale urban street view image acquisition includes data collection and data cleaning. First, the sampling points are determined on the city streets at intervals of 40 meters, and the longitude and latitude coordinates are determined. Then, the API of the map software Street View is legally called through Python via HTTP URL, and one image is collected from the front, back, left, and right perspectives of each sampling point. The horizontal and vertical angles of the line of sight and the viewpoint position data are input to obtain the street view, coordinates, angles and other information of each sample point, so as to realize the batch collection of street view images corresponding to the sample points. Finally, the images with poor clarity and taken too early are deleted to complete the data cleaning.

[0073] Before data collection, OSM data and GIS were combined to calculate the road network topology structure for each sampling point perspective to ensure that all street view images were parallel to the spatial long axis of the street and to prevent the deformation of street images from affecting subsequent feature extraction.

[0074] Preferably, the map software street view API legally called by Python includes but is not limited to Baidu Street View, Google Street View, etc.

[0075] S2, uses machine learning models to extract key spatial features.

[0076] Through precise preparation of the data set, model tuning and retraining, the machine learning algorithm SegNet is used to extract four key spatial features in street view images: street scale, building color, material texture, and natural elements, to obtain quantitative measurements of each spatial element in the street view.

[0077] Among them, the street frame scale is measured by sky visibility. The streets with low sky visibility have small street frame scale, and the streets with high sky visibility have large street frame scale.

[0078] Architectural color measurement includes two aspects: color harmony and the number of shop sign colors. First, the color type of street view data is calibrated for areas with typical architectural colors. Then, based on machine learning algorithms, color information is extracted from images to build an architectural color background library and an automatic color harmony evaluation model, so as to achieve the evaluation of architectural color harmony in large-scale street view data and the statistics of the number of shop sign colors.

[0079] Material mechanism measurement includes two aspects: material consistency and the proportion of traditional materials. Special calibration and model training are carried out on the building materials in the representative historical urban environment. Then, based on the intelligent recognition results of the model, the consistency of the materials used in the buildings in the street view images and the proportion of traditional materials such as brick, wood, and stone are evaluated.

[0080] Natural elements are measured by green view rate. Streets with high green view rate have more natural elements, while streets with low green view rate have fewer natural elements. Special plant type calibration has been carried out for China's existing built environment to ensure that different tree species and vegetation types can be accurately distinguished during the recognition process, and then the green view rate in street view images can be accurately calculated.

[0081] S3, invite experts to evaluate a small sample of images.

[0082] Representative images were selected from the large-scale street view images for small-scale scoring. Experts were invited to compare the sample images in pairs. The comparison results were then converted into image scores using the ELO rating system, thereby obtaining a small-scale quantitative score of the impact of spatial features on preference evaluation.

[0083] During the score conversion process, the ELO scoring algorithm first assigns each sample photo an initial score of 1000, and then updates the score based on the results of each comparison; when the result becomes stable after multiple iterations, the final score of the sample photo is obtained.

[0084] S4. Use artificial neural networks to train historical urban area integrity assessment models and carry out large-scale assessments.

[0085] The evaluation model is trained with subjective scores and objective indicator measurements, and then the model is used to conduct large-scale landscape integrity assessment and calculate the landscape integrity value of each sampling point. A 30m buffer zone is set on the left and right sides of the street segments, and the average integrity of all points in the buffer zone is calculated to obtain the integrity evaluation of the street segments.

[0086] The evaluation model uses artificial neural network technology (ANN) in the field of machine learning to better handle multiple interactions; cross validation (k-fold cross validation) is used for model selection, and randomly generated sub-samples are repeatedly used for training and validation. The evaluation model's evaluation of the integrity of the historical urban area is theoretically close to the professional judgment of designers, but it can quickly and efficiently analyze all streets in the area and carry out large-scale and refined integrity evaluation.

[0087] The following embodiment further illustrates the method by taking Shanghai as an example.

[0088] See also Figure 2,Step S1, large-scale urban street view image acquisition.

[0089] The specific steps are as follows: taking Shanghai as an example, in ARCGIS software, using the road network data of the central urban landscape area, a total of 19,735 sampling points are determined at intervals of 40 meters, and the longitude and latitude coordinates of each point are determined; secondly, combining OSM data and GIS to calculate the perspective of each sampling point based on the road network topology structure, to ensure that all street view images are parallel to the spatial long axis of the street, to prevent the deformation of street pictures from affecting subsequent feature extraction; then, this embodiment uses the street view images provided by existing databases (such as Baidu Maps) to replace manual photography to quickly obtain large quantities of street view images; finally, images that do not show uniform seasons are cleaned to avoid interference of street views in different seasons on the overall spatial quality. Specifically, this embodiment uses Python to legally call the Baidu Street View API via HTTP URL, inputs the horizontal and vertical angles of the line of sight and the viewpoint location data, and obtains the street view, coordinates, angle and other information of each sample point. See. Figure 3 In this embodiment, one image is collected from each sampling point at the front, rear, left, and right perspectives, and the image resolution is 480×360 pixels, and a total of about 78,940 valid city street scene photos are obtained.

[0090] Step S2, extract key spatial features of landscape elements based on machine learning. 1,000 representative point images are selected from the acquired street view images, and the machine learning algorithm SegNet is used to extract four key spatial features of street scale, building color, material texture, and natural elements in the street view images, laying the foundation for the next step of urban landscape integrity assessment model training. The specific method of quantifying the above four indicators is as follows:

[0091] (1) Streetscape scale: Select photos in the long axis direction and apply a convolutional neural network (CNN) model to judge the sky visibility. The model is implemented using the Python programming language and the TensorFlow framework. It adopts an encoder-decoder architecture, in which the encoder part is responsible for extracting features from the input image, while the decoder part is responsible for reconstructing the sky area. The model is optimized using the mean square error as the loss function and can accurately identify and quantify the proportion of sky in the image. See Figure 4 , streets with low sky visibility have small street outlines, and streets with high sky visibility have large street outlines.

[0092] Specifically, the calculation process of street scale characteristics is as follows:

[0093] The sky visibility is measured and transferred by scoring, that is, by calculating the ratio of the visible sky area to the total area of ​​the photo taken along the long axis (parallel to the street) of a certain sampling point, as the calculation result of the "street scale" feature integrity factor. The result value range is [0,1], where 0 means that there is no sky visible in the long axis direction of the image (completely closed space), and the "street scale" indicator score is 0, while 1 means that the entire sky is visible in the image (open space), and the "street scale" indicator score is 1. The specific formula is as follows:

[0094]

[0095] Among them, S a represents the number of pixels in the image that are identified as sky, T a Indicates the total number of pixels in the image.

[0096] The measurement of sky visibility uses a convolutional neural network (CNN) model to automatically identify the sky in the image. The model is implemented using the Python programming language and the TensorFlow framework. It uses a novel encoder-decoder architecture, in which the encoder part is responsible for extracting features from the input image, while the decoder part is responsible for reconstructing the sky area. The model is optimized using the mean square error as the loss function, and can accurately identify and quantify the proportion of sky in the image. The loss function used in the optimization process is shown below:

[0097]

[0098] in, represents the predicted sky area of ​​the model for the input image xi, y i represents the true sky area label, N is the number of training samples, and θ is the parameter set of the model.

[0099] (2) Architectural color: including color harmony and number of shop sign colors. Select vertical photos to calibrate the color type of the building facade in the image, extract color information from the image based on machine learning algorithms, build an architectural color background library and train a color harmony automatic evaluation model. Figure 5 and Figure 6 First, Deeplab is used to extract the building facades in street view images, and then shadow detection and color correction are performed to obtain clearer, objective building color photos that are not affected by lighting. Then, the colors in the images are extracted to build a building color background library to train the AI ​​model for automatic evaluation of color harmony. Figure 7 and Figure 8Based on the extracted color background and combined with expert scoring, an AI model for automated evaluation of color harmony was trained to conduct a large-scale evaluation of the color harmony of Shanghai’s central urban area and the number of store sign colors.

[0100] Specifically, the calculation process of architectural color features is as follows:

[0101] Including the number of shop sign colors and color harmony. First, use Deeplab to extract the building facades in the street view image, and then use shadow detection and color correction to obtain clearer, objective building color photos that are not affected by light. Then extract the colors in the image to build a building color background library. Among them, color extraction uses the RGB color model to divide the colors, and divides the three channels of red (Red), green (Green) and blue (Blue) into 10 intervals respectively. Each interval contains 26 consecutive integer values ​​in the range of 0-255, forming a background library containing 1000 color types.

[0102] Shop sign color number calculation process: Automatically identify shop signs on building facades in street view images, extract colors, compare them with colors in the background library, and count the number of colors.

[0103] Color harmony calculation process: Based on the extracted color background library and according to color matching principles (such as contrast, saturation, brightness, etc.), the color harmony of the building facades in the sample is evaluated to train the AI ​​model for automatic evaluation of color harmony, thereby realizing the evaluation of building color harmony in large-scale street view data.

[0104] The "architectural color" index is calculated by combining the measurement results of the number of store signs' colors and the degree of color harmony. The average of the number of store signs' colors in 1,000 randomly sampled images of the style and appearance areas is used as the benchmark value. The store sign's color score is the ratio of the measured data to the benchmark. The result is normalized within the numerical range of [0,1] to form a transformation from the store sign's color number measurement to the "architectural color" index score. Similarly, the color harmony is given by the AI ​​model and normalized to a score in the numerical range of [0,1]. Finally, the decision tree algorithm is used to calculate the contribution of the two feature scores to the style and appearance, and the corresponding weights are obtained; the weighted average of the two scores is used as the calculation result of the "architectural color" style and appearance integrity factor. The specific formula is as follows:

[0105] BC=C S ×ω1+H S ×ω2

[0106] Among them, C S represents the normalized score of the store sign color number, H S represents the color harmony score, ω1 and ω2 are the weights of the store sign color score and color harmony score respectively.

[0107] (3) Material mechanism: including material consistency and traditional material proportion. Select vertical photos to calibrate and train the model for building materials in representative historical urban environments. For details, see Fig. 9 First, the street view images were perspective corrected, the semantic segmentation model was used to extract the building elements, and an adaptive rectangle segmentation method was used to extract the material units. Then, the material units were manually calibrated to construct a large sample training set of building materials in the historical city, with the annotated types including brick, concrete, glass, metal, paint, stone and wood. The CNN model was trained to automatically classify the material rectangles, and then the facade material composition in the street view images was obtained. Fig.10 and Fig.11 The trained model is used to identify and analyze building materials in large-scale street view images. Based on the intelligent recognition results of the model, the consistency of materials used in buildings in street view images and the proportion of traditional materials such as brick, wood, and stone are evaluated.

[0108] Specifically, the calculation process of material mechanism characteristics is as follows:

[0109] It includes two aspects: the proportion of traditional materials and material consistency. First, the street view image is perspective corrected, the semantic segmentation model is used to extract architectural elements, and the adaptive rectangular segmentation method is used to extract material units. Subsequently, the material units are manually calibrated to construct a large sample training set of architectural materials in the historical style area. The annotation types include common materials such as bricks, concrete, glass, metal, paint, stone and wood. The CNN model is trained to automatically classify the material rectangles, and then the material composition of the facade in the street view image is obtained. The trained model is used for the recognition and analysis of architectural materials in large-scale street view images, and the proportion of three traditional materials, bricks, wood and stone, in the image is calculated.

[0110] Among them, in view of the different distances of buildings in different street view images, the building facades will account for different proportions in the image, and the same unit size cannot be used to directly segment the building facades. An adaptive rectangular segmentation method is adopted. First, the street view image is perspective corrected, and the building facades in the street view image are extracted using the semantic segmentation model of Deeplab, and binarized (the building facades are white, and the rest are black); then, according to the number of pixels in the facade part, the appropriate segmentation unit size is selected (that is, the length and width of the segmentation rectangle, such as 100*100 or 80*80 or 60*60, etc.). Optionally, the image with a smaller unit size is enlarged in pixels. This allows each unit to clearly reflect the characteristics of a certain material and adapt to building facade images of different sizes.

[0111] Based on the identified building facade material types and corresponding areas, the distribution density of each material on the building facade is calculated by the area ratio of each material, and then combined with the material distribution uniformity calculated by entropy, the material consistency result is obtained. The specific public announcement is as follows:

[0112]

[0113] Among them, ρ i is the distribution density of the ith material, Mi is the area of ​​the ith material, Mt is the total area of ​​the building facade material, and H is the entropy value.

[0114] The measurement results of traditional material proportion and material consistency were normalized to [0,1], and the decision tree algorithm was used to calculate the contribution of the two feature scores to the landscape, and the corresponding weights were obtained. The weighted average of the two scores was used as the calculation result of the "material texture" landscape integrity factor.

[0115] (4) Natural elements: Select photos in the long axis direction and apply the convolutional neural network (CNN) model to judge the green view rate. First, the existing built environment is specially calibrated for plant types to ensure that different tree species and vegetation types can be accurately distinguished during the recognition process. Then, a deep learning method based on the U-Net architecture is used to train the model using ResNet-50 to achieve accurate segmentation of green vegetation in the image and accurately calculate the green view rate in the street view image. Fig.12 Streets with high green view rates have more natural elements, while streets with low green view rates have fewer natural elements.

[0116] Specifically, the calculation process of natural elements includes the following steps:

[0117] The green view rate is used for measurement and score transfer, that is, the ratio of the visible area of ​​plants to the total area of ​​the photos taken along the short axis (perpendicular to the street) of a certain sampling point is calculated as the calculation result of the "natural element" feature integrity factor. The result value range is [0,1], where 0 means there are no plants in the image and the "natural element" indicator score is 0, while 1 means that the image is completely filled with plants and the "natural element" indicator score is 1. The specific formula is as follows:

[0118]

[0119] Among them, P a represents the visible area of ​​plants, that is, the number of pixels marked as plant areas after being identified by the CNN model; T a Indicates the total number of pixels in the image.

[0120] The green view rate is measured by automatically identifying plants in images using a convolutional neural network (CNN) model. First, the common plant types in China's existing built environment are calibrated to ensure that different tree species and vegetation types can be accurately distinguished during the identification process. Then, a deep learning method based on the U-Net architecture is used to train the model using ResNet-50 to achieve accurate segmentation of green vegetation in the image, thereby accurately calculating the plant area in the street view image and achieving the measurement of the green view rate.

[0121] See also Fig.13 In step S3, the 1000 representative point images used for feature extraction in step S2 are simultaneously scored by experts. The scoring program is written in JAVA language. First, each sample photo is given an initial score of 1000, and then experts are invited to compare the sample photos in pairs. The photo scores are updated according to the results of each comparison. After multiple iterations and stabilization, the final scores of the sample photos are obtained. The final scores of the photos are used as the expert evaluation results of the style and appearance integrity, laying the foundation for the next step of training a large-scale historical urban style and appearance integrity evaluation model that is close to professional judgment.

[0122] See also Fig.14 and Fig.15 , step S4, the evaluation model is trained with the subjective score and the objective index measurement results to carry out a large-scale historical urban style integrity assessment. Specifically, considering the complex interaction between the style integrity and spatial characteristics of the historical urban area, the model uses artificial neural network (ANN) training in the field of machine learning, and cross validation (k-fold cross validation) is used for model selection. The randomly generated sub-samples are repeatedly used for training and validation. The prediction error rate (percent incorrect predictions) of the evaluation model generated is 0.21, and the evaluation model is theoretically close to the professional judgment of the designer in the evaluation of the urban style integrity. At the same time, it can quickly and efficiently analyze all the streets in the area. Subsequently, a large-scale and refined integrity assessment is carried out in the style area of ​​Shanghai's central urban area, and the style integrity value of each sampling point is calculated. Finally, a buffer zone of 30m is set on the left and right sides of the street segment, and the average integrity of all points in the buffer zone is calculated to obtain the integrity evaluation of the street segment, and the results are visualized through ArcGIS.

[0123] In summary, this method has the following characteristics:

[0124] (1) Efficient acquisition of massive data. This method uses large-scale street view image capture technology to significantly reduce the time for data collection to 10% of the original time, greatly improving work efficiency. With the help of an intelligent evaluation model based on artificial neural networks, a comprehensive and accurate analysis of the target area can be completed in a very short time. Compared with the judgment results of professional designers, the accuracy rate has reached more than 90%, realizing large-scale and accurate calculation of the integrity of the historical urban landscape. At the same time, this method that combines street view data with machine learning technology has the ability of dynamic real-time monitoring. This real-time nature not only helps urban managers respond to changes quickly, but also ensures the continuity and effectiveness of urban landscape protection work.

[0125] (2) Construction of the evaluation system. This method uses deep learning technology to systematically extract and quantify the key spatial features of the historical city, and automatically assigns corresponding weights to these features through an algorithm. This process ensures the objective evaluation and accurate measurement of the impact of various factors on the integrity of the historical city's appearance. This method has good scalability and can be quickly promoted and used. By using machine learning technology to build a model to learn from the subjective experience and professional judgment of experts, high-precision and professional appearance integrity evaluation can be achieved even in small and medium-sized cities with relatively weak professional technology, ensuring the consistency and accuracy of the evaluation.

[0126] (3) Optimization of factor measurement. This method combines the expert evaluation results of small-scale samples to construct a historical urban landscape integrity evaluation model based on artificial neural networks. Through the dual drive of knowledge and data, the evaluation model is optimized in the field of landscape integrity evaluation. This enables the model to not only traverse the analysis area quickly and efficiently, but also has expert-like evaluation and diagnosis capabilities, thereby enabling a refined historical urban landscape integrity evaluation on a large scale.

[0127] (4) Great social value. This method is based on the street view data acquisition and machine learning technology to evaluate the integrity of the style and appearance of the historical city, which is also of great significance for the protection and inheritance of the style and appearance of the historical city. This method can accurately identify the architectural style and cultural characteristics in a specific historical block and evaluate its integrity. It can timely feedback the degree of preservation of these important contextual elements in the urban renewal process to urban planners, which not only helps to maintain the unique style and appearance of the historical city, but also promotes the preservation and inheritance of the cultural characteristics of the architecture and urban landscape of various places.

[0128] Example 2

[0129] See also Fig.16 Based on Example 1, this embodiment provides a historical urban area style integrity evaluation system based on deep learning, including:

[0130] A data acquisition module, used to acquire urban street view image data at multiple points;

[0131] A feature extraction module is used to extract the sky ratio and green view rate of the urban street view image using a pre-trained convolutional neural network model for the urban street view image data, as streetscape scale spatial features and natural element spatial features, respectively; construct a color background library by extracting color information from the urban street view image; calculate the number of store sign colors and color harmony of the urban street view image as architectural color spatial features; and calculate the proportion of traditional materials and material consistency of the urban street view image as material mechanism spatial features by constructing a sample set of architectural materials in the historical urban area;

[0132] An expert scoring module, used for obtaining expert scores based on an ELO scoring algorithm for the urban street view image data;

[0133] The large-scale historical style integrity evaluation module builds and trains an evaluation model based on artificial neural networks based on various spatial features and small-scale expert scoring data sets, and uses the evaluation model to realize the integrity evaluation of the historical urban style.

[0134] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for evaluating the integrity of historical urban areas based on deep learning, characterized in that: The steps include: Obtain city street view image data at multiple points; For the urban street view image data, a pre-trained model is used to extract the sky ratio and green view rate of the urban street view images as the streetscape scale spatial characteristics and natural element spatial characteristics, respectively. A color background library is constructed by extracting color information from the urban street view images, and the number of shop sign colors and color harmony in the urban street view images are calculated as architectural color space characteristics. By constructing a sample set of architectural materials in the historical urban area, the proportion of traditional materials and material consistency in the urban street view images are calculated as material mechanism space characteristics.

2. According to the method for evaluating the integrity of historical urban areas based on deep learning in claim 1, it is characterized in that: The process of obtaining urban street view image data at multiple points includes: For the target city streets, set sampling points at preset intervals and obtain the latitude and longitude coordinate information of the sampling points; Based on the latitude and longitude coordinate information of the sampling point, a city street view image parallel to the long axis direction of the street space and in the vertical direction at the sampling point is obtained through road network topology calculation; Cleaning processing is performed on the city street scene images showing different seasons to obtain the city street scene image data.

3. According to the method for evaluating the integrity of historical urban areas based on deep learning in claim 1, it is characterized in that: The process of acquiring the streetscape scale spatial characteristics includes the following steps: Based on the urban street view image data in the long-axis direction, the sky ratio is predicted by using a model based on a convolutional neural network and an encoder-decoder architecture to obtain the streetscape scale spatial features, wherein the encoder is used to extract the features of the input urban street view image, and the decoder is used to reconstruct the sky area.

4. According to the method for evaluating the integrity of historical urban areas based on deep learning in claim 1, it is characterized in that: The process of acquiring the spatial characteristics of natural elements includes the following steps: Based on the urban street view image data in the long-axis direction, a model based on a convolutional neural network and a U-Net architecture is used to segment green vegetation and predict the green view rate of the street view image as the spatial feature of the natural element.

5. According to the method for evaluating the integrity of historical urban areas based on deep learning in claim 1, it is characterized in that: The process of acquiring the architectural color space features comprises the following steps: Based on the vertical urban street view image data, the color type of the building facade in the image is calibrated. Through shadow detection and color correction, the urban street view image data with weakened lighting effects is obtained. By extracting the image color to build a color background library, the color number information of the store sign is obtained. Based on the color background library and the expert ratings of the color harmony of the pictures, a color harmony evaluation model is trained; The color harmony evaluation model is used to obtain the predicted color harmony, and the color space characteristics of the building are obtained by combining the color number information of the shop sign.

6. The method for evaluating the integrity of historical urban landscape based on deep learning according to claim 1 is characterized in that: The process of acquiring the material mechanism spatial characteristics includes the following steps: Based on the urban street view image data in the vertical direction, the architectural element information is extracted through perspective correction and semantic segmentation, the material unit is extracted through adaptive rectangular segmentation, and the architectural material sample set is obtained through calibration; Based on the building material sample set, a classification model based on a convolutional neural network is trained to obtain the composition information of the facade materials in the urban street view images, and the proportion of traditional materials and the material consistency score are obtained as the material mechanism spatial characteristics.

7. According to the method for evaluating the integrity of historical urban areas based on deep learning in claim 1, it is characterized in that: Also includes: For the urban street view image data, obtaining expert scores based on the ELO scoring algorithm; Based on the various spatial features and small-scale expert rating data sets, an evaluation model based on artificial neural networks is constructed and trained, and the evaluation model is used to evaluate the integrity of the historical urban area.

8. The method for evaluating the integrity of historical urban areas based on deep learning according to claim 7 is characterized in that: The process of using the evaluation model to evaluate the integrity of the historical urban area includes the following steps: The evaluation model is used to calculate the style integrity of each sampling point, a buffer zone is set for the street segment, the average style integrity of the sampling points in the buffer zone is calculated as the style integrity of the street segment, and a visualization signal is output.

9. A historical urban area style integrity evaluation system based on deep learning, characterized in that: include: A data acquisition module, used to acquire urban street view image data at multiple points; The feature extraction module is used to extract the sky ratio and green view rate of the urban street view image using a pre-trained convolutional neural network model for the urban street view image data, as the streetscape scale spatial feature and natural element spatial feature respectively, to construct a color background library by extracting color information from the urban street view image, to calculate the number of shop sign colors and color harmony of the urban street view image as the architectural color space feature, and to calculate the traditional material ratio and material consistency of the urban street view image as the material mechanism spatial feature by constructing a sample set of architectural materials in the historical urban area.

10. The historical urban area style integrity evaluation system based on deep learning according to claim 9 is characterized in that: Also includes: An expert scoring module, used for obtaining expert scores based on an ELO scoring algorithm for the urban street view image data; A large-scale historical style integrity evaluation module constructs and trains an evaluation model based on artificial neural networks based on the various spatial features and small-scale expert scoring data sets, and uses the evaluation model to achieve historical urban style integrity evaluation.

Images