A historical urban style integrity evaluation method and system based on deep learning

Abstract

This invention relates to a method and system for evaluating the integrity of historical urban landscape based on deep learning. The method includes: acquiring urban street view image data from multiple locations; extracting the sky ratio and green view rate of the urban street view images as street-scale spatial features and natural element spatial features, respectively; calculating the number of shop sign colors and color harmony as architectural color spatial features; and calculating the proportion and consistency of traditional materials as material mechanism spatial features; obtaining expert scores based on the ELO scoring algorithm for the urban street view image data; constructing and training an evaluation model based on an artificial neural network based on the various spatial features and a small-scale expert score dataset; and using the evaluation model to evaluate the integrity of the historical urban landscape.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for evaluating the integrity of historical urban landscape based on deep learning. Background Technology

[0002] In the process of urbanization, the phenomenon of "one-size-fits-all" cities and similar appearances still exists in large numbers. Urban construction urgently needs to shift from focusing on the "quantity" of urban space supply to improving the "quality" of the urban spatial environment. Urban landscape, as a long-term comprehensive accumulation of social culture (style) and material form (appearance), has become one of the important connotations of high-quality urban development in the future. Especially for historic urban areas, the complete preservation of spatial historical features can better convey the cultural connotations and historical heritage of the area to tourists and residents. As an important indicator for evaluating the degree of preservation of urban spatial historical features, the scientific evaluation of the integrity of the historic urban landscape is a key step in the renewal of old cities and the enhancement of urban characteristics, and is crucial for urban landscape management and scheme evaluation.

[0003] However, traditional assessments of historical district integrity rely on subjective questionnaires and human judgment, making them susceptible to the influence of personal experience and perspectives, resulting in significant discrepancies and a lack of objectivity. Given the diversity and complexity of the factors considered in assessing the integrity of historical districts, and the generally large spatial scale of the assessed objects, the limitations of existing methods in both concept and operation are becoming increasingly apparent: large-scale analyses at the traditional urban scale, such as quantitative assessments based on morphometrics, struggle to maintain the precision required for detailed analyses at a finer scale, while traditional analytical methods at a finer scale are often time-consuming and labor-intensive, making them difficult to apply on a large scale at the urban level.

[0004] In summary, the current historical urban landscape is characterized by a wide research area, diverse elements, and complex relationships between these elements. The calculation of its completeness faces the following three technical challenges:

[0005] (1) It requires a large amount of data support. Traditional manual surveying and statistical methods are time-consuming, inefficient and costly.

[0006] (2) An evaluation system needs to be constructed. There are many factors that affect the urban landscape and they are interrelated and complex. Traditional methods lack a systematic evaluation framework or fail to clearly define the weight of each indicator, resulting in a lack of objectivity in the evaluation process.

[0007] (3) The influencing factors need to be measured objectively and accurately. Traditional methods rely too much on questionnaires and personal experience when measuring influencing factors, which leads to a high degree of subjectivity.

[0008] Therefore, there is an urgent need for a method that can efficiently, quickly, and scientifically achieve a refined evaluation of the integrity of the historical urban landscape on a large scale, so as to help to make the planning and design analysis of historical urban areas more scientific. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and system for evaluating the integrity of historical urban areas based on deep learning. This method fully extracts multi-scale spatial features from urban street view images as the basis for evaluating the integrity of historical urban areas.

[0010] The objective of this invention can be achieved through the following technical solutions:

[0011] One aspect of the present invention provides a method for evaluating the integrity of historical urban landscape based on deep learning, comprising the following steps:

[0012] Acquire urban street view image data from multiple locations;

[0013] For the urban street view image data, the sky ratio and green view rate of the urban street view images are extracted using a pre-trained model and used as the spatial features of street block scale and natural elements, respectively. A color background library is constructed by extracting color information from the urban street view images, and the number of colors of shop signs and color harmony in the urban street view images are calculated as the spatial features of building color. By constructing a sample set of building materials in historical urban areas, the proportion of traditional materials and material consistency in the urban street view images are calculated as the spatial features of material mechanism.

[0014] As a preferred technical solution, the process of acquiring urban street view image data from multiple locations includes:

[0015] For the target city streets, sampling points are set at preset intervals to obtain the latitude and longitude coordinates of the sampling points;

[0016] Based on the latitude and longitude coordinates of the sampling points, urban street scene images of the sampling points are obtained by calculating the road network topology, which are parallel to the long axis of the street space and perpendicular to the road network topology.

[0017] The city street scene images, representing different seasons, are cleaned to obtain the city street scene image data.

[0018] As a preferred technical solution, the process of obtaining the street block-scale spatial features includes the following steps:

[0019] Based on urban street view image data along the long axis, the sky octagon is predicted using a model based on a convolutional neural network and an encoder-decoder architecture to obtain the spatial features at the street block scale. The encoder is used to extract features from the input urban street view image, and the decoder is used to reconstruct the sky region.

[0020] As a preferred technical solution, the process of acquiring the spatial characteristics of natural elements includes the following steps:

[0021] Based on urban street view image data along the long axis, a model based on convolutional neural networks and U-Net architecture is used to segment green vegetation and predict the green view rate of the street view image, which is then used as the spatial feature of the natural elements.

[0022] As a preferred technical solution, the process of obtaining the architectural color space characteristics includes the following steps:

[0023] Based on vertical urban street view image data, the color type of building facades in the images is calibrated. Through shadow detection and color correction, urban street view image data with weakened lighting effects is obtained. By extracting image colors, a color baseline library is constructed to obtain the color information of shop signs.

[0024] Based on the aforementioned color background library and expert scores for image color harmony, a color harmony evaluation model is trained.

[0025] Using the color harmony evaluation model, the predicted color harmony is obtained, and combined with the color number information of the shop sign, the architectural color space characteristics are obtained.

[0026] As a preferred technical solution, the process of obtaining the spatial characteristics of the material mechanism includes the following steps:

[0027] Based on vertical urban street view image data, architectural element information is extracted through perspective correction and semantic segmentation, material units are extracted through adaptive rectangular segmentation, and a set of architectural material samples is obtained through calibration.

[0028] Based on the aforementioned building material sample set, a classification model based on a convolutional neural network is trained to obtain the composition information of facade materials in urban street scene images, and to obtain the proportion of traditional materials and material consistency scores, which serve as the spatial features of the material mechanism.

[0029] As a preferred technical solution, the traditional materials include brick, wood and stone.

[0030] As a preferred technical solution, the following are also included:

[0031] For the aforementioned city street view image data, obtain expert scores based on the ELO scoring algorithm;

[0032] Based on the aforementioned spatial features and small-scale expert rating datasets, an evaluation model based on artificial neural networks is constructed and trained, and the evaluation model is used to evaluate the integrity of the historical urban landscape.

[0033] As a preferred technical solution, the evaluation model training process includes the following steps:

[0034] The evaluation model is trained by repeatedly using randomly generated subsamples, and model selection is achieved by cross-validation.

[0035] As a preferred technical solution, the process of evaluating the integrity of the historical urban landscape using the aforementioned evaluation model includes the following steps:

[0036] The evaluation model is used to calculate the landscape integrity of each sampling point. A buffer zone is set for the street line segment, and the mean of the landscape integrity of the sampling points within the buffer zone is calculated as the landscape integrity of the street line segment. A visual signal is output.

[0037] Another aspect of the present invention provides a deep learning-based system for evaluating the integrity of historical urban landscape, comprising:

[0038] The data acquisition module is used to acquire urban street view image data from multiple locations;

[0039] The feature extraction module is used to extract the sky ratio and green view rate of the urban street view image data using a pre-trained convolutional neural network model, which are respectively used as the spatial features of street block scale and natural element space features. By extracting color information from the urban street view image, a color background library is constructed, and the number of colors of shop signs and color harmony of the urban street view image are calculated as architectural color space features. By constructing a sample set of building materials in historical urban areas, the proportion of traditional materials and material consistency of the urban street view image are calculated as material mechanism space features.

[0040] As a preferred technical solution, the following are also included:

[0041] The expert scoring module is used to obtain expert scores based on the ELO scoring algorithm for the urban street view image data.

[0042] The large-scale historical landscape integrity evaluation module constructs and trains an evaluation model based on artificial neural networks based on the various spatial features and small-scale expert rating datasets, and uses the evaluation model to evaluate the integrity of the historical urban landscape.

[0043] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0044] (1) Fully extract the multi-dimensional spatial features of urban street scene images: This invention fully extracts the spatial features of street block scale, natural element spatial features, architectural color spatial features and material texture spatial features, and combines the above multi-scale spatial features to prepare for the subsequent evaluation of the integrity of the historical urban area.

[0045] (2) High efficiency: This invention utilizes large-scale street view image capture technology to significantly reduce data collection time and greatly improve work efficiency. With the help of an intelligent evaluation model based on artificial neural networks, a comprehensive analysis of the target area can be completed in a very short time. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the historical urban landscape integrity evaluation method based on deep learning in this embodiment.

[0047] Figure 2 This is a schematic diagram of the large-scale urban street view image acquisition process in the embodiment;

[0048] Figure 3 This is a schematic diagram of the urban street view image capture process in the embodiment;

[0049] Figure 4 This is a schematic diagram of the "street block scale - sky visibility" measurement results for the urban landscape area in the example.

[0050] Figure 5 This is a schematic diagram illustrating the process of constructing a building color baseline library based on street view data and machine learning in the embodiment.

[0051] Figure 6 This is the base color map of the public activity center-type landscape area in the embodiment;

[0052] Figure 7 This is a schematic diagram of the measurement results of "building color - color harmony" in the urban landscape area in the example.

[0053] Figure 8 This is a schematic diagram illustrating the measurement results of "building color - number of shop sign colors" in the urban landscape area in the example.

[0054] Figure 9 This is a schematic diagram illustrating the process of constructing a building material baseline library based on street view data and machine learning in the embodiment.

[0055] Figure 10 This is a schematic diagram of the measurement results of "material mechanism-material consistency" in the urban landscape area in the example.

[0056] Figure 11 This is a schematic diagram of the measurement results of "material mechanism - proportion of traditional materials" in the urban landscape area in the example;

[0057] Figure 12 This is a schematic diagram of the measurement results of "natural elements - green view rate" in the urban landscape area in the example;

[0058] Figure 13 This is a schematic diagram of the small-scale sample image scoring process in the embodiment;

[0059] Figure 14 This is a schematic diagram of the landscape integrity assessment model based on artificial neural networks in the embodiment.

[0060] Figure 15 This is a schematic diagram illustrating the landscape integrity scoring results at various sampling points in the urban landscape area, as shown in the example.

[0061] Figure 16 This is a schematic diagram of the historical urban landscape integrity evaluation system based on deep learning in the embodiment. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0063] The following are definitions of some of the terms involved in this invention:

[0064] Historic District Features: Historic district features refer to the unique external spatial image of a specific historic district, which is composed of visible material elements (such as building block scale, building color, material texture, natural elements, etc.).

[0065] Historic District Preservation: Historic district preservation is an indicator that evaluates the degree to which the historical features of a city space are preserved by comprehensively considering various aspects such as street scale, building color, material texture, and natural elements.

[0066] Street View Image: Street View Image (SVI) refers to 360-degree visual information of urban streets and their surrounding environment obtained through panoramic photography technology, which can provide a panoramic spatial view of the street that is close to a personal perspective.

[0067] Artificial Neural Networks (ANNs) are computational models that mimic the structure of neurons in the human brain, used to handle complex pattern recognition and prediction problems. They consist of a large number of simple processing units (neurons), and performance is optimized by adjusting connection weights.

[0068] ArcGIS: Geographic Information System platform, is an infrastructure for mapmaking that can be used to collect, organize, manage, analyze, communicate, and publish geographic information.

[0069] Example 1

[0070] To address the problems existing in the aforementioned prior art, this embodiment provides a method for evaluating the integrity of historical urban landscape based on deep learning. (See [link to relevant documentation]). Figure 1 The method includes the following steps:

[0071] S1, acquire large-scale urban street view images.

[0072] Large-scale urban street view image acquisition comprises two parts: data collection and data cleaning. First, sampling points are established on city streets at 40-meter intervals, and their latitude and longitude coordinates are determined. Then, using Python to legally call the street view API of map software via HTTP URL, one image is captured from the front, back, left, and right views of each sampling point. The horizontal and vertical angles of the line of sight and the viewpoint position are input to obtain the street view view, coordinates, angles, and other information for each sample point, achieving batch acquisition of street view images corresponding to each sample point. Finally, images with poor clarity or those taken too early are removed, completing the data cleaning process.

[0073] Before data collection, OSM data and GIS are combined to perform calculations based on the road network topology for each sampling point perspective to ensure that all street view images are parallel to the long axis of the street space, preventing the distortion of street images from affecting subsequent feature extraction.

[0074] Preferably, the map software street view APIs that are legally called by Python, including but not limited to Baidu Street View, Google Street View, etc.

[0075] S2 uses a machine learning model to extract key spatial features.

[0076] By accurately preparing the dataset and fine-tuning and retraining the model, the machine learning algorithm SegNet is used to extract four key spatial features from street view images: street scale, building color, material texture, and natural elements, so as to obtain quantitative measurements of each spatial element in the street view.

[0077] Street block scale is measured by sky visibility; streets with low sky visibility have small street block scales, while streets with high sky visibility have large street block scales.

[0078] Architectural color measurement includes two aspects: color harmony and the number of colors in shop signs. First, the color types of street view data are labeled for areas with typical architectural colors. Then, based on machine learning algorithms, color information is extracted from the images to construct a building color baseline library and an automatic color harmony evaluation model, so as to realize the evaluation of architectural color harmony and the statistics of the number of colors in shop signs in large-scale street view data.

[0079] Material mechanism measurement includes two aspects: material consistency and the proportion of traditional materials. Building materials in representative historical urban environments are specifically calibrated and modeled. Then, based on the model's intelligent recognition results, the consistency of materials used in buildings in street view images, as well as the proportion of traditional materials such as brick, wood, and stone, are evaluated.

[0080] Natural elements are measured using green view ratio. Streets with a high green view ratio have more natural elements, while streets with a low green view ratio have fewer natural elements. A specific plant type labeling method was used for China's existing built environment to ensure accurate differentiation of different tree species and vegetation types during the identification process, thereby accurately calculating the green view ratio in street view images.

[0081] S3, invite experts to evaluate a small sample of images.

[0082] Representative images are selected from the acquired large-scale street view images for small-scale scoring. Experts are invited to compare the sample images pairwise, and the comparison results are converted into image scores using the ELO rating system, thereby obtaining a small-scale quantitative score of the impact of spatial features on preference evaluation.

[0083] In the score conversion process, the ELO scoring algorithm first assigns an initial score of 1000 to each sample photo, and then updates the score based on the results of each comparison; when the results stabilize after multiple iterations, the final score of the sample photo is obtained.

[0084] S4 utilizes artificial neural networks to train a historical urban area integrity assessment model and conducts a large-scale assessment.

[0085] The evaluation model was trained using a combination of subjective scoring and objective indicators. The model was then used to conduct a large-scale assessment of the integrity of the streetscape, and the integrity value of each sampling point was calculated. A 30m buffer zone was set on the left and right sides of the street segment, and the average integrity value of all points within the buffer zone was calculated to obtain the integrity evaluation of the street segment.

[0086] The evaluation model is trained using Artificial Neural Networks (ANNs) from the field of machine learning to better handle multiple interaction relationships. It employs k-fold cross-validation for model selection, repeatedly using randomly generated subsamples for training and validation. Ultimately, the evaluation model's assessment of the integrity of historic city areas theoretically approaches the professional judgment of designers, but it can quickly and efficiently analyze all streets in the area, enabling large-scale and refined integrity evaluation.

[0087] The following embodiment uses Shanghai as an example to further illustrate this method.

[0088] See Figure 2Step S1: Acquisition of large-scale urban street view images.

[0089] The specific steps are as follows: Taking Shanghai as an example, using ArcGIS software, 19,735 sampling points were determined at 40-meter intervals based on the road network data of the central urban area, and the latitude and longitude coordinates of each point were determined. Secondly, combining OSM data and GIS, calculations based on the road network topology were performed on the perspective of each sampling point to ensure that all street view images are parallel to the long axis of the street, preventing the distortion of street images from affecting subsequent feature extraction. Subsequently, this embodiment uses street view images provided by existing databases (such as Baidu Maps) to replace manual photography, quickly acquiring a large number of street view images. Finally, images showing inconsistent seasonality are cleaned to avoid interference from different seasonal street views on the overall spatial quality. Specifically, this embodiment uses Python to legally call the Baidu Street View API via HTTP URL, inputting the horizontal and vertical angles of the line of sight and viewpoint position data to obtain the street view, coordinates, angle, and other information of each sample point. See [link to documentation]. Figure 3 In this embodiment, one image was captured from the front, back, left, and right perspectives of each sampling point, with an image resolution of 480×360 pixels, resulting in a total of approximately 78,940 valid urban street view photos.

[0090] Step S2: Extraction of key spatial features of urban landscape elements based on machine learning. 1000 representative images are selected from the acquired street view images. The SegNet machine learning algorithm is used to extract four key spatial features from the street view images: street scale, building color, material texture, and natural elements. This lays the foundation for training the urban landscape integrity assessment model in the next step. The specific methods for quantifying the above four indicators are as follows:

[0091] (1) Street Scale: Photos along the long axis were selected, and a convolutional neural network (CNN) model was applied to determine sky visibility. This model was implemented using Python and the TensorFlow framework, employing an encoder-decoder architecture. The encoder extracts features from the input image, while the decoder reconstructs the sky region. The model is optimized using mean squared error as the loss function, accurately identifying and quantifying the proportion of sky in the image. See also... Figure 4 Streets with low sky visibility have small street scales, while streets with high sky visibility have large street scales.

[0092] Specifically, the calculation process for street block scale features is as follows:

[0093] The measurement and scoring transfer is performed using sky visibility. Specifically, the proportion of the visible sky area to the total area of ​​a photograph taken along the long axis (parallel to the street) at a given sampling point is used as the result for calculating the "streetscape scale" element of landscape integrity. The result value ranges from [0,1], where 0 indicates no visible sky (completely enclosed space) in the image along the long axis, with a "streetscape scale" score of 0, while 1 indicates the entire sky is visible (open space), with a "streetscape scale" score of 1. The specific formula is as follows:

[0094]

[0095] Among them, S a T represents the number of pixels in the image that are identified as sky. a This indicates the total number of pixels in the image.

[0096] Sky visibility is measured using a convolutional neural network (CNN) model to automatically identify the sky in images. This model is implemented using Python and the TensorFlow framework and employs a novel encoder-decoder architecture. The encoder extracts features from the input image, while the decoder reconstructs the sky region. The model is optimized using mean squared error as the loss function, accurately identifying and quantifying the proportion of sky in the image. The loss function used in the optimization process is expressed as follows:

[0097]

[0098] in, y represents the predicted sky region of the model for the input image xi. i The labels represent the real sky regions, N is the number of training samples, and θ is the model's parameter set.

[0099] (2) Architectural Color: This includes two aspects: color harmony and the number of colors in shop signs. Vertical photos were selected to label the color types of building facades in the images. Color information was extracted from the images using machine learning algorithms to construct a building color baseline database and train an automatic color harmony evaluation model. See [link to relevant documentation] Figure 5 and Figure 6 First, DeepLab was used to extract building facades from street view images. Then, through shadow detection and color correction, clearer, objectively colored photos of buildings, unaffected by lighting conditions, were obtained. Subsequently, colors were extracted from the images to construct a building color baseline library, which was used to train an AI model for automated evaluation of color harmony. See also Figure 7 and Figure 8Based on the extracted color background and combined with expert scoring, an automated evaluation AI model for color harmony was trained to conduct a large-scale evaluation of color harmony and the number of colors in shop signs in the central urban area of ​​Shanghai.

[0100] Specifically, the calculation process for architectural color characteristics is as follows:

[0101] This includes two aspects: the number of colors in shop signs and color harmony. First, DeepLab is used to extract the building facades from street view images. Then, through shadow detection and color correction, clearer, objectively colored photos of buildings, unaffected by lighting, are obtained. Subsequently, colors are extracted from the images to construct a building color baseline library. Specifically, the color extraction uses the RGB color model for color segmentation, dividing the red, green, and blue channels into 10 intervals each. Each interval contains 26 consecutive integer values ​​within the range of 0-255, forming a baseline library containing 1000 color types.

[0102] The process of calculating the number of colors for shop signs is as follows: The shop signs on the building facades in the street view image are automatically identified, and the colors are extracted and compared with the colors in the background library to count the number of colors.

[0103] Color harmony calculation process: Based on the extracted color background library, the color harmony of building facades in the sample is evaluated according to color matching principles (such as contrast, saturation, brightness, etc.) to train an automated evaluation AI model for color harmony, thereby realizing the evaluation of building color harmony in large-scale street view data.

[0104] The "Architectural Color" index is calculated by combining the results of two aspects: the number of colors on shop signs and color harmony. The average of the number of colors on shop signs from 1000 randomly sampled images of the scenic area is used as the baseline value. The score for the number of colors on shop signs is the ratio of the measured data to the baseline. This result is normalized within the range of [0,1], transforming the measurement of shop sign colors into the "Architectural Color" index score. Similarly, color harmony is provided by an AI model and normalized to a score within the range of [0,1]. Finally, a decision tree algorithm is used to calculate the contribution of the two feature scores to the scenic area, obtaining the corresponding weights. The two scores are then weighted and averaged to obtain the result of the "Architectural Color" element for scenic area integrity. The specific formula is as follows:

[0105] BC = C S ×ω1+H S ×ω2

[0106] Among them, C S H represents the normalized score of the number of colors on the store sign. S The values ​​represent the color harmony score, with ω1 and ω2 being the weights of the color number score and color harmony score for the store sign, respectively.

[0107] (3) Material Mechanism: This includes two aspects: material consistency and the proportion of traditional materials. Vertical photographs were selected to calibrate and train models of building materials in representative historical urban environments. For details, see [link to relevant documentation]. Figure 9 First, perspective correction was performed on the street view images. A semantic segmentation model was used to extract building elements, and an adaptive rectangle segmentation method was used to extract material units. Subsequently, the material units were manually labeled to construct a large-sample training set of building materials in the historical urban area, with labeled types including brick, concrete, glass, metal, paint, stone, and wood. A CNN model was trained to automatically classify the material rectangles, thereby obtaining the facade material composition in the street view images. See [link to documentation] Figure 10 and Figure 11 The trained model is used for building material recognition and analysis in large-scale street view images. Based on the model's intelligent recognition results, the consistency of building materials used in street view images and the proportion of traditional materials such as brick, wood, and stone are evaluated.

[0108] Specifically, the calculation process for the material mechanism characteristics is as follows:

[0109] This study focuses on two aspects: the proportion of traditional materials and material consistency. First, perspective correction is applied to street view images. A semantic segmentation model is used to extract architectural elements, and an adaptive rectangle segmentation method is used to extract material units. Subsequently, the material units are manually labeled to construct a large-sample training set of architectural materials in historical districts. The labeled types include common materials such as brick, concrete, glass, metal, paint, stone, and wood. A CNN model is trained to automatically classify the material rectangles, thereby obtaining the facade material composition in the street view images. The trained model is then used for architectural material recognition and analysis in large-scale street view images, calculating the proportion of three traditional materials—brick, wood, and stone—in the images.

[0110] To address the issue that varying building distances in different street view images result in different proportions of building facades in the image, making it impossible to directly segment building facades using the same unit size, an adaptive rectangular segmentation method is employed. First, perspective correction is performed on the street view image. Then, the building facades are extracted from the street view image using a DeepLab semantic segmentation model and binarized (building facades are white, the rest is black). Subsequently, an appropriate segmentation unit size (i.e., the length and width of the segmentation rectangle, such as 100*100, 80*80, or 60*60, etc.) is selected based on the number of pixels in the facade portion. Optionally, pixels in images with smaller unit sizes are enlarged. This ensures that each unit clearly reflects the characteristics of a specific material while adapting to building facade images of different sizes.

[0111] Based on the identified building facade material types and corresponding areas, the distribution density of each material on the building facade is calculated by the proportion of area occupied by each material. This density is then combined with the material distribution uniformity calculated using entropy to obtain the material consistency result. The specific formula is as follows:

[0112]

[0113] Where, ρ i is the distribution density of the i-th material, Mi is the area of ​​the i-th material, Mt is the total area of ​​the building facade material, and H is the entropy value.

[0114] The results of the traditional material proportion and material consistency measurements are normalized to [0,1]. The decision tree algorithm is used to calculate the contribution of the two feature scores to the landscape and obtain the corresponding weights. The two scores are then weighted and averaged to obtain the result of the "material texture" landscape integrity element.

[0115] (4) Natural Elements: Photos along the long axis were selected, and a convolutional neural network (CNN) model was applied to determine the green view rate. First, the existing built environment was specifically labeled with plant types to ensure accurate differentiation of different tree species and vegetation types during the identification process. Then, a deep learning method based on the U-Net architecture was used, with ResNet-50 for model training, to achieve accurate segmentation of green vegetation in the images and precisely calculate the green view rate in the street view images. See [link to relevant documentation] Figure 12 Streets with high green view ratios have more natural elements, while streets with low green view ratios have fewer natural elements.

[0116] Specifically, the calculation process for natural elements includes the following steps:

[0117] The green view rate is used for measurement and scoring transfer. This involves calculating the proportion of visible vegetation area to the total area of ​​a photograph taken along the minor axis (perpendicular to the street) at a sampling point. This proportion is used as the result for calculating the "natural element" element of landscape integrity. The result value ranges from [0,1], where 0 indicates no vegetation in the image (natural element score of 0), and 1 indicates the image is completely filled with vegetation (natural element score of 1). The specific formula is as follows:

[0118]

[0119] Among them, P a T represents the visible area of ​​the plant, i.e., the number of pixels marked as the plant region after being identified by the CNN model; a This indicates the total number of pixels in the image.

[0120] The green view rate is measured using a convolutional neural network (CNN) model to automatically identify plants in images. First, common plant types in existing built environments in China are calibrated to ensure accurate differentiation of different tree species and vegetation types during the identification process. Then, a deep learning method based on the U-Net architecture is used, trained with ResNet-50 to accurately segment green vegetation in the images, thereby precisely calculating the plant area in street view images and measuring the green view rate.

[0121] See Figure 13 In step S3, the 1000 representative point images used for feature extraction in step S2 are simultaneously scored by experts. The scoring program is written in JAVA. First, each sample photo is assigned an initial score of 1000. Then, experts are invited to compare the sample images pairwise. The photo scores are updated based on the comparison results. After multiple iterations, the scores of the sample photos are stabilized and the final scores of the sample photos are obtained. The final scores of the photos are used as the expert evaluation results of the integrity of the historical district, laying the foundation for the next step of training a large-scale historical district integrity evaluation model that is close to professional judgment.

[0122] See Figure 14 and Figure 15 Step S4 involves training an evaluation model using both subjective ratings and objective indicator measurements to conduct a large-scale assessment of the integrity of the historical urban area's appearance. Specifically, considering the complex interaction between the integrity of the historical urban area's appearance and its spatial characteristics, the model is trained using an Artificial Neural Network (ANN) from the machine learning field. K-fold cross-validation is employed for model selection, and randomly generated subsamples are repeatedly used for training and validation. The resulting evaluation model achieves a percent incorrect prediction rate of 0.21, theoretically approaching the professional judgment of designers in assessing the integrity of the urban appearance. Simultaneously, it allows for rapid and efficient analysis of all streets within the area. Subsequently, a large-scale and detailed integrity assessment is conducted in the historical urban area of ​​Shanghai's central district, calculating the integrity value for each sampling point. Finally, a 30m buffer zone is set on the left and right sides of the street segments, and the average integrity value of all points within the buffer zone is calculated to obtain the integrity evaluation of the street segments. The results are then visualized using ArcGIS.

[0123] In summary, this method has the following characteristics:

[0124] (1) Efficient Acquisition of Massive Data. This method utilizes large-scale street view image capture technology, significantly reducing data collection time to 10% of the original time, greatly improving work efficiency. With the help of an intelligent evaluation model based on artificial neural networks, a comprehensive and accurate analysis of the target area can be completed in a very short time. Compared with the judgment results of professional designers, the accuracy rate reaches over 90%, realizing large-scale and accurate calculation of the integrity of the historical urban landscape. At the same time, this method, which combines street view data with machine learning technology, has the ability to monitor dynamically in real time. This real-time capability not only helps urban managers respond quickly to changes, but also ensures the continuity and effectiveness of urban landscape protection work.

[0125] (2) Construction of the evaluation system. This method uses deep learning technology to systematically extract and quantify the key spatial features of historical urban areas, and automatically assigns corresponding weights to these features through algorithms. This process ensures the objective assessment and accurate measurement of the impact of each factor on the integrity of the historical urban area's appearance. This method has good scalability and can be quickly promoted and used. By using machine learning technology to build a model that learns the subjective experience and professional judgment of experts, even in small and medium-sized cities with relatively weak professional technology, a high-precision and professional assessment of the integrity of the historical urban area's appearance can be achieved, ensuring the consistency and accuracy of the evaluation.

[0126] (3) Optimization of element measurement. This method combines the evaluation results of experts on small-scale samples to construct an evaluation model of the integrity of historical urban landscape based on artificial neural networks. Through the dual drive of knowledge and data, the evaluation model is optimized in the field of landscape integrity assessment. This enables the model to not only quickly and efficiently traverse the analysis area, but also to have evaluation and diagnostic capabilities similar to experts, thereby enabling refined evaluation of the integrity of historical urban landscape on a large scale.

[0127] (4) Significant social value. This method, based on street view data acquisition and machine learning technology, is an assessment method for the integrity of historical districts and is of great significance for the protection and inheritance of the historical district's character. This method can accurately identify the architectural styles and cultural characteristics within specific historical districts and assess their integrity. It provides timely feedback to urban planners on the degree of preservation of these important cultural elements during urban renewal, which not only helps maintain the unique character of historical districts but also promotes the preservation and inheritance of architectural and urban landscape cultural characteristics in various regions.

[0128] Example 2

[0129] See Figure 16 This embodiment, based on Embodiment 1, provides a deep learning-based system for evaluating the integrity of historical urban landscape, including:

[0130] The data acquisition module is used to acquire urban street view image data from multiple locations;

[0131] The feature extraction module is used to extract the sky ratio and green view rate of the urban street scene image data using a pre-trained convolutional neural network model, which are respectively used as the spatial features of the street block scale and the spatial features of natural elements. By extracting color information from the urban street scene image, a color background library is constructed, and the number of colors of shop signs and color harmony of the urban street scene image are calculated as the spatial features of building color. By constructing a sample set of building materials in historical urban areas, the proportion of traditional materials and material consistency of the urban street scene image are calculated as the spatial features of material mechanism.

[0132] The expert scoring module is used to obtain expert scores based on the ELO scoring algorithm for the urban street view image data.

[0133] A large-scale historical preservation integrity evaluation module is constructed and trained based on various spatial features and small-scale expert rating datasets to evaluate the integrity of the historical urban area.

[0134] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for evaluating the integrity of historical urban landscape based on deep learning, characterized in that, Includes the following steps: Acquire urban street view image data from multiple locations; For the urban street view image data, the sky ratio and green view rate of the urban street view images are extracted using a pre-trained model and used as the spatial features of street block scale and natural elements, respectively. A color background library is constructed by extracting color information from the urban street view images, and the number of colors of shop signs and color harmony in the urban street view images are calculated as the spatial features of building color. A sample set of building materials in historical urban areas is constructed, and the proportion of traditional materials and material consistency in the urban street view images are calculated as the spatial features of material mechanism. For the aforementioned city street view image data, obtain expert scores based on the ELO scoring algorithm; Based on the aforementioned spatial features and small-scale expert rating datasets, an evaluation model based on artificial neural networks is constructed and trained. This evaluation model is then used to assess the integrity of the historical urban landscape. The process of obtaining the architectural color space characteristics includes the following steps: Based on vertical urban street view image data, the color type of building facades in the images is calibrated. Through shadow detection and color correction, urban street view image data with weakened lighting effects is obtained. By extracting image colors, a color baseline library is constructed to obtain the color information of shop signs. Based on the aforementioned color background library and expert scores for image color harmony, a color harmony evaluation model is trained. Using the aforementioned color harmony evaluation model, the predicted color harmony is obtained. Combined with the information on the number of colors in the shop sign, the architectural color space characteristics are derived. The process of obtaining the spatial characteristics of the material mechanism includes the following steps: Based on vertical urban street view image data, architectural element information is extracted through perspective correction and semantic segmentation, material units are extracted through adaptive rectangular segmentation, and a set of architectural material samples is obtained through calibration. Based on the aforementioned building material sample set, a classification model based on a convolutional neural network is trained to obtain the composition information of facade materials in urban street scene images, and to obtain the proportion of traditional materials and material consistency scores, which serve as the spatial features of the material mechanism. The RGB color model is used for color division, dividing the red, green, and blue channels into 10 intervals each. Each interval contains 26 consecutive integer values ​​in the range of 0-255, forming a base library containing 1000 color types. The average of the number of shop sign colors in multiple randomly sampled images of the scenic area is used as the baseline value. The shop sign color score is the ratio of the measured data to the baseline. Normalization is performed to transform the shop sign color score measurement into a building color index score. Color harmony is provided by the model and normalized into a score. A decision tree algorithm is used to calculate the contribution of the shop sign color score and color harmony feature scores to the scenic area, obtaining corresponding weights. The two scores are then weighted and averaged to obtain the result of the calculation of the architectural color scenic integrity element. Based on the identified building facade material types and corresponding areas, the distribution density of each material on the building facade is calculated by the proportion of area occupied by each material. Combined with the material distribution uniformity calculated by entropy, the material consistency result is obtained.

2. The method for evaluating the integrity of historical urban landscape based on deep learning according to claim 1, characterized in that, The process of acquiring urban street view image data from multiple locations includes: For the target city streets, sampling points are set at preset intervals to obtain the latitude and longitude coordinates of the sampling points; Based on the latitude and longitude coordinates of the sampling points, urban street scene images of the sampling points are obtained by calculating the road network topology, which are parallel to the long axis of the street space and perpendicular to the road network topology. The city street scene images, representing different seasons, are cleaned to obtain the city street scene image data.

3. The method for evaluating the integrity of historical urban landscape based on deep learning according to claim 1, characterized in that, The process of obtaining the street block-scale spatial features includes the following steps: Based on urban street view image data along the long axis, the sky octagon is predicted using a model based on a convolutional neural network and an encoder-decoder architecture to obtain the spatial features at the street block scale. The encoder is used to extract features from the input urban street view image, and the decoder is used to reconstruct the sky region.

4. The method for evaluating the integrity of historical urban landscape based on deep learning according to claim 1, characterized in that, The process of acquiring the spatial characteristics of the natural elements includes the following steps: Based on urban street view image data along the long axis, a model based on convolutional neural networks and U-Net architecture is used to segment green vegetation and predict the green view rate of the street view image, which is then used as the spatial feature of the natural elements.

5. The method for evaluating the integrity of historical urban landscape based on deep learning according to claim 1, characterized in that, The process of evaluating the integrity of the historical district using the aforementioned evaluation model includes the following steps: The evaluation model is used to calculate the landscape integrity of each sampling point. A buffer zone is set for the street line segment, and the mean of the landscape integrity of the sampling points within the buffer zone is calculated as the landscape integrity of the street line segment. A visual signal is output.

6. A deep learning-based system for evaluating the integrity of historical urban landscape, characterized in that, The method for evaluating the integrity of historical urban landscape as described in any one of claims 1-5 includes: The data acquisition module is used to acquire urban street view image data from multiple locations; The feature extraction module is used to extract the sky ratio and green view rate of the urban street view image data using a pre-trained convolutional neural network model, which are respectively used as the spatial features of street block scale and natural element space features. By extracting color information from the urban street view image, a color background library is constructed, and the number of colors of shop signs and color harmony of the urban street view image are calculated as architectural color space features. By constructing a sample set of building materials in historical urban areas, the proportion of traditional materials and material consistency of the urban street view image are calculated as material mechanism space features.

7. The historical urban area integrity evaluation system based on deep learning according to claim 6, characterized in that, Also includes: The expert scoring module is used to obtain expert scores based on the ELO scoring algorithm for the urban street view image data. The large-scale historical landscape integrity evaluation module constructs and trains an evaluation model based on artificial neural networks based on the various spatial features and small-scale expert rating datasets, and uses the evaluation model to evaluate the integrity of the historical urban landscape.

Images