Urban street annual green vision rate calculation method based on multi-temporal streetscape images

Abstract

The invention discloses an urban street annual green vision rate calculation method based on multi-temporal streetscape images, and the method comprises the steps: generating streetscape sampling points according to road network data, collecting a plurality of temporal streetscape images of each streetscape sampling point covering all seasons of the whole year, carrying out the semantic segmentation of the streetscape images through a deep learning model, and obtaining the annual green vision rate of a city street. And extracting a vegetation region in the streetscape image, and performing mathematical expectation calculation on the green vision rates under different time phases to obtain the annual green vision rate of the urban street. According to the method, annual street view data are accurately collected and processed, and an effective means is provided for scientific evaluation of urban street greening by combining a deep learning technology and an image segmentation technology. According to the method, the urban planning department can be helped to know the greening condition more accurately, important decision support can be provided for subsequent urban greening improvement and street landscape optimization, and meanwhile, the method is high in automation degree and suitable for practical application of large-scale urban greening evaluation.

Description

Technical Field

[0001] The present invention relates to the field of geographic information technology, and in particular to a method for calculating the annual green view rate of urban streets based on multi-temporal street view images in combination with deep learning and image processing technologies. Background Art

[0002] Urban streets are important spaces for residents' daily public activities, and street greening is a core element of residents' landscape perception, which directly affects the walking index, environmental assessment, residents' health and economic benefits. Urban greening not only improves life satisfaction in high-density cities, but its structure also determines the quality and perception level of street space. Changes in greening structure reflect the trajectory of urban development. With the development of artificial intelligence vision technology, it has become possible to accurately identify street greening structures. Research on the spatiotemporal evolution of street greening helps optimize urban green space and promote high-quality and sustainable development of street space.

[0003] In the invention patent application with the publication number CN117036964A, a method for determining the greening level of urban roads is disclosed, which comprises: obtaining the road network data and surface coverage data of the target city, setting multiple street view sampling points in the target city according to the road network data, collecting multiple street view images at each street view sampling point, using the object classification model to perform semantic segmentation on each street view image, obtaining the classification results of the pixels corresponding to each object in each street view image, and determining the road greening level of the target city based on the green view rate and multi-weighted green space index of all street view sampling points, thereby improving the accuracy of determining the greening level of urban roads. However, it still has the following defects.

[0004] 1. Ignoring temporal changes: This method calculates the green view rate based on street view images at a fixed time, without considering the dynamic impact of factors such as seasons and time periods on the greening level, which may lead to overestimation or underestimation of the greening level in different seasons;

[0005] 2. Insufficient response to the spatiotemporal changes of greening: Since only image data of a single time period is used, this method is difficult to dynamically and continuously reflect the changing trend of urban street greening, and cannot effectively analyze seasonal changes within a year or changes in greening over many years;

[0006] 3. Lack of integration of time series data: This method only uses static image data and does not consider combining multi-temporal green view rate analysis to reflect the long-term trend of vegetation coverage. Summary of the invention

[0007] The purpose of the present invention is to provide a method for calculating the annual green view rate of urban streets based on multi-temporal street view images, which is used to accurately evaluate the greening conditions of urban streets and can take into account the impact of seasonal changes.

[0008] In order to achieve the above-mentioned invention object, the present invention adopts the following technical scheme:

[0009] A method for calculating the annual green view rate of urban streets based on multi-temporal street view images comprises the following steps:

[0010] S1, collect road network data, extract road center lines and generate street view sampling points according to preset distances;

[0011] S2, collecting street view images of each street view sampling point covering multiple phases of each season throughout the year;

[0012] S3, use the deep learning model to perform semantic segmentation on street view images and extract vegetation areas in street view images;

[0013] S4, calculating the green view rate of each street scene sampling point at each time phase;

[0014] S5, calculating the green view rate of each street view sampling point in each season;

[0015] S6. Calculate the annual green view rate of each street view sampling point;

[0016] S7. Calculate the annual green view rate of urban streets.

[0017] As a further improved technical solution of the present invention, in step S4, the green viewing rate of each street scene sampling point in each time phase is calculated by the following formula:

[0018]

[0019] Among them, GVI is the green view rate of a street view sampling point at a certain time phase, n is the number of street view images collected by the street view sampling point at this time phase, and Area Green Area is the number of pixels in the street view image that are identified as vegetation areas. All is the total number of pixels in the Street View image.

[0020] As a further improved technical solution of the present invention, in step S5, the green viewing rate of each street view sampling point in each season is calculated by the following formula:

[0021]

[0022] Among them, GVI season is the green view rate of a street view sampling point in a certain season, GVI1+GVI2+...GVI ns is the sum of the green view rates of the street view sampling point in each phase in this season, and ns is the number of phases of the street view sampling point in this season.

[0023] As a further improved technical solution of the present invention, in step S6, the annual green view rate of each street view sampling point is calculated by the following formula:

[0024] YGVI point =a×GVI spring +b×GVI summer +c×GVI autumn +d×GVI winter

[0025] Among them, YGVI point is the annual green view rate of a street view sampling point, GVI spring is the green view rate of the street view sampling point in spring, GVI summer is the green view rate of the street view sampling point in summer, GVI autumn is the green view rate of the street view sampling point in autumn, GVI winter is the green view rate of the street view sampling point in winter, and a, b, c, and d are the weights of the green view rate in each season.

[0026] As a further improved technical solution of the present invention, in step S7, the annual green view rate of the urban streets is calculated by the following formula:

[0027]

[0028] Among them, YGVI street is the annual green view rate of a street, YGVI point_I is the annual green view rate of a street view sampling point on the street, and N is the number of street view sampling points on the street.

[0029] As a further improved technical solution of the present invention, after collecting the road network data in step S1, it also includes road screening and simplification processing, and then extracting the road center line and generating street view sampling points according to the preset distance.

[0030] As a further improved technical solution of the present invention, step S2 includes stitching the street view images into a panoramic image and then collecting it.

[0031] As a further improved technical solution of the present invention, the deep learning model in step S3 is a deep learning semantic segmentation model Mask2Former pre-trained by the Cityscapes dataset.

[0032] Compared with the prior art, the technical effects of the present invention are:

[0033] Introducing multi-temporal data: This paper collects street view images of target city streets at different times to construct a multi-temporal data set, which not only considers the green view rate at a single moment, but also considers the seasonal changes in a year, and proposes the annual green view rate (YGVI). This method makes up for the limitations of the existing technology of calculating the green view rate at a fixed time phase, and provides a more dynamic and comprehensive greening assessment result.

[0034] More accurate greening level assessment: By calculating the green view index (GVI) at multiple time points and taking its mathematical expectation value to calculate the annual GVI, the annual GVI can more accurately reflect the street greening level, especially in areas with significant seasonal changes, and can effectively avoid misjudgment that may be caused by the single-phase method.

[0035] Spatiotemporal dynamic monitoring capability: The present invention can dynamically monitor the greening conditions of urban streets and reflect the spatiotemporal changing trends of greening. It can not only provide data support for the assessment of the current status of urban greening, but also provide more continuous and accurate information for long-term urban planning and greening management.

[0036] Automated processing of deep learning models: The present invention uses deep learning technology to realize automated processing of street view images and quantify green view rates on a large scale, which greatly reduces the burden of manual processing and improves monitoring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flowchart of a method for calculating the annual green view rate of urban streets based on multi-temporal street view images in a specific implementation mode of the present invention;

[0038] Figure 2 It is a detailed schematic diagram of Example 1. DETAILED DESCRIPTION

[0039] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any method or functional changes made by a person skilled in the art based on these embodiments are all within the protection scope of the present invention.

[0040] See also Figure 1 , a method for calculating the annual green view rate of urban streets based on multi-temporal street view images, comprising the following steps:

[0041] S1, collect road network data, extract road center lines and generate street view sampling points according to preset distances;

[0042] S2, collecting street view images of each street view sampling point covering multiple phases of each season throughout the year;

[0043] S3, use the deep learning model to perform semantic segmentation on street view images and extract vegetation areas in street view images;

[0044] S4, calculating the green view rate of each street scene sampling point at each time phase;

[0045] S5, calculating the green view rate of each street view sampling point in each season;

[0046] S6. Calculate the annual green view rate of each street view sampling point;

[0047] S7. Calculate the annual green view rate of urban streets.

[0048] It should be noted that "phase" in the present invention is defined as a time point.

[0049] Furthermore, in step S4, the green view rate of each street scene sampling point at each time phase is calculated by the following formula:

[0050]

[0051] Among them, GVI is the green view rate of a street view sampling point at a certain time phase, n is the number of street view images collected by the street view sampling point at this time phase, and Area Green Area is the number of pixels in the street view image that are identified as vegetation areas. All is the total number of pixels in the Street View image.

[0052] By calculating the percentage of the sum of the number of pixels identified as vegetation areas in several street view images collected at a certain street view sampling point at a certain time phase and the sum of the total number of pixels in several street view images, the green view rate GVI of a certain street view sampling point at a certain time phase can be obtained. The green view rate of each street view sampling point at each time phase can be calculated by cyclically executing the above calculation formula.

[0053] For example, there are two street view images collected by street view sampling point 1 in phase 1. The number of pixels identified as vegetation areas in one street view image is 10 million pixels, and the total number of pixels is 30 million pixels. The number of pixels identified as vegetation areas in the other street view image is 20 million pixels, and the total number of pixels is 30 million pixels. Then, the green view rate GVI1 of street view sampling point 1 in phase 1 = (1+2) / (3+3) = 50%.

[0054] Furthermore, in step S5, the green viewing rate of each street view sampling point in each season is calculated by the following formula:

[0055]

[0056] Among them, GVI season is the green view rate of a street view sampling point in a certain season, GVI1+GVI2+...GVI ns is the sum of the green view rates of the street view sampling point in each phase in this season, and ns is the number of phases of the street view sampling point in this season.

[0057] For example, the street view sampling point 1 has 6 sampling phases in spring, and the corresponding green view rates GVI1 = 50%, GVI2 = 45%, GVI3 = 45%, GVI4 = 50%, GVI5 = 46%, GVI6 = 46%.spring =(50%+45%+45%+50%+46%+46%) / 6=47%.

[0058] Furthermore, in step S6, the annual green view rate of each street view sampling point is calculated by the following formula:

[0059] YGVI point =a×GVI spring +b×GVI summer +c×GVI autumn +d×GVI winter

[0060] Among them, YGVI point is the annual green view rate of a street view sampling point, GVI spring is the green view rate of the street view sampling point in spring, GVI summer is the green view rate of the street view sampling point in summer, GVI autumn is the green view rate of the street view sampling point in autumn, GVI winter is the green view rate of the street view sampling point in winter, and a, b, c, and d are the weights of the green view rate in each season.

[0061] In the present invention, the seasonal division standard is determined by the historical seasonal change records of the city. Taking Nanjing as an example, April to May can be divided into spring, June to September can be divided into summer, October to November can be divided into autumn, and December to March can be divided into winter. season The calculation formula for GVI is spring , GVI summer , GVI autumn , GVI winter The specific value of .

[0062] The specific values ​​of a, b, c, and d are obtained by taking part in YGVI during the duration of each season. point The percentage of the total seasonal duration calculated is determined.

[0063] For example, the spring season in Nanjing lasts for two months, so YGVI is a popular activity in all seasons. point Calculate the case and participate in YGVI point The total season length is calculated to be 12 months, so a = 2 / 12 = 0.17. Similarly, b = 4 / 12 = 0.33, c = 2 / 12 = 0.17, and d = 4 / 12 = 0.33.

[0064] For example, in order to reduce the amount of calculation, when performing YGVI point When calculating, we can only consider the seasons with a longer time span. pointOnly summer and winter are considered in the calculation. Summer lasts for 4 months. point The total season length is calculated to be 8 months, then b = 4 / 8 = 0.5, and similarly d = 4 / 8 = 0.5. a and c are equal to 0, GVI springr , GVI autumn Not included in calculation.

[0065] Furthermore, in step S7, the annual green view rate of the urban streets is calculated by the following formula;

[0066]

[0067] Among them, YGVI street is the annual green view rate of a street, YGVI point_I is the annual green view rate of a street view sampling point on the street, and N is the number of street view sampling points on the street.

[0068] For example, there are three street view sampling points on a street, and the annual green view rate YGVI of the street view sampling points is point_1 =30%, YGVI point_2 =40%, YGVI point_3 = 20%, then the annual green vision rate of the street is YGVI street =(30%+40%+20%) / 3=30%.

[0069] Furthermore, after collecting the road network data in step S1, it also includes road screening and simplification processing, then extracting the road centerline and generating street view sampling points according to a preset distance.

[0070] Furthermore, step S2 includes stitching the street view images into a panoramic image and then collecting it.

[0071] Furthermore, the deep learning model in step S3 is a deep learning semantic segmentation model Mask2Former pre-trained by the Cityscapes dataset.

[0072] The present invention provides an effective means for scientific evaluation of urban street greening by accurately collecting and processing street view data throughout the year, combining deep learning technology and image segmentation technology. This method can not only help urban planning departments understand the greening status more accurately, but also provide important decision-making support for subsequent urban greening improvements and street landscape optimization. At the same time, the method provided by the present invention has a high degree of automation and is suitable for practical application in large-scale urban greening evaluation.

[0073] Example 1

[0074] See also Figure 2In this embodiment, the calculation of the annual green view rate of a street in Nanjing is taken as an example to describe the detailed process of the implementation of the present invention.

[0075] In this embodiment, the street view image comes from Baidu static image API, and the road network data comes from the OSM official website. The specific implementation steps are as follows:

[0076] 1. Data preparation and preprocessing

[0077] Download OSM road network data from the Open Street Map official website and filter data sources by attributes; select primary (first-level road), secondary (secondary road), tertiary (third-level road), residential (residential road), living_street (living street), and trunk (trunk) as the main data sources;

[0078] Buffer and rasterize the filtered data sources and convert them into vectorized simplified road networks, merge multi-lane roads into single lanes, and ensure that all roads are single-lane and interrupted at intersections;

[0079] In ArcGIS (ArcGIS is an existing geographic information system software), the Generate Points Along Line tool is used to generate street view sampling points on the road every 20 meters.

[0080] 2. Capture multi-temporal street view images

[0081] Use Python script to crawl the longitude and latitude of all street view sampling points, and call Baidu coordinate conversion API to convert the wgs84ll coordinate system (GPS coordinates) to the bd09ll coordinate system (Baidu coordinates);

[0082] Use the Baidu Map "Time Machine" function, define the URL parameters sent through the standard HTTP request through the Baidu Map API, input the converted coordinate system, and obtain the panoid (panoramic image ID) of the nearest time point on the street view sampling point. The panoid can uniquely determine the time axis of the street view image at the street view sampling point, and obtain the panoid and related information of all time points where the street view image exists based on the time axis;

[0083] The crawler technology is used to access Baidu static image API. Panoid is used to set the required crawling time and the set collection direction as parameters. The street view images collected from the four directions are rotated clockwise by 0°, 90°, 180° and 270° (i.e., images are collected from the north, east, south and west directions). The street view images collected from the four directions are stitched together to obtain a panoramic image with a human-centered perspective. When crawling, for each street view sampling point, a street view image is obtained according to its longitude, latitude and road azimuth point. The main street view image collection parameter settings are shown in Table 1.

[0084] Table 1

[0085]

[0086] 3. Vegetation area extraction based on deep learning

[0087] We use the deep learning semantic segmentation model Mask2Former pre-trained by the Cityscapes dataset (Cityscapes is a large-scale dataset for semantic understanding and object detection, specifically for urban scene analysis). The model performs well in urban scenes (such as the Cityscapes dataset) and is suitable for semantic segmentation of street scene images. We use the PyTorch environment to load the Mask2Former model, segment the collected street scene images, and identify various objects such as trees, sky, roads, and buildings.

[0088] The vegetation component obtained by semantic segmentation is used for subsequent calculations.

[0089] 4. Calculate the green view rate of each street view sampling point at each time phase

[0090] Substitute the vegetation component in the street view image obtained by semantic segmentation into the following formula for calculation:

[0091]

[0092] Among them, GVI is the green view rate of a street view sampling point at a certain time phase, n is the number of street view images collected by the street view sampling point at this time phase, and Area Green Area is the number of pixels in the street view image that are identified as vegetation areas. All is the total number of pixels in the Street View image.

[0093] By calculating the percentage of the sum of the number of pixels identified as vegetation areas in several street view images collected at a certain street view sampling point at a certain time phase and the sum of the total number of pixels in several street view images, the green view rate GVI of a certain street view sampling point at a certain time phase can be obtained. The green view rate of each street view sampling point at each time phase can be calculated by cyclically executing the above calculation formula.

[0094] 5. Calculate the green view rate of each street view sampling point in each season

[0095] Substitute the green view rate of the street view sampling point at each time phase into the following formula for calculation:

[0096]

[0097] Among them, GVI seasonis the green view rate of a street view sampling point in a certain season, GVI1+GVI2+...GVI ns is the sum of the green view rates of the street view sampling point in each phase in this season, and ns is the number of phases of the street view sampling point in this season.

[0098] 6. Calculate the annual green view rate of each street view sampling point

[0099] Substitute the green view rate of the street view sampling points in each season into the following formula for calculation:

[0100] YGVI point =a×GVI spring +b×GVI summer +c×GVI autumn +d×GVI winter

[0101] Among them, YGVI point is the annual green view rate of a street view sampling point, GVI spring is the green view rate of the street view sampling point in spring, GVI summer is the green view rate of the street view sampling point in summer, GVI autumn is the green view rate of the street view sampling point in autumn, GVI winter is the green view rate of the street view sampling point in winter, and a, b, c, and d are the weights of the green view rate in each season.

[0102] In the present invention, the seasonal division standard is determined by the historical seasonal change records of the city. For the seasonal division of Nanjing, April to May can be divided into spring, June to September into summer, October to November into autumn, and December to March into winter. season The calculation formula for GVI is spring , GVI summer , GVI autumn , GVI winter The specific value of .

[0103] The specific values ​​of a, b, c, and d are obtained by taking part in YGVI during the duration of each season. point The percentage of the total seasonal duration calculated is determined.

[0104] In this embodiment, in order to reduce the amount of calculation, when performing YGVI point Only seasons with longer time spans are considered in the calculation. point Only summer and winter are considered in the calculation. Summer lasts for 4 months. point The total season length is calculated to be 8 months, then b = 4 / 8 = 0.5, and similarly d = 4 / 8 = 0.5. a and c are equal to 0, GVI springr, GVI autumn Not included in calculation.

[0105] 7. Calculate the annual green view rate of urban streets

[0106] The annual green view rate YGVI of the street view sampling points point Expanded to the street scale, the spatial connection method is used to calculate the mean annual green view rate of all street view sampling points on the street to obtain the annual green view rate YGVI of a specific street. street .

[0107]

[0108] Among them, YGVI street is the annual green view rate of a street, YGVI point_I is the annual green view rate of a street view sampling point on the street, and N is the number of street view sampling points on the street.

[0109] The present invention can accurately reflect seasonal changes: by calculating the green view rate in different seasons throughout the year and dynamically monitoring the street greening conditions, it effectively avoids the evaluation bias caused by ignoring seasonal differences and provides a more realistic greening condition.

[0110] The present invention adopts an efficient evaluation based on the pedestrian perspective: using street view images as the data source, combined with deep learning technology to achieve automatic semantic segmentation, reduce manual intervention, improve the efficiency and accuracy of data processing, and better reflect the actual visual experience of urban residents.

[0111] The present invention has comprehensive spatial analysis and application potential: by spatially and attributely connecting the annual green view rate with the street, it provides a comprehensive view of the city's greening status. It has wide applicability and scalability, and can support decision-making in urban planning and management.

[0112] Finally, it should be noted that the above implementation modes are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned implementation modes, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned implementation modes, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various implementation modes of the present invention.

Claims

1. A method for calculating the annual green view rate of urban streets based on multi-temporal street view images, characterized in that: The following steps are involved: S1, collect road network data, extract road center lines and generate street view sampling points according to preset distances; S2, collecting street view images of each street view sampling point covering multiple phases of each season throughout the year; S3, use the deep learning model to perform semantic segmentation on street view images and extract vegetation areas in street view images; S4, calculating the green view rate of each street scene sampling point at each time phase; S5, calculating the green view rate of each street view sampling point in each season; S6. Calculate the annual green view rate of each street view sampling point; S7. Calculate the annual green view rate of urban streets.

2. The method for calculating the annual green view rate of urban streets based on multi-temporal street view images according to claim 1, characterized in that: In step S4, the green viewing rate of each street scene sampling point at each time phase is calculated by the following formula; Among them, GVI is the green view rate of a street view sampling point at a certain time phase, n is the number of street view images collected by the street view sampling point at this time phase, and Area Green Area is the number of pixels in the street view image that are identified as vegetation areas. All is the total number of pixels in the Street View image.

3. The method for calculating the annual green view rate of urban streets based on multi-temporal street view images according to claim 2, characterized in that: In step S5, the green viewing rate of each street view sampling point in each season is calculated by the following formula; Among them, GVI season is the green view rate of a street view sampling point in a certain season, GVI1+GVI2+...GVI ns is the sum of the green view rates of the street view sampling point in each phase in this season, and ns is the number of phases of the street view sampling point in this season.

4. The method for calculating annual green view rate of urban streets based on multi-temporal street view images according to claim 3, characterized in that: In step S6, the annual green view rate of each street view sampling point is calculated by the following formula; YGVI point =a×GVI spring +b×GVI summer +c×GVI autumn +d×GVI winter Among them, YGVI point is the annual green view rate of a street view sampling point, GVI spring is the green view rate of the street view sampling point in spring, GVI summer is the green view rate of the street view sampling point in summer, GVI autumn is the green view rate of the street view sampling point in autumn, GVI winter is the green view rate of the street view sampling point in winter, and a, b, c, and d are the weights of the green view rate in each season.

5. The method for calculating the annual green view rate of urban streets based on multi-temporal street view images according to claim 4, characterized in that: In step S7, the annual green view rate of the city streets is calculated by the following formula; Among them, YGVI street is the annual green view rate of a street, YGVI point_I is the annual green view rate of a street view sampling point on the street, and N is the number of street view sampling points on the street.

6. The method for calculating annual green view rate of urban streets based on multi-temporal street view images according to claim 1, characterized in that: After collecting the road network data in step S1, the process also includes road screening and simplification, then extracting the road centerline and generating street view sampling points at a preset distance.

7. The method for calculating annual green view rate of urban streets based on multi-temporal street view images according to claim 1, characterized in that: Step S2 includes stitching the street view images into a panoramic image and then collecting it.

8. The method for calculating annual green view rate of urban streets based on multi-temporal street view images according to claim 1, characterized in that: The deep learning model in step S3 is the deep learning semantic segmentation model Mask2Former pre-trained by the Cityscapes dataset.

Images