Street interface digital diagnosis and optimization method and system based on multi-agent sight line simulation

Abstract

The invention discloses a street interface digital diagnosis and optimization method and system based on multi-agent sight simulation, and the system comprises a data collection module which is used for obtaining the building image data of a target city street in a current state; the sand table construction module is used for combining the geographic data and the building image data of the target city street and constructing a digital sand table of a three-dimensional model of the target city street in a current state; the pedestrian trajectory acquisition module is used for acquiring original dynamic trajectories of pedestrians in the target city streets in the current state; the system comprises an intelligent agent module, an identification and measurement module, a diagnosis and optimization module and a display and interaction module. According to the method, the pedestrian movement track data of the target street are collected, and the simulation environment is created in the three-dimensional live-action digital sand table, so that the simulation accuracy and efficiency are improved, the digital diagnosis result is ensured to meet the actual situation of a real street and the needs of real work, and the scientificity and standardization of street interface digital diagnosis and optimization are improved.

Description

Technical Field

[0001] The present invention relates to a method and system for digital diagnosis and optimization, and in particular to a method and system for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation. Background Art

[0002] The street interface is a key optimization and control unit in the field of urban planning, which directly affects the smooth flow of people in the city, the beauty of the landscape and the quality of life of residents. At present, the research and practice on the street interface mainly focus on two aspects: one is the constituent elements of the street interface. The cognition, analysis and induction of the elements of the street interface are the basis for urban planning and design work; the other is the visual perception and use quality of the street interface by the public, which is used to guide the diagnosis and optimization of the street interface.

[0003] Traditional street interface design is usually based on experience and specifications, and is easily affected by planners' subjective experience and standard differences. However, the walking and visual perception of local residents on the streets are important aspects that need to be considered in street interface diagnosis. In other words, traditional street interface design lacks recognition and diagnostic response to the differences in the users of different target streets, and lacks scientificity and systematicity. Summary of the invention

[0004] Purpose of the invention: The first purpose of the present invention is to provide a method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation that takes into account the walking and visual perception of local residents on the streets and is not subject to the subjective influence of planners.

[0005] The second object of the present invention is to provide a system for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation.

[0006] Technical solution: The present invention discloses a method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation, comprising the following steps:

[0007] S1: Collect the building image data of the target city street in the current state;

[0008] S2: Combining the geographic data and building image data of the target city streets, a digital sand table of the three-dimensional model of the target city streets in the current state is constructed;

[0009] S3: Collect the original dynamic trajectories of pedestrians in the target city streets in the current state, obtain the original dynamic trajectory datasets of pedestrians in different streets in the target city, process the original dynamic trajectory datasets of pedestrians and obtain the typical trajectory datasets of pedestrians walking in the target city streets and the typical residence point datasets of pedestrians staying in the target city streets;

[0010] S4: Establish a multi-agent of pedestrians and place the multi-agent in the current digital sandbox. The multi-agent includes two types: walking type and resident type. A walking type agent is assigned to each typical trajectory data based on a typical trajectory data set, and a resident type agent is assigned to each typical resident point based on a typical resident point data set;

[0011] S5: Run the intelligent agent in the digital sandbox and extract the visual field image during the operation of the intelligent agent;

[0012] S6: Extract the street facade images of the buildings in the digital sandbox, and determine the key interface images of the target city streets in combination with the visual field images;

[0013] S7: Collect open source urban image datasets and annotate the indicator elements in the open source urban image datasets, use the annotated open source urban image datasets to train the deep semantic segmentation algorithm model, obtain a target model for identifying indicator elements in urban street key interface images, and calculate the diagnostic indicators of urban street key interface images based on the indicator elements;

[0014] S8: Compare the diagnostic indicators of the city street key interface image with the audit indicators of the city street key interface control requirements; optimize and adjust the city street key interfaces that do not meet the control requirements, input the optimized and adjusted city street key interfaces into the digital sandbox, and return to step S4 until the control requirements are met; use the city street key interfaces that meet the control requirements as the street interface optimization plan;

[0015] S9: Display the three-dimensional model of the street interface optimization solution in the digital sandbox, and display the diagnostic indicators of the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city street in the current state respectively;

[0016] S10: The user enters the three-dimensional model of the street interface optimization solution through VR virtual reality glasses to roam and perform human-computer interaction display.

[0017] Furthermore, the building image data in step S1 includes building top photography data, building oblique photography data, building outline data, and building facade real scene data of the target city street.

[0018] Furthermore, the steps of obtaining a typical trajectory dataset and a typical residence point dataset in step S3 are as follows:

[0019] Calculate the similarity between any two pedestrians’ original dynamic trajectories and build a graph neural network of pedestrians’ original dynamic trajectory dataset based on the similarity;

[0020] Based on the graph neural network, the original dynamic trajectory dataset of pedestrians in the target street area is clustered to obtain multiple clusters. The original dynamic trajectories of pedestrians contained in the multiple clusters are aggregated to obtain a typical trajectory dataset in the target street area.

[0021] Traverse the original dynamic trajectory dataset of pedestrians in the target street area and identify the pedestrian's stop location; set a threshold, and when the pedestrian stays at a certain location for a time exceeding the threshold, mark the location as a stop point;

[0022] All the obtained stay points are clustered based on the spatial distance and time interval between the stay points to obtain multiple clusters. The central position or average position of the stay points in the cluster is taken as the typical stay point. The typical stay points of multiple clusters constitute the typical stay point data set.

[0023] Furthermore, the similarity calculation steps are as follows:

[0024] Align the time steps between the sequences of multiple pedestrian original dynamic trajectories based on the interpolation method;

[0025] For any two sequences A and B of the original dynamic trajectories of pedestrians, the Manhattan distance between any point in sequence A and any point in sequence B is calculated. The Manhattan distance between each point in sequence A and sequence B constitutes a Manhattan distance matrix; where the i-th point in sequence A and the jth point in sequence B The Manhattan distance d M (A i ,B j ) is calculated as follows:

[0026]

[0027] The Manhattan distance is used to calculate the corresponding point in sequence A (A i (x Ai ,y Ai )) and the jth point in sequence B The cumulative distance D(A i ,B j ), cumulative distance D(A i ,B j ) is calculated as follows:

[0028] D(A1,B10=d M (A1,B1)

[0029] D(A i ,B j ) = d M (A i ,B j)+min(D(A i-1 ,B j ),D(A i ,B j-1 ),D(A i-1 ,B j-1 ))

[0030] Where i>1, j>1, D(A1,B1) is the value at the upper left position in the cumulative distance matrix, and D(A i-1 ,B j )、D(A i ,B j-1 ) and D(A i-1 ,B j-1 ) respectively represent the current position in the cumulative distance matrix (A i ,B j )’s values ​​at the three adjacent positions to the left, above, and above the left;

[0031] The cumulative distance between each point in sequence A and sequence B constitutes a cumulative distance matrix;

[0032] The similarity is the value in the last row and last column of the cumulative distance matrix.

[0033] Furthermore, step S4 also includes: inserting a visual frustum into the agent to obtain a current visual field when the agent is running.

[0034] Furthermore, the method of extracting the field of view image in step S5 is: starting from the running time of the self-walking intelligent agent, the field of view image of the walking intelligent agent is extracted at a set interval time; the resident intelligent agent has a 360° horizontal look, and starting from the running time, the field of view image of the resident intelligent agent is extracted at a set angle interval.

[0035] Furthermore, in step S6, the method of determining the target city street key interface image is as follows: establishing a grid of the building facade image along the street, numbering the grids, and calculating the frequency of the grids appearing in the visual field image;

[0036] The visibility of the grid is divided into high-visibility grid and low-visibility grid using the natural breakpoint method according to the frequency;

[0037] According to the proportion of high-visibility grids in the grids of the building facade images along the street, the building facade images along the street whose proportion of high-visibility grids exceeds a preset value are defined as key interface units of urban streets, and the buildings corresponding to the key interface units of urban streets in the three-dimensional real-life digital sand table of the target city streets are searched, and the facade images of the buildings are marked;

[0038] Identify the building facade images along the street in the visual image, define the building facade images containing signs as the key interface images of the urban street, and establish a database of key interface images of the urban street.

[0039] Further, the index elements in step S7 include buildings, windows, street furniture and facilities, sidewalks, motorways, plants and the sky;

[0040] The method for obtaining the target model in step S7 is: dividing the annotated open source urban image dataset into a training set and a test set according to a specified ratio, inputting the training set into the deep semantic segmentation algorithm model for multiple rounds of training and using the test set in each round of training to test the trained model, and saving the deep semantic segmentation algorithm model obtained from the round of training with the highest test accuracy as the target model.

[0041] Furthermore, the optimization adjustment described in step 8 refers to adjusting the indicator elements corresponding to the building facades in the digital sandbox based on the indicators that do not meet the management and control requirements.

[0042] Based on the same inventive concept, the present invention also provides a system for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation, comprising:

[0043] A data acquisition module is used to obtain the building image data of the target city street in the current state;

[0044] A sand table construction module is used to combine the geographic data and building image data of the target city streets and construct a digital sand table of the three-dimensional model of the target city streets in the current state;

[0045] Pedestrian trajectory collection module, used to collect the original dynamic trajectory of pedestrians in the target city streets in the current state;

[0046] The agent module is used to construct a pedestrian agent, place the agent in the current digital sandbox, control the agent to run in the digital sandbox according to the original dynamic trajectory, and extract the visual field image during the agent's operation;

[0047] The recognition and measurement module is used to extract the street facade images of the buildings in the digital sandbox, determine the key interface images of the target city streets in combination with the visual field images, identify the index elements in the key interface images of the city streets, and calculate the diagnostic index of the key interface images of the city streets based on the index elements;

[0048] The diagnosis and optimization module compares the diagnosis index of the key interface image of the urban street with the audit index of the key interface control requirements of the urban street, takes the key interface of the urban street that meets the control requirements as the street interface optimization plan, optimizes and adjusts the key interface of the urban street that does not meet the control requirements, inputs the optimized and adjusted key interface of the urban street into the digital sandbox, and repeats the steps of the intelligent body module, the identification and measurement module, and the diagnosis and optimization module until the control requirements are met;

[0049] The display and interaction module displays the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city streets in the current state in the digital sandbox. Users enter the three-dimensional model of the street interface optimization solution through VR virtual reality glasses to roam around.

[0050] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) The present invention collects the movement trajectory data of pedestrians on the target street and creates a simulation environment in a three-dimensional real-life digital sandbox, realizes the translation and simulation of pedestrian multi-agent movement and line-of-sight simulation rules, realizes the precise collection of crowd movement on the target street, improves the accuracy and efficiency of simulation, and ensures that the results of digital diagnosis meet the actual situation of the real street and the needs of real work;

[0051] (2) The semantic segmentation algorithm based on deep learning in the present invention identifies and extracts the building part of the intelligent body's visual field image, and determines the key interface units of urban streets according to the visual field image raster results, thereby avoiding inconsistent standards in determining the key interface units of urban streets, improving the scientificity and standardization of digital diagnosis and optimization of street interfaces, and significantly improving the efficiency compared with traditional work;

[0052] (3) The present invention displays the optimization scheme and the current street interface through a digital sandbox. Users can roam and interact with the display through VR virtual reality glasses, and observe the movement trajectories and visual fields of multiple agents in the target street, thereby improving the flexibility and immediacy of the optimization work. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of the method of the present invention;

[0054] Figure 2 A schematic diagram of pedestrian trajectory clustering and typical residence point identification in a street according to the present invention;

[0055] Figure 3 A schematic diagram of the intelligent agent of the present invention running in a digital sandbox;

[0056] Figure 4 Flow chart of the system of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0058] Example 1

[0059] The method of digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation described in the present invention is as follows: Figure 1 As shown, the following steps are included:

[0060] S1: Collect the building image data of the target city streets in the current state. The image data includes the building top photography data, building oblique photography data, building outline data and building facade real scene data of the target city streets. Preferably, a drone equipped with a Beidou positioning system and a decimeter-level precision laser radar point cloud data acquisition system is used to collect the building top photography data and building oblique photography data of the target city streets, and a handheld video probe with a resolution of more than 2k and a scanning range of more than 200 meters is used to collect the building outline data and building facade real scene data of the target city streets.

[0061] S2: Combine the geographic data and building image data of the target city streets to construct a three-dimensional real-life digital sand table of the target city streets in the current state. The geographic data includes geographic information vector data of traffic road networks, land boundaries and infrastructure information; preferably, before combining the geographic data and building image data of the target city streets, the geographic data is first entered into the geographic information platform, and the geography 2000 projection coordinate system is uniformly used to calibrate and spatially align the geographic data. The geographic data and building image data are fused to form a three-dimensional real-life digital sand table of the target city streets. Preferably, the building image data is input into the real-life modeling software of the prior art, the three-dimensional model of the building is reconstructed and monomerized, and a real-life model of the building is generated. Then, the real-life model of the building is input into the geographic information platform, and spatially matched with the geographic data to obtain a three-dimensional real-life digital sand table of the target city streets.

[0062] S3: Figure 2 As shown, the pedestrian trajectories in the target city streets in the current state are collected, and a typical trajectory dataset of pedestrians walking in the target city streets and a typical residence point dataset of pedestrians staying in the target city streets are obtained.

[0063] Set up probes that can completely cover the streets of the target city, and collect real-time video data in the streets of the target city through the probes; preferably, the probes are equipped with target detection and tracking algorithms to monitor the stereoscopic view. According to the tracking frame and foothold in the real-time video data, the original dynamic trajectory of pedestrians with personnel numbers and distance marks is obtained, and all the collected original dynamic trajectories of pedestrians are stored in the digital sandbox to obtain the original dynamic trajectory data set of pedestrians in different streets in the target city.

[0064] In the digital sandbox, calculate the similarity of any two pedestrians' original dynamic trajectories. The calculation steps are as follows:

[0065] Firstly, the time steps between the sequences of multiple pedestrians' original dynamic trajectories are aligned based on the interpolation method, and then the Manhattan distance between the sequence points of any two pedestrians' original dynamic trajectories at each time step is calculated to form a Manhattan distance matrix.

[0066] For any two pedestrian original dynamic trajectory sequences A and B, the Manhattan distance between any point in sequence A and any point in sequence B is calculated. The Manhattan distance between each point in sequence A and sequence B constitutes the Manhattan distance matrix. and the jth point in sequence B The Manhattan distance d M (A i ,B j ) is calculated as follows:

[0067]

[0068] The i-th point in the corresponding sequence A is calculated based on the Manhattan distance and the jth point in sequence B The cumulative distance D(A i ,B j ), cumulative distance D(A i ,B j ) is calculated as follows:

[0069] D(A1,B1)=d M (A1,B1)

[0070] D(A i ,B j ) = d M (A i ,B j )+min(D(A i-1 ,B j ),D(A i ,B j-1 ),D(A i-1 ,B j-1 ))

[0071] Where i>1, j>1, D(A1,B1) is the value at the upper left position in the cumulative distance matrix, and D(A i-1 ,B j )、D(A i ,B j-1 ) and D(A i-1 ,B j-1 ) respectively represent the current position in the cumulative distance matrix (A i ,Bj ), that is, when calculating the cumulative distance matrix, first calculate the data in the first row and first column of the cumulative distance matrix according to the Manhattan distance, that is, calculate D(A1,B1), and then calculate the distance of the adjacent points according to D(A1,B1), and so on.

[0072] The cumulative distance between each point in sequence A and sequence B constitutes a cumulative distance matrix. The value in the last row and last column of the cumulative distance matrix is ​​taken as the similarity. The value is inversely proportional to the similarity, that is, the closer the value is to 0, the higher the similarity.

[0073] Based on the similarity between any two pedestrians' original dynamic trajectories, a graph neural network of the pedestrian's original dynamic trajectory dataset is constructed. Each pedestrian's original dynamic trajectory is used as a node in the graph neural network, and the location information, pedestrian speed and timestamp of the pedestrian's original dynamic trajectory are used as node features; the connection between any two nodes is the edge of the graph neural network, and the similarity between any two pedestrians' original dynamic trajectories is used as the edge feature of the graph neural network.

[0074] Based on the graph neural network, the original dynamic trajectory data set of pedestrians in the target street area is clustered to obtain multiple clusters, and the original dynamic trajectories of pedestrians contained in the multiple clusters are aggregated to obtain a typical trajectory data set in the target street area. Clustering means grouping similar original dynamic trajectory data of pedestrians into one group; preferably, the aggregation method can be selected as linear fitting.

[0075] Traverse the original dynamic trajectory dataset of pedestrians in the target street area and identify the pedestrian's stop position. Set a threshold. When the pedestrian stays at a certain location for a period of time exceeding the threshold, mark the location as a stop point. Cluster all the obtained stop points based on the spatial distance and time interval between the stop points to obtain multiple clusters. The center position or average position of the stop point in the cluster is taken as a typical stop point. The typical stop points of multiple clusters constitute a typical stop point dataset. Preferably, the clustering method can be selected as the dbscan algorithm.

[0076] S4: Figure 3As shown, a multi-agent of pedestrians is established and placed in the current digital sandbox. The multi-agent includes two types: walking and resident. A walking agent is assigned to each typical trajectory data based on the typical trajectory data set, and a resident agent is assigned to each typical resident point based on the typical resident point data set. The walking direction and walking speed of the walking agent are defined. The walking direction is consistent with the tangent direction of the typical trajectory assigned to the walking agent, and the walking speed is consistent with the average speed of the cluster to which the typical trajectory assigned to the walking agent belongs; the position and time attributes of the resident agent are defined. The position is determined according to the coordinates of the typical resident point assigned to the resident agent, and the time attribute is determined according to the starting timestamp of the typical resident point assigned to the resident agent.

[0077] Insert the visual cone into the agent to obtain the current visual field when the agent is running. Define the viewpoint plane position, viewpoint height, line of sight direction, horizontal observation azimuth and vertical horizontal observation azimuth of the visual cone; the viewpoint plane position is consistent with the current position of the agent during operation, the viewpoint height is consistent with the average height of an adult of 175cm, the line of sight direction is consistent with the walking direction of the agent during operation, the horizontal observation azimuth is consistent with the horizontal viewing angle of 160° of the general visual range of an adult, and the vertical horizontal observation azimuth is consistent with the general vertical range of 135° of an adult.

[0078] S5: Run the intelligent agent in the digital sandbox, extract the visual field images during the operation of the intelligent agent, and store the obtained visual field images by number. Starting from the running time of the self-propelled intelligent agent, the visual field images of the walking intelligent agent are extracted at the set interval time; the resident intelligent agent has a 360° horizontal look, and starting from the running time, the visual field images of the resident intelligent agent are extracted at the set angle interval. Preferably, the visual field images are numbered according to the serial number of the corresponding intelligent agent and the order of extraction, and the visual field images are distributed and stored using a cloud server.

[0079] S6: Extract the street facade images of the buildings in the digital sandbox, and determine the key interface images of the target city streets in combination with the visual image. Extract the street facade images of the buildings in the digital sandbox, establish grids for the street facade images of the buildings, and number the grids. Input the grids and visual images into a cloud computing platform with a computing power of 10TFLOPS, use a semantic segmentation algorithm based on deep learning to identify and extract the buildings in the visual image, use a computer vision algorithm to sequentially compare the buildings in the grid and the visual image, record the frequency of the grid appearing in the visual image, and use the natural breakpoint method to divide the visibility of the grid into two categories: high visibility grid and low visibility grid according to the frequency.

[0080] The proportion of high-visibility grids in the grid of each building's street facade image is calculated, and the building street facade images whose high-visibility grid proportion exceeds a preset value are defined as key interface units of urban streets. Preferably, the preset value can be adjusted according to actual conditions, and the corresponding building is located in the three-dimensional real-life digital sandbox of the target city street according to the building serial number in the grid number, and the building facade image is marked.

[0081] A computer vision algorithm is used to identify street facade images of buildings in the field of view image, images containing urban street key interface units are defined as urban street key interface images, and an urban street key interface image database is established, preferably including urban street key interface images and serial numbers of the buildings contained in each image.

[0082] S7: Collect an open source city image dataset, preferably, obtain the open source city image dataset from the official website of cityscapedataset; annotate the indicator elements in the open source city images, where the indicator elements include buildings, windows, street furniture and facilities, sidewalks, motor lanes, plants and the sky, and divide the annotated open source city image dataset into a training set and a test set according to a specified ratio, preferably, the specified ratio is 8:2; input the training set into the deep semantic segmentation algorithm model for multiple rounds of training, preferably, 50-200 rounds of training, and use the test set in each round of training to test the trained model, and save the deep semantic segmentation algorithm model obtained from the round of training with the highest test accuracy as the target model. Use the target model to identify the indicator elements in the images of key interfaces of urban streets, and calculate the diagnostic indicators of key interfaces of urban streets based on the indicator elements. The specific description of the indicator elements is shown in Table 1 below:

[0083] Table 1 Index element description table

[0084] Urban interface index elements Indicator Description Building facade transparency The ratio of the number of pixels of windows in the viewshed image to the number of pixels of the rest of the building Street space enclosure The ratio of the number of pixels of buildings in the view area image to the number of pixels of the entire image Green View Rate The ratio of the number of plant pixels in the field of view image to the number of pixels in the entire image Sky visibility The ratio of the number of sky pixels in the field of view image to the number of pixels in the entire image Richness of interface elements The ratio of the number of pixels of street furniture and furniture in the viewshed image to the number of pixels in the image as a whole Street walkability The ratio of the number of sidewalk pixels to the number of motorway pixels in the view image

[0085] S8: Compare the diagnostic indicators of the key interfaces of urban streets with the audit indicators of the control requirements of the key interfaces of urban streets, mark the key interfaces of urban streets that do not meet the control requirements in the digital sand table, and display the indicators that do not meet the control requirements and the values ​​of the control requirements indicators in such key interfaces of urban streets, and then optimize and adjust the window layout of the building facades, the types, quantity and layout of plants and street furniture, and the width of the sidewalks in the digital sand table model, input the optimized and adjusted key interfaces of urban streets into the three-dimensional real-life digital sand table of the target urban streets, and return to step S4 until the control requirements are met. The key interfaces of urban streets that meet the control requirements are used as the street interface optimization plan.

[0086] S9: Display the three-dimensional model of the street interface optimization solution in the digital sandbox, and respectively display the diagnostic indicators of the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city street in the current state. Preferably, the three-dimensional model of the street interface optimization solution and the diagnostic indicators of the three-dimensional model of the target city street in the current state are respectively displayed in the form of a radar chart.

[0087] S10: The user enters the three-dimensional model of the street interface optimization solution through VR virtual reality glasses to roam and perform human-computer interaction display.

[0088] The present invention uses a semantic segmentation algorithm based on deep learning to identify and extract the architectural part of the intelligent body's field of view image, and determines the key interface units of urban streets according to the field of view image raster results, thereby avoiding inconsistent standards in the determination of key interface units of urban streets, improving the scientificity and standardization of digital diagnosis and optimization of street interfaces, and shortening the work process to 2 hours, which is more than 3 times more efficient than traditional work.

[0089] Example 2

[0090] The system for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation described in the present invention includes a data acquisition module, a sandbox construction module, a pedestrian trajectory acquisition module, an agent module, a recognition and measurement module, a diagnosis and optimization module, and a display and interaction module.

[0091] The data acquisition module is used to obtain the building image data of the target city streets in the current state. The image data includes the building top photography data, building oblique photography data, building outline data and building facade real scene data of the target city streets. Preferably, a drone equipped with a Beidou positioning system and a decimeter-level precision laser radar point cloud data acquisition system is used to collect the building top photography data and building oblique photography data of the target city streets, and a handheld video probe with a resolution of more than 2k and a scanning range of more than 200 meters is used to collect the building outline data and building facade real scene data of the target city streets.

[0092] The sand table construction module is used to combine the geographic data and building image data of the target city streets and construct a digital sand table of the three-dimensional model of the target city streets in the current state. The geographic data includes geographic information vector data of traffic road network, land boundary and infrastructure information; preferably, before combining the geographic data and building image data of the target city streets, the geographic data is first entered into the geographic information platform, and the geography 2000 projection coordinate system is uniformly used to calibrate and spatially align the geographic data; the geographic data and building image data are fused to form a digital sand table of the three-dimensional real scene of the target city streets. Preferably, the building image data is input into the real scene modeling software of the prior art, the three-dimensional model of the building is reconstructed and monomerized, and the building real scene model is generated, and then the building real scene model is input into the geographic information platform, and spatially matched with the geographic data to obtain the three-dimensional real scene digital sand table of the target city streets.

[0093] The pedestrian trajectory collection module is used to collect the original dynamic trajectories of pedestrians in the target city streets in the current state.

[0094] Set up probes that can completely cover the streets of the target city, and collect real-time video data in the streets of the target city through the probes; preferably, the probes are equipped with target detection and tracking algorithms to monitor the stereoscopic view. According to the tracking frame and foothold in the real-time video data, the original dynamic trajectory of pedestrians with personnel numbers and distance marks is obtained, and all the collected original dynamic trajectories of pedestrians are stored in the digital sandbox to obtain the original dynamic trajectory data set of pedestrians in different streets in the target city.

[0095] In the digital sandbox, calculate the similarity of any two pedestrians' original dynamic trajectories. The calculation steps are as follows:

[0096] Firstly, the time steps between the sequences of multiple pedestrians' original dynamic trajectories are aligned based on the interpolation method, and then the Manhattan distance between the sequence points of any two pedestrians' original dynamic trajectories at each time step is calculated to form a Manhattan distance matrix.

[0097] For any two sequences A and B of the original dynamic trajectories of pedestrians, the Manhattan distance between any point in sequence A and any point in sequence B is calculated. The Manhattan distance between each point in sequence A and sequence B constitutes the Manhattan distance matrix. i (x Ai ,y Ai )) and the jth point in sequence B (B i (x Bi ,y Bi ))'s Manhattan distance d M (A i ,B j ) is calculated as follows:

[0098] d M (A i ,B j )=|x Ai -x Bj |+|y Ai -y Bj |

[0099] The Manhattan distance is used to calculate the corresponding point in sequence A (A i (x Ai ,y Ai )) and the jth point in sequence B (B i (x Bi ,y Bi ))'s cumulative distance D(A i ,B j ), cumulative distance D(A i ,B j ) is calculated as follows:

[0100] D(A1,B1)=d M (A1,B1)

[0101] D(A i ,B j ) = d M (A i ,B j )+min(D(A i-1 ,B j ),D(A i ,B j-1 ),D(A i-1 ,B j-1 ))

[0102] Where i>1, j>1, D(A1,B1) is the value at the upper left position in the cumulative distance matrix, and D(A i-1 ,B j )、D(A i ,B j-1 ) and D(A i-1 ,B j-1 ) respectively represent the current position in the cumulative distance matrix (A i ,B j )’s values ​​at the three adjacent positions on the left, above, and above the left; that is, when calculating the cumulative distance matrix, first calculate the data in the first row and first column of the cumulative distance matrix according to the Manhattan distance, that is, calculate D(A1,B1), and then calculate the distances of the adjacent points according to D(A1,B1), and so on.

[0103] The cumulative distance between each point in sequence A and sequence B constitutes a cumulative distance matrix. The value in the last row and last column of the cumulative distance matrix is ​​taken as the similarity. The value is inversely proportional to the similarity, that is, the closer the value is to 0, the higher the similarity.

[0104] Based on the similarity between any two pedestrians' original dynamic trajectories, a graph neural network of the pedestrian's original dynamic trajectory dataset is constructed. Each pedestrian's original dynamic trajectory is used as a node in the graph neural network, and the location information, pedestrian speed and timestamp of the pedestrian's original dynamic trajectory are used as node features; the connection between any two nodes is the edge of the graph neural network, and the similarity between any two pedestrians' original dynamic trajectories is used as the edge feature of the graph neural network.

[0105] Based on the graph neural network, the original dynamic trajectory data set of pedestrians in the target street area is clustered to obtain multiple clusters, and the original dynamic trajectories of pedestrians contained in the multiple clusters are aggregated to obtain a typical trajectory data set in the target street area. Clustering means grouping similar original dynamic trajectory data of pedestrians into one group; preferably, the aggregation method can be selected as linear fitting.

[0106] Traverse the original dynamic trajectory dataset of pedestrians in the target street area and identify the pedestrian's stop position. Set a threshold. When the pedestrian stays at a certain location for a period of time exceeding the threshold, mark the location as a stop point. Cluster all the obtained stop points based on the spatial distance and time interval between the stop points to obtain multiple clusters. The center position or average position of the stop point in the cluster is taken as a typical stop point. The typical stop points of multiple clusters constitute a typical stop point dataset. Preferably, the clustering method can be selected as the dbscan algorithm.

[0107] The agent module is used to construct a pedestrian agent, place the agent in the current digital sandbox, control the agent to run in the digital sandbox according to the original dynamic trajectory, and extract the visual field image during the agent's operation. The multi-agent includes two types: walking and resident. A walking agent is assigned to each typical trajectory data based on the typical trajectory data set, and a resident agent is assigned to each typical resident point based on the typical resident point data set. Define the walking direction and walking speed of the walking agent. The walking direction is consistent with the tangent direction of the typical trajectory assigned to the walking agent, and the walking speed is consistent with the average speed of the cluster to which the typical trajectory assigned to the walking agent belongs; define the position and time attributes of the resident agent. The position is determined according to the coordinates of the typical resident point assigned to the resident agent, and the time attribute is determined according to the starting timestamp of the typical resident point assigned to the resident agent.

[0108] Insert the visual cone into the agent to obtain the current visual field when the agent is running. Define the viewpoint plane position, viewpoint height, line of sight direction, horizontal observation azimuth and vertical horizontal observation azimuth of the visual cone; the viewpoint plane position is consistent with the current position of the agent during operation, the viewpoint height is consistent with the average height of an adult of 175cm, the line of sight direction is consistent with the walking direction of the agent during operation, the horizontal observation azimuth is consistent with the horizontal viewing angle of 160° of the general visual range of an adult, and the vertical horizontal observation azimuth is consistent with the general vertical range of 135° of an adult.

[0109] Run the agent in the digital sandbox, extract the visual field images during the running process of the agent, and store the obtained visual field images by number. Starting from the running time of the self-propelled agent, extract the visual field images of the walking agent at the set interval time; the resident agent looks around 360° horizontally, and starting from the running time, extract the visual field images of the resident agent at the set angle interval. Preferably, the visual field images are numbered according to the serial number of the corresponding agent and the order of extraction, and the visual field images are distributed and stored using a cloud server.

[0110] The recognition and measurement module is used to extract the street facade images of the buildings in the digital sand table, determine the key interface images of the target city streets in combination with the sight picture images, identify the indicator elements in the key interface images of the city streets, and calculate the diagnostic indicators of the key interface images of the city streets based on the indicator elements. The street facade images of the buildings in the digital sand table are extracted, and the key interface images of the target city streets are determined in combination with the sight picture images. The street facade images of the buildings in the digital sand table are extracted, and the street facade images of the buildings are gridded and numbered. The grids and the sight picture images are input into a cloud computing platform with a computing power of 10TFLOPS, and the semantic segmentation algorithm based on deep learning is used to identify and extract the buildings in the sight picture images. The computer vision algorithm is used to sequentially compare the buildings in the grid and the sight picture images, and the frequency of the grid appearing in the sight picture images is recorded. According to the frequency, the visibility of the grid is divided into two categories: high visibility grids and low visibility grids using the natural breakpoint method.

[0111] The proportion of high-visibility grids in the grid of each building's street facade image is calculated, and the building street facade images whose high-visibility grid proportion exceeds a preset value are defined as key interface units of urban streets. Preferably, the preset value can be adjusted according to actual conditions, and the corresponding building is located in the three-dimensional real-life digital sandbox of the target city street according to the building serial number in the grid number, and the building facade image is marked.

[0112] A computer vision algorithm is used to identify street facade images of buildings in the field of view image, images containing urban street key interface units are defined as urban street key interface images, and an urban street key interface image database is established, preferably including urban street key interface images and serial numbers of the buildings contained in each image.

[0113] Collect an open source city image dataset, preferably, obtain an open source city image dataset from the official website of cityscapedataset; annotate the indicator elements in the open source city image, wherein the indicator elements include buildings, windows, street furniture and facilities, sidewalks, motor lanes, plants and the sky, and divide the annotated open source city image dataset into a training set and a test set according to a specified ratio, preferably, the specified ratio is 8:2; input the training set into the deep semantic segmentation algorithm model for multiple rounds of training, preferably, 50-200 rounds of training, and use the test set in each round of training to test the trained model, and save the deep semantic segmentation algorithm model obtained in the round of training with the highest test accuracy as the target model. Use the target model to identify the indicator elements in the key interface images of urban streets, and calculate the diagnostic indicators of the key interfaces of urban streets based on the indicator elements. The specific description of the indicator elements is consistent with Table 1 in Example 1.

[0114] The diagnosis and optimization module compares the diagnostic indicators of the key interface images of urban streets with the audit indicators of the key interface management and control requirements of urban streets, takes the key interfaces of urban streets that meet the management and control requirements as the street interface optimization plan, optimizes and adjusts the key interfaces of urban streets that do not meet the management and control requirements, inputs the optimized and adjusted key interfaces of urban streets into the digital sandbox, and re-performs the steps of the intelligent body module, the identification and measurement module, and the diagnosis and optimization module until the management and control requirements are met. Preferably, the key interfaces of urban streets that do not meet the management and control requirements are marked in the digital sandbox, and the indicators that do not meet the management and control requirements and the values ​​of the management and control requirement indicators in such key interfaces of urban streets are displayed respectively. The optimization and adjustment refers to the optimization and adjustment of the buildings, windows, street furniture and facilities, sidewalks, motor vehicle lanes, plants and sky on the building facades in the digital sandbox model.

[0115] The display and interaction module displays the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city street in the current state in the digital sand table, and the user enters the three-dimensional model of the street interface optimization solution through VR virtual reality glasses to roam. Preferably, the diagnostic indicators of the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city street in the current state are respectively displayed in the digital sand table. Preferably, the diagnostic indicators of the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city street in the current state are respectively displayed in the form of a radar chart.

Claims

1. A method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation, characterized by: The following steps are involved: S1: Collect the building image data of the target city street in the current state; S2: Combining the geographic data and building image data of the target city streets, a digital sand table of the three-dimensional model of the target city streets in the current state is constructed; S3: Collect the original dynamic trajectories of pedestrians in the target city streets in the current state, obtain the original dynamic trajectory datasets of pedestrians in different streets in the target city, process the original dynamic trajectory datasets of pedestrians and obtain the typical trajectory datasets of pedestrians walking in the target city streets and the typical residence point datasets of pedestrians staying in the target city streets; S4: Establish a multi-agent of pedestrians and place the multi-agent in the current digital sandbox. The multi-agent includes two types: walking type and resident type. A walking type agent is assigned to each typical trajectory data based on a typical trajectory data set, and a resident type agent is assigned to each typical resident point based on a typical resident point data set; S5: Run the agent in the digital sandbox and extract the visual field image during the operation of the agent; S6: Extract the street facade images of the buildings in the digital sandbox, and determine the key interface images of the target city streets in combination with the visual field images; S7: Collect open source urban image datasets and annotate the indicator elements in the open source urban image datasets, use the annotated open source urban image datasets to train the deep semantic segmentation algorithm model, obtain a target model for identifying indicator elements in urban street key interface images, and calculate the diagnostic indicators of urban street key interface images based on the indicator elements; S8: Compare the diagnostic indicators of the city street key interface image with the audit indicators of the city street key interface control requirements; optimize and adjust the city street key interface that does not meet the control requirements, input the optimized and adjusted city street key interface into the digital sandbox, and return to step S4 until the control requirements are met; the one that meets the control requirements is used as the street interface optimization plan; S9: Display the three-dimensional model of the street interface optimization solution in the digital sandbox, and display the diagnostic indicators of the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city street in the current state respectively; S10: The user enters the three-dimensional model of the street interface optimization solution through VR virtual reality glasses to roam and perform human-computer interaction display.

2. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 1, characterized in that: The building image data in step S1 includes building top photography data, building oblique photography data, building outline data, and building facade real scene data of the target city street.

3. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 1, characterized in that: The steps for obtaining a typical trajectory dataset and a typical residence point dataset in step S3 are as follows: Calculate the similarity between any two pedestrians’ original dynamic trajectories and build a graph neural network of pedestrians’ original dynamic trajectory dataset based on the similarity; Based on the graph neural network, the original dynamic trajectory dataset of pedestrians in the target street area is clustered to obtain multiple clusters. The original dynamic trajectories of pedestrians contained in the multiple clusters are aggregated to obtain a typical trajectory dataset in the target street area. Traverse the original dynamic trajectory dataset of pedestrians in the target street area and identify the pedestrians' stop locations; Set a threshold. When the pedestrian stays at a certain location for a period of time exceeding the threshold, the location is marked as a stay point. All the obtained stay points are clustered based on the spatial distance and time interval between the stay points to obtain multiple clusters. The central position or average position of the stay points in the cluster is taken as the typical stay point. The typical stay points of multiple clusters constitute the typical stay point data set.

4. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 3 is characterized in that: The similarity calculation steps are as follows: Align the time steps between the sequences of multiple pedestrian original dynamic trajectories based on the interpolation method; For any two sequences A and B of the original dynamic trajectories of pedestrians, the Manhattan distance between any point in sequence A and any point in sequence B is calculated. The Manhattan distance between each point in sequence A and sequence B constitutes a Manhattan distance matrix; where the i-th point in sequence A and the jth point in sequence B The Manhattan distance d M (A i ,B j ) is calculated as follows: The i-th point in the corresponding sequence A is calculated based on the Manhattan distance and the jth point in sequence B The cumulative distance D(A i ,B j ), cumulative distance D(A i ,B j ) is calculated as follows: D(A1,B1)=d M (A1,B1) D(A i ,B j )=d M (A i ,B j )+min(D(A i-1 ,B j ),D(A i ,B j-1 ),D(A i-1 ,B j-1 )) Where i>1, j>1, D ( A1,B1 ) is the value at the upper left position in the cumulative distance matrix, D(A i-1 ,B j )、D(A i ,B j-1 ) and D(A i-1 ,B j-1 ) respectively represent the current position in the cumulative distance matrix (A i ,B j )’s values ​​at the three adjacent positions to the left, above, and above the left; The cumulative distance between each point in sequence A and sequence B constitutes a cumulative distance matrix; The similarity is the value in the last row and last column of the cumulative distance matrix.

5. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 1, characterized in that: Step S4 also includes: inserting a visual cone into the agent to obtain the current visual field when the agent is running.

6. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 1, characterized in that: The method of extracting the field of view image described in step S5 is: starting from the running time of the self-propelled intelligent agent, the field of view image of the walking intelligent agent is extracted at the set interval time; the resident intelligent agent has a 360° horizontal look, and starting from the running time, the field of view image of the resident intelligent agent is extracted at the set angle interval.

7. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 1, characterized in that: The method of determining the target city street key interface image in step S6 is: establishing a grid for the building facade image along the street, numbering the grids, and calculating the frequency of the grids appearing in the visual field image; The visibility of the grid is divided into high-visibility grid and low-visibility grid using the natural breakpoint method according to the frequency; According to the proportion of high-visibility grids in the grids of the building facade images along the street, the building facade images along the street whose proportion of high-visibility grids exceeds a preset value are defined as key interface units of urban streets, and the buildings corresponding to the key interface units of urban streets in the three-dimensional real-life digital sand table of the target city streets are searched, and the facade images of the buildings are marked; Identify the building facade images along the street in the visual image, define the building facade images containing signs as the key interface images of the urban street, and establish a database of key interface images of the urban street.

8. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 1, characterized in that: The index elements in step S7 include buildings, windows, street furniture and facilities, sidewalks, motorways, plants and the sky; The method for obtaining the target model in step S7 is: dividing the annotated open source urban image dataset into a training set and a test set according to a specified ratio, inputting the training set into the deep semantic segmentation algorithm model for multiple rounds of training and using the test set in each round of training to test the trained model, and saving the deep semantic segmentation algorithm model obtained from the round of training with the highest test accuracy as the target model.

9. The method for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation according to claim 1, characterized in that: The optimization adjustment described in step 8 refers to adjusting the indicator elements corresponding to the building facades in the digital sandbox based on the indicators that do not meet the management and control requirements.

10. A system for digital diagnosis and optimization of street interfaces based on multi-agent line of sight simulation, characterized in that: include, A data acquisition module is used to obtain the building image data of the target city street in the current state; A sand table construction module is used to combine the geographic data and building image data of the target city streets and construct a digital sand table of the three-dimensional model of the target city streets in the current state; A pedestrian trajectory collection module is used to collect the original dynamic trajectories of pedestrians in the target city streets in the current state; The agent module is used to construct a pedestrian agent, place the agent in the current digital sandbox, control the agent to run in the digital sandbox according to the original dynamic trajectory, and extract the visual field image during the agent's operation; The recognition and measurement module is used to extract the street facade images of the buildings in the digital sandbox, determine the key interface images of the target city streets in combination with the visual field images, identify the index elements in the key interface images of the city streets, and calculate the diagnostic index of the key interface images of the city streets based on the index elements; The diagnosis and optimization module compares the diagnosis index of the key interface image of the urban street with the audit index of the key interface control requirements of the urban street, takes the key interface of the urban street that meets the control requirements as the street interface optimization plan, optimizes and adjusts the key interface of the urban street that does not meet the control requirements, inputs the optimized and adjusted key interface of the urban street into the digital sandbox, and repeats the steps of the intelligent body module, the identification and measurement module, and the diagnosis and optimization module until the control requirements are met; The display and interaction module displays the three-dimensional model of the street interface optimization solution and the three-dimensional model of the target city streets in the current state in the digital sandbox. Users enter the three-dimensional model of the street interface optimization solution through VR virtual reality glasses to roam around.

Images