Method and system for digital diagnosis and optimization of street interface based on multi-agent line-of-sight simulation

Abstract

The application discloses a kind of based on street interface digital diagnosis and optimization of method and system of multi-agent sight simulation, system includes data acquisition module, for obtaining the building image data of target city street under current state;Sand table construction module is used to combine the geographical data and building image data of target city street, and constructs the digital sand table of three-dimensional model of target city street under current state;Pedestrian trajectory acquisition module is used to collect the original dynamic trajectory of pedestrian in target city street under current state;Agent module, identification and measure module, diagnosis and optimization module and display and interaction module.The application collects pedestrian moving trajectory data of target street and creates simulation environment in three-dimensional real scene digital sand table, improves the accuracy and efficiency of simulation, ensures that the result of digital diagnosis meets the actual situation of real street and the needs of real work, and improves the scientificity and standardization of street interface digital diagnosis and optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to a method and system for digital diagnosis and optimization, in particular to a method and system for street interface digital diagnosis and optimization based on multi-agent sight simulation. BACKGROUND

[0002] The street interface is a key optimization and control unit in the field of urban planning, which directly affects the smoothness of urban people flow, the beauty of landscape and the quality of life of residents. At present, the research and practice of street interface mainly focus on two aspects, one is the constituent elements of street interface, the cognition, analysis and induction of street interface elements are the basis for carrying out urban planning and design work, the other is the sight perception and use quality of people to street interface, which is used to guide the diagnosis and optimization of street interface.

[0003] The traditional street interface design is usually based on experience and specification, which is easily affected by the subjective experience and standard difference of planners, while the walking and visual perception of local residents in the street is an important link that needs to be considered in the diagnosis of street interface, that is, the traditional street interface design lacks the recognition and diagnosis response of the difference of different target street users, and lacks scientificity and systematicness. SUMMARY

[0004] The first object of the present application is to provide a method for street interface digital diagnosis and optimization based on multi-agent sight simulation, which considers the walking and visual perception of local residents in the street and is not affected by the subjective influence of planners.

[0005] The second object of the present application is to provide a system for street interface digital diagnosis and optimization based on multi-agent sight simulation.

[0006] Technical scheme: The present application discloses a method for street interface digital diagnosis and optimization based on multi-agent sight simulation, comprising the following steps:

[0007] S1: collecting building image data of a target city street in a current state;

[0008] S2: combining geographical data and building image data of the target city street, constructing a digital sand table of a three-dimensional model of the target city street in the current state;

[0009] S3: collecting original dynamic trajectories of pedestrians in the target city street in the current state, obtaining original dynamic trajectory data sets of pedestrians in different streets in the target city, processing the original dynamic trajectory data sets of pedestrians and obtaining typical trajectory data sets of pedestrians walking in the target city street and typical residence point data sets of pedestrians staying in the target city street;

[0010] S4: Establish a multi-agent of pedestrians, and place the multi-agent into the current digital sand table, the multi-agent including a walking type and a staying type, a walking type agent is assigned to each typical trajectory data based on a typical trajectory data set, and a staying type agent is assigned to each typical staying point based on a typical staying point data set;

[0011] S5: Run the agent in the digital sand table, and extract the visual field picture image in the running process of the agent;

[0012] S6: Extract the building facade image along the street in the digital sand table, and determine the target city street key interface image in combination with the visual field picture image;

[0013] S7: Collect an open source city image data set, and label index elements in the open source city image data set, train a deep semantic segmentation algorithm model using the labeled open source city image data set, obtain a target model for identifying index elements in a city street key interface image, and calculate a diagnostic index of the city street key interface image according to the index elements;

[0014] S8: Compare the diagnostic index of the city street key interface image with an audit index of a city street key interface control requirement; optimize and adjust the city street key interface that does not meet the control requirement, input the optimized and adjusted city street key interface into the digital sand table, return to step S4 until the control requirement is met; and take the city street key interface that meets the control requirement as a street interface optimization scheme;

[0015] S9: Display the three-dimensional model of the street interface optimization scheme in the digital sand table, and display the diagnostic index of the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state, respectively;

[0016] S10: The user enters the three-dimensional model of the street interface optimization scheme through the VR virtual reality glasses to roam, and performs human-computer interaction display.

[0017] Further, the building image data in step S1 includes building top photography data, building oblique photography data, building contour data, and building facade real scene data of the target city street.

[0018] Further, the steps of obtaining the typical trajectory data set and the typical staying point data set in step S3 are as follows:

[0019] Calculate the similarity of any two pedestrian original dynamic trajectories, and construct a graph neural network of the pedestrian original dynamic trajectory data set based on the similarity;

[0020] Based on graph neural networks, the original dynamic trajectory dataset of pedestrians in the target street area is clustered to obtain multiple clusters. The original dynamic trajectories of pedestrians contained in the multiple clusters are then aggregated to obtain a typical trajectory dataset in the target street area.

[0021] Traverse the original dynamic trajectory dataset of pedestrians within the target street area to identify the stopping locations of pedestrians; set a threshold, and when the time a pedestrian spends at a certain location exceeds the threshold, that location is marked as a stopping point;

[0022] All the obtained stop points are clustered based on the spatial distance and time interval between them, resulting in multiple clusters. The center or average position of the stop points in each cluster is taken as the typical stop point. The typical stop points of multiple clusters constitute the typical stop point dataset.

[0023] Furthermore, the similarity calculation steps are as follows:

[0024] Align the time steps between sequences of multiple original dynamic trajectories of pedestrians using interpolation;

[0025] For any two original pedestrian trajectories, sequences A and B, calculate the Manhattan distance between any point in sequence A and any point in sequence B. The Manhattan distances between each point in sequences A and B form a Manhattan distance matrix; where the i-th point in sequence A... and the j-th point in sequence B Manhattan is far from d M (A i B j The calculation formula for ) is as follows:

[0026]

[0027] The i-th point (A) in the corresponding sequence A is calculated based on the Manhattan distance. i (x Ai ,y Ai and the j-th point in sequence B The cumulative distance D(A) i B j ), cumulative distance D(A i B j The calculation formula for ) is as follows:

[0028] D(A1,B10=d M (A1, B1)

[0029] D(A i B j )=d M (A i B j)+min(D(A i-1 B j ),D(A i B j-1 ),D(A i-1 B j-1 ))

[0030] In the formula, i > 1, j > 1, and D(A1, B1) is the value at the top left of the cumulative distance matrix. i-1 B j ), D(A i B j-1 ) and D(A i-1 B j-1 ) represent the current position (A) in the cumulative distance matrix. i B j The values ​​of the three adjacent positions to the left, above, and upper left of the ().

[0031] The cumulative distance between each point in sequence A and sequence B constitutes the cumulative distance matrix;

[0032] The similarity is the value in the last row and last column of the cumulative distance matrix.

[0033] Furthermore, step S4 also includes: placing a visual frustum in the agent to acquire the current field of view during the operation of the agent.

[0034] Furthermore, the method for extracting the field of view images in step S5 is as follows: starting from the running time of the walking agent, extract the field of view images of the walking agent at a set interval; for the stationary agent, perform a 360° horizontal panoramic view, and starting from the running time, extract the field of view images of the stationary agent at a set angle interval.

[0035] Furthermore, the method for determining the key interface images of the target city streets in step S6 is as follows: a grid is created from the street facade images of the buildings, the grids are numbered, and the frequency of the grids appearing in the view image is calculated.

[0036] Based on frequency, the visibility of the raster is divided into high-visibility raster and low-visibility raster using the natural breakpoint method;

[0037] Based on the proportion of high-visibility grids in the grid of the building facade image along the street, the building facade images along the street with a proportion of high-visibility grids exceeding a preset value are defined as key interface units of the city street. The corresponding buildings of the key interface units of the city street are searched in the 3D real-scene digital sand table of the target city street, and the facade images of the buildings are marked.

[0038] Identify the building facade image in the visual field picture image, define the building facade image containing the mark as the city street key interface image, and establish a city street key interface image database.

[0039] Further, the index elements in step S7 include buildings, windows, street furniture and facilities, sidewalks, motor vehicle lanes, plants and the sky.

[0040] The target model is obtained in step S7 in the following manner: the annotated open source city image dataset is divided into a training set and a test set according to a specified ratio, the training set is input into a deep semantic segmentation algorithm model for multi-round training, and the test set is used to test the model obtained by training in each round of training, and the deep semantic segmentation algorithm model obtained by training with the highest test accuracy in one round of training is saved as the target model.

[0041] Further, the optimization adjustment in step 8 is based on the index that does not meet the control requirements, and adjusts the index elements corresponding to the building facade in the digital sand table.

[0042] Based on the same inventive concept, the present application also provides a system for street interface digital diagnosis and optimization based on multi-agent visual simulation, comprising,

[0043] The data acquisition module is used to acquire building image data of the target city street in the current state.

[0044] The sand table construction module is used to combine the geographic data and building image data of the target city street, and construct a digital sand table of the three-dimensional model of the target city street in the current state.

[0045] The pedestrian trajectory acquisition module is used to acquire the original dynamic trajectory of pedestrians in the target city street in the current state.

[0046] The agent module is used to construct a pedestrian agent, place the agent into the current digital sand table, control the agent to run in the digital sand table according to the original dynamic trajectory, and extract the visual field picture image in the running process of the agent.

[0047] The identification and measurement module is used to extract the building facade image in the digital sand table, determine the target city street key interface image in combination with the visual field picture image, identify the index elements in the city street key interface image, and calculate the diagnostic index of the city street key interface image according to the index elements.

[0048] The diagnosis and optimization module compares the diagnosis indexes of the urban street key interface image with the audit indexes of the urban street key interface control requirements, takes the urban street key interface meeting the control requirements as the street interface optimization scheme, optimizes and adjusts the urban street key interface not meeting the control requirements, inputs the optimized and adjusted urban street key interface into the digital sand table, and performs the steps of the agent module, the identification and measurement module and the diagnosis and optimization module again until the control requirements are met;

[0049] The display and interaction module displays the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target urban street in the current state in the digital sand table, and the user enters the three-dimensional model of the street interface optimization scheme to roam through the VR virtual reality glasses.

[0050] Advantages: Compared with the prior art, the present application has the following remarkable advantages: (1) The present application collects the pedestrian moving track data of the target street and creates a simulation environment in the three-dimensional real scene digital sand table, realizes the translation and simulation of the pedestrian multi-agent motion and the line of sight simulation rule, realizes the accurate collection of the crowd movement of the target street, improves the accuracy and efficiency of the simulation, and ensures that the results of the digital diagnosis meet the actual situation of the real street and the needs of the real work;

[0051] (2) The present application identifies and extracts the building part of the agent visual field picture image based on the semantic segmentation algorithm of deep learning, and determines the urban street key interface unit according to the visual field image grid result, avoids the inconsistency of the standards in the determination of the urban street key interface unit, improves the scientificity and standardization of the street interface digital diagnosis and optimization, and the working efficiency is obviously improved compared with the traditional one;

[0052] (3) The present application displays the optimization scheme and the current street interface through the digital sand table, the user can roam and man-machine interaction display through the VR virtual reality glasses, and observe the moving track and visual field picture of the multi-agent in the target street, which improves the flexibility and immediacy of the optimization work. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is the flow chart of the method of the present application;

[0054] Figure 2 is the schematic diagram of the pedestrian track clustering and typical residence point identification in the street of the present application;

[0055] Figure 3 is the schematic diagram of the agent running in the digital sand table of the present application;

[0056] Figure 4 is the flow chart of the system of the present application. DETAILED DESCRIPTION

[0057] The technical solutions of the present application will be further described below with reference to the drawings.

[0058] Embodiment 1

[0059] A method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation, as shown in the present application, comprises the following steps: Figure 1

[0060] S1: Collect building image data of the target city street in the current state. The image data includes building top photography data, building oblique photography data, building contour data and building facade real scene data of the target city street. Preferably, the building top photography data and the building oblique photography data of the target city street are collected by using a UAV equipped with a Beidou positioning system and a decimeter-level precision laser radar point cloud data collection system, and the building contour data and the building facade real scene data of the target city street are collected by using a handheld video probe with a resolution of 2k or more and a scanning range of 200 meters or more.

[0061] S2: Combine the geographic data and the building image data of the target city street to construct a three-dimensional real scene digital sand table of the target city street in the current state. The geographic data includes geographic information vector data of traffic network, plot boundary and infrastructure information; preferably, before combining the geographic data and the building image data of the target city street, the geographic data is first entered into a geographic information platform and corrected and spatially aligned using the Geodetic 2000 projection coordinate system. The geographic data and the building image data are fused to form a three-dimensional real scene digital sand table of the target city street, preferably, the building image data is input into the existing real scene modeling software, the three-dimensional model of the building is reconstructed and individualized to generate a building real scene model, and then the building real scene model is input into the geographic information platform to be spatially matched with the geographic data to obtain the three-dimensional real scene digital sand table of the target city street.

[0062] S3: As shown in the present application, collect the pedestrian trajectories in the target city street in the current state, and obtain the typical trajectory data set of the pedestrians walking in the target city street and the typical residence point data set of the pedestrians staying in the target city street. Figure 2

[0063] A probe capable of completely covering the target city street is erected, and real-time video data in the target city street is collected through the probe; preferably, the probe is provided with a target detection and tracking algorithm, which can monitor the stereoscopic view. According to the tracking frame and the landing point in the real-time video data, the original dynamic trajectory of the pedestrian with a personnel number and a distance identifier is obtained, all the collected original dynamic trajectories of the pedestrians are stored in the digital sand table, and the original dynamic trajectory data set of the pedestrians in different streets in the target city is obtained.

[0064] ​​In a digital sandbox, calculate the similarity between the original dynamic trajectories of any two pedestrians. The calculation steps are as follows:

[0065] First, the time steps of multiple pedestrian original dynamic trajectory sequences are aligned using interpolation. Then, the Manhattan distance between any two pedestrian original dynamic trajectory sequence points at each time step is calculated to form a Manhattan distance matrix.

[0066] For any two original pedestrian trajectories, sequences A and B, calculate the Manhattan distance between any point in sequence A and any point in sequence B. The Manhattan distances between each point in sequences A and B form a Manhattan distance matrix. The i-th point in sequence A... and the j-th point in sequence B Manhattan is far from d M (A i B j The calculation formula for ) is as follows:

[0067]

[0068] The i-th point in sequence A is calculated based on the Manhattan distance. and the j-th point in sequence B The cumulative distance D(A) i B j ), cumulative distance D(A i B j The calculation formula for ) is as follows:

[0069] D(A1,B1)=d M (A1, B1)

[0070] D(A i B j )=d M (A i B j )+min(D(A i-1 B j ),D(A i B j-1 ),D(A i-1 B j-1 ))

[0071] In the formula, i > 1, j > 1, and D(A1, B1) is the value at the top left of the cumulative distance matrix. i-1 B j ), D(A i B j-1 ) and D(A i-1 B j-1 ) represent the current position (A) in the cumulative distance matrix. i Bj The values ​​of the three adjacent positions to the left, above, and upper left of the cumulative distance matrix are calculated by first obtaining the data of the first row and first column of the cumulative distance matrix based on the Manhattan distance, i.e., calculating D(A1,B1). Then, the distance to the adjacent point is calculated based on D(A1,B1), and so on.

[0072] The cumulative distance between each point in sequence A and sequence B forms a cumulative distance matrix. The value in the last row and last column of the cumulative distance matrix is ​​used as the similarity score. Furthermore, the value is inversely proportional to the similarity score; that is, the closer the value is to 0, the higher the similarity score.

[0073] A graph neural network (GNN) for a dataset of original pedestrian trajectories is constructed based on the similarity between any two original pedestrian trajectories. Each original pedestrian trajectory is treated as a node in the GNN, and the location information, speed, and timestamp of the original pedestrian trajectory are used as node features. The connection between any two nodes is used as an edge in the GNN, and the similarity between any two original pedestrian trajectories is used as the edge feature of the GNN.

[0074] Based on graph neural networks, the original dynamic trajectory dataset of pedestrians within the target street area is clustered to obtain multiple clusters. The original dynamic trajectories of pedestrians contained in each cluster are then aggregated to obtain a typical trajectory dataset within the target street area. Clustering involves grouping similar original dynamic trajectory data of pedestrians into a group; preferably, linear fitting can be used as the aggregation method.

[0075] The system iterates through the original dynamic trajectory dataset of pedestrians within the target street area to identify pedestrian stopping locations. A threshold is set; when a pedestrian spends more than the threshold at a certain location, that location is marked as a stopping point. All obtained stopping points are clustered based on the spatial distance and time interval between them, resulting in multiple clusters. The center or average position of the stopping points within each cluster is used as a typical stopping point. The typical stopping points from multiple clusters constitute a typical stopping point dataset. Preferably, the clustering method can be the dbscan algorithm.

[0076] S4: As Figure 3As shown, the multi-agent of pedestrians is established, and the multi-agent is placed in the current digital sand table. The multi-agent includes walking type and staying type, a walking type agent is assigned to each typical trajectory data based on a typical trajectory data set, and a staying type agent is assigned to each typical staying point based on a typical staying point data set. The walking direction and walking speed of the walking type agent are defined, the walking direction is consistent with the tangent direction of the typical trajectory to which the walking agent is assigned, and the walking speed is consistent with the average speed of the cluster to which the typical trajectory to which the walking agent is assigned belongs; the position and time attribute of the staying type agent are defined, the position is determined according to the coordinates of the typical staying point to which the staying type agent is assigned, and the time attribute is determined according to the starting time stamp of the typical staying point to which the staying type agent is assigned.

[0077] The view cone is placed in the agent for obtaining the current view picture of the agent running. The view point plane position, view point height, view line direction, horizontal observation azimuth angle and vertical horizontal observation azimuth angle of the view cone are defined; the view point plane position is consistent with the current position in the running process of the agent, the view point height is consistent with the average height of adults 175 cm, the view line direction is consistent with the walking direction in the running process of the agent, the horizontal observation azimuth angle is consistent with the general horizontal visual angle of adults 160°, and the vertical horizontal observation azimuth angle is consistent with the general vertical range of adults 135°.

[0078] S5: running the agent in the digital sand table, extracting the view picture image in the running process of the agent, and numbering and storing the obtained view picture image. Starting from the running time of the walking type agent, the view picture image of the walking type agent is extracted at a set interval; the staying type agent looks around 360° horizontally, and the view picture image of the staying type agent is extracted at a set angle interval starting from the running time. Preferably, the view picture image is numbered according to the serial number of the corresponding agent and the order of extraction, and the view picture image is distributed saved using a cloud server.

[0079] S6: extracting the building facade image along the street in the digital sand table, and determining the key interface image of the target city street in combination with the view picture image. The building facade image along the street in the digital sand table is extracted, the building facade image along the street is established into a grid, and the grid is numbered. The grid and the view picture image are input into a cloud computing platform with a computing power of 10TFLOPS, a semantic segmentation algorithm based on deep learning is used to identify and extract the building in the view picture image, a computer vision algorithm is used to sequentially compare the buildings in the grid and the view picture image, the frequency of the grid appearing in the view picture image is recorded, and the visibility of the grid is divided into two categories of high visibility grid and low visibility grid according to the frequency using the natural breakpoint method.

[0080] The proportion of high-visibility grids in the grid of the building facade image along the street is calculated, and the building facade image along the street whose high-visibility grid proportion exceeds a preset value is defined as a city street key interface unit. Preferably, the preset value can be adjusted according to the actual situation. The corresponding building is located in the target city street three-dimensional real scene digital sand table according to the building serial number in the grid number, and the building facade image is marked.

[0081] Using a computer vision algorithm, the building facade image along the street in the view picture image is recognized, the image containing the city street key interface unit is defined as the city street key interface image, and a city street key interface image database is established. Preferably, it includes the city street key interface image and the serial number of the building contained in each image.

[0082] S7: Collecting an open source city image dataset, preferably from the cityscapedataset official website; labeling the index elements in the open source city image, wherein the index elements include buildings, windows, street furniture and facilities, sidewalks, motor vehicle lanes, plants and sky; dividing the labeled open source city image dataset into a training set and a test set according to a specified ratio, preferably 8:2; inputting the training set into a deep semantic segmentation algorithm model for multiple rounds of training, preferably 50-200 rounds of training; in each round of training, using the test set to test the trained model, and saving the deep semantic segmentation algorithm model obtained in the round of training with the highest test accuracy as the target model. Using the target model to identify the index elements in the city street key interface image, and calculating the diagnostic indicators of the city street key interface according to the index elements. The specific description of the index elements is shown in Table 1 as follows:

[0083] Table 1: Index element description table

[0084] Urban interface indicator elements Indicator description Building facade transparency Ratio of the number of pixels of windows to the number of pixels of other parts of the building in the viewshed picture image Street space enclosure Ratio of the number of pixels of the building to the total number of pixels of the image in the viewshed picture image Green view ratio Ratio of the number of pixels of plants to the total number of pixels of the image in the viewshed picture image Sky visibility Ratio of the number of pixels of the sky to the total number of pixels of the image in the viewshed picture image Interface element richness Ratio of the number of pixels of street furniture and implementation to the total number of pixels of the image in the viewshed picture image Street walkability Ratio of the number of pixels of the sidewalk to the number of pixels of the carriageway in the viewshed picture image

[0085] S8: Comparing the diagnostic indicators of the city street key interface with the audit indicators of the city street key interface control requirements, marking the city street key interface that does not meet the control requirements in the digital sand table, and respectively displaying the index that does not meet the control requirements and the value of the control requirement index in such city street key interface. Then, the window layout of the building facade in the digital sand table model, the type, number and layout of plants and street furniture, and the width of the sidewalk are optimized and adjusted. The optimized and adjusted city street key interface is input into the target city street three-dimensional real scene digital sand table, and step S4 is returned until the control requirements are met. The city street key interface that meets the control requirements is used as a street interface optimization scheme.

[0086] S9: display the three-dimensional model of the street interface optimization scheme in the digital sand table, and respectively show the diagnostic indicators of the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state. Preferably, the diagnostic indicators of the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state are respectively shown in the form of radar charts.

[0087] S10: the user enters the street interface optimization scheme three-dimensional model through the VR virtual reality glasses to roam and perform human-computer interaction display.

[0088] The semantic segmentation algorithm based on deep learning of the application identifies and extracts the building part of the agent visual field picture image, and determines the city street key interface unit according to the visual field image grid result, avoids inconsistent standards in the determination of the city street key interface unit, improves the scientificity and standardization of the street interface digital diagnosis and optimization, and shortens the working process to 2 hours, which is more than 3 times higher than the traditional working efficiency.

[0089] Embodiment 2

[0090] The system for street interface digital diagnosis and optimization based on multi-agent visual line simulation provided by the application comprises a data acquisition module, a sand table construction module, a pedestrian trajectory acquisition module, an agent module, an identification and measurement module, a diagnosis and optimization module, and a display and interaction module.

[0091] The data acquisition module is used to acquire the building image data of the target city street in the current state. The image data comprises the top photography data of the building of the target city street, the oblique photography data of the building, the building contour data, and the building facade real scene data. Preferably, the top photography data and the oblique photography data of the building of the target city street are acquired by using a UAV equipped with a Beidou positioning system and a decimeter-level precision laser radar point cloud data acquisition system, and the building contour data and the building facade real scene data are acquired by using a handheld video probe with a resolution of 2k or above and a scanning range of 200 meters or above.

[0092] The sand table construction module is used for combining geographical data and building image data of the target urban street, and constructing a digital sand table of a three-dimensional model of the target urban street in a current state. The geographical data includes geographical information vector data of traffic network, land boundary and infrastructure information. Preferably, before the geographical data and the building image data of the target urban street are combined, the geographical data is first recorded into a geographical information platform, and is uniformly corrected and spatially aligned by adopting a geodetic 2000 projection coordinate system. The geographical data and the building image data are fused to form a three-dimensional real scene digital sand table of the target urban street. Preferably, the building image data is input into a real scene modeling software of the prior art, a three-dimensional model of the building is reconstructed and individualized, and a building real scene model is generated. Then, the building real scene model is input into the geographical information platform, and is spatially matched with the geographical data to obtain the three-dimensional real scene digital sand table of the target urban street.

[0093] The pedestrian trajectory acquisition module is used for acquiring original dynamic trajectories of pedestrians in the target urban street in the current state.

[0094] The probe capable of completely covering the target urban street is erected, and real-time video data in the target urban street is acquired by the probe. Preferably, the probe is provided with a target detection and tracking algorithm, and is capable of monitoring a stereoscopic visual angle. According to a tracking frame and a landing point in the real-time video data, original dynamic trajectories of pedestrians with personnel numbers and distance identifiers are obtained, and all the acquired original dynamic trajectories of pedestrians are stored into the digital sand table to obtain original dynamic trajectory data sets of pedestrians in different streets in the target urban street.

[0095] In the digital sand table, the similarity of any two original dynamic trajectories of pedestrians is calculated. The calculation steps are as follows:

[0096] Firstly, the time steps between the sequences of the plurality of original dynamic trajectories of pedestrians are aligned based on an interpolation method, and then the Manhattan distance of each time step between the sequence points of any two original dynamic trajectories of pedestrians is calculated to form a Manhattan distance matrix.

[0097] For any two sequences A and B of original dynamic trajectories of pedestrians, the Manhattan distance of any point in the sequence A and any point in the sequence B is calculated, and the Manhattan distance between each point in the sequence A and the sequence B forms a Manhattan distance matrix. The Manhattan distance d i (x Ai ,y Ai ) of the i-th point (A i (x Bi ,y Bi ) in the sequence A and the j-th point (B M (x i ,y j ) in the sequence B is calculated according to the following formula:

[0098] d M (A i ,B j )=|x Ai -x Bj |+|y Ai -y Bj |

[0099] The cumulative distance D(A i ,B Ai ) of the ith point (A Ai (x i ,y Bi )) in sequence A and the jth point (B Bi (x i ,y j )) in sequence B is calculated based on Manhattan distance, and the calculation formula of the cumulative distance D(A i ,B j ) is as follows:

[0100] D(A1,B1)=d M (A1,B1)

[0101] D(A i ,B j )=d M (A i ,B j )+min(D(A i-1 ,B j ),D(A i ,B j-1 ),D(A i-1 ,B j-1 ))

[0102] wherein i>1, j>1, D(A1,B1) is the value of the left uppermost position in the cumulative distance matrix, D(A i-1 ,B j ), D(A i ,B j-1 ) and D(A i-1 ,B j-1 ) respectively represent the values of the three adjacent positions of the left side, the upper side and the left upper side of the current position (A i ,B j ) in the cumulative distance matrix; that is, when the cumulative distance matrix is calculated, the data of the first row and the first column of the cumulative distance matrix is calculated according to Manhattan distance, that is, D(A1,B1) is calculated, then the distances of the adjacent points of D(A1,B1) are calculated according to D(A1,B1), and so on.

[0103] The cumulative distance between each point in sequence A and sequence B constitutes a cumulative distance matrix. The value in the last row and last column of the cumulative distance matrix is taken as the similarity. The value is inversely proportional to the similarity, that is, the closer the value is to 0, the higher the similarity.

[0104] Based on the similarity between any two original dynamic trajectories of pedestrians, a graph neural network of the original dynamic trajectory dataset of pedestrians is constructed. Each original dynamic trajectory of a pedestrian is taken as a node in the graph neural network, and the position information, pedestrian speed and time stamp of the original dynamic trajectory of the pedestrian are taken as the node features; the connection between any two nodes is the edge of the graph neural network, and the similarity between any two original dynamic trajectories of pedestrians is taken as the feature of the edge of the graph neural network.

[0105] Based on the graph neural network, the original dynamic trajectory dataset of pedestrians in the target street area is clustered to obtain a plurality of clustering clusters, and the original dynamic trajectories contained in the plurality of clustering clusters are aggregated respectively to obtain a typical trajectory dataset in the target street area. Clustering is to divide similar original dynamic trajectory data of pedestrians into a group; preferably, the aggregation mode can be selected as linear fitting.

[0106] The original dynamic trajectory dataset of pedestrians in the target street area is traversed to identify the stay positions of pedestrians. A threshold is set, and when the time of a pedestrian staying at a position exceeds the threshold, the position is marked as a stay point. Based on the spatial distance and time interval between the stay points, all the obtained stay points are clustered to obtain a plurality of clustering clusters, and the center position or average position of the stay points in the clustering cluster is taken as a typical stay point. The typical stay points of the plurality of clustering clusters constitute a typical stay point dataset. Preferably, the clustering mode can be selected as the dbscan algorithm.

[0107] An agent module is used to construct a pedestrian agent, place the agent into the current digital sand table, control the agent to run in the digital sand table according to the original dynamic trajectory, and extract the view picture image in the running process of the agent. The multi-agent includes two types of walking and staying, a walking agent is assigned to each typical trajectory based on the typical trajectory dataset, and a staying agent is assigned to each typical stay point based on the typical stay point dataset. The walking direction and walking speed of the walking agent are defined, the walking direction is consistent with the tangent direction of the typical trajectory to which the walking agent is assigned, and the walking speed is consistent with the average speed of the clustering cluster to which the typical trajectory belongs; the position and time attribute of the staying agent are defined, the position is determined according to the coordinates of the typical stay point to which the staying agent is assigned, and the time attribute is determined according to the start time stamp of the typical stay point to which the staying agent is assigned.

[0108] A view cone is placed in the agent for obtaining the current view picture of the agent running. The view point plane position, view point height, view line direction, horizontal observation azimuth angle and vertical horizontal observation azimuth angle of the view cone are defined; the view point plane position is consistent with the current position in the agent running process, the view point height is consistent with the average height of adults, i.e. 175 cm, the view line direction is consistent with the walking direction in the agent running process, the horizontal observation azimuth angle is consistent with the general horizontal view angle of adults, i.e. 160°, and the vertical horizontal observation azimuth angle is consistent with the general vertical range of adults, i.e. 135°.

[0109] The agent in the digital sand table is run, the view picture image in the agent running process is extracted, and the obtained view picture image is numbered and stored. Starting from the running time of the walking type agent, the view picture image of the walking type agent is extracted at a set interval; the 360° horizontal view of the resident type agent is extracted at a set angle interval starting from the running time. Preferably, the view picture image is numbered according to the serial number of the corresponding agent and the order of extraction, and the view picture image is distributed and saved using a cloud server.

[0110] The recognition and measurement module is used to extract the building facade image along the street in the digital sand table, determine the target city street key interface image in combination with the view picture image, recognize the index elements in the city street key interface image, and calculate the diagnostic index of the city street key interface image according to the index elements. The building facade image along the street in the digital sand table is extracted, the target city street key interface image is determined in combination with the view picture image. The building facade image along the street in the digital sand table is extracted, the building facade image along the street is established into a grid, and the grid is numbered. The grid and the view picture image are input into a cloud computing platform with a computing power of 10 TFLOPS, a semantic segmentation algorithm based on deep learning is used to recognize and extract the building in the view picture image, a computer vision algorithm is used to sequentially compare the buildings in the grid and the view picture image, the frequency of the grid appearing in the view picture image is recorded, and the visibility of the grid is divided into two categories, i.e. high visibility grid and low visibility grid, according to the frequency using the natural breakpoint method.

[0111] The proportion of high visibility grid in each building facade image along the street is calculated, the building facade image along the street with a high visibility grid proportion exceeding a preset value is defined as a city street key interface unit, and preferably, the preset value can be adjusted according to the actual situation. The corresponding building is located in the target city street three-dimensional real scene digital sand table according to the building serial number in the grid number, and the building facade image is marked.

[0112] The computer vision algorithm is used to identify the building facade image in the field of vision picture image, the image containing the key interface unit of the city street is defined as the key interface image of the city street, and a key interface image database of the city street is established, preferably, which includes the key interface image of the city street and the serial number of the building contained in each image.

[0113] An open source city image dataset is collected, preferably, from the official website of cityscapedataset; the index elements in the open source city image are labeled, wherein the index elements include buildings, windows, street furniture and facilities, sidewalks, motor vehicle lanes, plants and sky, the labeled open source city image dataset is divided into a training set and a test set according to a specified ratio, preferably, the specified ratio is 8:2; the training set is input into a deep semantic segmentation algorithm model for multiple rounds of training, preferably, 50-200 rounds of training are performed, the test set is used to test the model obtained by training in each round of training, and the deep semantic segmentation algorithm model obtained by the round of training with the highest test accuracy is saved as a target model. The index elements in the key interface image of the city street are identified using the target model, and the diagnostic indicators of the key interface of the city street are calculated according to the index elements. The specific description of the index elements is consistent with Table 1 in Embodiment 1.

[0114] The diagnostic and optimization module compares the diagnostic indicators of the key interface image of the city street with the audit indicators of the control requirements of the key interface of the city street, takes the key interface of the city street meeting the control requirements as the street interface optimization scheme, optimizes and adjusts the key interface of the city street not meeting the control requirements, inputs the optimized and adjusted key interface of the city street into the digital sand table, and re-performs the steps of the agent module, the identification and measurement module and the diagnostic and optimization module until the control requirements are met. Preferably, the key interface of the city street not meeting the control requirements is marked in the digital sand table, and the values of the indicators not meeting the control requirements and the control requirement indicators in such key interface of the city street are displayed respectively. The optimization and adjustment are performed on the buildings, windows, street furniture and facilities, sidewalks, motor vehicle lanes, plants and sky of the building facade in the digital sand table model.

[0115] The display and interaction module displays the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state in the digital sand table, and the user enters the three-dimensional model of the street interface optimization scheme through the VR virtual reality glasses to roam. Preferably, the diagnostic indicators of the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state are displayed in the digital sand table respectively. Preferably, the diagnostic indicators of the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state are displayed in the form of radar charts respectively.

Claims

1. A method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation, characterized in that: The method comprises the following steps: S1: collecting building image data of a target city street in a current state; S2: combining geographical data and building image data of the target city street to construct a digital sand table of a three-dimensional model of the target city street in the current state; S3: collecting original dynamic trajectories of pedestrians in the target city street in the current state, obtaining original dynamic trajectory data sets of pedestrians in different streets in the target city, processing the original dynamic trajectory data sets of pedestrians, and obtaining a typical trajectory data set of pedestrians walking in the target city street and a typical residence point data set of pedestrians staying in the target city street; S4: establishing a multi-agent of pedestrians, placing the multi-agent into the current digital sand table, the multi-agent including two types of walking and staying, assigning a walking agent to each typical trajectory data based on the typical trajectory data set, and assigning a staying agent to each typical residence point based on the typical residence point data set; S5: running the agents in the digital sand table, and extracting view picture images in the running process of the agents; S6: extracting building facade images along streets in the digital sand table, and determining key interface images of the target city street in combination with the view picture images; S7: collecting an open source city image data set, labeling index elements in the open source city image data set, training a deep semantic segmentation algorithm model using the labeled open source city image data set, obtaining a target model for identifying index elements in the key interface images of the city street, and calculating diagnostic indexes of the key interface images of the city street according to the index elements; S8: comparing the diagnostic indexes of the key interface images of the city street with audit indexes of control requirements of the key interface of the city street, optimizing and adjusting the key interface of the city street that does not meet the control requirements, inputting the optimized and adjusted key interface of the city street into the digital sand table, returning to step S4 until the control requirements are met, and taking the key interface of the city street that meets the control requirements as a street interface optimization scheme; S9: displaying a three-dimensional model of the street interface optimization scheme in the digital sand table, and respectively displaying diagnostic indexes of the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state; S10: a user enters the three-dimensional model of the street interface optimization scheme to roam through VR virtual reality glasses, and performs human-computer interaction display.

2. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 1, characterized in that: The building image data in step S1 includes building top photography data, building oblique photography data, building contour data, and building facade real scene data of the target city street.

3. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 1, characterized in that: In step S3, the steps of obtaining the typical trajectory data set and the typical residence point data set are as follows: calculating the similarity of any two original dynamic trajectories of pedestrians, constructing a graph neural network of the original dynamic trajectory data set of pedestrians based on the similarity; based on the graph neural network, clustering the original dynamic trajectory data set of pedestrians in the target street area to obtain multiple clustering clusters, and aggregating the original dynamic trajectories of pedestrians contained in the multiple clustering clusters to obtain a typical trajectory data set in the target street area; traversing the original dynamic trajectory data set of pedestrians in the target street area, and identifying the residence positions of the pedestrians; Setting a threshold, when the time of the pedestrian staying at a certain position exceeds the threshold, marking the position as a stay point; Clustering all the obtained stay points based on the spatial distance and time interval between the stay points to obtain a plurality of clustering clusters, taking the center position or average position of the stay points in the clustering cluster as a typical stay point, and the typical stay points of the plurality of clustering clusters constitute a typical stay point data set.

4. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 3, characterized in that: The similarity calculation step is as follows: Align the time steps between the sequences of the plurality of original dynamic trajectories of pedestrians based on an interpolation method; For any two original pedestrian trajectories, sequences A and B, calculate the Manhattan distance between any point in sequence A and any point in sequence B. The Manhattan distances between each point in sequences A and B form a Manhattan distance matrix; where the i-th point in sequence A... and the j-th point in sequence B Manhattan is far from d M (A i B j The calculation formula for ) is as follows: The cumulative distance D(A i ,B j ) of the i-th point in sequence A and the j-th point in sequence B is calculated based on Manhattan distance The formula for calculating the cumulative distance D(A i ,B j ) is as follows:​ D(A1,B1) = d M (A1,B1) D(A i ,B j ) = d M (A i ,B j ) + min(D(A i-1 ,B j ), D(A i ,B j-1 ), D(A i-1 ,B j-1 )) where i > 1, j > 1, D ( A1,B1 ) is the value of the left uppermost position in the accumulated distance matrix, D(A i-1 ,B j ), D(A i ,B j-1 ) and D(A i-1 ,B j-1 ) represent the values of the three adjacent positions left, above and left above of the current position (A i ,B j ) in the accumulated distance matrix, respectively; The cumulative distance between each point in sequence A and sequence B constitutes a cumulative distance matrix. The similarity is the value in the last row and last column of the cumulative distance matrix.

5. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 1, characterized in that: In step S4, a visual cone is placed in the agent to obtain the current visual field picture of the agent during operation.

6. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 1, characterized in that: In step S5, the visual field picture image is extracted in the following manner: from the start of the operation of the walking agent, the visual field picture image of the walking agent is extracted at a set interval; the stay agent looks around 360° horizontally, and the visual field picture image of the stay agent is extracted at a set angle interval from the start of the operation.

7. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 1, characterized in that: In step S6, the target city street key interface image is determined in the following manner: a grid is established for the building facade along the street, the grid is numbered, and the frequency of the grid appearing in the visual field picture image is calculated; According to the frequency, the visibility of the grid is divided into high-visibility grid and low-visibility grid using the natural breakpoint method; According to the proportion of high-visibility grid in the grid of building facade along the street, the building facade along the street whose high-visibility grid proportion exceeds a preset value is defined as a city street key interface unit, the corresponding building in the target city street three-dimensional real scene digital sand table is searched, and the building facade image is identified; The building facade image in the visual field picture image is identified, the building facade image containing the identification is defined as the city street key interface image, and a city street key interface image database is established.

8. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 1, characterized in that: The index elements in step S7 include buildings, windows, street furniture and facilities, sidewalks, motor vehicle lanes, plants, and the sky; In step S7, the target model is obtained in the following manner: the labeled open source city image dataset is divided into a training set and a test set according to a specified proportion, the training set is input into a deep semantic segmentation algorithm model for multiple rounds of training, and the test set is used to test the model obtained by training in each round of training, and the deep semantic segmentation algorithm model obtained by the round of training with the highest test accuracy is saved as the target model.

9. The method for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation according to claim 1, characterized in that: The optimization adjustment in step 8 is based on the index that does not meet the control requirements to adjust the index elements corresponding to the building facade in the digital sand table.

10. A system for street interface digital diagnosis and optimization based on multi-agent line-of-sight simulation, characterized in that: It comprises, A data acquisition module for acquiring building image data of the target city street in the current state; A sand table construction module for constructing a three-dimensional model digital sand table of the target city street in the current state in combination with the geographic data and building image data of the target city street; A pedestrian trajectory acquisition module for acquiring original dynamic trajectories of pedestrians in the target city street in the current state; The intelligent agent module is used to build a pedestrian intelligent agent, put the intelligent agent into the current digital sand table, control the intelligent agent to run in the digital sand table according to the original dynamic trajectory, and extract the visual field picture image in the running process of the intelligent agent. The identification and measurement module is used to extract the building facade image along the street in the digital sand table, determine the target city street key interface image in combination with the visual field picture image, identify the index elements in the city street key interface image, and calculate the diagnostic index of the city street key interface image according to the index elements. The diagnosis and optimization module compares the diagnostic index of the city street key interface image with the audit index of the control requirement of the city street key interface, takes the city street key interface meeting the control requirement as the street interface optimization scheme, optimizes and adjusts the city street key interface not meeting the control requirement, inputs the optimized and adjusted city street key interface into the digital sand table, and re-performs the steps of the intelligent agent module, the identification and measurement module and the diagnosis and optimization module until the control requirement is met. The display and interaction module displays the three-dimensional model of the street interface optimization scheme and the three-dimensional model of the target city street in the current state in the digital sand table, and the user enters the three-dimensional model of the street interface optimization scheme to roam through the VR virtual reality glasses.

Images