High-precision automatic identification and classification method for styles and features along street based on three-dimensional topology network

Abstract

The invention discloses a high-precision automatic identification and classification method for styles and features along a street based on a three-dimensional topology network. The method comprises the steps of current street scene image data acquisition, three-dimensional form point cloud data preprocessing, three-dimensional topology network construction, street type identification, type identification result verification and man-machine interaction display feedback. According to the method, street three-dimensional space forms are automatically identified and classified by using point cloud data of the street three-dimensional space forms and adopting a graph neural network technology. According to the method, street space type identification in the urban planning field can be coped with, a street three-dimensional space form type database based on the graph neural network technology is established, automatic identification and classification of street three-dimensional space form types are realized, the accuracy of urban street three-dimensional space identification and classification is improved in an accurate quantification mode, and the method is suitable for popularization and application. And a powerful support and a reliable basis are provided for refined management and control of the urban street space three-dimensional form.

Description

Technical Field

[0001] The invention relates to the field of urban planning, and in particular to a high-precision automatic recognition and classification method for streetscapes based on a three-dimensional topological network. Background Art

[0002] Under the current rapid development background of China, the urbanization process is advancing at an unprecedented speed, which has brought economic growth and urban prosperity, but also caused a series of urban problems such as "one thousand cities look the same" and dilapidated appearance. As an important part of urban public space, urban street space is not only used as the main place for residents' daily public activities, but also an important window for people to understand the historical changes of the city and the lives of citizens, which is related to the quality of life of residents. This makes the identification and classification of the three-dimensional spatial morphological types of urban streets particularly critical. The three-dimensional spatial morphological recognition and classification technology of urban streets can provide detailed urban spatial information, which can help planners make more scientific decisions when conducting urban design and avoid unreasonable planning and construction to a certain extent. When facing the risk of demolition and reconstruction of historical buildings and traditional blocks caused by rapid urbanization, the three-dimensional spatial morphological recognition and classification technology of urban streets can record the morphology and appearance of existing streets and buildings in detail, help formulate protection plans, and effectively protect the historical appearance and cultural heritage of the city. The three-dimensional spatial morphological recognition and classification technology of urban streets can provide strong support in both the planning and design of new cities and the protection and renewal of old cities. This wide applicability makes the three-dimensional spatial morphological recognition and classification technology of urban streets an important tool in urban planning and management.

[0003] Currently, the common methods for identifying and classifying the three-dimensional morphology of urban street space are: one is traditional field survey, which obtains the corresponding three-dimensional spatial morphological data of the street by measuring spatial morphological indicators such as road length and building height on site and taking photos on both sides of the street, and then uses three-dimensional modeling to summarize the three-dimensional spatial morphological types of the street. This method has problems such as scattered data, large workload, and large manual measurement errors, and cannot accurately restore the three-dimensional morphology of the street space; the other is to identify the three-dimensional spatial morphological types of the street by image sampling and semantic segmentation of street scene pictures. This method has image recognition and semantic segmentation errors, and at the same time cannot fully express the characteristics of the three-dimensional spatial morphological types of the street. The results are quite arbitrary and lack interpretability. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a high-precision automatic recognition and classification method for streetscape based on a three-dimensional topological network. The present invention can perform efficient data analysis and processing on streetscape and automatically determine its style type.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] Step 1: Data Collection

[0007] Use a 3D stereo scanner with GPS positioning, effective pixels of 6400 DPI or more, and accuracy of 100mm or more to collect 3D point cloud data of each street side in the target urban area. Use a portable panoramic image acquisition device with GPS positioning, color depth of 48 bits or more, and resolution of 8K or more to collect street view image data on both sides of the street in the target urban area.

[0008] Step 2: Data preprocessing

[0009] The GPS positioning data collected along the streets is used to perform initial alignment between the 3D morphology point cloud data and the street view image data. Feature points are extracted from the 3D morphology point cloud data and the street view image data, and the street view image data is stretched and deformed to align the feature points of the 3D morphology point cloud data and the street view image data. The ICP algorithm is used to map the street view image data to the coordinates of each 3D point in the 3D morphology point cloud data, and the dot matrix of the 3D morphology point cloud data and the street view image data are aligned.

[0010] Step 3: 3D topology network construction

[0011] Using a graphics workstation equipped with a graphics processing unit with a video memory of more than 24GB and a video memory bit width of more than 384 bits, the nodes of the three-dimensional topological network are established according to the three-dimensional point coordinates of the three-dimensional morphological point cloud data on each street and side of the street. According to the point coordinate information, each node extracts the image color and image texture information of the street view image data after dot matrix alignment at the coordinate, as well as the surface curvature and normal direction information of the three-dimensional morphological point cloud data at the coordinate. Each node is connected in pairs to establish the connection of the three-dimensional topological network. Each connection contains the difference values ​​of the nodes at both ends in terms of three-dimensional point coordinates, image color, image texture, surface curvature, and normal direction. The global optimization method is used to screen the data of node and connection information.

[0012] Step 4: Street type identification

[0013] Through the graph neural network, the node topology feature matrix and the line topology feature matrix are established for each street and each side according to the nodes and lines in the three-dimensional topological network. A graph convolution layer is constructed to aggregate the topological similarity features of the node and its adjacent nodes respectively, and the graph convolution layer is stacked according to the adjacency relationship to further obtain a wider range of adjacency information. An output layer is added to output the number and proportion of nodes of the adjacency relationship of each type of topological features on each side of each street. A street three-dimensional morphology classification model is established through a fully connected neural network. According to the training of the street three-dimensional morphology type training database, the number and proportion of nodes of the adjacency relationship of each type of topological features on each side of each street are input, and then the specific type of each street and each side of the street is determined.

[0014] Step 5: Type recognition result verification

[0015] The data of each street side along each street is extracted at a ratio of no less than 20% as the street 3D morphology type verification data set. The difference between each full-map topological feature of each street side along each street in the verification data set and each full-map topological feature of each sample data in the training database is calculated, and the mean of the difference is calculated. When the mean of the difference is the smallest, if the recognition result of the 3D morphology type of the street to be verified in the verification data set is the same as the 3D morphology type of the street of the corresponding sample in the training database, the verification is passed; otherwise, the verification fails, and the process returns to step 4 and adjusts the parameter interval values ​​of each full-map topological feature of this type of sample data.

[0016] Step 6: Human-computer interaction display feedback

[0017] The data from the three-dimensional morphological type spectrum database of the target urban street space is imported into a holographic sandbox display device equipped with a wearable three-dimensional motion capture system and a large screen with a resolution of 8K or above, to display the type composition of the overall three-dimensional morphology of the target city's street space, as well as the specific types of each street and each side of the street, to conduct human-computer virtual interaction and record feedback information.

[0018] Furthermore, in the step 2, the ICP algorithm is used to map the street view image data to the three-dimensional point coordinates of the three-dimensional morphological point cloud data, and the target points are extracted from the street view image data at a spatial spacing of no more than 100 mm, and the three-dimensional point coordinates in the three-dimensional morphological point cloud data closest to the spatial coordinates of the target points are obtained respectively. When multiple target points are mapped to the same three-dimensional point coordinate, only the target point closest to the three-dimensional point coordinate is retained.

[0019] Furthermore, in the step three, the global optimization method is used to perform data screening of node and line information, which is to construct constraint conditions, that is, the values ​​of the node's three-dimensional point coordinates, image color, image texture, surface curvature, and normal direction data cannot be empty, otherwise the node and related lines are deleted; the absolute length of the line is not greater than 200 mm, otherwise the line is deleted; the line must not have an intersection with other lines in the same plane, otherwise the line with a larger absolute length is deleted.

[0020] Furthermore, in step 4, a node topology feature matrix is ​​established based on the nodes and lines in the three-dimensional topology network, which is to collect image color, image texture, surface curvature, and normal direction indicators of adjacent nodes with lines. When the difference of each indicator of adjacent nodes is less than 15%, it is determined that the two nodes have the same specific node topology feature type, and the topological similarity between each node topology feature type is calculated according to the indicator interval. The node topology feature type enters the attributes of each node to establish a node topology feature matrix.

[0021] Beneficial effects of the present invention:

[0022] 1. The present invention combines three-dimensional morphological data with neural network technology to construct a three-dimensional morphological training database, and determines the three-dimensional morphological type of street space through morphological parameters, thus breaking through the problem of large errors in three-dimensional morphological data in traditional spatial modeling, identification and classification; reducing the amount of data that needs to be collected in traditional spatial surveys, and simplifying the process of street three-dimensional morphological identification and classification. In step four, the three-dimensional spatial morphology of the street is automatically identified and classified through the graph neural network combined with the three-dimensional topological network. Applying graph neural network technology to the identification and classification of street three-dimensional spatial morphology can effectively break through the subjectivity of human judgment in traditional spatial typology, making the entire process more intelligent and automated.

[0023] 2. The present invention establishes a three-dimensional topological network through three-dimensional morphological point cloud data and street view image data. In step 2, the ICP algorithm is used to map the street view image data to the three-dimensional morphological point cloud data, and the error is controlled within 100 mm. Data screening and adjustment are performed in combination with distance constraints, thereby realizing a fine simulation of the three-dimensional morphology of urban street space in a computer three-dimensional network, and maximizing the accuracy and efficiency of three-dimensional morphological recognition and classification.

[0024] 3. The present invention uses graph convolutional neural network technology to perform feature aggregation on nodes and lines in a three-dimensional topological network. In step three, when the difference in various indicators of adjacent nodes with lines is less than 15%, data integration is performed in the form of averaging to form feature point clusters. When the difference in node indicators within each feature point cluster is less than 15%, it is classified into one feature point cluster. Through the integration of graph convolutional neural networks, while ensuring data integrity, the data structure is optimized, the amount of data calculation is reduced, and the efficiency of street three-dimensional morphology recognition and classification is improved.

[0025] 4. In step 5 of the present invention, no less than 20% of the street-side data of each street is combined with the street 3D morphology training database. Through interactive feedback and round-by-round optimization, when the difference in various full-map topological features between the verification data set and the sample data in the training database is minimized, the street 3D morphology type to be verified is identified as the same as the street 3D morphology type of the corresponding sample in the training database.

[0026] 5. In step six of the present invention, a holographic sand table display device equipped with a wearable three-dimensional motion capture system and a large screen with a resolution of 8K or above is used to display the spatial distribution of three-dimensional morphological types of urban streets, thereby improving the display effect, and recording feedback information through human-computer virtual interaction, thereby enriching the ways for the public to participate in urban planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the method of the present invention;

[0028] Figure 2 It is a schematic diagram of the data collection and preprocessing of street view images on both sides. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] A high-precision automatic recognition and classification method for streetscapes based on a three-dimensional graph network. Figure 1-2 As shown, the following steps are included:

[0031] 1. Data collection. Use a 3D stereo scanner with GPS positioning, effective pixels of 6400 DPI or more, and accuracy of 100 mm or more to collect 3D point cloud data of the streets along the streets within the target urban area. Use a portable panoramic image acquisition device with GPS positioning, color depth of 48 bits or more, and resolution of 8K or more to collect street view image data on both sides of the streets within the target urban area.

[0032] Among them, the three-dimensional point cloud data of each street side in the target urban area shall be collected. Before the data collection process, the equipment shall be calibrated to ensure the accuracy of the data and remove the debris that may affect the scanning quality. Cloudy daytime shall be selected for data collection as much as possible to avoid the influence of shadows and light changes on data collection. The formulated data collection scanning path shall ensure that all important areas are covered to avoid scanning duplicate or missing areas.

[0033] 2. Data preprocessing. Use the GPS positioning data collected along the streets to perform initial alignment of the 3D point cloud data and the street view image data. Extract feature points from the 3D point cloud data and the street view image data, stretch and deform the street view image data, and align the feature points of the 3D point cloud data and the street view image data. Use the ICP algorithm to map the street view image data to the coordinates of each 3D point in the 3D point cloud data, and align the dot matrix of the 3D point cloud data and the street view image data.

[0034] The use of the ICP algorithm to map the street view image data to the three-dimensional point coordinates of the three-dimensional morphological point cloud data is to extract the target points from the street view image data at a spatial spacing of no more than 100 mm, and obtain the three-dimensional point coordinates in the three-dimensional morphological point cloud data closest to the spatial coordinates of the target points. When multiple target points are mapped to the same three-dimensional point coordinate, only the target point closest to the three-dimensional point coordinate is retained.

[0035] 3. Construction of three-dimensional topological network. Using a graphics workstation equipped with a graphics processing unit with a video memory of more than 24GB and a video memory bit width of more than 384 bits, the nodes of the three-dimensional topological network are established according to the three-dimensional point coordinates of the three-dimensional morphological point cloud data on each street and each side of the street. According to the point coordinate information, each node extracts the image color and image texture information of the street view image data after dot matrix alignment at the coordinate, as well as the surface curvature and normal direction information of the three-dimensional morphological point cloud data at the coordinate. Each node is connected in pairs to establish the connection of the three-dimensional topological network. Each connection contains the difference values ​​of the nodes at both ends in terms of three-dimensional point coordinates, image color, image texture, surface curvature, and normal direction. Use global optimization methods to screen the data of node and connection information.

[0036] The use of the global optimization method to perform data screening of node and line information is to construct constraint conditions, namely, the values ​​of the node's three-dimensional point coordinates, image color, image texture, surface curvature, and normal direction data cannot be empty, otherwise the node and related lines will be deleted; the absolute length of the line is not greater than 200 mm, otherwise the line will be deleted; the line must not have intersections with other lines in the same plane, otherwise the line with a larger absolute length will be deleted.

[0037] 4. Street type identification. Through the graph neural network, each street and each side of the street is used to establish a node topology feature matrix and a line topology feature matrix based on the nodes and lines in the three-dimensional topological network. Construct a graph convolution layer to aggregate the topological similarity features of the node and its adjacent nodes respectively, and stack the graph convolution layer according to the adjacency relationship to further obtain a wider range of adjacency information. Add an output layer to output the number and proportion of nodes of the adjacency relationship of each type of topological features on each side of each street. A street three-dimensional morphology classification model is established through a fully connected neural network. According to the training of the street three-dimensional morphology type training database, the number and proportion of nodes of the adjacency relationship of each type of topological features on each side of each street are input to determine the specific type of each street and each side of the street.

[0038] The node topology feature matrix is ​​established based on the nodes and lines in the three-dimensional topology network, which collects the image color, image texture, surface curvature, and normal direction indicators of the adjacent nodes with lines. When the difference of each indicator of the adjacent nodes is less than 15%, it is determined that the two nodes have the same specific node topology feature type, and the topological similarity between each node topology feature type is calculated according to the indicator interval. The node topology feature type enters the attributes of each node to establish the node topology feature matrix.

[0039] In the process of establishing the street three-dimensional morphology classification model through the fully connected neural network, the data is first preprocessed, the number of nodes and their proportions of the adjacency relationship of each type of topological features in each street and each side along the street are extracted and standardized, and then a fully connected neural network is constructed, the input layer is designed to accept these feature vectors, the middle layer uses appropriate activation functions and regularization techniques to prevent overfitting, and the output layer is configured to output the probability distribution of each three-dimensional morphology type, and then the model is trained using a training database, and the model performance is optimized using cross-validation and early stopping mechanisms. The classification effect of the model can be analyzed based on the performance of the evaluation model on the validation set through indicators such as confusion matrix, precision, recall rate and F1 score, and the network structure and hyperparameters are adjusted as needed to improve the classification accuracy and generalization ability of the model.

[0040] 5. Verification of type recognition results. Extract data from each street side at a ratio of no less than 20% as a street 3D morphology type verification data set. Calculate the difference between each full-map topological feature of each street side in the verification data set and each full-map topological feature of each sample data in the training database, and calculate the mean of the difference. When the mean of the difference is the smallest, if the 3D morphology type recognition result of the street to be verified in the verification data set is the same as the 3D morphology type of the corresponding sample in the training database, the verification passes; otherwise, the verification fails, and returns to step 4 and adjusts the parameter interval values ​​of each full-map topological feature of this type of sample data.

[0041] The calculation method comprises the following steps: calculating the difference between each full-map topological feature of each street side in the verification data set and each full-map topological feature of each sample data in the training database, and calculating the mean difference. In the process of calculating the mean difference, the mean difference is calculated according to the weight of each full-map topological feature. More than 30 experts who have been engaged in the urban and rural planning industry for more than ten years are invited to score or rank the importance of each feature, normalize the score to a weight range, calculate the weight based on the expert evaluation results, and further calculate the mean difference.

[0042] 6. Human-computer interaction display feedback. The data of the target urban street space three-dimensional morphological type spectrum database is imported into a holographic sand table display device equipped with a wearable three-dimensional motion capture system and a large screen with a resolution of 8K or above, to display the type composition of the overall three-dimensional morphology of the target city's street space, as well as the specific types of each street and each side of the street, to conduct human-computer virtual interaction and record feedback information.

[0043] The human-computer virtual interaction and recording of feedback information, during which a barrier-free user interaction interface supporting sign language and voice interaction is utilized, has a data processing module to process the street space three-dimensional morphological type spectrum data imported from the database, has a data display module to display the overall street space three-dimensional morphology of the target city and the specific style types of each street and each side of the street through a holographic sandbox device, and has a feedback recording module to record the user's interactive behavior and feedback information.

[0044] Example

[0045] The technical solution of the present invention will be described in detail below by taking the high-precision automatic recognition and classification of streetscapes in a certain area of ​​Nanjing as an example.

[0046] The technical solution of the present invention will be described in detail below by taking the high-precision automatic recognition and classification of streetscapes in a certain area of ​​Nanjing as an example.

[0047] (1) Take the streets in a certain area of ​​Nanjing as the target streets, and collect the 3D point cloud data of each side of the street. During the collection process, a portable panoramic image acquisition device with GPS positioning, color depth of 48 bits or more, and resolution of 8K or more is required to collect street view image data on both sides of the street within the target area. The process of collecting the 3D point cloud data of each side of the street should include:

[0048] (1.1) During the data collection process, the equipment should be calibrated first to ensure the accuracy of the data. At the same time, debris along the street that may affect the scanning process and data accuracy should be removed. In terms of the selection of scanning environment, data collection should be carried out on a sunny day with a cloudy sky as much as possible to avoid the influence of shadows, light changes, climate changes, etc. on the data collection process.

[0049] (1.2) During the data collection process, a clear and well-defined data collection scanning path must be developed to ensure that all important areas along the street are covered. At the same time, it is also necessary to avoid scanning duplicate or missing areas, which would cause inaccurate and incomplete data along the street.

[0050] (2) Using the collected GPS positioning data along the streets, perform initial alignment of the three-dimensional point cloud data along the streets with the street view image data. Specifically, it includes:

[0051] (2.1) When aligning the acquired 3D point cloud data along the street with the street view image, the corresponding feature points should be extracted from the two types of data respectively, and the street view image data should be stretched and deformed to ensure that the feature points of the 3D point cloud data and the street view image data are aligned.

[0052] (2.2) After the feature points are aligned, the IPC algorithm is used to map the data carried by the street view image to the coordinates of each three-dimensional point in the three-dimensional morphological point cloud data, and the dot matrix of the three-dimensional morphological point cloud data and the street view image data is aligned.

[0053] (2.3) The IPC algorithm used maps the street view image data to the three-dimensional point coordinates of the three-dimensional morphological point cloud data. The target point is extracted from the street view image data at a spatial spacing of no more than 100 mm, and the three-dimensional point coordinates of the three-dimensional morphological point cloud data closest to the target point spatial coordinates are obtained. When multiple target points are mapped to the three-dimensional point coordinates of the same three-dimensional morphological point cloud data, only the target point closest to the three-dimensional point coordinates in spatial distance is retained.

[0054] (3) Using the aligned 3D point coordinates of the 3D morphological point cloud data along the streets, a 3D topological network node is established. Specifically, it includes:

[0055] (3.1) Use a graphics workstation equipped with a graphics processing unit with a video memory of 24 GB or more and a video memory bit width of 384 bits or more to establish a three-dimensional topological network node using the spatial coordinates of each three-dimensional point in the three-dimensional morphological point cloud data of each street along each side of the street.

[0056] (3.2) In the established 3D network topology, according to the spatial coordinate information of each node, the street view image data information after dot matrix alignment is extracted, including the image color and image texture information corresponding to this spatial point in the street view image. At the same time, the surface curvature and normal direction information of the 3D morphological point cloud data at the spatial coordinate point should also be extracted.

[0057] (3.3) After extracting the street view image data and the three-dimensional morphological point cloud data, the three-dimensional topological network nodes are connected in pairs to establish three-dimensional network topological connections. In the process of connecting the two pairs, the global optimization technology is used to screen the data of the node and connection information.

[0058] The global optimization method used in (3.4) is to screen nodes and lines according to the constructed constraints. The coordinates of the three-dimensional topological network nodes, image color, image surface texture information, surface curvature, and normal direction data cannot be empty, otherwise the node and its related lines will be deleted. When the absolute length of the spatial line is greater than 200mm, the spatial line will be deleted. When the line has an intersection with other lines in the same plane, the line with the larger absolute length will be deleted.

[0059] (4) Use graph neural network technology to establish a street three-dimensional morphological classification model database, and then determine the specific type of each street and each side of the street through database training.

[0060] (4.1) By using graph neural network technology, each street and each side of the street is based on the nodes and lines in the three-dimensional topological network to establish a node topological feature matrix and a line topological feature matrix, that is, to collect the image color, image texture, surface curvature, and normal direction indicators of adjacent nodes with lines. When the difference in each indicator is less than 15%, it can be determined that the two nodes have the same specific node topological feature type, and the topological similarity between each node topological feature type is calculated according to the indicator interval. The node topological feature type enters the attributes of each node to establish a node topological feature matrix.

[0061] (4.2) Construct a graph convolution layer to aggregate the topological similarity features of nodes and their adjacent nodes respectively, and stack the graph convolution layers according to the adjacency relationship to obtain more adjacency information, so as to output the number and proportion of nodes of each type of topological feature of the adjacency relationship on each street and each side of the street.

[0062] (4.3) To establish a street 3D morphology classification model through a full neural network, the data must first be preprocessed to extract and standardize the number and proportion of nodes in the adjacency relationship of each type of topological feature on each street and each side of the street, and then a fully connected neural network is constructed. The model is then trained using a training database, and cross-validation and early stopping mechanisms are used to optimize model performance. The network structure and hyperparameters are adjusted as needed to improve the classification accuracy and generalization ability of the model. Finally, the specific type of each street and each side of the street is determined by combining the training of the street 3D morphology classification model database and the number and proportion of nodes in the adjacency relationship of each type of topological feature on each street and each side of the street.

[0063] (5) Establish a street 3D morphology type verification data set to verify the recognition results. Specifically include:

[0064] (5.1) Data from each street side shall be extracted at a ratio of no less than 20% to establish a street 3D morphology type verification data set for verification.

[0065] (5.2) Calculate the difference between the full-map topological features of each street and each side of the street in the verification data set and the full-map topological features of each sample data in the training database, and calculate the mean of the difference. The calculation involves calculating the mean of the difference based on the weights of each full-map topological feature. This can be done by inviting more than 30 experts who have been engaged in the urban and rural planning industry for more than ten years to score or rank the importance of each feature, normalizing the scores to a weight range, assigning weights based on the comprehensive expert evaluation results, and further calculating the mean of the difference.

[0066] (5.3) When the mean difference is the smallest, if the recognition result of the three-dimensional spatial morphological type of the street to be verified in the test data set is consistent with the three-dimensional spatial morphological type of the street of the corresponding sample in the three-dimensional morphological type training database, the verification is passed; otherwise, the verification fails, and the process returns to step 4 and adjusts the value intervals of the various full-map topological feature parameters of this type of sample data.

[0067] (6) Display the three-dimensional morphological type pedigree database of the target urban area in a human-computer interactive manner, including:

[0068] (6.1) The data of the target urban street space three-dimensional morphological type spectrum database is imported into a holographic sand table display device equipped with a wearable three-dimensional motion capture system and a large screen with a resolution of 8K or above, to display the type composition of the overall three-dimensional morphology of the target city’s street space.

[0069] (6.2) In the process of human-computer virtual interaction, an accessible user interaction interface that supports sign language and voice interaction is used. A data processing module is provided to process the three-dimensional morphological type spectrum data of the street space imported from the database. A data display module is provided to display the overall three-dimensional morphology of the street space of the target city and the specific style types of each street and each side of the street through a holographic sand table device. A feedback recording module is provided to record the user's interactive behavior and feedback information.

[0070] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A high-precision automatic recognition and classification method for streetscape features based on a three-dimensional topological network, characterized in that: The steps include: Step 1: Data Collection Use a 3D stereo scanner with GPS positioning, effective pixels of 6400 DPI or more, and an accuracy of 100mm or more to collect 3D point cloud data of the streets along the streets within the target urban area; use a portable panoramic image acquisition device with GPS positioning, color depth of 48 bits or more, and resolution of 8K or more to collect street view image data on both sides of the streets within the target urban area; Step 2: Data preprocessing Using the collected GPS positioning data along the streets, the three-dimensional point cloud data and the street view image data are initially aligned; Extracting feature points from the three-dimensional morphological point cloud data and the street view image data, stretching and deforming the street view image data, and aligning the feature points of the three-dimensional morphological point cloud data and the street view image data; The street view image data is mapped to the three-dimensional point coordinates of the three-dimensional morphological point cloud data using the ICP algorithm, and the dot matrix of the three-dimensional morphological point cloud data and the street view image data are aligned; Step 3: 3D topology network construction Using a graphics workstation equipped with a graphics processing unit with a video memory of more than 24GB and a video memory bit width of more than 384 bits, the nodes of the three-dimensional topological network are established based on the three-dimensional point coordinates of the three-dimensional morphological point cloud data of each street and each side of the street; Each node extracts the image color and image texture information of the street view image data after dot matrix alignment at the coordinate, and the surface curvature and normal direction information of the three-dimensional point cloud data at the coordinate according to the point coordinate information; Each node is connected in pairs to establish the connection of the three-dimensional topological network; each connection contains the difference values ​​of the nodes at both ends in three-dimensional point coordinates, image color, image texture, surface curvature, and normal direction; the global optimization method is used to screen the data of node and connection information; Step 4: Street type identification Through the graph neural network, each street and each side of the street is used to establish a node topology feature matrix and a line topology feature matrix according to the nodes and lines in the three-dimensional topology network; a graph convolution layer is constructed to aggregate the topological similarity features of the node and its adjacent nodes respectively, and the graph convolution layer is stacked according to the adjacency relationship to further obtain a wider range of adjacency information; an output layer is added to output the number and proportion of nodes of the adjacency relationship of each type of topological features on each side of each street; a street three-dimensional morphology classification model is established through a fully connected neural network, and according to the training of the street three-dimensional morphology type training database, the number and proportion of nodes of the adjacency relationship of each type of topological features on each side of each street are input, and then the specific type of each street and each side of the street is determined; Step 5: Type recognition result verification The data of each street side along each street are extracted at a ratio of not less than 20% as a street 3D morphology type verification data set; the difference between each full-map topological feature of each street side along each street in the verification data set and each full-map topological feature of each sample data in the training database is calculated, and the mean of the difference is calculated; when the mean of the difference is the smallest, if the recognition result of the 3D morphology type of the street to be verified in the verification data set is the same as the 3D morphology type of the street of the corresponding sample in the training database, the verification is passed; otherwise, the verification fails, and the process returns to step 4 and adjusts the parameter interval values ​​of each full-map topological feature of the sample data of this type; Step 6: Human-computer interaction display feedback The data from the three-dimensional morphological type spectrum database of the target urban street space is imported into a holographic sandbox display device equipped with a wearable three-dimensional motion capture system and a large screen with a resolution of 8K or above, to display the type composition of the overall three-dimensional morphology of the target city's street space, as well as the specific types of each street and each side of the street, to conduct human-computer virtual interaction and record feedback information.

2. The method for identifying the three-dimensional street morphology type based on a three-dimensional topological network according to claim 1, characterized in that: In the step 2, the street view image data is mapped to the three-dimensional point coordinates of the three-dimensional morphological point cloud data using the ICP algorithm, and the target points are extracted from the street view image data at a spatial interval of no more than 100 mm, and the three-dimensional point coordinates in the three-dimensional morphological point cloud data closest to the spatial coordinates of the target points are respectively obtained; when multiple target points are mapped to the same three-dimensional point coordinates, only the target point closest to the three-dimensional point coordinates is retained.

3. The method for identifying the three-dimensional street morphology type based on a three-dimensional topological network according to claim 2, characterized in that: In the step three, the global optimization method is used to screen the data of node and line information, which is to construct constraint conditions, that is, the values ​​of the node's three-dimensional point coordinates, image color, image texture, surface curvature, and normal direction data cannot be empty, otherwise the node and related lines are deleted; the absolute length of the line is not greater than 200 mm, otherwise the line is deleted; the line must not have an intersection with other lines in the same plane, otherwise the line with a larger absolute length is deleted.

4. The method for identifying the three-dimensional street morphology type based on a three-dimensional topological network according to claim 3, characterized in that: In the step 4, a node topology feature matrix is ​​established based on the nodes and lines in the three-dimensional topology network, and the image color, image texture, surface curvature, and normal direction indicators of adjacent nodes with lines are collected. When the difference between the indicators of adjacent nodes is less than 15%, it is determined that the two nodes have the same specific node topology feature type, and the topological similarity between the node topology feature types is calculated according to the indicator interval; the node topology feature type is entered into the attributes of each node to establish a node topology feature matrix.

Images