A method, device, equipment, medium and product for calculating the shape similarity of buildings for any single view

Abstract

The present application discloses a method, apparatus, device, medium and product for calculating the shape similarity of buildings for any single view, which relates to the field of urban three-dimensional computing services. The method includes: obtaining a single-view image of a building, and extracting the internal feature lines and external boundary feature lines of the building in the single-view image of the building; obtaining the three-dimensional shape features of the building in the single-view image of the building based on the internal feature lines and external boundary feature lines; forming a three-dimensional shape feature pair from the three-dimensional shape features of the buildings in any two single-view images of the buildings; inputting the three-dimensional shape feature pair into a similarity measurement model for the shape similarity of buildings, and using the similarity measurement model to calculate the shape similarity of the buildings in any two single-view images of the buildings. The present application realizes the calculation of the similarity of the target shapes of buildings in any single view under a unified standard, and provides technical support for realizing the query and calculation of buildings in the urban real scene.

Description

Technical Field

[0001] This application relates to the field of urban three-dimensional computing services, and particularly to a method, device, equipment, medium and product for calculating the shape similarity of buildings for any single view. Background Art

[0002] Similarity measurement plays an important role in the fields of quantitative research, computer vision and pattern recognition, and is the core technology for realizing key downstream tasks such as target recognition, perception, and retrieval. With the acceleration of the urbanization process, in-depth exploration of the form and structure of buildings has become an important way to understand the urban landscape and architectural features.

[0003] In terms of the overall form of a building, as the interface where the building directly interacts with the external environment, the building facade is the most intuitive external manifestation of the building form, contains most of the elements of the building's overall form, and is an essential and important part in the process of its quantitative analysis.

[0004] However, current research on building shapes mostly focuses on planar shapes and spatial structures. Although current quantitative analysis of building forms has received some attention and research, it usually uses statistical parameter indicators or fuzzy, qualitative and subjective methods such as building styles to describe and evaluate building facades, and there are still relatively few quantitative theoretical methods for characterizing the geometric form and topological relationship of their facades. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, equipment, medium and product for calculating the shape similarity of buildings for any single view, which can realize the calculation of the similarity of building target shapes in any two single-view images of buildings under a unified standard.

[0006] To achieve the above purpose, this application provides the following solutions:

[0007] In the first aspect, this application provides a method for calculating the shape similarity of buildings for any single view, including:

[0008] Obtain a single-view image of a building, and extract the internal feature lines and external boundary feature lines of the building in the single-view image of the building;

[0009] Based on the internal feature lines and external boundary feature lines, obtain the three-dimensional shape features of the building in the single-view image of the building; form a three-dimensional shape feature pair of the three-dimensional shape features of the buildings in any two single-view images of buildings;

[0010] Input the three-dimensional shape feature pair into the similarity measurement model of building shape similarity, and use the similarity measurement model to calculate the shape similarity of the buildings in any two single-view images of buildings;

[0011] Among them, the similarity measurement model is obtained by iteratively training a preset deep learning network model using a sample data set, and the sample data set is constructed based on three-dimensional shape feature pairs of buildings in any two single-view images of buildings and the similarity between any two single-view images of buildings.

[0012] Optionally, the obtaining of the single-view image of the building and the extraction of the internal feature lines and external boundary feature lines of the building in the single-view image of the building specifically include:

[0013] Performing edge detection on the single-view image of the building to obtain an edge line pixel connected region;

[0014] Performing semantic segmentation on the single-view image of the building to obtain the pixel region of the building;

[0015] Obtaining the internal feature lines of the building according to the edge line pixel connected region and the pixel region of the building;

[0016] Determining the building area boundary according to the pixel region of the building to obtain the external boundary feature lines of the building.

[0017] Optionally, the obtaining of the three-dimensional shape features of the building in the single-view image of the building based on the internal feature lines and external boundary feature lines specifically includes:

[0018] Superimposing and merging the internal feature lines and external boundary feature lines to obtain a preliminary feature line set of the building;

[0019] Eliminating the preliminary feature lines or duplicate preliminary feature lines smaller than the threshold in the preliminary feature line set to obtain a simplified feature line set;

[0020] Performing polyline splitting on the simplified feature lines in the simplified feature line set to obtain the split simplified feature line segments, simplified feature line endpoints, and the connection relationship between the split simplified feature line segments and simplified feature line endpoints;

[0021] Constructing a global three-dimensional shape feature map with the split simplified feature line endpoints as nodes and the split simplified feature line segments as edges;

[0022] According to the connection relationship between the split simplified feature line segments and simplified feature line endpoints, judging whether it is a non-planar intersection point. If it is a non-planar intersection point, then constructing local three-dimensional shape features based on the local estimation method of the vanishing point as the node attributes of the global three-dimensional shape feature map to obtain the three-dimensional shape features of the building in the single-view image of the building.

[0023] Optionally, the similarity metric model includes a vector extraction structure and a similarity metric structure;

[0024] The vector extraction structure obtains two metric vectors based on the three-dimensional shape feature pair, and transmits the two metric vectors to the similarity metric structure;

[0025] The similarity metric structure interacts with the two received metric vectors, and uses the metric result obtained by the interaction as the predicted local partition shape similarity; the predicted local partition shape similarity relationship is obtained based on the predicted local partition shape similarity;

[0026] Based on the predicted local partition shape similarity relationship, the global shape similarity relationship is calculated to obtain the predicted global shape similarity as the building shape similarity.

[0027] Optionally, before inputting the three-dimensional shape feature pair into the similarity metric model of the building shape similarity and calculating the building shape similarity in any two single-view images of the building using the similarity metric model, it further includes: constructing the global shape similarity and local shape similarity of the single-view image of the building based on the three-dimensional shape feature pair;

[0028] The global shape similarity and local shape similarity are used as labels of the similarity metric model, and the similarity metric model is optimized with the goal of minimizing the difference between the local shape similarity and the predicted local partition shape similarity, and between the global shape similarity and the predicted global shape similarity.

[0029] Optionally, the constructing the global shape similarity and local shape similarity of the single-view image of the building based on the three-dimensional shape feature pair specifically includes:

[0030] Calculating the three-dimensional shape feature pair through the WL graph kernel, and using the obtained similarity value as the global shape similarity;

[0031] Determine the building target area according to the external boundary feature line, and calculate the circumscribed rectangle of the building target area;

[0032] Divide the circumscribed rectangle into 9 shape sub-blocks, and calculate the WL similarity between each shape sub-block of the three-dimensional shape feature pair according to the WL graph kernel respectively to obtain the WL similarity matrix;

[0033] Perform dimensionality transformation on the WL similarity matrix to obtain the WL similarity matrix in the form of an 81-dimensional vector as the local shape similarity.

[0034] In a second aspect, the present application provides a device for calculating the building shape similarity for any single view, including:

[0035] A feature line acquisition module, configured to acquire a single-view image of a building and extract internal feature lines and external boundary feature lines of the building in the single-view image of the building;

[0036] A feature pair acquisition module, configured to acquire three-dimensional shape features of the building in the single-view image of the building based on the internal feature lines and the external boundary feature lines; and form a three-dimensional shape feature pair from the three-dimensional shape features of the building in any two single-view images of the building;

[0037] A building shape similarity calculation module, configured to input the three-dimensional shape feature pair into a similarity measurement model of building shape similarity, and use the similarity measurement model to calculate the building shape similarity between any two single-view images of the building;

[0038] Wherein, the similarity measurement model is obtained by iteratively training a preset deep learning network model using a sample data set, and the sample data set is constructed based on three-dimensional shape feature pairs formed by three-dimensional shape features of the building in any two single-view images of the building, and the similarity between the buildings in any two single-view images of the building.

[0039] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for calculating the building shape similarity for any single view described in any one of the above.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for calculating the building shape similarity for any single view described in any one of the above are implemented.

[0041] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method for calculating the building shape similarity for any single view described in any one of the above are implemented.

[0042] According to the specific embodiments provided by the present application, the following technical effects are disclosed in the present application:

[0043] The present application provides a method, apparatus, device, medium and product for calculating the shape similarity of buildings for any single view. By obtaining the single-view image of the building and extracting the internal feature lines and external boundary feature lines, using image processing techniques, the key information that can characterize the shape of the building is effectively extracted from a single image. By capturing the internal structure and external contour of the building, the present application realizes the comprehensive extraction of the shape features of the building, providing a rich data source for the calculation of shape similarity. Based on the extracted internal feature lines and external boundary feature lines, the three-dimensional shape features of the building are constructed, and these features are input into a pre-trained similarity metric model for similarity calculation, realizing the conversion from a two-dimensional image to three-dimensional shape features and the mapping from three-dimensional shape features to shape similarity scores. The similarity metric model is iteratively trained with a large number of sample data, and can automatically learn and capture the subtle differences between the building shapes, so as to accurately calculate the shape similarity of the buildings in any two single-view images.

[0044] In summary, the present application realizes the extraction of the key shape features of the building from a single-view image, solves the problem that it is difficult to accurately calculate the shape similarity under a single-view image, and realizes fast and accurate similarity evaluation based on deep learning technology. It not only broadens the application scenarios of calculating the shape similarity of buildings, but also provides strong technical support for fields such as urban planning, architectural design, and image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0046] Figure 1 It is an application environment diagram of a method for calculating the shape similarity of buildings for any single view in an embodiment of the present application;

[0047] Figure 2 It is a schematic flowchart of a method for calculating the shape similarity of buildings for any single view provided in an embodiment of the present application;

[0048] Figure 3 It is a process diagram of extracting building feature lines in any two single-view images of buildings provided in an embodiment of the present application;

[0049] Figure 4 It is a process diagram of extracting building shape features in any two single-view images of buildings provided in an embodiment of the present application;

[0050] Figure 5Schematic diagram of functional modules of a building shape similarity calculation device for any single view provided by an embodiment of the present application;

[0051] Figure 6 Process diagram of constructing a similarity measurement model provided by an embodiment of the present application;

[0052] Figure 7 Schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0053] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0054] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the drawings and specific implementation manners.

[0055] The method for calculating the building shape similarity for any single view provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set separately, integrated on the server 104, placed on the cloud or other servers. The terminal 102 can send the single-view image of the building to the server 104. After receiving the single-view image of the building, for the single-view image of the building, the server 104 extracts the internal feature lines and external boundary feature lines of the building in the single-view image of the building; based on the internal feature lines and external boundary feature lines, obtains the three-dimensional shape features of the building in the single-view image of the building; forms a three-dimensional shape feature pair from the three-dimensional shape features of the building in any two single-view images of the building; inputs the three-dimensional shape feature pair into the similarity measurement model of the building shape similarity, and uses the similarity measurement model to calculate the building shape similarity between any two single-view images of the building; wherein, the similarity measurement model is obtained by iteratively training a preset deep learning network model using a sample data set, and the sample data set is constructed based on the three-dimensional shape feature pairs formed by the three-dimensional shape features of the building in any two single-view images of the building, and the similarity between the buildings in any two single-view images of the building. The server 104 can feedback the obtained building shape similarity to the terminal 102. In addition, in some embodiments, the method for calculating the building shape similarity for any single view can also be implemented separately by the server 104 or the terminal 102. For example, the terminal 102 can directly process the single-view image of the building, or the server 104 can obtain the single-view image of the building from the data storage system and process the single-view image of the building.

[0056] In an exemplary embodiment, as Figure 2 shown, a method for calculating the building shape similarity for any single view is provided. This method is executed by a computer device, and can be specifically executed separately by a computer device such as a terminal or a server, or jointly executed by a terminal and a server. In the embodiments of the present application, taking this method applied to Figure 1 the server 104 in

[0057] Step 201, obtain a single-view image of a building, and extract the internal feature lines and external boundary feature lines of the building in the single-view image of the building.

[0058] Step 202, obtain the three-dimensional shape features of the building in the single-view image of the building based on the internal feature lines and external boundary feature lines; form a three-dimensional shape feature pair from the three-dimensional shape features of the building in any two single-view images of the building.

[0059] Step 203: input the three-dimensional shape feature pair into a similarity measurement model of building shape similarity, and use the similarity measurement model to calculate the building shape similarity in any two single-view images of the building.

[0060] Among them, the similarity measurement model is obtained by iteratively training a preset deep learning network model using a sample data set, and the sample data set is constructed based on a three-dimensional shape feature pair composed of the three-dimensional shape features of the buildings in any two single-view images of the buildings, and the similarity of the buildings in any two single-view images of the buildings.

[0061] By implementing the above steps 201 to 203 , it is possible to calculate the similarity of the shapes of building targets in any two single-view images of the building.

[0062] Among them, implementing this embodiment, such as Figure 3 As shown, obtaining a single-view image of a building, and extracting the internal feature lines and the external boundary feature lines of the building in the single-view image of the building, specifically includes:

[0063] Based on RINDNet (Edge Detection for Discontinuity in Reflectance, Illumination, Normal and Depth), edge detection is performed on the single-view image of the building to obtain the edge line pixel connected area, which serves as the basis for the subsequent extraction of linear information of the building feature lines.

[0064] Based on FasterRCNN+SAM (Faster Region-based Convolutional Neural Network, fast regional convolutional neural network; Semantic Approximation Module, semantic approximation module), semantic segmentation of building areas in single-view images of buildings is performed to obtain the pixel area of ​​the building; semantic recognition of building areas in single-view images of buildings is achieved, providing a semantic basis for the subsequent recognition of building feature lines.

[0065] According to the edge line pixel connected area and the pixel area of ​​the building, the internal characteristic line of the building is obtained; according to the pixel area of ​​the building, the building area boundary is determined to obtain the external boundary characteristic line of the building.

[0066] The internal feature lines and the external boundary feature lines are superimposed and merged to obtain a preliminary feature line set of the building; preliminary feature lines smaller than a threshold or repeated preliminary feature lines in the preliminary feature line set are eliminated to obtain a simplified feature line set.

[0067] Perform a polyline split on the simplified feature lines in the simplified feature line set to obtain the split simplified feature line segments, the endpoints of the simplified feature lines, and the connection relationships between the split simplified feature line segments and the endpoints of the simplified feature lines.

[0068] In another exemplary embodiment of the present application, as Figure 4 shown, taking the endpoints of the split simplified feature lines as nodes and the split simplified feature line segments as edges, construct a global three-dimensional shape feature map to represent the global structure of the building shape in the form of graph data.

[0069] The present application uses a two-intersecting-line exhaustive search method for image vanishing point estimation. The main principle of this method is to first obtain the first vanishing point of the image using two parallel lines in the image, then uniformly extract the second vanishing point samples on the equivalent sphere, and finally calculate the third vanishing point through the cross product of the first and second vanishing points, and realize the optimization estimation of the vanishing point and camera parameters for any single view through the iterative process of the assumed camera parameters in the algorithm. Among them, it is assumed that the point in the image is the camera origin, the initial focal length is 50 pixels, and the iteration interval is 50 pixels. Through the iterative operation of this method, a set of orthogonal vanishing point groups vp1(v1, u1), vp2(v2, u2), vp3(v3, u3) can be obtained each time. Among them, v1 represents the horizontal coordinate of the vanishing point vp1 in the image, u1 represents the vertical coordinate of the vanishing point vp1 in the image; v2 represents the horizontal coordinate of the vanishing point vp2 in the image, u2 represents the vertical coordinate of the vanishing point vp2 in the image; v3 represents the horizontal coordinate of the vanishing point vp3 in the image, u3 represents the vertical coordinate of the vanishing point vp3 in the image.

[0070] The specific grouping of the feature line directions in the present application is as follows: Connect the midpoints of all line segments in the extracted building feature line group to the three vanishing points, calculate the angles between the line segments and the connecting lines, and record the angles between each line segment and the vanishing points as R = [r1, r2, r3]. Take the minimum angle in R as r. If r is less than 0.15, it is considered that the line segment is orthogonal to the direction of the vanishing point. Otherwise, it will be marked as an unknown direction. Finally, select the minimum number of iterations of the average value of each angle r in each orthogonal direction, regard the vanishing point group obtained in this iteration as the best vanishing point group of the image, and retain the iterative generated camera parameters and the direction categories of the line segments.

[0071] The intersection points are extracted as the endpoints of the feature lines obtained according to the building shape information and their belonging relationships, and the information such as the vanishing points and the directions of the feature lines in the acquired images. Among all the endpoints of the building feature lines, the three-dimensional geometric intersection points on the building shape are screened out, and the planar edge intersection points that do not have three-dimensional space features due to texture materials on the building appearance plane are removed. Specifically, first, the endpoints belonging to multiple branches are extracted into the candidate intersection point list of the building shape. Then, for any candidate intersection point, according to the recorded belonging branch numbers, the directions of all the line segments in its intersection line list are obtained from the line segment direction category vector. If there are only line segments with the same known direction in the intersection line directions, then this intersection point is removed from the candidate intersection point set, the intersection points of the same known direction intersection lines are removed, and the misaligned intersection points caused by detection and occlusion of non-planar structures are eliminated. After the above screening is completed, only the intersection points obtained by the intersection of two or more known direction or unknown direction line segments are retained in the candidate intersection points as the final geometric intersection point set of the building shape.

[0072] According to the connection relationship between the simplified feature line segments and the endpoints of the simplified feature lines after splitting, it is judged whether it is a non-planar intersection point. If it is a non-planar intersection point, then a local three-dimensional shape feature is constructed based on the local estimation method of the vanishing point as the node attribute of the global three-dimensional shape feature map, forming a building global shape feature expression with three-dimensional invariance, and obtaining the three-dimensional shape feature of the building in the single-view image of the building.

[0073] In another exemplary embodiment of the present application, a building shape similarity measurement model based on the three-dimensional shape feature of the building is constructed, and a similarity measurement model for the three-dimensional shape feature of the building is built. Based on the measurement relationship between the structures of the three-dimensional shape feature of the building itself, the domain knowledge of the measurement relationship between the three-dimensional shapes of the building characterized by the features is used to construct a weakly supervised learning method with inaccurate labels. Then, through the formulation of the model training rules, the model is guided by the large-scale sample label knowledge to establish the similarity measurement parameters and mapping relationships that conform to the similarity between the building shapes.

[0074] As Figure 5 shown, the similarity measurement model includes a vector extraction structure and a similarity measurement structure; the vector extraction structure obtains two measurement vectors according to the pair of three-dimensional shape features, and transmits the two obtained measurement vectors to the similarity measurement structure.

[0075] The similarity measurement structure interacts with the two received measurement vectors, and takes the measurement result obtained by the interaction as the predicted local partition shape similarity; the predicted local partition shape similarity relationship is obtained according to the predicted local partition shape similarity.

[0076] Calculate the global shape similarity relationship based on the predicted local partition shape similarity relationship to obtain the predicted global shape similarity as the building shape similarity.

[0077] Among them, to implement this implementation mode, first, in the preparation stage of the vector extraction structure, the three-dimensional shape features of the input building target are globally learned and dimensionally transformed through a layer of GCN (Graph Convolutional Network). In the second layer, the local shape information with obvious features in the building target shape is strengthened through GAT (Graph Attention Network) convolution, providing more prominent feature shape information for the target during global measurement and increasing the feature expression of the local shape during local measurement. Then, the global comprehensive information is obtained through GCN to further learn the features of the data. When extracting the global vector, the global average pooling structure is used to obtain the feature vectors of the input graph at different levels, and finally, the multi-level feature vectors are superimposed based on the local feature enhancement information.

[0078] The similarity measurement module takes the two measurement vectors obtained after the three-dimensional shape features of the two building targets pass through the vector extraction module as input. In the measurement process, the NTN (Neural Tensor NetWork) interaction structure proposed in the Similarity Graph Neural Network (SimGNN) is introduced. The measurement result obtained from the first interaction of the two targets in the module is used as the predicted local partition shape similarity between the two targets and participates in the model learning and training at the same time. Then, based on the obtained predicted local partition shape similarity relationship, the global shape similarity relationship between the two targets is gradually learned and deduced to obtain the predicted global shape similarity as the building shape similarity.

[0079] Among them, implementing this implementation mode also includes: constructing the global shape similarity and local shape similarity of the single-view image of the building based on the three-dimensional shape features.

[0080] The global shape similarity and local shape similarity are used as the labels of the similarity measurement model, and the similarity measurement model is optimized with the goal of minimizing the local shape similarity and the predicted local partition shape similarity, and the global shape similarity and the predicted global shape similarity.

[0081] In another exemplary embodiment of the present application, to construct the global shape similarity and local shape similarity of the single-view image of the building based on the three-dimensional shape features, it further includes:

[0082] In the weakly supervised learning method of inaccurate labels, two parts of labels, namely global shape similarity and local shape similarity, are constructed based on the WL (Weisfiler-Lehman) graph kernel metric results of three-dimensional shape feature pairs.

[0083] For the global shape similarity label, the WL graph kernel is used to calculate the three-dimensional shape feature pairs, and the obtained similarity value is used as the global shape similarity.

[0084] For the local shape similarity, the target area of the building is determined according to the external boundary feature lines, and the circumscribed rectangle of the building target area is calculated; the circumscribed rectangle is divided into 9 shape sub-blocks, and the WL similarity between each shape sub-block of the three-dimensional shape feature pairs is calculated respectively according to the WL graph kernel to obtain the WL similarity matrix; the dimension of the WL similarity matrix is transformed to obtain the WL similarity matrix in the form of an 81-dimensional vector as the local shape similarity.

[0085] During the training process, in order to ensure the randomness of the sample composition and avoid overfitting learning of the model to a certain type of label, the present application adopts a proportional uniform mixing method for the design of the strong and weak label combination rules. That is, after determining the mixing ratio, first randomly shuffle the images in the dataset, and form weak sample pairs by pairing each uncombined image with the image after it. Assuming a 1:1 mixing case, then the next group of each group of weak samples is a strong sample pair composed of the image after the uncombined image. Then, during the training process, the model is trained in the order of the samples formed.

[0086] The present application also provides an application scenario, which applies the above-mentioned method for calculating the building shape similarity for any single view. Specifically: The method for calculating the building shape similarity for any single view provided in this embodiment can be applied to the building comparison and analysis scenario in the field of urban planning and management. This scenario generally includes four main links: data collection, feature extraction, similarity calculation, and decision-making analysis. In the data collection link, the system obtains a large number of single-view images of buildings through methods such as drones, satellite images, or ground photography. These images constitute the basic data for subsequent analysis. Subsequently, entering the feature extraction link, using the method proposed in the present application, the internal feature lines and external boundary feature lines of the building are automatically extracted from each single-view image, and further three-dimensional shape features are generated. These features accurately and comprehensively reflect the shape characteristics of the building. In the similarity calculation link, the three-dimensional shape features of any two buildings are formed into a feature pair and input into a pre-trained similarity measurement model. The model quickly and accurately calculates the shape similarity of the buildings in these two images through deep learning algorithms and outputs the building shape similarity. Finally, in the decision-making analysis link, based on the similarity calculation results, urban planners or managers can conduct in-depth comparative analysis of the buildings. For example, in the old city renovation project, the shape similarity between the new design plan and the existing buildings can be calculated to evaluate whether the design plan is coordinated with the surrounding environment; in the disaster risk assessment, potential vulnerable areas or structural defects can be identified by comparing the building shape features.

[0087] The method for calculating the building shape similarity for any single view provided in this embodiment is a key link in the building comparison and analysis scenario for feature extraction and similarity calculation. Through automated feature extraction and efficient similarity calculation, this method can significantly improve the efficiency and accuracy of urban planning and management work, providing strong support for the development and safety of the city. In the specific implementation process, according to different application scenarios and requirements, the specific implementation method and parameter settings of the method can be flexibly adjusted to achieve the best analysis effect.

[0088] Based on the same inventive concept, the embodiments of the present application also provide a device for implementing the above-mentioned method for calculating the building shape similarity for any single view. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the device for calculating the building shape similarity for any single view provided below can refer to the limitations on the method for calculating the building shape similarity for any single view in the above text and will not be elaborated here.

[0089] In an exemplary embodiment, as Figure 6 shown, a device for calculating the building shape similarity for any single view is provided, including:

[0090] A feature line acquisition module 301 is configured to acquire a single-view image of a building and extract internal feature lines and external boundary feature lines of the building in the single-view image of the building.

[0091] A feature pair acquisition module 302 is configured to acquire three-dimensional shape features of the building in the single-view image of the building based on the internal feature lines and the external boundary feature lines; and form a three-dimensional shape feature pair with the three-dimensional shape features of the building in any two single-view images of the building.

[0092] A building shape similarity calculation module 303 is configured to input the three-dimensional shape feature pair into a similarity measurement model of building shape similarity, and calculate the building shape similarity between any two single-view images of the building by using the similarity measurement model.

[0093] Wherein, the similarity measurement model is obtained by iteratively training a preset deep learning network model with a sample data set, and the sample data set is constructed based on three-dimensional shape feature pairs composed of three-dimensional shape features of the building in any two single-view images of the building, and the similarity between the buildings in any two single-view images of the building.

[0094] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as shown in Figure 7 the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store single-view image processing data of the building. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for calculating the similarity of the shape of a building for any single view.

[0095] Those skilled in the art can understand that Figure 7The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0096] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0097] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0098] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0099] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random-access memories (ReRAM), magnetoresistive random-access memories (MRAM), ferroelectric random-access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0100] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0101] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0102] In this article, specific examples are used to elaborate on the principles and implementation manners of the present application. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for calculating building shape similarity for any single view, characterized in that: The building shape similarity calculation method for any single view includes: Acquire a single-view image of a building, and extract internal feature lines and external boundary feature lines of the building in the single-view image of the building; The three-dimensional shape features of the building in the single-view image of the building are obtained based on the internal feature lines and the external boundary feature lines, specifically including: superimposing and merging the internal feature lines and the external boundary feature lines to obtain a preliminary feature line set of the building; eliminating preliminary feature lines or repeated preliminary feature lines that are less than a threshold in the preliminary feature line set to obtain a simplified feature line set; performing polyline splitting on the simplified feature lines in the simplified feature line set to obtain simplified feature line segments after splitting, simplified feature line endpoints, and a connection relationship between the simplified feature line segments after splitting and the simplified feature line endpoints after splitting; constructing a global three-dimensional shape feature map with the simplified feature line endpoints after splitting as nodes and the simplified feature line segments after splitting as edges; judging whether it is a non-planar intersection according to the connection relationship between the simplified feature line segments after splitting and the simplified feature line endpoints, if it is a non-planar intersection, constructing a local three-dimensional shape feature based on a local estimation method of a vanishing point as a node attribute of the global three-dimensional shape feature map, and obtaining the three-dimensional shape features of the building in the single-view image of the building; forming a three-dimensional shape feature pair with the three-dimensional shape features of the building in any two single-view images of the building; Inputting the three-dimensional shape feature pair into a similarity measurement model of building shape similarity, and using the similarity measurement model to calculate the building shape similarity in any two single-view images of the building; Among them, the similarity measurement model is obtained by iteratively training a preset deep learning network model using a sample data set, and the sample data set is constructed based on a three-dimensional shape feature pair composed of the three-dimensional shape features of the buildings in any two single-view images of the buildings, and the similarity of the buildings in any two single-view images of the buildings.

2. The method for calculating building shape similarity based on any single view according to claim 1, characterized in that: The acquiring of a single-view image of a building and extracting internal feature lines and external boundary feature lines of the building in the single-view image of the building specifically includes: Perform edge detection on the single-view image of the building to obtain the edge line pixel connected area; Perform semantic segmentation of the building area on the single-view image of the building to obtain the pixel area of ​​the building; According to the edge line pixel connected area and the building pixel area, the internal characteristic line of the building is obtained; The building area boundary is determined according to the pixel area of ​​the building, and the external boundary feature line of the building is obtained.

3. The method for calculating building shape similarity based on any single view according to claim 1, characterized in that: The similarity measurement model includes a vector extraction structure and a similarity measurement structure; The vector extraction structure obtains two measurement vectors according to the three-dimensional shape feature pair, and transmits the two measurement vectors to the similarity measurement structure; The similarity measurement structure interacts with the two received measurement vectors, and uses the measurement result obtained by the interaction as the predicted local partition shape similarity; and obtains the predicted local partition shape similarity relationship according to the predicted local partition shape similarity; The global shape similarity relationship is calculated based on the predicted local partition shape similarity relationship to obtain the predicted global shape similarity as the building shape similarity.

4. The method for calculating building shape similarity based on any single view according to claim 1, characterized in that: The step of inputting the three-dimensional shape feature pair into a similarity measurement model of building shape similarity and using the similarity measurement model to calculate the building shape similarity in any two single-view images of the building also includes: constructing a global shape similarity and a local shape similarity of the single-view images of the building based on the three-dimensional shape feature pair; The global shape similarity and local shape similarity are used as labels of the similarity measurement model, and the similarity measurement model is optimized with the goal of minimizing the local shape similarity and the predicted local partition shape similarity, and the global shape similarity and the predicted global shape similarity.

5. The method for calculating building shape similarity based on any single view according to claim 4, characterized in that: The constructing the global shape similarity and the local shape similarity of the single view image of the building based on the three-dimensional shape feature pair specifically includes: The three-dimensional shape feature pairs are calculated through the WL graph, and the obtained similarity value is used as the global shape similarity; Determine the building target area according to the external boundary feature line, and calculate the circumscribed rectangle of the building target area; The bounding rectangle is divided into 9 shape sub-blocks, and the WL similarity between each shape sub-block of the 3D shape feature pair is calculated according to the WL kernel to obtain the WL similarity matrix; The WL similarity matrix is ​​transformed in dimension to obtain a WL similarity matrix in the form of an 81-dimensional vector as the local shape similarity.

6. A device for calculating building shape similarity for any single view, characterized in that: The building shape similarity calculation device for any single view comprises: A feature line acquisition module, used to acquire a single-view image of a building and extract internal feature lines and external boundary feature lines of the building in the single-view image of the building; The feature pair acquisition module is used to acquire the three-dimensional shape features of the building in the single-view image of the building based on the internal feature lines and the external boundary feature lines, specifically comprising: superimposing and merging the internal feature lines and the external boundary feature lines to obtain a preliminary feature line set of the building; eliminating preliminary feature lines or repeated preliminary feature lines that are less than a threshold value in the preliminary feature line set to obtain a simplified feature line set; performing polyline splitting on the simplified feature lines in the simplified feature line set to obtain simplified feature line segments after splitting, simplified feature line endpoints, and a connection relationship between the simplified feature line segments after splitting and the simplified feature line endpoints after splitting; constructing a global three-dimensional shape feature map with the simplified feature line endpoints after splitting as nodes and the simplified feature line segments after splitting as edges; judging whether it is a non-planar intersection according to the connection relationship between the simplified feature line segments after splitting and the simplified feature line endpoints, if it is a non-planar intersection, constructing a local three-dimensional shape feature based on a local estimation method of a vanishing point as a node attribute of the global three-dimensional shape feature map to obtain the three-dimensional shape features of the building in the single-view image of the building; forming a three-dimensional shape feature pair with the three-dimensional shape features of the building in any two single-view images of the building; A building shape similarity calculation module, used for inputting the three-dimensional shape feature pair into a similarity measurement model of building shape similarity, and calculating the building shape similarity in any two single-view images of the building using the similarity measurement model; Among them, the similarity measurement model is obtained by iteratively training a preset deep learning network model using a sample data set, and the sample data set is constructed based on a three-dimensional shape feature pair composed of the three-dimensional shape features of the buildings in any two single-view images of the buildings, and the similarity of the buildings in any two single-view images of the buildings.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the building shape similarity calculation method for any single view described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for calculating the building shape similarity for any single view described in any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for calculating the building shape similarity for any single view described in any one of claims 1 to 5 is implemented.

Images