Data processing method and system for real estate surveying and mapping

Abstract

The present application relates to the field of real estate surveying and mapping technology, and specifically discloses a data processing method and system for real estate surveying and mapping, which respectively collects aerial images and ground measurement data of real estate to be modeled by drones and surveying and mapping equipment, and uses artificial intelligence technology based on deep learning to process aerial images and ground measurement data, respectively extracts multi-level joint features of aerial images and high-dimensional semantic embedding features of ground measurement data, and then performs cross-modal gated joint encoding on the two to comprehensively mine the spatial layout and structural information of the real estate to be modeled, thereby intelligently generating real estate surveying and mapping renderings. In this way, the efficiency and accuracy of real estate surveying and mapping can be effectively improved, the dependence on manual operations can be reduced, and the automation and intelligence of the surveying and mapping process can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of real estate surveying and mapping, and more specifically, to a data processing method and system for real estate surveying and mapping. Background Art

[0002] With the acceleration of urbanization and the booming real estate market, the demand for real estate surveying and mapping is growing. Traditional real estate surveying and mapping relies on manual on-site measurement and hand-drawn drawings, which is not only time-consuming and inefficient, but also prone to human errors, resulting in inaccurate surveying and mapping results. In addition, under complex terrain conditions, traditional methods are difficult to achieve all-round and multi-angle measurements, especially in high-rise buildings, densely populated residential areas or areas with complex terrain. These problems are particularly prominent.

[0003] In recent years, with the rapid development of drone technology, remote sensing technology and artificial intelligence, real estate surveying and mapping technology has ushered in new developments. Nowadays, aerial photography using drones has become an effective means of quickly acquiring geographic information. Drones can carry high-definition cameras or other sensor devices to shoot target areas from the air, thereby obtaining high-resolution remote sensing images. However, image data obtained by drones alone is difficult to accurately reflect the detailed information of ground buildings, especially for key attributes such as building size and height. Therefore, combining ground measurement data to fuse real estate multimodal information has become an important way to improve the accuracy of real estate surveying and mapping.

[0004] However, since drone aerial photography data and ground measurement data have different forms of expression, there is often a certain information gap between the two. How to effectively integrate these two different types of data to obtain more accurate real estate space information has become a technical problem that needs to be solved urgently.

[0005] Therefore, an intelligent data processing method for real estate surveying and mapping is expected. Summary of the invention

[0006] The present application provides a data processing method and system for real estate surveying and mapping, which can effectively improve the efficiency and accuracy of real estate surveying and mapping, reduce dependence on manual operations, and improve the automation and intelligence of the surveying and mapping process.

[0007] In a first aspect, a data processing method for real estate surveying and mapping comprises:

[0008] Acquire aerial images of the property to be modeled collected by a drone, and ground measurement data of the property to be modeled collected by a surveying and mapping device;

[0009] Performing multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial view of the real estate to be modeled;

[0010] Using a ground measurement embedding coding matrix, high-dimensional embedding coding is performed on the ground measurement data of the property to be modeled to obtain a ground measurement high-dimensional embedding coding vector;

[0011] Performing cross-modal gated joint encoding on the aerial photography perspective fine-grained interactive saliency fusion feature map of the property to be modeled and the ground measurement high-dimensional embedding coding vector to obtain an aerial photography-measurement multi-modal fusion feature map of the property to be modeled;

[0012] Generate a real estate surveying and mapping rendering based on the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled;

[0013] The fine-grained interactive fusion feature map of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement are cross-modal gated joint encoding to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, including:

[0014] Performing local semantic enhancement based on the self-attention mechanism on the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled to obtain a local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled;

[0015] The local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement are subjected to cross-modal conditional fusion based on a gating mechanism to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

[0016] In a possible implementation, multi-scale feature extraction and multi-scale feature fine-grained joint perception are performed on the aerial image of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial view of the real estate to be modeled, including:

[0017] Inputting the aerial image of the real estate to be modeled into a multi-scale image feature extractor based on a dilated pyramid network to obtain a shallow feature map of the aerial perspective of the real estate to be modeled and a deep feature map of the aerial perspective of the real estate to be modeled;

[0018] Fine-grained bidirectional global attention fusion is performed on the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled.

[0019] In a possible implementation, fine-grained bidirectional global attention fusion is performed on the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled to obtain a fine-grained interactive salient fusion feature map of the aerial perspective of the real estate to be modeled, including:

[0020] Performing feature fine-grained deconstruction on the shallow feature map of the aerial photography perspective of the real estate to be modeled and the deep feature map of the aerial photography perspective of the real estate to be modeled to obtain a set of local feature matrices of shallow channel dimensions of the aerial photography perspective of the real estate and a set of local feature matrices of deep channel dimensions of the aerial photography perspective of the real estate;

[0021] Performing global attention interactive optimization on the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective to obtain a set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective optimized by one-way global attention and a set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective optimized by one-way global attention;

[0022] The set of local feature matrices of the shallow channel dimension of the one-way global attention optimization of real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the one-way global attention optimization of real estate aerial photography perspective are respectively coupled along the channel dimension to obtain a shallow feature map of the one-way global interaction optimization of real estate aerial photography perspective and a deep feature map of the one-way global interaction optimization of real estate aerial photography perspective;

[0023] The position-weighted sum between the one-way global interactive optimization of the real estate aerial photography perspective shallow feature map and the one-way global interactive optimization of the real estate aerial photography perspective deep feature map is calculated to obtain the fine-grained interactive significant fusion feature map of the real estate aerial photography perspective to be modeled.

[0024] In a possible implementation, the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective are subjected to global attention interactive optimization to obtain a set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective optimized by one-way global attention and a set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective optimized by one-way global attention, including:

[0025] Using each local feature matrix of the shallow channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective as a query feature matrix, and using the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a first converter structure to obtain the one-way global attention optimized set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective;

[0026] Each local feature matrix of the deep channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective is used as a query feature matrix, and the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a unidirectional global attention interaction module based on a second converter structure to obtain the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective optimized by the unidirectional global attention.

[0027] In a possible implementation, each local feature matrix of the shallow channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective is used as a query feature matrix, and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a unidirectional global attention interaction module based on a first converter structure to obtain the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective optimized by the unidirectional global attention, including:

[0028] Calculate the matrix product of the query feature matrix and the transposed matrix of each key feature matrix in the set of key feature matrices respectively, and then divide the matrix product by the square root of the scale of the key feature matrix and pass it through a softmax function to obtain a set of attention weight matrices;

[0029] Each attention weight matrix in the set of attention weight matrices is multiplied with the query feature matrix respectively, and then the multiplication results are summed by position point to obtain the one-way global attention optimized real estate aerial photography perspective shallow channel dimension local feature matrix.

[0030] In a possible implementation, the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled is subjected to local semantic enhancement based on a self-attention mechanism to obtain a local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled, including:

[0031] Performing point convolution encoding and spatial convolution encoding on the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled to obtain a spatial deepening feature map of the aerial photography perspective of the real estate to be modeled;

[0032] Constructing a pseudo-query feature matrix and a pseudo-key feature matrix based on the aerial photography perspective spatial deepening feature graph of the real estate to be modeled;

[0033] The product of the pseudo-query feature matrix and the transposed matrix of the pseudo-key feature matrix is ​​calculated to obtain the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled.

[0034] In a possible implementation, constructing a pseudo-query feature matrix and a pseudo-key feature matrix based on the aerial photography perspective space deepening feature map of the real estate to be modeled includes:

[0035] Copy the aerial perspective spatial deepening feature map of the real estate to be modeled to obtain a backup aerial perspective spatial deepening feature map of the real estate to be modeled;

[0036] The feature shapes of the backup aerial perspective space deepening feature map of the real estate to be modeled and the aerial perspective space deepening feature map of the real estate to be modeled are reshaped respectively to obtain the backup aerial perspective space deepening feature matrix of the real estate to be modeled and the aerial perspective space deepening feature matrix of the real estate to be modeled as the simulated query feature matrix and the simulated key feature matrix.

[0037] In a possible implementation, the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement are subjected to cross-modal conditional fusion based on a gating mechanism to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, including:

[0038] Input the local semantic autocorrelation library matrix of the aerial view of the real estate to be modeled into the attention gating unit based on the GELU function to obtain the weight matrix of the local semantic autocorrelation library of the aerial view of the real estate to be modeled;

[0039] Taking the ground measurement high-dimensional embedding coding vector as the value feature vector, calculating the product between the weight matrix of the local semantic autocorrelation library of the aerial photography perspective of the real estate to be modeled and the value feature vector to obtain a cross-modal interactive fusion condition weight vector;

[0040] The cross-modal interactive fusion condition weight vector is used as the weight vector, and the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled is weighted channel by channel to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

[0041] In a second aspect, a data processing system for real estate surveying and mapping is provided, comprising:

[0042] A data acquisition module, used to acquire aerial images of the real estate to be modeled collected by a drone, and ground measurement data of the real estate to be modeled collected by a surveying and mapping device;

[0043] A feature fine-grained joint perception module is used to perform multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial view of the real estate to be modeled;

[0044] A high-dimensional embedding coding module, used for performing high-dimensional embedding coding on the ground measurement data of the property to be modeled using a ground measurement embedding coding matrix to obtain a ground measurement high-dimensional embedding coding vector;

[0045] A cross-modal gated joint encoding module is used to perform cross-modal gated joint encoding on the aerial photography perspective fine-grained interactive significant fusion feature map of the real estate to be modeled and the ground measurement high-dimensional embedding coding vector to obtain an aerial photography-measurement multi-modal fusion feature map of the real estate to be modeled;

[0046] A real estate surveying and mapping rendering generation module is used to generate a real estate surveying and mapping rendering based on the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled;

[0047] Wherein, the cross-modal gated joint encoding module is used to:

[0048] Performing local semantic enhancement based on the self-attention mechanism on the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled to obtain a local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled;

[0049] The local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement are subjected to cross-modal conditional fusion based on a gating mechanism to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

[0050] In a possible implementation, the feature fine-grained joint perception module is used to:

[0051] Inputting the aerial image of the real estate to be modeled into a multi-scale image feature extractor based on a dilated pyramid network to obtain a shallow feature map of the aerial perspective of the real estate to be modeled and a deep feature map of the aerial perspective of the real estate to be modeled;

[0052] Fine-grained bidirectional global attention fusion is performed on the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled.

[0053] The present application provides a data processing method and system for real estate surveying and mapping, which respectively collects aerial images and ground measurement data of the real estate to be modeled by drones and surveying and mapping equipment, uses artificial intelligence technology based on deep learning to process the aerial images and ground measurement data, respectively extracts the multi-level joint features of the aerial images and the high-dimensional semantic embedding features of the ground measurement data, and then performs cross-modal gated joint encoding on the two to comprehensively mine the spatial layout and structural information of the real estate to be modeled, thereby intelligently generating real estate surveying and mapping renderings. In this way, the efficiency and accuracy of real estate surveying and mapping can be effectively improved, the dependence on manual operations can be reduced, and the automation and intelligence of the surveying and mapping process can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0055] Figure 1 This is a schematic flow chart of a data processing method for real estate surveying and mapping according to an embodiment of the present application.

[0056] Figure 2 A data flow diagram of a data processing method for real estate surveying and mapping according to an embodiment of the present application.

[0057] Figure 3 In the data processing method for real estate surveying in an embodiment of the present application, a schematic flowchart of performing multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled.

[0058] Figure 4 In the data processing method for real estate surveying and mapping of an embodiment of the present application, a schematic flowchart is provided for performing cross-modal gated joint encoding on the fine-grained interactive salient fusion feature map of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedded coding vector of the ground measurement to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

[0059] Figure 5 This is a schematic block diagram of a data processing system for real estate surveying according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0061] In recent years, with the advancement of science and technology, especially the rapid development of drones, remote sensing technology and AI, the real estate surveying and mapping industry is undergoing innovation. Nowadays, drones are widely used for high-altitude photography, and their high-definition cameras or other sensing devices can efficiently capture geographic information in a large area and form high-definition remote sensing images. However, the data collected by drones alone is not enough to describe the actual conditions of ground buildings in detail, especially in terms of specific parameters such as the size and height of the buildings. Therefore, combining the measurement data from the ground with the image data from drones and integrating multi-source information has become a key measure to improve the accuracy of real estate surveying and mapping.

[0062] However, there is an essential difference in the form of expression between the images taken by drones and the data measured on the ground, which has caused a certain information gap. How to effectively integrate these two different sources of data to obtain more accurate spatial information has become a major technical challenge that needs to be overcome.

[0063] In response to the above technical problems, the technical concept of this application is: using drones and surveying equipment to collect aerial images and ground measurement data of the real estate to be modeled, respectively, using deep learning-based artificial intelligence technology to process the aerial images and ground measurement data, respectively extracting the multi-level joint features of the aerial images and the high-dimensional semantic embedding features of the ground measurement data, and then performing cross-modal gated joint encoding on the two to fully mine the spatial layout and structural information of the real estate to be modeled, thereby intelligently generating real estate surveying and mapping renderings. In this way, the efficiency and accuracy of real estate surveying and mapping can be effectively improved, the dependence on manual operations can be reduced, and the automation and intelligence of the surveying and mapping process can be improved.

[0064] Figure 1 This is a schematic flow chart of a data processing method for real estate surveying and mapping according to an embodiment of the present application. Figure 2 This is a data flow diagram of the data processing method for real estate surveying and mapping according to an embodiment of the present application. Figure 1 and Figure 2As shown, the data processing method for real estate surveying and mapping includes: S1, obtaining an aerial image of the real estate to be modeled collected by an unmanned aerial vehicle, and ground measurement data of the real estate to be modeled collected by a surveying and mapping device; S2, performing multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled; S3, using a ground measurement embedding coding matrix to perform high-dimensional embedding coding on the ground measurement data of the real estate to be modeled to obtain a ground measurement high-dimensional embedding coding vector; S4, performing cross-modal gated joint coding on the fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled and the ground measurement high-dimensional embedding coding vector to obtain an aerial-measurement multi-modal fusion feature map of the real estate to be modeled; S5, generating a real estate surveying and mapping rendering based on the aerial-measurement multi-modal fusion feature map of the real estate to be modeled.

[0065] In the above-mentioned data processing method for real estate surveying and mapping, the S1 obtains the aerial image of the real estate to be modeled collected by the drone, and the ground measurement data of the real estate to be modeled collected by the surveying and mapping equipment. It should be understood that the aerial image and the ground measurement data respectively provide information of different dimensions required for real estate modeling. Aerial images are taken from the air by drones, which can cover a wide area and provide an intuitive display of the overall layout, surrounding environment and structural form of the real estate. The ground measurement data accurately measures the specific size, elevation, location and other physical properties of the real estate through professional surveying and mapping equipment (such as total station, laser rangefinder, GPS, etc.). The combination of the two can ensure the comprehensiveness and accuracy of real estate surveying and mapping.

[0066] In the above-mentioned data processing method for real estate surveying and mapping, the S2 performs multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain the fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled. It should be understood that by performing multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain the fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled, it can help us to more comprehensively understand and capture the detailed information in the aerial image. Specifically, multi-scale feature extraction can capture targets or objects of different sizes, such as different parts of a building, roads, vegetation, etc., which is crucial for understanding and depicting the overall layout of the real estate. Multi-scale feature fine-grained joint perception further enhances the effect of feature extraction, which not only focuses on global information, but also emphasizes the importance of local details. Through fine-grained perception, the system can identify and enhance those subtle features that are particularly important for real estate modeling, such as the location of doors and windows, the edges of buildings, etc. Such processing helps to generate more accurate and detailed real estate surveying and mapping renderings. By combining multi-scale feature extraction with fine-grained joint perception, we can ensure that the features extracted from aerial images not only cover the macroscopic architectural layout, but also retain the microscopic detail changes, thus providing a solid foundation for the subsequent generation of real estate surveying and mapping renderings. This ultimately improves the efficiency and accuracy of real estate surveying and mapping, reduces the need for manual intervention, and improves the automation and intelligence level of the entire process.

[0067] Optionally, in one embodiment of the present application, Figure 3 This is a schematic flow chart of the data processing method for real estate surveying in the embodiment of the present application, which performs multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain the fine-grained interactive significant fusion feature map of the aerial view of the real estate to be modeled. Figure 3 As shown, multi-scale feature extraction and multi-scale feature fine-grained joint perception are performed on the aerial image of the property to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the property to be modeled, including: S21, inputting the aerial image of the property to be modeled into a multi-scale image feature extractor based on a diaphanous pyramid network to obtain a shallow feature map of the aerial perspective of the property to be modeled and a deep feature map of the aerial perspective of the property to be modeled; S22, performing fine-grained bidirectional global attention fusion on the shallow feature map of the aerial perspective of the property to be modeled and the deep feature map of the aerial perspective of the property to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the property to be modeled.

[0068] In the above-mentioned data processing method for real estate surveying, the S21, the aerial image of the real estate to be modeled is input into a multi-scale image feature extractor based on a dilute pyramid network to obtain a shallow feature map of the aerial perspective of the real estate to be modeled and a deep feature map of the aerial perspective of the real estate to be modeled. It should be understood that, considering that the aerial images collected by drones often contain rich spatial information, including targets of different scales (such as buildings, roads, vegetation, etc.) and complex backgrounds. Therefore, in order to fully capture the real estate spatial layout information in the aerial image of the real estate to be modeled, the present application uses a multi-scale image feature extractor based on a dilute pyramid network to perform multi-level feature extraction on the aerial image of the real estate to be modeled. The dilute pyramid network (ASPP) can effectively extract feature representations of different scales by using convolution kernels of different apertures to perform sliding convolution processing on the image, capture the local edge detail information of the real estate to be modeled and the overall layout information of the building, thereby obtaining a shallow feature map of the aerial perspective of the real estate to be modeled and a deep feature map of the aerial perspective of the real estate to be modeled.

[0069] In the above-mentioned data processing method for real estate surveying and mapping, the S22 performs fine-grained bidirectional global attention fusion on the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled to obtain the fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled. It should be understood that traditional image multi-level feature fusion methods usually adopt simple splicing or positional addition, which cannot fully utilize the correlation dependency between features at different levels, resulting in poor fusion effect. In order to solve this problem, the present application proposes a fine-grained bidirectional attention fusion strategy, which introduces a bidirectional attention mechanism to coordinately optimize the correlation between the deep features and the shallow features of the real estate to be modeled, thereby achieving deep fusion of deep and shallow features.

[0070] Optionally, in one embodiment of the present application, fine-grained bidirectional global attention fusion is performed on the shallow feature map of the aerial photography perspective of the real estate to be modeled and the deep feature map of the aerial photography perspective of the real estate to be modeled to obtain the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled, including: fine-grained feature deconstruction of the shallow feature map of the aerial photography perspective of the real estate to be modeled and the deep feature map of the aerial photography perspective of the real estate to be modeled to obtain a set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective and a set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective; global attention interactive optimization is performed on the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective to obtain a unidirectional global attention optimized real estate aerial photography perspective. A set of local feature matrices of shallow channel dimensions of real estate aerial photography perspective and a set of local feature matrices of deep channel dimensions of real estate aerial photography perspective optimized by one-way global attention are prepared; the set of local feature matrices of shallow channel dimensions of real estate aerial photography perspective optimized by one-way global attention and the set of local feature matrices of deep channel dimensions of real estate aerial photography perspective optimized by one-way global attention are respectively coupled with features along the channel dimension to obtain a shallow feature map of real estate aerial photography perspective optimized by one-way global interaction and a deep feature map of real estate aerial photography perspective optimized by one-way global interaction; the position-weighted sum between the shallow feature map of real estate aerial photography perspective optimized by one-way global interaction and the deep feature map of real estate aerial photography perspective optimized by one-way global interaction is calculated to obtain the fine-grained interactive significant fusion feature map of real estate aerial photography perspective to be modeled.

[0071] Optionally, in one embodiment of the present application, the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective are subjected to global attention interactive optimization to obtain a set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective optimized by one-way global attention and a set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective optimized by one-way global attention, including: using each local feature matrix of the shallow channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective as a query feature matrix, using the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective as a set of key feature matrices, and converting the query feature matrix into a key feature matrix. The matrix and the set of the key feature matrix are input into the one-way global attention interaction module based on the first converter structure to obtain the set of the one-way global attention optimized real estate aerial photography perspective shallow channel dimension local feature matrix; each of the real estate aerial photography perspective deep channel dimension local feature matrices in the set of the real estate aerial photography perspective deep channel dimension local feature matrices is used as the query feature matrix, and the set of the real estate aerial photography perspective shallow channel dimension local feature matrices is used as the set of key feature matrices, and the set of the query feature matrix and the key feature matrix is ​​input into the one-way global attention interaction module based on the second converter structure to obtain the one-way global attention optimized real estate aerial photography perspective deep channel dimension local feature matrix set.

[0072] Optionally, in one embodiment of the present application, each local feature matrix of the shallow channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective is used as a query feature matrix, and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a one-way global attention interaction module based on a first converter structure to obtain the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective optimized by the one-way global attention, including: respectively calculating the matrix product of the query feature matrix and the transposed matrix of each key feature matrix in the set of key feature matrices, and then dividing the matrix product by the square root of the scale of the key feature matrix and passing it through a softmax function to obtain a set of attention weight matrices; respectively multiplying each attention weight matrix in the set of attention weight matrices with the query feature matrix, and then summing the multiplication results by position points to obtain the one-way global attention optimized local feature matrix of the shallow channel dimension of the real estate aerial photography perspective.

[0073] In summary, in the embodiment of the present application, the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled are processed by the following attention joint perception formula to obtain the fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled, wherein the attention joint perception formula is:

[0074] ;

[0075] in, represents the shallow feature map of the aerial view of the real estate to be modeled, represents the deep feature map of the aerial view of the real estate to be modeled, Indicates deconstruction processing, and They represent the first, second, and third shallow feature maps of the aerial photography perspective of the real estate to be modeled along the channel dimension. and The local feature matrix of shallow channel dimension of the aerial photography perspective of a real estate, The value of is the number of channels of the shallow feature map of the aerial photography perspective of the real estate to be modeled, They respectively represent the first, second, and third deep feature maps of the aerial photography perspective of the real estate to be modeled along the channel dimension. and The local feature matrix of the deep channel dimension of the real estate aerial photography perspective, represents the matrix multiplication operation, represents the scale of the local feature matrix of the deep channel dimension of the real estate aerial photography perspective, and the scale of the local feature matrix of the deep channel dimension of the real estate aerial photography perspective is the same as the scale of the local feature matrix of the shallow channel dimension of the real estate aerial photography perspective, represents the transpose of a matrix, is the normalized exponential function, and They respectively represent the first, second, and third local feature matrices in the set of the shallow channel dimension of the one-way global attention optimization real estate aerial photography perspective. and A one-way global attention optimizes the local feature matrix of the shallow channel dimension of the real estate aerial photography perspective, and They respectively represent the first, second, and third local feature matrices in the set of the one-way global attention optimization real estate aerial photography perspective deep channel dimension local feature matrix. and One-way global attention optimization of the local feature matrix of the deep channel dimension of the real estate aerial photography perspective, represents feature aggregation processing, Indicates the shallow feature map of the one-way global interactive optimization of the real estate aerial photography perspective, Indicates the deep feature map of the one-way global interactive optimization of real estate aerial photography perspective, a and b For different weight coefficients, It represents the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled.

[0076] Specifically, the shallow feature map of the aerial view of the property to be modeled and the deep feature map of the aerial view of the property to be modeled are firstly fine-grainedly deconstructed, and each feature map is decomposed into a set of local feature matrices of the channel dimension along the channel dimension, so as to analyze the local information in the feature map more carefully. Then, after the fine-grained deconstruction of the feature map, the multi-head attention mechanism in the Transformer structure is used to measure the correlation between the two sets of local feature matrices of the channel dimension, strengthen the mutual relationship between the features, and more comprehensively capture the global bidirectional semantic association information between the shallow features and deep features of the property to be modeled by performing one-way semantic interaction between each channel dimension local feature matrix and all channel dimension local feature matrices in the other set. Then, the attention-optimized local feature matrices are recombined along the channel dimension to form a complete shallow feature map and deep feature map of the property to be modeled, and the two are fused by element-by-element weighted sum to generate a fine-grained interactive salient fusion feature map of the aerial view of the property to be modeled containing rich interactive information.

[0077] In the above-mentioned data processing method for real estate surveying and mapping, the S3 uses a ground measurement embedding coding matrix to perform high-dimensional embedding coding on the ground measurement data of the real estate to be modeled to obtain a ground measurement high-dimensional embedding coding vector. It should be understood that since the ground measurement data usually contains a large number of data items, such as length, width, height, angle, etc. In order to convert these data items into a data form compatible with the features of the aerial image, so as to facilitate subsequent multimodal data fusion processing, the present application further uses a ground measurement embedding coding matrix to perform high-dimensional embedding coding on the ground measurement data of the real estate to be modeled, maps it to a high-dimensional space, and learns the intrinsic correlation between each data item in the ground measurement data to generate a ground measurement high-dimensional embedding coding vector that can reflect the spatial structure and physical properties of the real estate to be modeled. In an embodiment of the present application, the ground measurement embedding coding matrix is ​​trained by a Word2Vec model.

[0078] In the above-mentioned data processing method for real estate surveying and mapping, the S4 performs cross-modal gated joint encoding on the aerial photography perspective fine-grained interactive significant fusion feature map of the real estate to be modeled and the ground measurement high-dimensional embedding coding vector to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled. It should be understood that considering that aerial images contain rich visual information, it is helpful to understand the overall structure and environmental relationship of the real estate; while the ground measurement data provides accurate numerical information of the real estate, which helps to determine the specific size and location of the real estate. Therefore, in order to make full use of the complementary advantages of the two types of data so as to more accurately restore the real situation of the real estate, the present application proposes a cross-modal gated joint encoding method, which imitates the self-attention mechanism of the Transformer structure and combines the gating unit to achieve the cross-modal fusion of the aerial photography perspective fine-grained interactive significant fusion feature map of the real estate to be modeled and the ground measurement high-dimensional embedding coding vector.

[0079] Optionally, in one embodiment of the present application, Figure 4 In the data processing method for real estate surveying and mapping of the embodiment of the present application, a schematic flow chart of cross-modal gated joint encoding is performed on the aerial photography perspective fine-grained interactive salient fusion feature map of the real estate to be modeled and the ground measurement high-dimensional embedding coding vector to obtain the aerial photography-measurement multi-modal fusion feature map of the real estate to be modeled. Figure 4 As shown, the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement are cross-modal gated joint encoding to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, including: S41, local semantic enhancement based on the self-attention mechanism is performed on the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled to obtain the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled; S42, cross-modal conditional fusion based on the gating mechanism is performed on the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

[0080] Optionally, in one embodiment of the present application, the fine-grained interactively significant fused feature map of the aerial perspective of the real estate to be modeled is subjected to local semantic enhancement based on a self-attention mechanism to obtain a local semantic autocorrelation library matrix of the aerial perspective of the real estate to be modeled, including: performing point convolution encoding and spatial convolution encoding on the fine-grained interactively significant fused feature map of the aerial perspective of the real estate to be modeled to obtain a spatial deepening feature map of the aerial perspective of the real estate to be modeled; constructing an imitation query feature matrix and an imitation key feature matrix based on the spatial deepening feature map of the aerial perspective of the real estate to be modeled; and calculating the product between the imitation query feature matrix and the transposed matrix of the imitation key feature matrix to obtain the local semantic autocorrelation library matrix of the aerial perspective of the real estate to be modeled.

[0081] Optionally, in one embodiment of the present application, a simulated query feature matrix and a simulated key feature matrix are constructed based on the aerial photography perspective space deepening feature map of the property to be modeled, including: copying the aerial photography perspective space deepening feature map of the property to be modeled to obtain a backup aerial photography perspective space deepening feature map of the property to be modeled; and reshaping the feature shapes of the backup aerial photography perspective space deepening feature map of the property to be modeled and the aerial photography perspective space deepening feature map of the property to be modeled, respectively, to obtain a backup aerial photography perspective space deepening feature matrix of the property to be modeled and a simulated key feature matrix of the property to be modeled as the simulated query feature matrix and the simulated key feature matrix.

[0082] Optionally, in one embodiment of the present application, the local semantic autocorrelation library matrix of the aerial perspective of the property to be modeled and the high-dimensional embedded coding vector of the ground measurement are cross-modal conditionally fused based on a gating mechanism to obtain the aerial photography-measurement multimodal fusion feature map of the property to be modeled, including: inputting the local semantic autocorrelation library matrix of the aerial perspective of the property to be modeled into an attention gating unit based on a GELU function to obtain a weight matrix of the local semantic autocorrelation library of the aerial perspective of the property to be modeled; using the high-dimensional embedded coding vector of the ground measurement as a value feature vector, calculating the product between the weight matrix of the local semantic autocorrelation library of the aerial perspective of the property to be modeled and the value feature vector to obtain a cross-modal interactive fusion conditional weight vector; using the cross-modal interactive fusion conditional weight vector as a weight vector, performing channel-by-channel weighting on the fine-grained interactive significant fusion feature map of the aerial perspective of the property to be modeled to obtain the aerial photography-measurement multimodal fusion feature map of the property to be modeled.

[0083] In summary, in the embodiment of the present application, the ground measurement high-dimensional embedding coding vector and the aerial photography perspective fine-grained interactive significant fusion feature map of the real estate to be modeled are processed by the following cross-modal joint coding formula to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, wherein the cross-modal joint coding formula is:

[0084] ;

[0085] in, represents the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled, Representation layer normalization operation, represents point convolution, Represents a convolution operation based on a 3×3 convolution kernel, It represents the spatial deepening feature map of the aerial view of the property to be modeled. Indicates a copy operation. Represents the backup aerial photography perspective spatial deepening feature map of the real estate to be modeled. represents the feature shape reshaping, represents the transpose of a matrix, represents the matrix multiplication operation, represents the Gaussian error linear unit, Represents the weight matrix of the local semantic autocorrelation library from the aerial photography perspective of the real estate to be modeled, represents the high-dimensional embedding encoding vector of the ground measurement, represents the cross-modal interaction fusion condition weight vector, Represents the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

[0086] First, the fine-grained interactive salient fusion feature map of the aerial photography perspective of the real estate to be modeled is subjected to point convolution encoding and spatial convolution encoding to extract its deep spatial features and enhance its spatial expression ability. The query feature matrix and the key feature matrix are constructed based on this. The product of the query feature matrix and the transposed key feature matrix is ​​calculated to reveal its internal correlation and construct the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled. Then, the attention gating unit based on the GELU function is used to dynamically adjust the autocorrelation weight to achieve adaptive selection and enhancement of features and generate a weight matrix. Furthermore, the weight matrix is ​​multiplied by the high-dimensional embedding encoding vector of the ground measurement to obtain a cross-modal interactive fusion conditional weight vector, thereby achieving conditional fusion between modalities and enhancing the cross-modal association between the semantic embedding features of the ground measurement data and the visual features of the real estate. Finally, the cross-modal interactive fusion conditional weight vector is used to weight the fine-grained interactive salient fusion feature map of the aerial photography perspective of the real estate to be modeled channel by channel to integrate the multimodal features of real estate modeling, thereby generating a multimodal fusion feature map of aerial photography-measurement of the real estate to be modeled.

[0087] In the above-mentioned data processing method for real estate surveying and mapping, the S5 generates a real estate surveying and mapping rendering based on the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled. Optionally, in one embodiment of the present application, the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled is input into a real estate surveying and mapping rendering generator based on a diffusion model to obtain a real estate surveying and mapping rendering. It should be understood that the diffusion model destroys the training data by gradually adding noise, and then the training model reversely removes the noise and gradually restores the original data, so as to learn the inherent distribution law of the data, so as to generate high-quality images. In this way, the real estate surveying and mapping rendering generator based on the diffusion model can gradually refine the feature representation according to the input aerial photography-measurement multimodal fusion feature map of the real estate to be modeled through forward diffusion and reverse denoising process, and generate a high-precision real estate surveying and mapping rendering.

[0088] In a preferred example, inputting the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled into a real estate surveying and mapping rendering generator based on a diffusion model to obtain a real estate surveying and mapping rendering includes:

[0089] Determine the maximum eigenvalue of the aerial photography-measurement multimodal fusion of the real estate to be modeled and the minimum eigenvalue of the aerial photography-measurement multimodal fusion of the real estate to be modeled of the aerial photography-measurement multimodal fusion feature map;

[0090] Calculate the mean value of the aerial photography-measurement multimodal fusion of the real estate to be modeled and the standard deviation of the aerial photography-measurement multimodal fusion of the real estate to be modeled of the feature set of the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, and calculate the quotient of the mean value of the aerial photography-measurement multimodal fusion of the real estate to be modeled and the standard deviation of the aerial photography-measurement multimodal fusion of the real estate to be modeled to obtain the statistical normalization value of the aerial photography-measurement multimodal fusion of the real estate to be modeled;

[0091] Calculate the inverse of each eigenvalue of the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, multiply it by the difference between the maximum eigenvalue of the aerial photography-measurement multimodal fusion of the real estate to be modeled and the minimum eigenvalue of the aerial photography-measurement multimodal fusion of the real estate to be modeled, and then subtract it from the statistical normalization value of the aerial photography-measurement multimodal fusion of the real estate to be modeled to obtain the aerial photography-measurement multimodal fusion distribution approximation feature map of the real estate to be modeled;

[0092] Calculate an exponential function with a natural constant as the base, with each eigenvalue of the aerial photography-measurement multimodal fusion distribution approximation feature map of the real estate to be modeled as an exponent to obtain an approximate feature map of the aerial photography-measurement multimodal fusion distribution of the real estate to be modeled;

[0093] Performing point addition on the aerial photography-measurement multimodal fusion distribution class approximate feature map of the property to be modeled and the aerial photography-measurement multimodal fusion statistical normalized value of the property to be modeled, and calculating the base-two logarithm of the absolute value of each eigenvalue of the point-added vector to obtain an optimized aerial photography-measurement multimodal fusion feature map of the property to be modeled;

[0094] The optimized aerial photography-measurement multimodal fusion feature map of the real estate to be modeled is input into the real estate surveying and mapping rendering generator based on the diffusion model to obtain the real estate surveying and mapping rendering.

[0095] Among them, the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled The optimization expression is:

[0096] ;

[0097] in, Indicates the maximum eigenvalue of the aerial photography-measurement multimodal fusion of the real estate to be modeled, It represents the minimum eigenvalue of the multi-modal fusion of aerial photography and measurement of the real estate to be modeled. represents the mean value of the aerial photography-measurement multimodal fusion of the real estate to be modeled, represents the standard deviation of the aerial photography-measurement multimodal fusion of the property to be modeled, The aerial photography-measurement multimodal fusion statistical normalization value of the property to be modeled, and is the difference between the maximum eigenvalue of the aerial photography-measurement multimodal fusion of the real estate to be modeled and the minimum eigenvalue of the aerial photography-measurement multimodal fusion of the real estate to be modeled, represents the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, Indicates the calculation of the inverse of each eigenvalue of the feature map, It means point multiplication by position. Add by position, Subtract by position, Represents the approximate feature map of the aerial photography-measurement multimodal fusion distribution of the real estate to be modeled, Represents the optimized aerial photography-measurement multimodal fusion feature map of the property to be modeled.

[0098] Here, in the preferred example, since the aerial photography perspective fine-grained interactive significant fusion feature map of the property to be modeled and the ground measurement high-dimensional embedding coding vector respectively represent the image shallow-deep fine-grained interactive significant fusion features of the aerial image of the property to be modeled and the ground measurement data high-dimensional embedding features of the property to be modeled, when they are input into the gated cross-modal joint encoder based on the imitation converter structure, the feature modality difference and feature dimension non-correspondence will make the aerial photography-measurement multimodal fusion feature map of the property to be modeled also have a diversified set expression distribution of cross-modal interactive fusion. Therefore, it is expected to improve the balance between the regression mapping accuracy and completeness when the aerial photography-measurement multimodal fusion feature map of the property to be modeled is input into the real estate surveying and mapping rendering generator based on the diffusion model for generating regression, thereby improving the accuracy of the obtained real estate surveying and mapping rendering.

[0099] Based on this, by performing random statistical normalization on the diversified feature set of the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, the confidence space constructed based on the overall eigenvalue of the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled is approximated by the standardized continuous probability density distribution relative to the response hypothesis test of each eigenvalue of the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, thereby establishing the target reachability of the diversified feature distribution of the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled to a unified regression target, so as to achieve a balanced executableness between mapping accuracy and mapping completeness in the generation regression process based on the diversified feature distribution of the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, and improve the accuracy of the real estate surveying and mapping renderings obtained by inputting the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled into the real estate surveying and mapping renderings generator based on the diffusion model.

[0100] In summary, the data processing method for real estate surveying and mapping according to the embodiment of the present application is explained, which uses drones and surveying equipment to collect aerial images and ground measurement data of the real estate to be modeled, respectively, and uses deep learning-based artificial intelligence technology to process the aerial images and ground measurement data, respectively extracting the multi-level joint features of the aerial images and the high-dimensional semantic embedding features of the ground measurement data, and then through cross-modal gated joint encoding of the two, the spatial layout and structural information of the real estate to be modeled are fully mined, thereby intelligently generating real estate surveying and mapping renderings. In this way, the efficiency and accuracy of real estate surveying and mapping can be effectively improved, the dependence on manual operations can be reduced, and the automation and intelligence of the surveying and mapping process can be improved.

[0101] Figure 5 Schematic block diagram of a data processing system for real estate surveying and mapping according to an embodiment of the present application. Figure 5 As shown, the data processing system 100 for real estate surveying and mapping includes: a data acquisition module 110, which is used to acquire aerial images of the real estate to be modeled collected by a drone, and ground measurement data of the real estate to be modeled collected by a surveying and mapping device; a feature fine-grained joint perception module 120, which is used to perform multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial images of the real estate to be modeled to obtain a fine-grained interactive salient fusion feature map of the real estate to be modeled; a high-dimensional embedding coding module 130, which is used to perform high-dimensional embedding coding on the ground measurement data of the real estate to be modeled using a ground measurement embedding coding matrix to obtain a ground measurement high-dimensional embedding coding vector; a cross-modal gated joint coding module 140, which is used to perform cross-modal gated joint coding on the fine-grained interactive salient fusion feature map of the real estate to be modeled and the ground measurement high-dimensional embedding coding vector to obtain an aerial-measurement multi-modal fusion feature map of the real estate to be modeled; and a real estate surveying and mapping rendering generation module 150, which is used to generate a real estate surveying and mapping rendering based on the aerial-measurement multi-modal fusion feature map of the real estate to be modeled.

[0102] Optionally, in one embodiment of the present application, the cross-modal gated joint encoding module is used to: perform local semantic enhancement based on a self-attention mechanism on the fine-grained interactively significant fusion feature map of the aerial perspective of the property to be modeled to obtain a local semantic autocorrelation library matrix of the aerial perspective of the property to be modeled; perform cross-modal conditional fusion based on a gating mechanism on the local semantic autocorrelation library matrix of the aerial perspective of the property to be modeled and the ground measurement high-dimensional embedding coding vector to obtain the aerial photography-measurement multimodal fusion feature map of the property to be modeled.

[0103] Optionally, in one embodiment of the present application, the feature fine-grained joint perception module is used to: input the aerial image of the property to be modeled into a multi-scale image feature extractor based on a diaphanous pyramid network to obtain a shallow feature map of the aerial perspective of the property to be modeled and a deep feature map of the aerial perspective of the property to be modeled; perform fine-grained bidirectional global attention fusion on the shallow feature map of the aerial perspective of the property to be modeled and the deep feature map of the aerial perspective of the property to be modeled to obtain a fine-grained interactively significant fused feature map of the aerial perspective of the property to be modeled.

[0104] Here, those skilled in the art will appreciate that the specific operations of each step in the data processing system for real estate surveying and mapping have been described in detail above. Figures 1 to 4 The data processing method for real estate surveying and mapping has been described in detail, and therefore, its repeated description will be omitted.

[0105] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0106] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

Claims

1. A data processing method for real estate surveying and mapping, characterized in that: include: Acquire aerial images of the property to be modeled collected by a drone, and ground measurement data of the property to be modeled collected by a surveying and mapping device; Performing multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial view of the real estate to be modeled; Using a ground measurement embedding coding matrix, high-dimensional embedding coding is performed on the ground measurement data of the property to be modeled to obtain a ground measurement high-dimensional embedding coding vector; Performing cross-modal gated joint encoding on the aerial photography perspective fine-grained interactive salient fusion feature map of the property to be modeled and the ground measurement high-dimensional embedding coding vector to obtain an aerial photography-measurement multi-modal fusion feature map of the property to be modeled; Generate a real estate surveying and mapping rendering based on the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled; The fine-grained interactive fusion feature map of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement are cross-modal gated joint encoding to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, including: Performing local semantic enhancement based on the self-attention mechanism on the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled to obtain a local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled; Performing cross-modal conditional fusion based on a gating mechanism on the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled; The local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement are subjected to cross-modal conditional fusion based on a gating mechanism to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled, including: Input the local semantic autocorrelation library matrix of the aerial view of the real estate to be modeled into the attention gating unit based on the GELU function to obtain the weight matrix of the local semantic autocorrelation library of the aerial view of the real estate to be modeled; Taking the ground measurement high-dimensional embedding coding vector as the value feature vector, calculating the product between the weight matrix of the local semantic autocorrelation library of the aerial photography perspective of the real estate to be modeled and the value feature vector to obtain a cross-modal interactive fusion condition weight vector; The cross-modal interactive fusion condition weight vector is used as the weight vector, and the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled is weighted channel by channel to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

2. The data processing method for real estate surveying and mapping according to claim 1, characterized in that: The aerial image of the real estate to be modeled is subjected to multi-scale feature extraction and multi-scale feature fine-grained joint perception to obtain a fine-grained interactive significant fusion feature map of the aerial view of the real estate to be modeled, including: Inputting the aerial image of the real estate to be modeled into a multi-scale image feature extractor based on a dilated pyramid network to obtain a shallow feature map of the aerial perspective of the real estate to be modeled and a deep feature map of the aerial perspective of the real estate to be modeled; Fine-grained bidirectional global attention fusion is performed on the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled.

3. The data processing method for real estate surveying and mapping according to claim 2, characterized in that: Performing fine-grained bidirectional global attention fusion on the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled, including: Performing feature fine-grained deconstruction on the shallow feature map of the aerial photography perspective of the real estate to be modeled and the deep feature map of the aerial photography perspective of the real estate to be modeled to obtain a set of local feature matrices of shallow channel dimensions of the aerial photography perspective of the real estate and a set of local feature matrices of deep channel dimensions of the aerial photography perspective of the real estate; Performing global attention interactive optimization on the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective to obtain a set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective optimized by one-way global attention and a set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective optimized by one-way global attention; The set of local feature matrices of the shallow channel dimension of the one-way global attention optimization of real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the one-way global attention optimization of real estate aerial photography perspective are respectively coupled along the channel dimension to obtain a shallow feature map of the one-way global interaction optimization of real estate aerial photography perspective and a deep feature map of the one-way global interaction optimization of real estate aerial photography perspective; The position-weighted sum between the one-way global interactive optimization of the real estate aerial photography perspective shallow feature map and the one-way global interactive optimization of the real estate aerial photography perspective deep feature map is calculated to obtain the fine-grained interactive significant fusion feature map of the real estate aerial photography perspective to be modeled.

4. The data processing method for real estate surveying and mapping according to claim 3 is characterized in that: The set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective are optimized by global attention interaction to obtain a set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective optimized by one-way global attention and a set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective optimized by one-way global attention, including: Using each local feature matrix of the shallow channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective as a query feature matrix, and using the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a first converter structure to obtain the one-way global attention optimized set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective; Each local feature matrix of the deep channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective is used as a query feature matrix, and the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a unidirectional global attention interaction module based on a second converter structure to obtain the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective optimized by the unidirectional global attention.

5. The data processing method for real estate surveying and mapping according to claim 4, characterized in that: Each local feature matrix of the shallow channel dimension of the real estate aerial photography perspective in the set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective is used as a query feature matrix, and the set of local feature matrices of the deep channel dimension of the real estate aerial photography perspective is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a one-way global attention interaction module based on a first converter structure to obtain the one-way global attention optimized set of local feature matrices of the shallow channel dimension of the real estate aerial photography perspective, including: Calculate the matrix product of the query feature matrix and the transposed matrix of each key feature matrix in the set of key feature matrices respectively, and then divide the matrix product by the square root of the scale of the key feature matrix and pass it through a softmax function to obtain a set of attention weight matrices; Each attention weight matrix in the set of attention weight matrices is multiplied with the query feature matrix respectively, and then the multiplication results are summed by position point to obtain the one-way global attention optimized real estate aerial photography perspective shallow channel dimension local feature matrix.

6. The data processing method for real estate surveying and mapping according to claim 5, characterized in that: The fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled is subjected to local semantic enhancement based on the self-attention mechanism to obtain a local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled, including: Performing point convolution encoding and spatial convolution encoding on the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled to obtain a spatial deepening feature map of the aerial photography perspective of the real estate to be modeled; Constructing a pseudo-query feature matrix and a pseudo-key feature matrix based on the aerial photography perspective spatial deepening feature graph of the real estate to be modeled; The product of the pseudo-query feature matrix and the transposed matrix of the pseudo-key feature matrix is ​​calculated to obtain the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled.

7. The data processing method for real estate surveying and mapping according to claim 6, characterized in that: Based on the aerial photography perspective spatial deepening feature graph of the real estate to be modeled, a pseudo-query feature matrix and a pseudo-key feature matrix are constructed, including: Copy the aerial perspective spatial deepening feature map of the real estate to be modeled to obtain a backup aerial perspective spatial deepening feature map of the real estate to be modeled; The feature shapes of the backup aerial perspective space deepening feature map of the real estate to be modeled and the aerial perspective space deepening feature map of the real estate to be modeled are reshaped respectively to obtain the backup aerial perspective space deepening feature matrix of the real estate to be modeled and the aerial perspective space deepening feature matrix of the real estate to be modeled as the simulated query feature matrix and the simulated key feature matrix.

8. A data processing system for real estate surveying and mapping, characterized in that: include: A data acquisition module, used to acquire aerial images of the real estate to be modeled collected by a drone, and ground measurement data of the real estate to be modeled collected by a surveying and mapping device; A feature fine-grained joint perception module is used to perform multi-scale feature extraction and multi-scale feature fine-grained joint perception on the aerial image of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial view of the real estate to be modeled; A high-dimensional embedding coding module, used for performing high-dimensional embedding coding on the ground measurement data of the property to be modeled using a ground measurement embedding coding matrix to obtain a ground measurement high-dimensional embedding coding vector; A cross-modal gated joint encoding module is used to perform cross-modal gated joint encoding on the aerial photography perspective fine-grained interactive significant fusion feature map of the real estate to be modeled and the ground measurement high-dimensional embedding coding vector to obtain an aerial photography-measurement multi-modal fusion feature map of the real estate to be modeled; A real estate surveying and mapping rendering generation module is used to generate a real estate surveying and mapping rendering based on the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled; Wherein, the cross-modal gated joint encoding module is used to: Performing local semantic enhancement based on the self-attention mechanism on the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled to obtain a local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled; Performing cross-modal conditional fusion based on a gating mechanism on the local semantic autocorrelation library matrix of the aerial photography perspective of the real estate to be modeled and the high-dimensional embedding coding vector of the ground measurement to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled; The cross-modal gated joint encoding module is used to: Input the local semantic autocorrelation library matrix of the aerial view of the real estate to be modeled into the attention gating unit based on the GELU function to obtain the weight matrix of the local semantic autocorrelation library of the aerial view of the real estate to be modeled; Taking the ground measurement high-dimensional embedding coding vector as the value feature vector, calculating the product between the weight matrix of the local semantic autocorrelation library of the aerial photography perspective of the real estate to be modeled and the value feature vector to obtain a cross-modal interactive fusion condition weight vector; The cross-modal interactive fusion condition weight vector is used as the weight vector, and the fine-grained interactive significant fusion feature map of the aerial photography perspective of the real estate to be modeled is weighted channel by channel to obtain the aerial photography-measurement multimodal fusion feature map of the real estate to be modeled.

9. The data processing system for real estate surveying and mapping according to claim 8, characterized in that: The feature fine-grained joint perception module is used to: Inputting the aerial image of the real estate to be modeled into a multi-scale image feature extractor based on a dilated pyramid network to obtain a shallow feature map of the aerial perspective of the real estate to be modeled and a deep feature map of the aerial perspective of the real estate to be modeled; Fine-grained bidirectional global attention fusion is performed on the shallow feature map of the aerial perspective of the real estate to be modeled and the deep feature map of the aerial perspective of the real estate to be modeled to obtain a fine-grained interactive significant fusion feature map of the aerial perspective of the real estate to be modeled.

Images