A multi-source data fusion application method

Abstract

The present invention discloses a multi-source data fusion application method, including: Step 1, processing point cloud data to obtain target point cloud data; Step 2, processing images to obtain target images; Step 3, matching the target point cloud data and the target images to obtain modeling data; Step 4, model reconstruction: based on the modeling data, constructing a three-dimensional model and a two-dimensional model of the measured area; Step 5, model modification: modifying the two-dimensional model and the three-dimensional model; Step 6, model display: integrally displaying the two-dimensional spatial data of the two-dimensional model and the three-dimensional spatial data of the three-dimensional model. The structure of the present invention is simple and reasonably designed. It uses a method of first rough segmentation and then fine-tuning to process point cloud data, uses learnable weights to fuse historical images and new images, and then fuses multi-source point cloud data and multi-source images to obtain modeling data, respectively establishing a two-dimensional model and a three-dimensional model, and integrating the display of two-dimensional spatial data and three-dimensional spatial display.

Description

Technical Field

[0001] The present invention belongs to the field of geographic information technology, and particularly relates to a method for fusing and applying multi-source data. Background Art

[0002] In modern society, the types and quantities of data collected by people are constantly increasing. These data usually come from different sources, including sensors, social media, mobile devices, and so on. How to fuse these data for analysis and application has become an important technical challenge.

[0003] The integration of 2D and 3D is a new technology that can fuse data from different sources into a unified model for better visualization and analysis.

[0004] In the aspect of geographic information acquisition and display, it is of great significance to achieve multi-source data fusion, comprehensively utilize geological data, form a comprehensive application platform, intuitively analyze the geographical environment, and manage data scenarios. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method for fusing and applying multi-source data in view of the deficiencies in the above-mentioned prior art. The method has a simple structure and reasonable design, fuses multi-source point cloud data and multi-source images to obtain modeling data, respectively establishes a 2D model and a 3D model, integrates 2D spatial data and 3D spatial display, and the 2D spatial data and 3D spatial data are in the same coordinate system.

[0006] To solve the above technical problem, the technical solution adopted by the present invention is: a method for fusing and applying multi-source data, characterized in that:

[0007] Step 1: Obtain target point cloud data: Input the first point cloud data, the second point cloud data, and the historical point cloud data into a coarse segmentation network. According to the relationship between pixels and object region features learned by the coarse segmentation network, enhance the description of pixel features to obtain a coarse segmentation result; fuse the coarse segmentation result and the historical point cloud data to obtain a fused point cloud, and input the fused point cloud into a fine-tuning network. The fine-tuning network outputs an accurate segmentation result to obtain the target point cloud data;

[0008] Step 2: Obtain target images: Respectively perform feature extraction on the first new image, the second new image, and the historical image to obtain a first feature map F 1 , a second feature map F 2 , and a third feature map F 3 . Fuse the first feature map F 1 and the second feature map F 2 to obtain a fused feature map F R; Output learnable weights based on the multi-layer perceptron (MLP) model, and fuse the third feature map F based on the learnable weights 3 and the fused feature map F R , to obtain the target image;

[0009] Step 3: Match the target point cloud data and the target image to obtain the modeling data;

[0010] Step 4: Model reconstruction: Based on the modeling data, construct a three-dimensional model and a two-dimensional model of the area to be measured;

[0011] Step 5: Model modification: Modify the two-dimensional model and the three-dimensional model;

[0012] Step 6: Model display: Integrally display the two-dimensional spatial data of the two-dimensional model and the three-dimensional spatial data of the three-dimensional model.

[0013] The above-mentioned multi-source data fusion application method is characterized in that: The specific method of Step 1 is:

[0014] Step 101: The first aerial device collects the first newly added point cloud data of the area to be measured, preprocesses the first newly added point cloud data, corrects the first newly added point cloud data, and obtains the first point cloud data;

[0015] Step 102: The first vehicle-mounted device collects the second newly added point cloud data of the area to be measured, preprocesses the second newly added point cloud data, corrects the second newly added point cloud data, and obtains the second point cloud data;

[0016] Step 103: Obtain the historical point cloud data of the area to be measured;

[0017] Step 104: The first point cloud data, the second point cloud data, and the historical point cloud data form a point cloud data set. Divide the point cloud data set into a training set, a validation set, and a test set, and add corresponding labels to the point cloud data in the training set;

[0018] Step 105: Construct a segmentation network, input the point cloud data in the training set into the segmentation network to obtain a predicted segmentation result, and adjust the network parameters of the segmentation network according to the predicted segmentation result until the training stop condition is met, and obtain a trained segmentation network;

[0019] Step 106: Input the first point cloud data and the second point cloud data into the segmentation network respectively to obtain a rough segmentation result, fuse the rough segmentation result and the historical point cloud data to obtain a fused point cloud, and slice the fused point cloud;

[0020] Step 107: Construct a fine-tuning network: Input the sliced data into the fine-tuning network, and the fine-tuning network outputs an accurate segmentation result to obtain the target point cloud data.

[0021] The above-mentioned multi-source data fusion application method is characterized in that: The specific method of step two is as follows:

[0022] Step 201, The second aviation equipment collects the first new image of the measured area, performs aerial triangulation encryption on the preprocessed first new image, and obtains the first new image;

[0023] Step 202, The second vehicle-mounted equipment collects the second new image of the measured area, performs aerial triangulation encryption on the preprocessed second new image, and obtains the second new image;

[0024] Step 203, Obtain the historical image of the measured area;

[0025] Step 204, Respectively perform feature extraction on the first new image and the second new image to obtain the first feature map F 1 and the second feature map F 2 , perform feature extraction on the historical image to obtain the third feature map F 3 ;

[0026] Step 205, Use the BCA network to fuse the first feature map F 1 and the second feature map F 2 to obtain the fused feature map F R ;

[0027] Step 206, Construct a multi-layer perceptron mlp model;

[0028] Step 207, Perform global max pooling on the fused feature map F R to obtain the first pooling value, and use the first pooling value to construct the first channel dimension vector C FR ;

[0029] Step 208, Perform global average pooling on the third feature map F 3 to obtain the second pooling value, and use the second pooling value to construct the second channel dimension vector C F3 ;

[0030] Step 209, Input the first channel dimension vector C FR and the second channel dimension vector C F3 into the multi-layer perceptron mlp model, and the mlp model outputs the weight ω 1 ;

[0031] Step 2010, Based on the formula F = ω 1 ·F R +(1 - ω 1 )·F 3 perform weighted fusion on the fused feature map F R and the third feature map F 3 to obtain the target image.

[0032] The above-mentioned multi-source data fusion application method is characterized in that the specific method for matching the target point cloud data and the target image is as follows:

[0033] Step 301, determine the matching domain: Use the CornerNet model to read the corner points of the m-th target image, form a target box from the corner points, and convert to obtain the ground coordinates X of each corner point j , and the connection of the ground coordinates of each corner point forms the matching domain;

[0034] Step 302, determine the matching line: Divide one side of the target box into n equal parts to obtain n + 1 parallel lines, randomly select a point on each parallel line, and convert to obtain the ground coordinates X of each point d , and the ground coordinates X of each point d are connected to form the first matching line;

[0035] Step 303, in the target point cloud, find the target point cloud subset corresponding to the matching domain, and determine at least 3 coordinate points corresponding to the ground coordinates X in the target point cloud subset d , form the second matching line, and calculate the rotation angle θ between the first matching line and the second matching line;

[0036] Step 304, rotate the target point cloud subset by the rotation angle θ, and add the rotated target point cloud subset to the feature point set of the m-th target image to obtain the complete feature point set of the m-th target image;

[0037] Step 305, repeat steps 301-304 to complete the matching of all target point cloud data and target images.

[0038] The above-mentioned multi-source data fusion application method is characterized in that the modified two-dimensional model in step five includes: detecting and filling the invalid value area of the two-dimensional model; repairing the noise area of the two-dimensional model; removing the suspended matter of the two-dimensional model; setting the number of layers, corresponding multiple colors to multiple layer data, and performing layered coloring rendering on the two-dimensional model.

[0039] The above-mentioned multi-source data fusion application method is characterized in that the modified three-dimensional model in step five includes: detecting and filling the invalid value area of the three-dimensional model; repairing the noise area of the three-dimensional model; removing the suspended matter of the three-dimensional model; setting the number of layers, corresponding multiple colors to multiple layer data, and performing layered coloring rendering on the three-dimensional model; performing automatic equalization of light and color on the three-dimensional model.

[0040] The above-mentioned multi-source data fusion application method is characterized in that: in step one, the segmentation network adopts a V-net network, and at the end of the encoder of the V-net network, a pooling layer, a 1×1 convolutional layer, and three 3×3 convolutional layers with different dilation rates are sequentially arranged.

[0041] The above-mentioned multi-source data fusion application method is characterized in that: in step 106, the fine-tuning network adopts a U-net network, and the U-net network uses the weighted sum of the boundary loss function and the binary cross-entropy loss as the overall loss function.

[0042] The above-mentioned multi-source data fusion application method is characterized in that: the two-dimensional spatial data includes vector data, elevation data, image data, and oblique data; the three-dimensional spatial data includes model data and oblique data.

[0043] The above-mentioned multi-source data fusion application method is characterized in that: the preprocessing of the point cloud data in step one includes fully automatic point cloud filtering, and the fully automatic point cloud filtering includes adaptive filtering, leveling filtering, smoothing filtering, fusion filtering, general filtering, elevation reduction filtering, and profile filtering.

[0044] The present invention has the following advantages compared with the prior art:

[0045] 1. The structure of the present invention is simple, the design is reasonable, and the implementation and use operations are convenient.

[0046] 2. When processing point cloud data, the present invention is divided into two stages. In the first stage, a 3D convolutional segmentation network is used to segment the point cloud data. The segmentation result is fused with the historical point cloud data in the channel dimension and input into the fine-tuning network. Then, the 2D convolutional fine-tuning network is used to fine-tune the segmentation result, so that the network can make full use of the features of 3D convolution and 2D convolution, thereby improving the segmentation accuracy.

[0047] 3. The present invention uses a BCA network to perform feature fusion on the first newly added image and the second newly added image to obtain a fused feature map. Based on the fused feature map and the pooling value of the historical image, the network weights are updated, and then the entire multi-layer perceptron mlp model is updated. Finally, a target image of the same size as the input is obtained, and the use effect is good.

[0048] 4. In the present invention, the two-dimensional spatial data and the three-dimensional spatial data are integrally displayed.

[0049] In summary, the present invention has a simple structure and a reasonable design. It uses a method of first rough segmentation and then fine-tuning to process point cloud data, uses learnable weights to fuse historical images and newly added images, and fuses multi-source point cloud data and multi-source images to obtain modeling data. Two-dimensional models and three-dimensional models are respectively established, and the two-dimensional spatial data and the three-dimensional display are integrated.

[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0051] Figure 1 It is a flowchart of the method for obtaining the target point cloud data of the present invention.

[0052] Figure 2 It is a flowchart of the method for obtaining the target image of the present invention.

[0053] Figure 3 It is a flowchart of the method of the present invention. Detailed Embodiments

[0054] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments of the present invention.

[0055] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0056] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0057] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0058] For ease of description, spatial relative terms, such as "above", "over", "on the upper surface", "upper", etc., may be used herein to describe the spatial positional relationship of one device or feature to other devices or features as shown in the figures. It should be understood that the spatial relative terms are intended to encompass different orientations in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figure is inverted, a device described as "above" or "over" other devices or structures will then be positioned "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both the orientations of "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the corresponding explanations are made for the spatial relative descriptions used herein.

[0059] As Figures 1-3 shown, this embodiment includes a multi-source data fusion application method, which is characterized in that:

[0060] Step 1: Obtain target point cloud data: Input the first point cloud data, the second point cloud data, and the historical point cloud data into the coarse segmentation network. According to the relationship between the pixels and the object region features learned by the coarse segmentation network, enhance the description of the pixel features to obtain a coarse segmentation result; fuse the coarse segmentation result and the historical point cloud data to obtain a fused point cloud, and input the fused point cloud into the fine-tuning network. The fine-tuning network outputs an accurate segmentation result to obtain the target point cloud data.

[0061] In a possible embodiment, the specific method for obtaining the target point cloud data is:

[0062] Step 101: The first aerial device collects the first new point cloud data of the measured area, preprocesses the first new point cloud data, and corrects the first new point cloud data to obtain the first point cloud data;

[0063] Step 102: The first vehicle-mounted device collects the second new point cloud data of the measured area, preprocesses the second new point cloud data, and corrects the second new point cloud data to obtain the second point cloud data;

[0064] Preprocessing the point cloud data includes full-automatic point cloud filtering, and full-automatic point cloud filtering includes adaptive filtering, leveling filtering, smoothing filtering, fusion filtering, general filtering, elevation reduction filtering, and profile filtering.

[0065] Step 103: Obtain the historical point cloud data of the measured area;

[0066] Step 104: The first point cloud data, the second point cloud data, and the historical point cloud data form a point cloud data set. The point cloud data set is divided into a training set, a validation set, and a test set, and corresponding labels are added to the point cloud data in the training set.

[0067] In actual use, the methods for preprocessing the first newly added point cloud data and the second newly added point cloud data include normalization and denoising using Gaussian filtering.

[0068] Normalization is to scale the point cloud data to the same scale range, subtract the mean and then divide by the standard deviation. Denoising using Gaussian filtering is to perform denoising processing on the point cloud data using Gaussian filtering.

[0069] The data set is divided into a training set, a validation set, and a test set according to the ratio of 8:1:1.

[0070] It should be noted that the labels corresponding to the point cloud data are used to characterize the categories of each coordinate point therein. Example: The corresponding labels can be set to three categories, namely target A, target B, target C, and background that need to be recognized, or two categories, namely target and background.

[0071] Taking the identification of buildings as an example, each coordinate point in the point cloud data can be labeled as building and background, the buildings can be identified, and then each coordinate point can be labeled as building or background.

[0072] In a possible embodiment, taking the data set composed of the point cloud data obtained once as an example, the data set is divided, with a total of 800 images. Among them, 640 are used for model training, 80 are used for test verification, and 80 are used for model testing.

[0073] In a possible embodiment, the point cloud data can be obtained by a lidar. The point cloud data obtained by the lidar is transmitted to a remote controller on the ground in real time, and the remote controller on the ground is then transmitted to terminal devices such as mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs) in real time, so as to realize the acquisition and real-time transmission of the point cloud data, and the terminal device performs point cloud data segmentation. The acquisition method of the point cloud data in the embodiments of the present application is not limited in any way.

[0074] Step 105: Construct a segmentation network, input the point cloud data in the training set into the segmentation network to obtain a predicted segmentation result, and adjust the network parameters of the segmentation network according to the predicted segmentation result until the training stop condition is met, and a trained segmentation network is obtained;

[0075] Step 106: Input the first point cloud data and the second point cloud data into the segmentation network respectively to obtain a rough segmentation result. Fuse the rough segmentation result with the historical point cloud data to obtain a fused point cloud, and slice the fused point cloud.

[0076] Step 107: Construct a fine-tuning network: Input the sliced data into the fine-tuning network, and the fine-tuning network outputs an accurate segmentation result to obtain the target point cloud data.

[0077] In the first stage, a segmentation network with 3D convolution is used to segment the point cloud data. The segmentation result is fused with the historical point cloud data in the channel dimension and input into the fine-tuning network. Then, a fine-tuning network with 2D convolution is used to fine-tune the segmentation result, enabling the network to fully utilize the features of 3D convolution and 2D convolution, thereby improving the segmentation accuracy.

[0078] Adding historical point cloud data to the training set can fully express the local geometric feature distribution information. Therefore, when the segmentation network performs semantic segmentation on the first point cloud data and the second point cloud data, it can have good adaptability and improve the segmentation accuracy. Using the segmentation network to extract the key points of the point cloud data can ensure the representativeness of the point cloud data while reducing the number of point clouds, and can improve the efficiency of matching and fusion in step three.

[0079] Fuse the rough segmentation result with the historical point cloud data. When the point cloud of the rough segmentation result is sparse, make full use of the rich features of buildings widely existing in the historical point cloud data to achieve effective positioning of the buildings.

[0080] Use the fine-tuning network to subdivide pixels and further optimize the segmentation result of the segmentation network.

[0081] In this embodiment, the segmentation network in step one adopts a V-net network. At the end of the encoder of the V-net network, a pooling layer, a 1×1 convolutional layer, and three 3×3 convolutional layers with different dilation rates are sequentially set.

[0082] In this embodiment, the fine-tuning network in step one adopts a U-net network. The U-net network uses the weighted sum of the boundary loss function and the binary cross-entropy loss as the overall loss function.

[0083] The U-net network refers to a convolutional network used for biomedical image segmentation. The V-net network refers to a fully convolutional neural network used for three-dimensional medical image segmentation.

[0084] Step two: Process the image to obtain the target image.

[0085] Step 201: The second aviation equipment collects the first new image of the measured area, and performs aerial triangulation encryption on the preprocessed first new image to obtain the first new image.

[0086] Step 202: The second vehicle-mounted device collects the second newly added image of the measured area, performs aerial triangulation encryption on the preprocessed second newly added image, and obtains the second newly added image;

[0087] Step 203: Obtain the historical image of the measured area;

[0088] Step 204: Respectively perform feature extraction on the first newly added image and the second newly added image to obtain the first feature map F 1 and the second feature map F 2 , perform feature extraction on the historical image to obtain the third feature map F 3 .

[0089] The feature extraction module is built with a deep neural network, and its feature extraction method is well-known to those skilled in the art. Therefore, its specific implementation method will not be described in detail.

[0090] Step 205: Use the BCA network to fuse the first feature map F 1 and the second feature map F 2 . The BCA network uses a convolutional layer to fuse the first feature map F 1 and the second feature map F 2 . The BCA network uses a cross-entropy loss function for supervision and an attention loss function to strengthen the cooperation between edge detection and semantic segmentation. The weights of the cross-entropy loss function and the attention loss function are 1 and 0.5 respectively.

[0091] The full English name of BCA is: Boundary guided Context Aggregation module, and BCA is translated as Boundary Guided Network.

[0092] Step 206: Construct a multi-layer perceptron mlp model;

[0093] The full English name of MLP is Multilayer Perceptron, and MLP is translated as Multilayer Perceptron, which is a feedforward artificial neural network model.

[0094] Step 207: Perform global max pooling on the fused feature map F R to obtain the first pooling value, and use the first pooling value to construct the first channel dimension vector C FR ;

[0095] Step 208: Perform global average pooling on the third feature map F 3 to obtain the second pooling value, and use the second pooling value to construct the second channel dimension vector C F3 ;

[0096] Step 209: The first channel dimension vector C FRand the second channel dimension vector C F3 Input the multi-layer perceptron (MLP) model, and the MLP model outputs the weight ω 1 ;

[0097] Step 2010: Based on the formula F = ω 1 ·F R +(1 - ω 1 )·F 3 Perform weighted fusion on the fused feature map F R and the third feature map F 3 to obtain the target image.

[0098] First, perform pooling operations on the fused feature map F R and the third feature map F 3 with the same number of channels respectively, construct channel dimension vectors using the obtained pooling values, input the first channel dimension vector C FR and the second channel dimension vector C F3 into the multi-layer perceptron (MLP) model, and the MLP model outputs the weight ω 1 .

[0099] Obtain the learnable weight ω 1 based on the pooling values. Therefore, if the image features are different, the pooling values are different, and the corresponding learnable weight ω 1 is different. Thus, update the learnable weight ω 1 based on the multi-layer perceptron (MLP) model. Finally, obtain the target image with the same size as the input, and it has a good usage effect.

[0100] Affected by the shooting angle, the first newly added image collected by the second aerial device has rich information in its top area, but its side has scarce or even empty information due to the shooting angle or occlusion. The second newly added image collected by the second vehicle-mounted device is the opposite. The top area of the second newly added image has scarce or even empty information, but the side information is complete and rich. Therefore, fusing the first newly added image and the second newly added image can provide geometric constraint information with more abundant details. At the same time, fusing historical images can make full use of the rich features of buildings widely existing in historical images when the feature information of the first newly added image and the second newly added image is sparse, avoid the occurrence of blank image information, and achieve effective positioning of buildings.

[0101] Step Three: The specific method for matching the target point cloud data and the target image to obtain the modeling data is as follows:

[0102] Step 301: Determine the matching domain: Use the CornerNet model to read the corner points of the m-th target image, form a target box from the corner points, and convert to obtain the ground coordinates X j of each corner point. The connection lines of the ground coordinates of each corner point form the matching domain.

[0103] It should be noted that the CornerNet model predicts two sets of heatmaps through a convolutional network, which represent the upper-left corner position and the lower-right corner position of the target image respectively. The upper-left corner position and the lower-right corner position are the corner points, and the upper-left corner position and the lower-right corner position form the target box.

[0104] Step 302, Determine the matching line: Divide one side of the target box into n equal parts to obtain n + 1 parallel lines. Randomly select a point on each parallel line and convert to obtain the ground coordinates X of each point d of each point, the ground coordinates X d Connect them to form the first matching line;

[0105] Step 303, In the target point cloud, find the target point cloud subset corresponding to the matching domain, and determine at least 3 coordinate points corresponding to the ground coordinates X d in the target point cloud subset to form the second matching line, and calculate the rotation angle θ between the first matching line and the second matching line;

[0106] Step 304, Rotate the target point cloud subset by the rotation angle θ, and add the rotated target point cloud subset to the feature point set of the m-th target image to obtain the complete feature point set of the m-th target image;

[0107] Step 305, Repeat steps 301 - 304 to complete the matching of all target point cloud data and target images.

[0108] Rotate the target point cloud data corresponding to the building so that it can be matched and fused with the target image corresponding to the building, thereby completing the fusion of the aerial image. Under the building framework provided by the target image, fill in the centimeter-level accurate information brought by the target point cloud data, which can improve the accuracy of building positioning.

[0109] Step Four, Model reconstruction: Based on the modeling data, construct the 3D model and 2D model of the measured area; It should be noted that the model reconstruction is to establish the 3D model and 2D model of the buildings in the measured area based on the two-dimensional and three-dimensional integration technology.

[0110] Step Five, Model modification: Modify the 2D model and 3D model;

[0111] Modifying the 2D model includes: detecting and filling the invalid value area of the 2D model; repairing the noise area of the 2D model; clearing the suspended matter of the 2D model; setting the number of layers, corresponding multiple colors to multiple layer data respectively, and performing layered coloring rendering on the 2D model.

[0112] The modified three-dimensional model includes: detecting and filling invalid value areas of the three-dimensional model; repairing noise areas of the three-dimensional model; removing suspended matter from the three-dimensional model; setting the number of layers, corresponding multiple colors to multiple layer data respectively, and performing layered coloring rendering on the three-dimensional model; performing automatic equalization of light and color on the three-dimensional model.

[0113] Step Six, Model Display: Integrate and display the two-dimensional spatial data of the two-dimensional model and the three-dimensional spatial data of the three-dimensional model based on the two-dimensional and three-dimensional integration technology.

[0114] The two-dimensional spatial data includes vector data, elevation data, image data, and oblique data; the three-dimensional spatial data includes model data and oblique data.

[0115] The two-dimensional and three-dimensional integration technology is a new generation of GIS technology. The full English name of GIS is Geographic Information System, which is translated as Geographic Information System. Simply put, the two-dimensional and three-dimensional integration technology can integrate the two-dimensional spatial data and three-dimensional spatial data in GIS on the same platform. Among them, the established two-dimensional data can be directly used on the three-dimensional platform, and the two-dimensional spatial data can be directly visualized in the three-dimensional scene without any conversion processing. At the same time, during the use of this two-dimensional and three-dimensional integration technology, users can directly operate on the building data on the two-dimensional GIS map, and the three-dimensional GIS map of the building will also be generated synchronously in the three-dimensional scene.

[0116] For example: When the user creates the location data of a new building on the two-dimensional GIS map, through the two-dimensional and three-dimensional integration technology, the topological relationship of the building in the two-dimensional scene can be established, and by obtaining the elevation data of the building in the two-dimensional scene, the three-dimensional GIS map of the building will be generated synchronously.

[0117] Among them, the content not described in detail in the specification belongs to the prior art well-known to those skilled in the art.

[0118] The above are only embodiments of the present invention and do not impose any limitations on the present invention. Any simple modifications, changes, and equivalent structural changes made to the above embodiments according to the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A multi-source data fusion application method, characterized in that: Step 1: Obtain target point cloud data: Input the first point cloud data and the second point cloud data into a rough segmentation network, enhance the description of pixel features by learning the relationship between pixels and object region features according to the rough segmentation network, and obtain a rough segmentation result; fuse the rough segmentation result and historical point cloud data to obtain a fused point cloud, input the fused point cloud into a fine-tuning network, and the fine-tuning network outputs an accurate segmentation result to obtain the target point cloud data; Step 2: Obtain the target image: Extract features from the first newly added image, the second newly added image, and the historical image respectively to obtain the first feature map , the second feature map and the third feature map . Fuse the first feature map and the second feature map to obtain the fused feature map ; Output learnable weights based on the multi-layer perceptron (MLP) model, and fuse the third feature map and the fused feature map based on the learnable weights to obtain the target image; Step 201: The second aviation device collects the first new image of the measured area, performs aerial triangulation encryption on the preprocessed first new image to obtain the first new image; Step 202: The second vehicle-mounted device collects the second new image of the measured area, performs aerial triangulation encryption on the preprocessed second new image to obtain the second new image; Step 203: Obtain the historical image of the measured area; Step 204: Perform feature extraction on the first newly added image and the second newly added image respectively to obtain a first feature map and a second feature map , perform feature extraction on the historical image to obtain a third feature map ; Step 205: Use the BCA network to fuse the first feature map and the second feature map to obtain a fused feature map ; Step 206: Construct a multi-layer perceptron (MLP) model; Step 207. Perform global max pooling on the fused feature map to obtain a first pooling value, and construct a first channel dimension vector using the first pooling value ; Step 208: Perform global average pooling on the third feature map to obtain a second pooling value, and construct a second-channel dimension vector using the second pooling value ; Step 209: Input the first channel dimension vector and the second channel dimension vector into the multi-layer perceptron (MLP) model, and the MLP model outputs weights ; Step 2010: Based on the formula perform weighted fusion on the fused feature map and the third feature map to obtain the target image ; Step 3: Match the target point cloud data and the target image to obtain modeling data; Step 301, determining the matching area: Use the CornerNet model to read the corner points of the m-th target image, form a target box with the corner points, and convert to obtain the ground coordinates of each corner point , and the connection lines of the ground coordinates of each corner point form the matching area; Step 302, determine the matching line: Divide one side of the target box into n equal parts to obtain n + 1 parallel lines. Randomly select a point on each parallel line and convert to obtain the ground coordinates of each point , the ground coordinates of each point Connect them to form the first matching line; Step 303: In the target point cloud, find the target point cloud subset corresponding to the matching domain, and determine at least three coordinate points corresponding to the ground coordinates in the target point cloud subset to form a second matching line, and calculate the rotation angle between the first matching line and the second matching line ; ; Step 304: Rotate the corresponding rotation angle of the target point cloud subset , and add the rotated target point cloud subset to the feature point set of the m-th target image to obtain the complete feature point set of the m-th target image; Step 305: Repeat Steps 301-304 to complete the matching of all target point cloud data and target images; Step 4: Model reconstruction: Based on the modeling data, construct a three-dimensional model and a two-dimensional model of the measured area; Step 5: Model modification: Modify the two-dimensional model and the three-dimensional model; Step 6: Model display: Integrally display the two-dimensional spatial data of the two-dimensional model and the three-dimensional spatial data of the three-dimensional model.

2. A multi-source data fusion application method according to claim 1, characterized in that: The specific method of Step 1 is: Step 101: The first aviation device collects the first new point cloud data of the measured area, preprocesses the first new point cloud data, corrects the first new point cloud data to obtain the first point cloud data; Step 102: The first vehicle-mounted device collects the second new point cloud data of the measured area, preprocesses the second new point cloud data, corrects the second new point cloud data to obtain the second point cloud data; Step 103: Obtain the historical point cloud data of the measured area; Step 104: The first point cloud data, the second point cloud data and the historical point cloud data form a point cloud data set. Divide the point cloud data set into a training set, a validation set and a test set, and add corresponding labels to the point cloud data in the training set; Step 105: Construct a segmentation network, input the point cloud data in the training set into the segmentation network to obtain a predicted segmentation result, adjust the network parameters of the segmentation network according to the predicted segmentation result until the training stop condition is met, and obtain a trained segmentation network; Step 106: Input the first point cloud data and the second point cloud data into the segmentation network respectively to obtain a rough segmentation result, fuse the rough segmentation result and the historical point cloud data to obtain a fused point cloud, and slice the fused point cloud; Step 107: Construct a fine-tuning network, input the sliced data into the fine-tuning network, and the fine-tuning network outputs an accurate segmentation result to obtain the target point cloud data.

3. A multi-source data fusion application method according to claim 1, characterized in that: The modification of the 2D model in step five includes: detecting and filling the invalid value areas of the 2D model; repairing the noise areas of the 2D model; removing the suspended matter in the 2D model; setting the number of layers, corresponding multiple colors to multiple layer data respectively, and performing rendering with layer coloring on the 2D model.

4. A multi-source data fusion application method according to claim 1, characterized in that: The modification of the 3D model in step five includes: detecting and filling the invalid value areas of the 3D model; repairing the noise areas of the 3D model; removing the suspended matter in the 3D model; setting the number of layers, corresponding multiple colors to multiple layer data respectively, and performing rendering with layer coloring on the 3D model; performing automatic light and color equalization processing on the 3D model.

5. A multi-source data fusion application method according to claim 2, characterized in that: The segmentation network in step one adopts a V-net network, and a pooling layer, a 1×1 convolutional layer, and three 3×3 convolutional layers with different dilation rates are sequentially arranged at the end of the encoder of the V-net network.

6. A multi-source data fusion application method according to claim 2, characterized in that: The fine-tuning network in step one adopts a U-net network, and the U-net network uses the weighted sum of the boundary loss function and the binary cross-entropy loss as the overall loss function.

7. A multi-source data fusion application method according to claim 1 or 4, characterized in that: The two-dimensional spatial data includes vector data, elevation data, image data, and oblique data; the three-dimensional spatial data includes model data and oblique data.

Images