Target detection method and system fusing DOM and DSM data

Abstract

The invention discloses a target detection method and system fusing DOM and DSM data, and belongs to the technical field of target detection, and the method comprises the steps: obtaining original image data, carrying out the preprocessing, and making DOM data of a target region; constructing a digital surface model based on the original image data to obtain DSM data of the target area; performing feature fusion on the DOM data and the DSM data through a graph fusion model to obtain multi-modal fusion data; and making a training set based on the multi-modal fusion data, training a target detection model, and performing building identification through the target detection model. According to the invention, the DOM data and the DSM data of the target area are constructed based on the original image data, and feature fusion is carried out through the graph fusion model to obtain the multi-modal fusion data to carry out building target detection training, so that information complementation is carried out on the data of different modals, the information amount of the data is enriched, and the accuracy of building target identification is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a target detection method and system integrating DOM and DSM data. Background Art

[0002] Building target detection has a wide range of applications in urban planning, map making, environmental monitoring and other fields. With the continuous development of deep learning and computer vision technology, target detection methods based on deep learning are becoming more and more popular. Among them, convolutional neural network (CNN) is one of the most commonly used deep learning models, which has powerful feature learning and classification capabilities and can effectively recognize images. In addition, classic deep learning models such as U-Net and MaskR-CNN are also often used in target detection tasks, bringing more possibilities for the application of building target detection.

[0003] At present, building target detection generally extracts building information from remote sensing images or aerial images. Although the accuracy of target detection can be improved by constructing a specific neural network model, it also increases the complexity of the algorithm. In the past two years, a small number of target detection methods that combine DOM data and DSM data have emerged, but they still use conventional machine learning classification models, and the classification accuracy still needs to be improved.

[0004] Therefore, a new building object detection method is needed to improve the detection accuracy. Summary of the invention

[0005] In view of this, the present invention proposes a target detection method and system integrating DOM and DSM data, which are used to solve the problem of low accuracy in building target detection.

[0006] In a first aspect, the present invention discloses a target detection method integrating DOM and DSM data, the method comprising:

[0007] Obtain the original image data taken by the drone and pre-process it to produce DOM data of the target area;

[0008] Build a digital surface model based on the original image data to obtain the DSM data of the target area;

[0009] The features of DOM data and DSM data are fused through the graph fusion model to obtain multimodal fusion data;

[0010] A training set is created based on multimodal fusion data and a target detection model is trained, and buildings are identified through the target detection model.

[0011] Based on the above technical solution, preferably, obtaining the original image data of the drone aerial photography and preprocessing it to prepare the DOM data of the target area specifically includes:

[0012] Preprocessing the original image to generate a single orthophoto set, and then stitching it to obtain a DOM map of the target area. The preprocessing includes distortion correction, image rotation, quick view creation and image format conversion.

[0013] Perform image enhancement processing on the DOM overall image to obtain DOM data of the target area.

[0014] On the basis of the above technical solution, preferably, the step of constructing a digital surface model based on the original image data to obtain the DSM data of the target area specifically includes:

[0015] Perform grayscale stretching on the original image and adjust the brightness;

[0016] Perform distortion correction processing on the adjusted image to eliminate the distortion difference and obtain image data without distortion difference;

[0017] Perform aerial triangulation on the distortion-free image data to construct a digital surface model;

[0018] The digital surface model is edited, checked, edged and mosaicked to obtain the DSM data of the target area.

[0019] On the basis of the above technical solution, preferably, the graph fusion model includes a feature extraction layer, a graph construction layer, a graph convolution layer, a graph attention layer, a fusion layer and an output layer connected in sequence.

[0020] On the basis of the above technical solution, preferably, the feature fusion of DOM data and DSM data by the graph fusion model to obtain multimodal fusion data specifically includes:

[0021] Feature points of DOM data and DSM data are extracted respectively through the feature extraction layer;

[0022] A graph structure is constructed through the graph construction layer, using the extracted feature points as nodes and the connections between the feature points as edges;

[0023] Propagate and update node features of the graph structure using a graph neural network through a graph convolution layer;

[0024] The attention mechanism is introduced through the graph attention layer, and the node features are weightedly aggregated by calculating the attention weights between nodes;

[0025] The fused node features are fused through the fusion layer to obtain multimodal fusion data;

[0026] The multimodal fusion data is output through the output layer.

[0027] On the basis of the above technical solution, preferably, the extracting feature points of the DOM data and the DSM data respectively through the feature extraction layer specifically includes:

[0028] For DSM data, based on the height threshold, the area with a height change greater than the first preset threshold is extracted, and the feature points with a slope greater than the second preset threshold are screened out from the area with a height change greater than the first preset threshold as height feature points; the edge or shape is extracted as the contour feature point based on the morphological operation; the local feature is extracted as the positioning feature point based on the feature descriptor;

[0029] For DOM data, color feature points are screened based on color thresholds; local texture features are extracted as positioning feature points based on statistical methods; edge shapes in the data are extracted as contour feature points based on edge detection; and ground object information in DOM data is extracted based on pattern recognition.

[0030] On the basis of the above technical solution, preferably, the step of constructing a graph structure by using the extracted feature points as nodes and the connection relationships between the feature points as edges through the graph construction layer specifically includes:

[0031] Determine the correlation between the surface height information in the DSM data and the surface texture and color in the DOM data, and obtain the correspondence between the height feature points of the DSM data and the color feature points and positioning feature points of the DOM data;

[0032] Match the height change information of the ground objects in the DSM data with the ground object information in the DOM data to obtain the corresponding relationship between the height feature points in the DSM data and the ground object information in the DOM data;

[0033] Match the corresponding relationship between the contour feature points in the DSM data and the contour feature points in the DOM data;

[0034] Match the corresponding relationship between the positioning feature points in the DSM data and the positioning feature points in the DOM data;

[0035] Each feature point is used as a node of the graph structure, the correlation degree of feature information between corresponding feature points is calculated, two feature points whose correlation degree of feature information is greater than a third preset threshold are connected to form an edge of the graph structure, and a graph structure is constructed.

[0036] In a second aspect of the present invention, a target detection system integrating DOM and DSM data is disclosed, the system comprising:

[0037] DOM data module: used to obtain the original image data of drone aerial photography and pre-process it to produce DOM data of the target area;

[0038] DSM data module: used to build a digital surface model based on the original image data to obtain the DSM data of the target area;

[0039] Feature fusion module: used to perform feature fusion on DOM data and DSM data through the graph fusion model to obtain multimodal fusion data;

[0040] Target detection module: used to create training sets based on multimodal fusion data and train target detection models, and identify buildings through target detection models.

[0041] In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor, at least one memory, a communication interface and a bus;

[0042] Wherein, the processor, memory, and communication interface communicate with each other via the bus;

[0043] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to the first aspect of the present invention.

[0044] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method described in the first aspect of the present invention.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1) The present invention constructs DOM data and DSM data of the target area based on the original image data, and performs feature fusion of the DOM data and the DSM data through a graph fusion model to obtain multimodal fusion data for building target detection training. Since the multimodal data is fused, the buildings in the image can be more accurately identified, thereby improving the accuracy of building target recognition.

[0047] 2) The present invention extracts feature points of DOM data and DSM data through a graph fusion model and constructs a graph structure between the two modalities. It can learn the dependency relationship and interaction information between different modalities, and use data from different modalities to complement information, make up for data missing caused by a single type of data, enrich the amount of data information, and improve the accuracy of feature expression.

[0048] 3) The present invention performs feature matching through height features, contour features, terrain information, positioning features, texture and color features, etc., associates corresponding feature points, and thus constructs a graph structure, which can provide a good initial value for the graph fusion model and improve the fusion efficiency of the graph fusion model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a flow chart of the target detection method of fusing DOM and DSM data of the present invention;

[0051] Figure 2 This is an example diagram of a high-definition DOM orthophoto of the present invention;

[0052] Figure 3 This is a schematic diagram of the structure of the PSPNet model used in the present invention.

[0053] Figure 4 This is an example diagram of feature extraction results based on the PSPNet model. DETAILED DESCRIPTION

[0054] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] See also Figure 1 The present invention proposes a target detection method that integrates DOM and DSM data, the method comprising:

[0056] S1. Obtain the original image data of the drone aerial photography and pre-process it to produce DOM data of the target area.

[0057] Step S1 specifically includes the following sub-steps:

[0058] S11. Use a drone to capture the required original images, pre-process the original images, generate a single orthophoto set, and then stitch them together to obtain a DOM map of the target area.

[0059] The specific preprocessing includes distortion correction, image rotation, quick view creation and image format conversion. After the preprocessing is completed, the single-chip orthophoto software is used to generate a single-chip orthophoto set, which is then stitched together to obtain the DOM map of the target area. Figure 2 Shown is an example of a high-definition DOM orthophoto.

[0060] S12, performing image enhancement processing on the DOM overall image to obtain DOM data of the target area.

[0061] Based on the DOM overall image, image enhancement technology is used for further processing to enhance the contrast of the image and improve the image quality. The present invention uses the local mean calculation method to enhance local features and provide better training data for subsequent network model training.

[0062] The local mean image enhancement formula can be expressed as: Where m x (i,j) represents the local mean, x(k,l) represents the pixel value at the coordinate (k,l) in the original image, n is the defined local window size, and (i,j) is the center coordinate of the local window. The function of this formula is to calculate the mean in the local area of ​​the image, and then use this mean to enhance the image. After image enhancement, the image can be used as valid DOM data and input into the network model.

[0063] S2. Build a digital surface model based on the original image data to obtain DSM data of the target area.

[0064] Step S2 specifically includes the following sub-steps:

[0065] S21, preprocessing the original image, stretching the image grayscale and adjusting the brightness;

[0066] S22, performing distortion correction processing on the pre-processed original image to eliminate the distortion difference and obtain image data without distortion difference;

[0067] S23, gradually executing the three steps of point cloud encryption, point cloud filtering and smoothing, performing aerial triangulation, and constructing a digital surface model corresponding to the image;

[0068] S24, the digital surface model is edited and checked, and then the edges are connected according to the principle of connecting the edges in the southeast, northwest and northeast directions, and the single-scene digital surface models after the edges are connected are mosaicked to obtain the DSM data of the target area.

[0069] S3. Feature fusion of DOM data and DSM data is performed through the graph fusion model to obtain multimodal fusion data.

[0070] The graph fusion model of the present invention includes a feature extraction layer, a graph construction layer, a graph convolution layer, a graph attention layer, a fusion layer and an output layer which are connected in sequence.

[0071] Among them, the feature extraction layer is used to extract feature points from DOM data and DSM data respectively.

[0072] Specifically, for DSM data, the area with a height change greater than a first preset threshold is extracted based on the height threshold, and the feature points with a slope greater than a second preset threshold are screened out from the area with a height change greater than the first preset threshold as height feature points; the edge or shape of the area is extracted as contour feature points based on morphological operations; local features are extracted as positioning feature points based on feature descriptors;

[0073] For DOM data, color feature points are screened based on color thresholds to distinguish roads or buildings, etc.; local texture features are extracted as positioning feature points based on statistical data; edge shapes in images are extracted as contour feature points based on edge detection; and ground object information in DOM data is extracted based on pattern recognition.

[0074] The present invention uses different methods to process and obtain different feature points, which will be used as nodes for constructing a graph structure. Enriching the types of nodes is helpful for data fusion.

[0075] Graph construction layer: used to construct a graph structure using the extracted feature points as nodes and the connection relationships between the feature points as edges.

[0076] Specifically, first, determine the correlation between the surface height information in the DSM data and the surface texture and color in the DOM data, and obtain the correspondence between the height feature points of the DSM data and the color feature points of the DOM data; match the height change information of the ground object in the DSM data with the ground object information in the DOM data, and obtain the correspondence between the height feature points in the DSM data and the ground object information in the DOM data;

[0077] Then, the corresponding relationship between the contour feature points in the DSM data and the contour feature points in the DOM data is matched; the corresponding relationship between the positioning feature points in the DSM data and the positioning feature points in the DOM data is matched;

[0078] Finally, each feature point is used as a node of the graph structure, the correlation degree of feature information between corresponding feature points is calculated, and two feature points whose correlation degree of feature information is greater than a third preset threshold are connected to form an edge of the graph structure, thereby constructing a graph structure.

[0079] Among them, when judging the degree of association of feature information between nodes to form the edge of the graph structure model, the nodes need to be classified, and the judgment methods between different types of feature points are different. For example, in terms of height information, it is necessary to judge the surface height information in the DSM data and the surface texture and color in the DOM data according to the corresponding relationship to obtain the relationship between the corresponding feature points; in terms of object shape, the height change information of the ground object in the DSM data can be matched with the ground object information in the DOM data to associate the corresponding feature points. After that, the entropy method can be used to calculate the weight of the edge, and the entropy value of the feature information and the building is used as the weight value. The weight of the edge is determined according to the degree of association of the feature information of the two nodes. According to the weight value of the feature information of the two nodes, a suitable weight threshold is set. If the weight value of the feature information is greater than the weight threshold, it can be considered that there is an edge between the two nodes. If it is less than the weight threshold, it is considered that there is no edge between the two nodes, thereby determining the connection relationship of the edge.

[0080] Weights can also be assigned based on the relationship of contextual information. For example, feature points with higher heights and more obvious textures have higher weights. In terms of task requirements, different feature point relationships have different overall weight values. For example, if the algorithm is used to detect buildings, the weight values ​​of the building's texture features and height information will be relatively increased during the graph structure construction process.

[0081] Based on the edge connection relationship determined above, data structures such as adjacency matrix or adjacency list can be used to represent the graph structure. For the adjacency matrix, an n×n matrix can be used to represent the weighted graph, where n is the number of nodes. Each element in the matrix represents the weight value of the edge between two nodes. For the adjacency list, a linked list can be used to represent the edges in the weighted graph, and each linked list contains two pieces of information: the two vertices it connects and the weight of the edge between them.

[0082] Graph convolution layer: used to propagate and update node features of the graph structure using a graph neural network.

[0083] Since the nodes of the present invention are feature information, when the feature information changes, the corresponding number of nodes and the connection relationship of the edges will also change, so the graph structure needs to be updated and maintained in a timely manner. Each feature point in the graph structure represents a valid feature, and the edges between points indicate the relationship between features.

[0084] Graph attention layer: used to introduce the attention mechanism and perform weighted aggregation of node features by calculating the attention weights between nodes.

[0085] The weighted aggregation method is to assign a weight to each individual, then multiply each individual by the corresponding weight and add them together to obtain a weighted sum as the aggregation result. This method can take into account the importance and differences of individuals and can better reflect the overall characteristics.

[0086] Attention weights can be calculated based on the relationship between nodes and feature similarity, so as to more specifically fuse information from different modalities. Through the attention mechanism, the graph fusion model can more flexibly handle the correlation or dependency between multimodal data and focus more on important modalities.

[0087] Fusion layer: used to fuse the fused node features to obtain multimodal fusion data.

[0088] The present invention updates the graph structure and fuses the feature nodes through a graph neural network or a graph convolutional network. After the graph structure fusion processing training, multimodal fusion data is obtained, which can effectively remove noise and avoid mutual influence between images, so that the feature extraction in the subsequent steps can achieve better results.

[0089] Output layer: used to output multimodal fusion data.

[0090] The graph fusion model of the present invention establishes connections between multimodal data and captures their correlation or dependency. It optimizes the feature propagation and fusion process of the graph structure through the graph neural network, which can better integrate multimodal data and extract useful information.

[0091] Assume there are N modes, and the characteristics of each mode are expressed as Where i = 1, 2, ..., N. The goal of the graph fusion model is to learn a comprehensive multimodal feature representation Z∈R d , where d is the feature dimension after fusion.

[0092] For each modality i, use the feature extractor f i (·) Extract modal features H i :

[0093] H i =f i (X i )

[0094] Graph neural network propagation is used to update the feature representation of graph structure nodes. Through multiple iterative propagation, the node features are gradually integrated and updated.

[0095] Z0=[H1,H2,...,H N ]

[0096] Z k+1 =GNN(Z k ,A)

[0097] Among them, Z0 is the initial node feature representation of the graph structure, Z kis the node feature representation after the kth iteration, GNN(·) is the graph neural network propagation function, and A is the adjacency matrix of the graph structure.

[0098] In the last layer of the graph neural network, the fused node features are fused to obtain the final multimodal feature representation Z:

[0099] Z=Fusion(Z k )

[0100] Fusion(·) represents the fusion operation of the fusion layer.

[0101] S4. Create a training set based on multimodal fusion data and train a target detection model, and use the target detection model to identify buildings.

[0102] The multimodal fusion data obtained through feature annotation processing is used to create a training set for target detection, and a target detection model that can automatically identify buildings is trained.

[0103] Step S4 specifically includes the following sub-steps:

[0104] S41, feature labeling of building targets in the multimodal fusion data, and using the feature-labeled images as a training set for a target detection model;

[0105] S42, using the multimodal fusion data as input, training the PSPNET model through the training set, observing the recognition results and error values ​​output by the PSPNET model, and selecting the model with accurate recognition results and error values ​​as the target detection model to avoid overfitting in the training process;

[0106] S43, select a set of test set data, input it into the target detection model obtained in S42, and observe the recognition accuracy and error value of the model. If the error value is small, it means that the trained network model can accurately recognize the buildings in the image;

[0107] S44, inputting the multimodal fusion data of the image to be tested into the target detection model to automatically extract the building target in the image.

[0108] The present invention adopts the trained PSPNET model as the target detection model. The PSPNet (Pyramid SceneParsing Network) model is a deep learning model for image semantic segmentation, which aims to classify each pixel in the image, thereby dividing the image into different semantic regions. The design of the PSPNet model mainly revolves around using pyramid pooling to obtain multi-scale information and improve the performance of semantic segmentation by fusing this information.

[0109] like Figure 3 The figure shows the structural diagram of the PSPNet model. The PSPNet model first uses a convolutional neural network (usually based on ResNet or VGG, etc.) as a feature extractor to extract features of the input image layer by layer through multiple convolutional layers. These features contain semantic information of different scales at different levels. In order to capture contextual information of different scales, the PSPNet model introduces a pyramid pooling module. This module divides the feature map into multiple regions of different sizes and performs a pooling operation on each region. Specifically, the PSPNet model uses hole convolutions of different sizes (with different expansion rates) to generate feature maps of different scales, and then performs pooling operations on these feature maps, average pooling or maximum pooling on the features in each region to obtain a feature vector of a fixed size. As shown in Figure 4 Shown is an example graph of feature extraction results of the PSPNet model.

[0110] In order to fuse multi-scale feature information into the final semantic segmentation result, the PSPNet model uses an upsampling operation. It restores the feature vector output by the pyramid pooling module to the original size of the input image through an upsampling operation, and fuses these features with the original feature map. This fusion operation can be achieved through a simple connection or convolution operation. Finally, the PSPNet model uses a classifier to classify the fused feature map at the pixel level. Usually, a 1x1 convolution layer or a fully connected layer is used to reduce the dimension of the features and output the probability distribution of each pixel belonging to different categories. Through the design of pyramid pooling and feature fusion, the PSPNet model can make full use of multi-scale contextual information and improve the performance of semantic segmentation. It has achieved good results in various image semantic segmentation tasks and has a wide range of application value for scene understanding, image analysis and other applications.

[0111] The server operating system used in the present invention is Windows 10, the programming language used is Python 3.8, and the programming environment is PyCharm Community; in terms of hardware, the computer processor is Intel (R) Core (TM) i9-10900K CPU @ 3.70GHz, and the running memory is 64GB. The best model after training is used to predict the test data set to obtain the building recognition result map.

[0112] The training set data used in the present invention is 300 multimodal fusion image data, and the test set data is 100 multimodal fusion image data. Both the training set and the test set data need to be feature labeled to verify the calculation results of the network model.

[0113] Corresponding to the above method embodiment, the present invention also proposes a target detection system integrating DOM and DSM data, the system comprising:

[0114] DOM data module: used to obtain the original image data of drone aerial photography and pre-process it to produce DOM data of the target area;

[0115] DSM data module: used to build a digital surface model based on the original image data to obtain the DSM data of the target area;

[0116] Feature fusion module: used to perform feature fusion on DOM data and DSM data through the graph fusion model to obtain multimodal fusion data;

[0117] Target detection module: used to create training sets based on multimodal fusion data and train target detection models, and identify buildings through target detection models.

[0118] The above system embodiments and method embodiments correspond one to one, and for a brief description of the system embodiments, please refer to the method embodiments.

[0119] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0120] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the method described in the embodiment of the present invention. The storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0121] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be distributed to multiple network units. A person skilled in the art may select some or all of the modules according to actual needs to achieve the purpose of the solution of this embodiment without creative effort.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A target detection method integrating DOM and DSM data, characterized in that: The method comprises: Obtain the original image data taken by the drone and pre-process it to produce DOM data of the target area; Build a digital surface model based on the original image data to obtain the DSM data of the target area; The features of DOM data and DSM data are fused through the graph fusion model to obtain multimodal fusion data; A training set is created based on multimodal fusion data and a target detection model is trained, and buildings are identified through the target detection model.

2. The target detection method of integrating DOM and DSM data according to claim 1, characterized in that: The method of obtaining the original image data of the drone aerial photography and preprocessing it to produce the DOM data of the target area specifically includes: Preprocessing the original image to generate a single orthophoto set, and then splicing to obtain a DOM map of the target area; the preprocessing includes distortion correction, image rotation, creation of a quick view and image format conversion; Perform image enhancement processing on the DOM overall image to obtain the DOM data of the target area 。 3. The target detection method of integrating DOM and DSM data according to claim 1, characterized in that: The method of constructing a digital surface model based on the original image data to obtain DSM data of the target area specifically includes: Perform grayscale stretching on the original image and adjust the brightness; Perform distortion correction processing on the adjusted image to eliminate the distortion difference and obtain image data without distortion difference; Perform aerial triangulation on the distortion-free image data to construct a digital surface model; The digital surface model is edited, checked, edged and mosaicked to obtain the DSM data of the target area.

4. The target detection method of fusing DOM and DSM data according to claim 1, characterized in that: The graph fusion model includes a feature extraction layer, a graph construction layer, a graph convolution layer, a graph attention layer, a fusion layer and an output layer which are connected in sequence.

5. The target detection method of fusing DOM and DSM data according to claim 4, characterized in that: The method of performing feature fusion of DOM data and DSM data through a graph fusion model to obtain multimodal fusion data specifically includes: Feature points of DOM data and DSM data are extracted respectively through the feature extraction layer; A graph structure is constructed through the graph construction layer, using the extracted feature points as nodes and the connections between the feature points as edges; Propagate and update node features of the graph structure using a graph neural network through a graph convolution layer; The attention mechanism is introduced through the graph attention layer, and the node features are weightedly aggregated by calculating the attention weights between nodes; The fused node features are fused through the fusion layer to obtain multimodal fusion data; The multimodal fusion data is output through the output layer.

6. The target detection method of fusing DOM and DSM data according to claim 5, characterized in that: The extracting of feature points from the DOM data and the DSM data respectively through the feature extraction layer specifically includes: For DSM data, extract the area where the height change is greater than the first preset threshold, and select the feature points whose slope is greater than the second preset threshold from the area where the height change is greater than the first preset threshold as the height feature points; extract the edge or shape as the contour feature point based on the morphological operation; extract the local feature as the positioning feature point based on the feature descriptor; For DOM data, color feature points are screened based on color thresholds; local texture features are extracted as positioning feature points based on statistical methods; edge shapes in the data are extracted as contour feature points based on edge detection; and ground object information in DOM data is extracted based on pattern recognition.

7. The target detection method of fusing DOM and DSM data according to claim 5, characterized in that: The step of constructing a graph structure by using the extracted feature points as nodes and the connection relationships between the feature points as edges through the graph construction layer specifically includes: Determine the correlation between the surface height information in the DSM data and the surface texture and color in the DOM data, and obtain the correspondence between the height feature points of the DSM data and the color feature points and positioning feature points of the DOM data; Match the height change information of the ground objects in the DSM data with the ground object information in the DOM data to obtain the corresponding relationship between the height feature points in the DSM data and the ground object information in the DOM data; Match the corresponding relationship between the contour feature points in the DSM data and the contour feature points in the DOM data; Match the corresponding relationship between the positioning feature points in the DSM data and the positioning feature points in the DOM data; Each feature point is used as a node of the graph structure, the correlation degree of feature information between the feature points is calculated, two feature points whose correlation degree of feature information is greater than a third preset threshold are connected to form an edge of the graph structure, and a graph structure is constructed.

8. A target detection system integrating DOM and DSM data, characterized in that: The system comprises: DOM data module: used to obtain the original image data of drone aerial photography and pre-process it to produce DOM data of the target area; DSM data module: used to build a digital surface model based on the original image data to obtain the DSM data of the target area; Feature fusion module: used to perform feature fusion on DOM data and DSM data through the graph fusion model to obtain multimodal fusion data; Target detection module: used to create training sets based on multimodal fusion data and train target detection models, and identify buildings through target detection models.

9. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; Wherein, the processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to any one of claims 1 to 7.

Images