Remote sensing image building identification method and system based on multi-neural network integration, and electronic equipment

Abstract

The invention belongs to the technical field of image recognition, and provides a remote sensing image building recognition method and system based on multi-neural network integration, and electronic equipment. The method comprises the steps of original edge network construction, two-class standard building data set construction, multi-class model parameter training, positioning information acquisition, remote sensing image acquisition, region related information extraction, main building type determination, airspace rule matching, parameter and rule lowering, model parameter updating and image recognition. According to the invention, the original edge network is constructed, so that the requirements on model parameters and computing power are reduced; through the trained multi-class model parameters, region related information extraction and main building type determination, adaptive matching of the model parameters according to different scene types is realized, the building identification accuracy is improved, and the volume of the edge model is reduced; through airspace rule matching, movement or flight of edge equipment is limited, and potential risks of personnel and buildings are reduced.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a remote sensing image building recognition method, system and electronic equipment based on multi-neural network integration. Background Art

[0002] With the rapid development of remote sensing technology, high-resolution remote sensing imagery is increasingly being used in urban planning, environmental monitoring, disaster assessment, and other fields. Accurate building extraction is a critical task in these applications, directly impacting the accuracy of subsequent data analysis and decision support. However, traditional deep learning models typically have complex network structures and high computational requirements, making them difficult to effectively deploy in resource-constrained environments such as embedded devices, drones, and low-power edge computing devices. To address this issue, designing lightweight deep learning models within limited computing resources that both meet the requirements for efficient inference and accurately extract buildings from complex and diverse remote sensing imagery has become a hot topic in both academia and industry. The design of lightweight models requires achieving a good balance between computational efficiency and accuracy while ensuring model performance by optimizing the network structure, number of parameters, and computational complexity. Furthermore, the data characteristics of high-resolution remote sensing imagery place higher demands on model design. Among various network models, the symmetric encoder-decoder structure of the U-Net is well-suited for pixel-level segmentation tasks. The encoder gradually extracts high-level features, while the decoder gradually restores spatial resolution. Combined with the detailed information in the feature maps, this helps accurately extract building boundaries, especially for complex shapes and small-scale buildings. Furthermore, the U-Net decoder incorporates skip connections, fusing shallow, high-resolution features from the encoder with deep, semantic features from the decoder. For building extraction from remote sensing imagery, this multi-scale feature fusion better captures building shape details and context. This ensures the highest possible accuracy and robustness in remote sensing image information extraction.

[0003] However, the existing recognition models have high computing power requirements and are not suitable for edge device deployment. In addition, the models commonly used are general models. The advantage of general models is that they do not need to be replaced and have a wide range of applications. However, the disadvantage of general models is that if the accuracy of multiple types of building recognition is required, a large amount of data is required for training. The trained model is too large and the model response speed is slow, which is not suitable for edge device deployment. Conventional recognition methods all use a single local model for recognition. This method has the defects of high computing power consumption and poor regional information perception. In addition, the existing recognition methods only apply the recognition results in a single way, usually performing obstacle avoidance and other operations through the identified buildings. However, the risks of different building areas are different, and flight restrictions on edge devices are often required. The lack of such rules and strategies leads to higher potential risks of edge devices to people and buildings. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a remote sensing image building recognition method, system and electronic equipment based on multi-neural network integration to solve the problems of high computing power requirements, high model parameters, single model parameters, poor regional information perception ability and lack of rule strategy restrictions in the existing methods.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A multi-neural network integrated remote sensing image building recognition method, comprising:

[0007] Build the Mobilenet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network;

[0008] Construct regular building datasets and multiple single-category building datasets;

[0009] After training the original edge network using a conventional building dataset, performing secondary training using each of the single-category building datasets to obtain several sets of targeted model parameters, and storing all of the targeted model parameters in a database on a cloud server to obtain several sets of model parameter packages;

[0010] Obtaining location information of the edge device and uploading the location information to the cloud server;

[0011] Acquiring a satellite remote sensing image of an area corresponding to the positioning information;

[0012] Analyzing the satellite remote sensing image using the information recognition model of the cloud server to obtain regional related information;

[0013] Determine the main building type in the area where the edge device is located using a main building statistical formula based on the relevant information of the area;

[0014] Match the area-related information with a preset airspace division rule engine to obtain the target area airspace rules;

[0015] Matching the model parameter package in the multi-model parameter library in the cloud server according to the main building type, and transferring the target area airspace rule and the matched model parameter package to the edge device;

[0016] Updating the model parameter package to the original building recognition model of the edge device to obtain an optimized building recognition model;

[0017] The optimized building recognition model is used to recognize images collected during the movement according to the airspace rules of the target area to obtain a building recognition result.

[0018] Preferably, constructing a Mobilenet encoder, a Transformer encoder, a U-Net decoder, and a segmentation head output connected in sequence to obtain the original edge network includes:

[0019] Using the Mobilenet encoder to perform multiple downsampling operations on the input image to generate feature maps of different scales, and obtain the highest layer feature map;

[0020] Using the Transformer encoder to perform sequence flattening, position encoding addition, and several layer encoding processing on the highest layer feature map to obtain a feature sequence containing global context information;

[0021] Using the U-Net decoder to perform 2D feature map reshaping, cascade fusion of features at different levels, skip connection, and step-by-step upsampling on the feature sequence to obtain a final feature map;

[0022] The final feature map is convolved and activated using the segmentation head output to obtain a building recognition result.

[0023] Preferably, conventional building datasets and multiple single-category building datasets are constructed, including:

[0024] Construct a single-class building dataset; the building types in the single-class building dataset include: residential buildings, office buildings, commercial buildings, industrial buildings, educational buildings, public buildings, and cultural and religious buildings;

[0025] Counting the proportion of each type of building data set to obtain building type proportion data;

[0026] Data is extracted from each of the single-type building datasets according to the building type proportion data to obtain the conventional building dataset; the proportion of each type of building in the conventional building dataset is the same as the building type proportion data.

[0027] Preferably, after the original edge network is trained using a conventional building dataset, secondary training is performed using each of the single-category building datasets to obtain several sets of targeted model parameters, and all of the targeted model parameters are stored in a database on a cloud server to obtain several sets of model parameter packages;

[0028] Labeling the buildings in the conventional building dataset and the single-category building dataset to obtain building data to be identified;

[0029] Training the original edge network using the conventional building dataset according to a preset number of iterations to obtain an initial universal recognition model;

[0030] removing the conventional building dataset, and using the single-category building dataset to train the initial universal recognition model to obtain the targeted model parameters;

[0031] The targeted model parameters and the building types corresponding to the targeted model parameters are bound and stored in the database to obtain the model parameter package.

[0032] Preferably, obtaining a satellite remote sensing image of an area corresponding to the positioning information includes:

[0033] Determine the longitude and latitude coordinates of the edge device according to the positioning information;

[0034] Obtaining the maximum moving distance of the edge device;

[0035] Using the latitude and longitude coordinates as the center position and the maximum moving distance as the radius to define the potential moving range;

[0036] Calling a remote sensing image acquisition interface to acquire an original remote sensing image corresponding to the latitude and longitude coordinates;

[0037] The original remote sensing image is cropped using the potential movement range as a mask to obtain the satellite remote sensing image.

[0038] Preferably, the process of constructing the information recognition model includes:

[0039] Using the U-Net++ network to perform region segmentation on the pre-collected satellite remote sensing image to obtain a single-channel semantic mask;

[0040] Performing regional segmentation on the satellite remote sensing image using the single-channel semantic mask to obtain a single-block image;

[0041] Labeling the single block image to obtain a labeled training set;

[0042] The ResNet50 network is trained using the labeled training set to obtain the trained information recognition model.

[0043] Preferably, determining the main building type in the area where the edge device is located using a main building statistical formula based on the area-related information includes:

[0044] Construct the main building statistical formula; the expression of the main building statistical formula is: Where T is the main building type; argmax(·) represents the building type corresponding to the maximum value; max(·) represents the maximum value; k is the adjustment coefficient; n i represents the number of type i buildings in the target area; P i is the value of type i building in the proportion data of the building type; P0 is the lowest statistical threshold; I(·) means that it is 1 if the internal conditions are met, otherwise it is 0;

[0045] Extracting building data from the relevant information of the area to obtain regional building information;

[0046] The regional building information is calculated using the main building statistical formula to obtain the main building type.

[0047] Preferably, the optimized building recognition model is used to recognize images collected during movement according to the target area airspace rule to obtain a building recognition result, including:

[0048] Using the image acquisition module of the edge device to acquire an image to be recognized;

[0049] Using the optimized building recognition model to identify the image to be identified, to obtain a building recognition result;

[0050] Matching the target area airspace rules according to the building recognition results to obtain a travel strategy;

[0051] When the control instruction received by the edge device conflicts with the travel strategy, the edge device is controlled to move or fly according to the travel strategy.

[0052] Preferably, a multi-neural network integrated remote sensing image building recognition system includes:

[0053] The network construction module is used to construct the MobileNet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network;

[0054] Dataset construction module, used to construct general building datasets and multiple single-category building datasets;

[0055] A model parameter training module is configured to train the original edge network using a conventional building dataset and then perform secondary training using each of the single-category building datasets to obtain several sets of targeted model parameters, and store all of the targeted model parameters in a database on a cloud server to obtain several sets of model parameter packages;

[0056] A positioning information acquisition module is used to obtain the positioning information of the edge device and upload the positioning information to the cloud server;

[0057] A remote sensing image acquisition module, configured to acquire a satellite remote sensing image of an area corresponding to the positioning information;

[0058] A regional information extraction module, configured to analyze the satellite remote sensing image using the information recognition model of the cloud server to obtain regional information;

[0059] A main building determination module, configured to determine the main building type in the area where the edge device is located using a main building statistical formula based on the area-related information;

[0060] An airspace rule matching module is used to match the area-related information with a preset airspace division rule engine to obtain the airspace rules of the target area;

[0061] A parameter and rule decentralization module is used to match the model parameter package in the multi-model parameter library in the cloud server according to the main building type, and to decentralize the target area airspace rules and the matched model parameter package to the edge device;

[0062] A model parameter updating module, configured to update the model parameter package to the original building recognition model of the edge device to obtain an optimized building recognition model;

[0063] The image recognition module is used to use the optimized building recognition model to recognize images collected during the movement according to the target area airspace rules to obtain building recognition results.

[0064] Preferably, an electronic device comprises: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor can execute the aforementioned multi-neural network integrated remote sensing image building recognition method.

[0065] The present invention discloses the following technical effects:

[0066] The present invention provides a remote sensing image building recognition method, system and electronic equipment based on multi-neural network integration. By constructing an original edge network, the problem that the existing recognition model requires high computing power and is not suitable for edge device deployment is solved, and the model parameters and computing power requirements are reduced. By training multiple types of model parameters, extracting regional related information and determining the main building type, the problems of difficult general model training, large model parameters, high computing power requirements, slow recognition speed and poor regional information perception effect of a single local model are solved, and the model parameters are adaptively matched according to different scene types. By matching airspace rules, the problem that the lack of such rule strategies leads to high potential risks of edge devices to people and buildings is solved, and hard restrictions on the movement or flight of edge devices are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 A schematic diagram of a process for remote sensing image building recognition based on multi-neural network integration provided by an embodiment of the present invention;

[0069] Figure 2 A schematic diagram of the original edge network workflow provided by an embodiment of the present invention;

[0070] Figure 3 A schematic diagram of the process of constructing a building dataset provided by an embodiment of the present invention;

[0071] Figure 4 A schematic diagram of a model parameter acquisition process according to an embodiment of the present invention;

[0072] Figure 5 A schematic diagram of a remote sensing image acquisition process according to an embodiment of the present invention;

[0073] Figure 6 A flowchart of the original edge network training provided by an embodiment of the present invention;

[0074] Figure 7 A schematic diagram of iterative loss provided by an embodiment of the present invention;

[0075] Figure 8 A schematic diagram of the mean intersection and comparison iteration provided in an embodiment of the present invention;

[0076] Figure 9 A schematic diagram of accuracy iteration provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0078] The purpose of the present invention is to provide a remote sensing image building recognition method, system and electronic equipment based on multi-neural network integration to solve the problems of high computing power requirements, high model parameters, single model parameters, poor regional information perception ability and lack of rule strategy restrictions in existing methods.

[0079] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0080] Figure 1 A schematic diagram of a remote sensing image building recognition process based on multi-neural network integration provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the present invention provides a remote sensing image building recognition method based on multi-neural network integration, comprising:

[0081] Step 100: Build the Mobilenet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network;

[0082] Step 200: Constructing a conventional building dataset and multiple single-category building datasets;

[0083] Step 300: After training the original edge network using a conventional building dataset, secondary training is performed using each of the single-class building datasets to obtain several sets of targeted model parameters, and all of the targeted model parameters are stored in a database on a cloud server to obtain several sets of model parameter packages;

[0084] Step 400: Obtaining the location information of the edge device and uploading the location information to the cloud server;

[0085] Step 500: Acquire a satellite remote sensing image of the area corresponding to the positioning information;

[0086] Step 600: Analyze the satellite remote sensing image using the information recognition model of the cloud server to obtain region-related information;

[0087] Step 700: Determine the main building type in the area where the edge device is located using a main building statistical formula based on the area-related information;

[0088] Step 800: Match the region-related information with a preset airspace division rule engine to obtain the target region airspace rules;

[0089] Step 900: Match the model parameter package in the multi-model parameter library in the cloud server according to the main building type, and transfer the target area airspace rule and the matched model parameter package to the edge device;

[0090] Step 1000: updating the model parameter package to the original building recognition model of the edge device to obtain an optimized building recognition model;

[0091] Step 1100: Using the optimized building recognition model according to the target area airspace rule, the images collected during the movement are recognized to obtain a building recognition result.

[0092] refer to Figure 2 , construct the Mobilenet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network, including:

[0093] Step 101: using the MobileNet encoder to perform multiple downsampling operations on the input image to generate feature maps of different scales, and obtain the highest-level feature map;

[0094] Step 102: Using the Transformer encoder, perform sequence flattening, position encoding, and several layer encoding processes on the highest-level feature map to obtain a feature sequence containing global context information;

[0095] Step 103: Using the U-Net decoder, reshape the feature sequence into a 2D feature map, perform cascade fusion of features at different levels, perform skip connections, and perform step-by-step upsampling to obtain a final feature map;

[0096] Step 104: Using the segmentation head output, perform convolution and activation processing on the final feature map to obtain a building recognition result.

[0097] refer to Figure 3, constructing regular building datasets and various single-class building datasets, including:

[0098] Step 201: Construct a single-class building dataset; the building types in the single-class building dataset include: residential buildings, office buildings, commercial buildings, industrial buildings, educational buildings, public buildings, and cultural and religious buildings;

[0099] Step 202: Count the proportion of each type of building data set to obtain building type proportion data;

[0100] Step 203: extracting data from each of the single-type building datasets according to the building type proportion data to obtain the conventional building dataset; the proportion of each type of building in the conventional building dataset is the same as the building type proportion data.

[0101] refer to Figure 4 After training the original edge network using a conventional building dataset, performing secondary training using each of the single-class building datasets to obtain several sets of targeted model parameters, and storing all of the targeted model parameters in a database on a cloud server to obtain several sets of model parameter packages;

[0102] Step 301: Labeling the buildings in the conventional building dataset and the single-category building dataset to obtain building data to be identified;

[0103] Step 302: training the original edge network using the conventional building dataset according to a preset number of iterations to obtain an initial universal recognition model;

[0104] Step 303: removing the conventional building dataset, and using the single-category building dataset to train the initial general recognition model to obtain the targeted model parameters;

[0105] Step 304: Bind the targeted model parameters and the building types corresponding to the targeted model parameters and store them in the database to obtain the model parameter package.

[0106] refer to Figure 5 , obtaining a satellite remote sensing image of an area corresponding to the positioning information, including:

[0107] Step 501: Determine the longitude and latitude coordinates of the edge device according to the positioning information;

[0108] Step 502: Obtain the farthest moving distance of the edge device;

[0109] Step 503: Using the latitude and longitude coordinates as the center position and the maximum moving distance as the radius to define a potential moving range;

[0110] Step 504: calling a remote sensing image acquisition interface to acquire an original remote sensing image corresponding to the latitude and longitude coordinates;

[0111] Step 505: cropping the original remote sensing image using the potential movement range as a mask to obtain the satellite remote sensing image.

[0112] Preferably, the process of constructing the information recognition model includes:

[0113] Using the U-Net++ network to perform region segmentation on the pre-collected satellite remote sensing image to obtain a single-channel semantic mask;

[0114] Performing regional segmentation on the satellite remote sensing image using the single-channel semantic mask to obtain a single-block image;

[0115] Labeling the single block image to obtain a labeled training set;

[0116] The ResNet50 network is trained using the labeled training set to obtain the trained information recognition model.

[0117] Furthermore, determining the main building type in the area where the edge device is located using a main building statistical formula based on the area-related information includes:

[0118] Construct the main building statistical formula; the expression of the main building statistical formula is: Where T is the main building type; argmax(·) represents the building type corresponding to the maximum value; max(·) represents the maximum value; k is the adjustment coefficient; n i represents the number of type i buildings in the target area; P i is the value of type i building in the proportion data of the building type; P0 is the lowest statistical threshold; I(·) means that it is 1 if the internal conditions are met, otherwise it is 0;

[0119] Extracting building data from the relevant information of the area to obtain regional building information;

[0120] The regional building information is calculated using the main building statistical formula to obtain the main building type.

[0121] Specifically, the image collected during the movement is identified using the optimized building recognition model according to the target area airspace rule to obtain a building recognition result, including:

[0122] Using the image acquisition module of the edge device to acquire an image to be recognized;

[0123] Using the optimized building recognition model to identify the image to be identified, to obtain a building recognition result;

[0124] Matching the target area airspace rules according to the building recognition results to obtain a travel strategy;

[0125] When the control instruction received by the edge device conflicts with the travel strategy, the edge device is controlled to move or fly according to the travel strategy.

[0126] Furthermore, a multi-neural network integrated remote sensing image building recognition system includes:

[0127] The network construction module is used to construct the MobileNet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network;

[0128] Dataset construction module, used to construct general building datasets and multiple single-category building datasets;

[0129] A model parameter training module is configured to train the original edge network using a conventional building dataset and then perform secondary training using each of the single-category building datasets to obtain several sets of targeted model parameters, and store all of the targeted model parameters in a database on a cloud server to obtain several sets of model parameter packages;

[0130] A positioning information acquisition module is used to obtain the positioning information of the edge device and upload the positioning information to the cloud server;

[0131] A remote sensing image acquisition module, configured to acquire a satellite remote sensing image of an area corresponding to the positioning information;

[0132] A regional information extraction module, configured to analyze the satellite remote sensing image using the information recognition model of the cloud server to obtain regional information;

[0133] A main building determination module, configured to determine the main building type in the area where the edge device is located using a main building statistical formula based on the area-related information;

[0134] An airspace rule matching module is used to match the area-related information with a preset airspace division rule engine to obtain the airspace rules of the target area;

[0135] A parameter and rule decentralization module is used to match the model parameter package in the multi-model parameter library in the cloud server according to the main building type, and to decentralize the target area airspace rules and the matched model parameter package to the edge device;

[0136] A model parameter updating module, configured to update the model parameter package to the original building recognition model of the edge device to obtain an optimized building recognition model;

[0137] The image recognition module is used to use the optimized building recognition model to recognize images collected during the movement according to the target area airspace rules to obtain building recognition results.

[0138] Optionally, an electronic device comprises: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor can execute the aforementioned multi-neural network integrated remote sensing image building recognition method.

[0139] Specifically, this embodiment adopts Mobile-netV2, which introduces an inverted residual block: using an extended bottleneck structure, the number of channels is first expanded, and then depth-separable convolution is used, and the ReLU activation in the bottleneck structure is removed to avoid feature information loss. The U-Net network based on the full convolutional architecture has significant receptive field limitations in image segmentation tasks: although its core convolution operation (W*X) can effectively extract local features, the progressive local information integration mechanism achieved by stacking convolution layers is difficult to effectively model the long-range spatial dependencies between pixels. Although the network can expand the theoretical receptive field while reducing the resolution of the feature map through MaxPooling downsampling, this discretized spatial compression process will inevitably lead to the loss of fine-grained features. In contrast, the Transformer's self-attention mechanism breaks through the local receptive field limitation of the convolution kernel by calculating the association weights between any two elements in the input sequence, and exhibits stronger global context modeling capabilities. However, the traditional visual Transformer (ViT) directly embeds patches into images. While this approach effectively captures global relationships, it lacks the local feature extraction capabilities inherent in convolutional neural networks (CNNs), resulting in insufficient representation of edge regions. This deficiency manifests itself in issues such as overlooked outlines of small objects, blurred object boundaries, and loss of texture features, ultimately impacting segmentation accuracy.

[0140] Furthermore, this embodiment proposes a hybrid architecture design strategy: the first half uses a lightweight MobileNet V2 as the feature extraction backbone, reducing computational complexity while retaining the inherent local detail capture advantage of U-Net. The second half introduces Transformer modules at the deep feature level, implementing global context modeling through a self-attention mechanism. This layered collaborative mechanism avoids the loss of detailed information caused by direct application of Transformers while effectively overcoming the limitations of traditional U-Net in modeling long-range dependencies.

[0141] Optionally, data preprocessing: This embodiment performs data augmentation on the training set, test set, and validation set, that is, performing a series of transformations on the original data, such as random rotation, scaling, flipping, cropping, etc., to generate more diverse training samples. This helps the model better learn the various changes and differences in the data, thereby improving the model's generalization ability. In this process, data augmentation can be regarded as a regularization technique, which introduces additional randomness and uncertainty during the training process, helping to reduce the model's overfitting of the training data. By introducing more diverse data samples, it is more difficult for the model to remember the specific details in the training set, thereby improving the model's generalization ability.

[0142] refer to Figure 6 , the core process of feature map processing:

[0143] 1) Mobilenet encoder extracts local features: The input image first passes through the Mobilenet backbone network, and then generates feature maps of different scales through multiple downsampling stages. The final highest-level feature map is 1 / 16 of the original image size.

[0144] 2) Transformer global modeling: Flatten the highest-level feature map C4 output by the Mobilenet encoder into a sequence and add a learnable positional encoding: Z0 = x patch +pos; where Z0 is the encoder output; x patch is the flattened sequence; pos is the position code.

[0145] 3) Through several layers of TransformerEncoder, a feature sequence containing global context information is output.

[0146] 4) Decoder feature fusion: Reshape the Transformer output into a 2D feature map, fuse features at different levels through cascade operations, use skip connections to bridge the encoder and decoder, and gradually upsample to restore spatial resolution.

[0147] 5) Segmentation head output: The final feature map is processed through 1×1 convolution and softmax or sigmoid is used to generate a segmentation probability map. The output size maintains the same resolution as the original input.

[0148] Specifically, Figure 7 value Figure 8 In the equation, Loss is the loss value; TrainLoss is the loss value of the training set; Test Loss is the loss value of the test set; Epoch is the number of training rounds; IoU is the intersection over union ratio; TrainIoU is the intersection over union ratio of the training set; TestIoU is the intersection over union ratio of the test set; Accuracy is the accuracy percentage; TrainAccuracy is the accuracy of the training set; TestAccuracy is the accuracy of the test set. Experimental results: The evaluation indicators selected are cross entropy loss function, average intersection over union ratio and pixel-level accuracy. Reference Figure 7 , Cross entropy loss function: Cross entropy loss is used to measure the difference between the predicted distribution and the true distribution. The smaller the value, the closer the model prediction is to the true label.

[0149]

[0150] Among them, L is the cross entropy loss value; t c is the probability distribution of category c in the true label; C is the total number of true labels; p c The model predicts the probability value that the sample belongs to category c.

[0151] refer to Figure 8 , the mean intersection union ratio is used to measure the overlap between the predicted results and the true labels:

[0152]

[0153] Among them, mIou is the mean intersection union ratio; IoU c is the intersection-over-union ratio of category c.

[0154] refer to Figure 9 , pixel-level accuracy is used to measure the proportion of pixels predicted correctly among all pixels:

[0155]

[0156] Where PA is the pixel-level accuracy; y t-pre represents the category label of the t-th pixel predicted by the model; y t is the true category label of the t-th pixel; N is the total number of pixel indices.

[0157] According to experimental results, the improved model of the U-Net network performed well in target detection tasks such as building extraction. After training for more than 100 times, the pixel-level classification accuracy of the network model exceeded 75% and the precision reached more than 90%.

[0158] Preferably, the positioning information of the edge device is obtained, and the positioning information includes longitude and latitude coordinates; the maximum moving distance R of the edge device is obtained; the longitude and latitude coordinates are used as the center position, and the maximum moving distance R is used as the radius to determine the range of the satellite remote sensing image to be obtained; the satellite remote sensing image acquisition interface is called to obtain a rectangular satellite remote sensing image containing the range, and the size of the rectangular satellite remote sensing image meets the range coverage requirement with the center position as the center and a side length of not less than 2R; a circular mask with the image pixel point corresponding to the center position as the center and a radius of R is established; the rectangular satellite remote sensing image is cropped based on the circular mask, and the image within the circular area is retained to obtain a circular satellite remote sensing image with the positioning position as the center and the maximum moving distance of the edge device as the radius.

[0159] For more specific information, refer to Table 1 for the main building types and representative buildings within each category. Street view images were obtained through map APIs, low-altitude remote sensing images were acquired using drones, and building images were downloaded in batches from image websites. Blurred images were removed using OpenCV's Laplace variance thresholding algorithm for clarity detection. Images with a similarity exceeding 95% were removed using the pHash algorithm. Images were resized to a fixed size, maintaining the aspect ratio or filling with black borders.

[0160] Table 1

[0161]

[0162] Optionally, high-resolution remote sensing images can be acquired in batches through the remote sensing satellite interface. The images are RGB three-channel, uniformly sized to 1024×1024 pixels, and stored in PNG format. A pre-trained U-Net++ network is used for preliminary region segmentation: the original remote sensing image is input into the U-Net++ network, which outputs a single-channel semantic mask in which the pixel value corresponds to the region category. The mask is morphologically processed to eliminate small noise and retain the complete outline of the building area.

[0163] Furthermore, connected domain analysis is used to identify independent building outlines based on the building areas in the semantic mask. A minimum enclosing rectangle is generated for each building outline, and a single-block image is cropped from the original image. The cropped image is resized or filled with a black background, and the background blocks not covered by the building are retained as negative samples. The single-block image is annotated with the building type: residential building, office building, commercial building, industrial building, educational building, public building, cultural and religious building; a JSON format annotation file is generated, containing the image path, category label and area coordinates. The fully connected classification head of the original ResNet50 network is removed and replaced with a custom classification layer, and the input layer is adapted to the single-block image size. The ResNet50 network is trained using the aforementioned image set, and the model is iteratively optimized using Adam and cross entropy losses.

[0164] Preferably, this embodiment sets exclusive airspace rules for different types of buildings: 1) Flight altitude restriction: the flight altitude over residential buildings, office buildings, commercial buildings and other buildings must be 5 to 10 meters higher than the roof; 2) Flying is prohibited within a horizontal distance of 10 meters around balconies and windows of residential buildings to prevent equipment collisions, etc.; 3) Industrial building areas are set as no-fly zones, and any flying activities are strictly prohibited to prevent safety accidents caused by equipment operation, cargo transportation, etc., especially in industrial areas involving flammable, explosive, toxic and harmful substances, the no-fly range is expanded to 100 meters around; 4) In areas with dense student activities such as campus playgrounds and squares in front of teaching buildings, flying below an altitude of 30 meters is prohibited; 5) Flying is strictly prohibited inside cultural and religious buildings, etc.

[0165] The beneficial effects of the present invention are as follows:

[0166] The present invention reduces the model parameters and computing power requirements by constructing an original edge network; through the training of multiple types of model parameters, extraction of regional related information and determination of the main building type, it realizes adaptive matching of model parameters according to different scene types, improves the accuracy of building recognition, and reduces the size of the edge model; through airspace rule matching, the movement or flight of edge devices is restricted, reducing the potential risks to people and buildings.

[0167] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0168] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A multi-neural network integrated remote sensing image building recognition method, characterized in that: include: Build the Mobilenet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network; Construct regular building datasets and multiple single-category building datasets; After training the original edge network using a conventional building dataset, performing secondary training using each of the single-category building datasets to obtain several sets of targeted model parameters, and storing all of the targeted model parameters in a database on a cloud server to obtain several sets of model parameter packages; Obtaining location information of the edge device and uploading the location information to the cloud server; Acquiring a satellite remote sensing image of an area corresponding to the positioning information; Analyzing the satellite remote sensing image using the information recognition model of the cloud server to obtain regional related information; Determine the main building type in the area where the edge device is located using a main building statistical formula based on the relevant information of the area; Match the area-related information with a preset airspace division rule engine to obtain the target area airspace rules; Matching the model parameter package in the multi-model parameter library in the cloud server according to the main building type, and transferring the target area airspace rule and the matched model parameter package to the edge device; Updating the model parameter package to the original building recognition model of the edge device to obtain an optimized building recognition model; The optimized building recognition model is used to recognize images collected during the movement according to the airspace rules of the target area to obtain a building recognition result.

2. The method for remote sensing image building recognition based on multi-neural network integration according to claim 1, characterized in that: Build the Mobilenet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network, including: Using the Mobilenet encoder to perform multiple downsampling operations on the input image to generate feature maps of different scales, and obtain the highest layer feature map; Using the Transformer encoder to perform sequence flattening, position encoding addition, and several layer encoding processing on the highest layer feature map to obtain a feature sequence containing global context information; Using the U-Net decoder to perform 2D feature map reshaping, cascade fusion of features at different levels, skip connection, and step-by-step upsampling on the feature sequence to obtain a final feature map; The final feature map is convolved and activated using the segmentation head output to obtain a building recognition result.

3. The method for remote sensing image building recognition based on multi-neural network integration according to claim 2, characterized in that: Construct general building datasets and various single-category building datasets, including: Construct a single-class building dataset; the building types in the single-class building dataset include: residential buildings, office buildings, commercial buildings, industrial buildings, educational buildings, public buildings, and cultural and religious buildings; Counting the proportion of each type of building data set to obtain building type proportion data; Data is extracted from each of the single-type building datasets according to the building type proportion data to obtain the conventional building dataset; the proportion of each type of building in the conventional building dataset is the same as the building type proportion data.

4. The method for remote sensing image building recognition based on multi-neural network integration according to claim 3, characterized in that: After training the original edge network using a conventional building dataset, performing secondary training using each of the single-category building datasets to obtain several sets of targeted model parameters, and storing all of the targeted model parameters in a database on a cloud server to obtain several sets of model parameter packages; Labeling the buildings in the conventional building dataset and the single-category building dataset to obtain building data to be identified; Training the original edge network using the conventional building dataset according to a preset number of iterations to obtain an initial universal recognition model; removing the conventional building dataset, and using the single-category building dataset to train the initial universal recognition model to obtain the targeted model parameters; The targeted model parameters and the building types corresponding to the targeted model parameters are bound and stored in the database to obtain the model parameter package.

5. The method for remote sensing image building recognition based on multi-neural network integration according to claim 4, characterized in that: Acquiring a satellite remote sensing image of an area corresponding to the positioning information, including: Determine the longitude and latitude coordinates of the edge device according to the positioning information; Obtaining the maximum moving distance of the edge device; Using the latitude and longitude coordinates as the center position and the maximum moving distance as the radius to define the potential moving range; Calling a remote sensing image acquisition interface to acquire an original remote sensing image corresponding to the latitude and longitude coordinates; The original remote sensing image is cropped using the potential movement range as a mask to obtain the satellite remote sensing image.

6. The method for remote sensing image building recognition using multiple neural network integration according to claim 5, characterized in that: The construction process of the information recognition model includes: Using the U-Net++ network to perform region segmentation on the pre-collected satellite remote sensing image to obtain a single-channel semantic mask; Performing regional segmentation on the satellite remote sensing image using the single-channel semantic mask to obtain a single-block image; Labeling the single block image to obtain a labeled training set; The ResNet50 network is trained using the labeled training set to obtain the trained information recognition model.

7. The method for remote sensing image building recognition using multiple neural network integration according to claim 6, characterized in that: Determine the main building type in the area where the edge device is located using a main building statistical formula based on the area-related information, including: Construct the main building statistical formula; the expression of the main building statistical formula is: Where T is the main building type; argmax(·) represents the building type corresponding to the maximum value; max(·) represents the maximum value; k is the adjustment coefficient; n i represents the number of type i buildings in the target area; P i is the value of type i building in the proportion data of the building type; P0 is the lowest statistical threshold; I(·) means that it is 1 if the internal conditions are met, otherwise it is 0; Extracting building data from the relevant information of the area to obtain regional building information; The regional building information is calculated using the main building statistical formula to obtain the main building type.

8. The method for remote sensing image building recognition using multiple neural network integration according to claim 7, characterized in that: The optimized building recognition model is used to recognize images collected during the movement according to the target area airspace rule to obtain a building recognition result, including: Using the image acquisition module of the edge device to acquire an image to be recognized; Using the optimized building recognition model to identify the image to be identified, to obtain a building recognition result; Matching the target area airspace rules according to the building recognition results to obtain a travel strategy; When the control instruction received by the edge device conflicts with the travel strategy, the edge device is controlled to move or fly according to the travel strategy.

9. A multi-neural network integrated remote sensing image building recognition system, characterized in that: include: The network construction module is used to construct the MobileNet encoder, Transformer encoder, U-Net decoder, and segmentation head output connected in sequence to obtain the original edge network; Dataset construction module, used to construct general building datasets and multiple single-category building datasets; A model parameter training module is configured to train the original edge network using a conventional building dataset and then perform secondary training using each of the single-category building datasets to obtain several sets of targeted model parameters, and store all of the targeted model parameters in a database on a cloud server to obtain several sets of model parameter packages; A positioning information acquisition module is used to obtain the positioning information of the edge device and upload the positioning information to the cloud server; A remote sensing image acquisition module, configured to acquire a satellite remote sensing image of an area corresponding to the positioning information; A regional information extraction module, configured to analyze the satellite remote sensing image using the information recognition model of the cloud server to obtain regional information; A main building determination module, configured to determine the main building type in the area where the edge device is located using a main building statistical formula based on the area-related information; An airspace rule matching module is used to match the area-related information with a preset airspace division rule engine to obtain the airspace rules of the target area; A parameter and rule decentralization module is used to match the model parameter package in the multi-model parameter library in the cloud server according to the main building type, and to decentralize the target area airspace rules and the matched model parameter package to the edge device; A model parameter updating module, configured to update the model parameter package to the original building recognition model of the edge device to obtain an optimized building recognition model; The image recognition module is used to use the optimized building recognition model to recognize images collected during the movement according to the target area airspace rules to obtain building recognition results.

10. An electronic device, characterized in that: include: At least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor can execute a multi-neural network integrated remote sensing image building recognition method according to any one of claims 1 to 8.

Images