Remote sensing image aircraft segmentation method and system based on aggregation regulation and control module

Abstract

The invention discloses a remote sensing image aircraft segmentation method and system based on an aggregation regulation and control module, and the method comprises the steps: obtaining a remote sensing image comprising an aircraft, inputting the remote sensing image to a preprocessing module, and obtaining an input image; inputting the input image into the first convolutional layer to obtain an input feature map; inputting to an aggregation regulation and control module comprising a linear layer, a plurality of convolution units and a lightweight linear layer based on the input feature map; inputting the input features into a linear layer and a lightweight linear layer respectively to correspondingly obtain an initial feature map and a weight matrix; inputting the initial feature maps into a convolution unit in sequence to obtain a plurality of extracted feature maps of different layers; based on all the extracted feature maps, performing aggregation based on the weight matrix to obtain an aggregated feature map; and inputting the aggregated feature map to a segmentation module to obtain an aircraft segmentation result of the remote sensing image. And the segmentation precision of the aircraft target in the remote sensing image under the complex background is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image aircraft segmentation method and system based on an aggregation control module. Background Art

[0002] At present, remote sensing image segmentation technology is widely used in many fields such as geographic information system (GIS), urban planning, environmental monitoring, disaster assessment and agricultural monitoring. Remote sensing images usually have large range, complex background and diverse targets. Especially in aerial remote sensing images, images often contain a lot of background noise, ground object occlusion and multi-scale targets, which brings great challenges to target segmentation. In particular, the target objects in remote sensing images, such as airplanes and ships, are usually small and have complex backgrounds. The ground objects such as clouds, mountains and rivers in the background are often similar to the characteristics of the target objects (such as color and texture), which can easily lead to mis-segmentation. With the development of deep learning technology, convolutional neural network (CNN) methods have gradually become the mainstream of remote sensing image segmentation. Typical deep learning segmentation networks such as fully convolutional networks (FCN), U-Net and SegNet have made significant progress in medical imaging, satellite imagery and remote sensing image segmentation tasks.

[0003] However, the performance of traditional CNN models will decline when processing images with large changes in target scale, complex background and inconsistent image resolution. For the segmentation of small objects such as airplanes, traditional convolution operations may have difficulty capturing fine target features. Therefore, how to effectively utilize multi-scale information, enhance the model's context perception ability, and improve the model's segmentation accuracy for small targets has become the focus of researchers.

[0004] Therefore, how to effectively utilize multi-scale information to improve the segmentation accuracy of aircraft targets in remote sensing images is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a remote sensing image aircraft segmentation method and system based on an aggregation control module, which utilizes multi-scale context information and structure perception capabilities to improve the segmentation accuracy of aircraft targets in remote sensing images under complex backgrounds.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A remote sensing image aircraft segmentation method based on an aggregation control module, comprising:

[0008] Acquire a remote sensing image including the aircraft and input it into a preprocessing module to obtain an input image;

[0009] Inputting the input image into the first convolutional layer to obtain an input feature map;

[0010] Based on the input feature map, an aggregation control module including a linear layer, a plurality of convolutional units and a lightweight linear layer is input;

[0011] The input features are respectively input into the linear layer and the light linear layer, and an initial feature map and a weight matrix are obtained correspondingly;

[0012] The initial feature map is sequentially input into the convolution unit to obtain a plurality of extracted feature maps of different layers;

[0013] Aggregating all the extracted feature maps based on the weight matrix to obtain an aggregated feature map;

[0014] The aggregated feature map is input into a segmentation module to obtain an aircraft segmentation result of the remote sensing image.

[0015] Preferably, the preprocessing module includes: a denoising unit, an image enhancement unit, a cropping unit, a scaling unit and a normalization unit;

[0016] The remote sensing image is input to the denoising unit for denoising to obtain a denoised image;

[0017] The denoised image is input to the image enhancement unit for image enhancement to obtain an enhanced image;

[0018] The enhanced image is input to the cropping unit for cropping to obtain a cropped image;

[0019] The cropped image is input to the scaling unit for scaling processing to obtain a scaled image;

[0020] The scaled image is input to the normalization unit for format conversion to obtain the input image.

[0021] Preferably, the plurality of convolutional units are sequentially connected and have the same structure, and all include: a deep convolutional layer and an activation layer;

[0022] The initial feature map is sequentially input into the deep convolution layer and the activation layer to obtain the extracted feature map of the corresponding layer.

[0023] Preferably, the method for obtaining the aggregate feature map is:

[0024] Based on the weights corresponding to the number of channels in the weight matrix consistent with the number of layers of the extracted feature map as corresponding layer perception weights;

[0025] The aggregated feature map is obtained by performing a weighted summation based on element-by-element multiplication of all the layer perception weights and the extracted feature maps corresponding to the number of layers.

[0026] Preferably, the segmentation module comprises: a second convolution layer, a deconvolution layer and an activation function layer;

[0027] The aggregated feature map is input into the second convolutional layer to obtain a processed feature map;

[0028] The processed feature map is input into the deconvolution layer for upsampling to obtain an upsampled feature map;

[0029] The upsampled feature map is input into the activation function layer to obtain the classification probability of each pixel in the remote sensing image;

[0030] The aircraft segmentation result is obtained based on the classification probability.

[0031] Preferably, obtaining the aircraft segmentation result based on the classification probability specifically includes:

[0032] Determine a segmentation result at position (i, j) in the remote sensing image based on the classification probability

[0033] Among them, P ij represents the classification probability at position (i, j) in the remote sensing image, Indicates setting threshold;

[0034] Based on all the segmentation results Get the final aircraft segmentation result R seg .

[0035] Preferably, the model training process is also included:

[0036] An aircraft segmentation model is formed based on the preprocessing module, the first convolutional layer, the aggregation control module and the segmentation module;

[0037] Obtain multiple remote sensing images including aircraft and their corresponding category labels as training sets;

[0038] The aircraft segmentation model is optimized and trained based on the training set and the total loss function to obtain a trained aircraft segmentation model.

[0039] Preferably, the total loss function L total Specifically:

[0040] L total =αL ce +βL d

[0041] Among them, L ce represents the cross entropy loss function, α represents the weight coefficient of the cross entropy loss function, L drepresents the Dice coefficient loss function, and β represents the weight coefficient of the Dice coefficient loss function.

[0042] Preferably, the cross entropy loss function L ce Specifically:

[0043]

[0044] Among them, H and W represent the height and width of the remote sensing image respectively, and N represents the number of categories. Indicates whether the remote sensing image position (i, j) belongs to category k, Represents the predicted probability of belonging to category k at the remote sensing image location (i, j);

[0045] The Dice coefficient loss function L d Specifically:

[0046]

[0047] Among them, y ij Represents the category label at the remote sensing image location (i, j).

[0048] A remote sensing image aircraft segmentation system based on an aggregation control module, comprising: a data processing module, a feature extraction module, a feature processing module, a multi-layer feature acquisition module, a feature map aggregation module and a segmentation result output module;

[0049] The data processing module is used to obtain a remote sensing image including an aircraft and input it to a preprocessing module to obtain an input image;

[0050] The feature extraction module is used to input the input image into the first convolution layer to obtain an input feature map;

[0051] The feature processing module is used to input the input feature map to an aggregation control module including a linear layer, a plurality of convolutional units and a lightweight linear layer; the input features are respectively input to the linear layer and the lightweight linear layer, and an initial feature map and a weight matrix are correspondingly obtained;

[0052] The multi-layer feature acquisition module is used to sequentially input the initial feature map into the convolution unit to obtain a plurality of extracted feature maps of different layers;

[0053] The feature map aggregation module is used to aggregate all the extracted feature maps based on the weight matrix to obtain an aggregated feature map;

[0054] The segmentation result output module is used to obtain the aircraft segmentation result of the remote sensing image based on the input of the aggregated feature map to the segmentation module.

[0055] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image aircraft segmentation method and system based on an aggregation control module, which has the following beneficial effects:

[0056] 1. Improve segmentation accuracy: The present invention introduces multi-level contextual information through an aggregation control module, so that the model can better capture the details and structural information of aircraft targets in remote sensing images, especially in complex backgrounds. It can effectively separate targets from backgrounds and reduce mis-segmentation. While improving the target segmentation accuracy, it enhances the ability to suppress complex backgrounds and improves the reliability of segmentation results.

[0057] 2. The present invention enhances the receptive field of the model by combining deep convolution with global average pooling operations, thereby improving the ability to capture a wide range of contextual information, so that small object targets in remote sensing images, such as airplanes, can be segmented more accurately.

[0058] 3. Through improved convolution and deconvolution operations, the present invention can be efficiently executed in a resource-constrained environment and is suitable for various remote sensing image analysis tasks. Moreover, through efficient parameter fine-tuning and the use of adapters, the resource requirements for model training and deployment are reduced, making it suitable for remote sensing image processing tasks in various computing environments. It has great technical value in target detection and segmentation tasks of aerial remote sensing images.

[0059] 4. The present invention significantly improves the segmentation accuracy by constructing an aggregation control module, combining multi-level context information and structural perception capabilities. By weighted fusion of features of different scales, it can better adapt to the segmentation tasks of complex backgrounds and multi-scale targets. Through the aggregation processing of multi-level context information, this method not only effectively suppresses background noise, but also improves segmentation robustness and improves the segmentation accuracy of small object targets.

[0060] 5. Efficient computing and resource utilization and good generalization ability: By using fine-tuning and adapter technology, the present invention can significantly reduce the computing resources required in the training and reasoning process while maintaining high accuracy, and is suitable for environments with limited computing resources such as embedded systems and edge computing platforms. Through multi-scale and multi-focus level context aggregation, the model has strong adaptability and can process remote sensing images of different resolutions and different environments, which enhances the generalization ability and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0062] Figure 1 A flow chart of a remote sensing image aircraft segmentation method based on an aggregation control module provided by the present invention.

[0063] Figure 2 This is a schematic diagram of the structure of the aggregation control module provided by the present invention.

[0064] Figure 3 Schematic diagram of the data processing process of the aggregation control module provided by the present invention.

[0065] Figure 4 This is a schematic diagram of the aircraft segmentation model structure provided by the present invention.

[0066] Figure 5 A schematic diagram of the structure of a remote sensing image aircraft segmentation system based on an aggregation control module provided by the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Example 1

[0069] like Figure 1 As shown, the embodiment of the present invention discloses a remote sensing image aircraft segmentation method based on an aggregation control module, comprising:

[0070] Acquire a remote sensing image including the aircraft and input it into a preprocessing module to obtain an input image;

[0071] Based on the input image, input it to the first convolution layer to obtain an input feature map;

[0072] Based on the input feature map, an aggregation control module including a linear layer, multiple convolutional units and a lightweight linear layer is input;

[0073] The input features are input to the linear layer and the light linear layer respectively, and the initial feature map and weight matrix are obtained accordingly;

[0074] The initial feature map is input into the convolution unit in sequence to obtain multiple extracted feature maps of different layers;

[0075] Based on all the extracted feature maps, aggregate them based on the weight matrix to obtain an aggregate feature map;

[0076] Based on the aggregated feature map, it is input into the segmentation module to obtain the aircraft segmentation result of the remote sensing image.

[0077] Example 2

[0078] The embodiment of the present invention discloses a remote sensing image aircraft segmentation method based on an aggregation control module, comprising:

[0079] Acquire a remote sensing image including the aircraft and input it into a preprocessing module to obtain an input image;

[0080] Preferably, the preprocessing module includes: a denoising unit, an image enhancement unit, a cropping unit, a scaling unit and a normalization unit;

[0081] The remote sensing image is input to a denoising unit for denoising to obtain a denoised image;

[0082] The denoised image is input to the image enhancement unit for image enhancement to obtain an enhanced image;

[0083] The enhanced image is input to the cropping unit for cropping operation to obtain a cropped image;

[0084] The cropped image is input to the scaling unit for scaling processing to obtain a scaled image;

[0085] The scaled image is input to the normalization unit for format conversion to obtain the input image.

[0086] Preferably, remote sensing images are often interfered by noise during the acquisition process, so denoising is required to improve the image quality. Common denoising methods include mean filtering, median filtering and Gaussian filtering. Assume that the original remote sensing image is I(x, y), where x and y are the spatial coordinates of the image, and each pixel value in the image may be affected by noise N(x, y), that is, the noisy remote sensing image I noisy (x,y) is:

[0087] I noisy (x,y)=I(x,y)+N(x,y).

[0088] Preferably, the purpose of denoising is to restore the original image I(x, y). In this embodiment, the denoising unit uses Gaussian filtering for denoising. Gaussian filtering is implemented by convolution operation. The denoised image I d (x,y) is:

[0089] I d (x,y)=(Inoisy *G);

[0090] Among them, * represents the convolution operation and G represents the filter kernel.

[0091] Preferably, the image enhancement unit can improve the contrast and details of the image, especially in remote sensing images, and can highlight certain specific features. Common enhancement methods include histogram equalization and contrast stretching. In this embodiment, the image enhancement unit uses contrast stretching to enhance the denoised image:

[0092]

[0093] Among them, I en (x,y) represents the enhanced image, represents the minimum pixel value in the denoised image, represents the maximum pixel value in the denoised image, C max Indicates the maximum value of the enhanced pixel value range, C min Indicates the minimum value of the enhanced pixel value range.

[0094] Preferably, in order to reduce the amount of calculation and focus on the target area, the enhanced image is sequentially input to the cropping unit and the scaling unit for processing. The cropping unit selects the cropping area of ​​the image: (x min ,y min ,x max ,y max ), where x min Indicates the minimum horizontal coordinate of the clipping area (that is, the horizontal coordinate of the left edge of the clipping area), y min Indicates the minimum ordinate of the clipping area (i.e. the ordinate of the upper boundary of the clipping area), x max Indicates the maximum horizontal coordinate of the clipping area (that is, the horizontal coordinate of the right edge of the clipping area), y max represents the maximum ordinate of the cropping area (i.e., the ordinate of the lower boundary of the cropping area); based on the cropping area, the enhanced image is cropped to obtain the cropped image I crop (x,y):

[0095] I crop (x,y)=I en (x,y)for x min ≤x≤x max ,y min ≤y≤y max .

[0096] Preferably, the cropped image I crop (x, y) is input to the scaling unit, and the target size is (W, H), then the scaled image I scaled (x',y') can be expressed as:

[0097] I scaled (x',y')=I crop (x,y)where(x',y')=scale(x,y,W,H)

[0098] Among them, scale represents the scale function, which is used to crop the image I crop The coordinates (x,y) in (x,y) are scaled to the dimensions (W,H) of the destination image.

[0099] Preferably, the scaled image is input to the normalization unit for format conversion, and converted into a format suitable for model input to obtain an input image. In this embodiment, the normalization unit converts the image into an input image of size C×H×W. Where C represents the number of channels (usually 3 for color images), and H and W are the height and width of the scaled image, respectively.

[0100] Preferably, standardized image data for input into the model can be obtained by performing a series of preprocessing operations on the remote sensing images, thereby ensuring data consistency and efficiency.

[0101] Based on the input image, it is input to the first convolutional layer to obtain the input feature map.

[0102] like Figure 2 As shown, based on the input feature map, it is input to an aggregation control module including a linear layer, multiple convolutional units and a lightweight linear layer.

[0103] The input features are input into the linear layer and the light linear layer respectively, and the initial feature map and weight matrix are obtained accordingly.

[0104] Preferably, the input feature map X is projected into the new feature space through a linear layer to obtain the initial feature map f proj Represents a linear projection operation.

[0105] Preferably, a fully connected layer in the lightweight linear layer performs a linear projection operation on the input feature map X and the extracted feature maps of each layer obtained by subsequent processing to obtain a weight matrix: Where Φ represents a fully connected layer.

[0106] The initial feature map is input into the convolution unit in sequence to obtain multiple extracted feature maps of different layers.

[0107] Preferably, the multiple convolution units are sequentially connected and have the same structure, and all include: a deep convolution layer and an activation layer;

[0108] The initial feature map is input into the deep convolutional layer and activation layer in sequence to obtain the extracted feature map of the corresponding layer.

[0109] Preferably, in this embodiment, the activation layer adopts GeLu activation function.

[0110] Preferably, a hierarchical context representation is obtained by stacking L convolutional units in series, and the l-th layer extracted feature map Z obtained by the l-th convolutional unit l for:

[0111] Z l =GeLU(DWConv(Z l-1 ));

[0112] l∈{1,...,L};

[0113] Among them, DWConv represents the deep convolution operation, GeLu represents the activation function,

[0114] Preferably, in this embodiment, the convolution kernel size of the depth convolution layer is k l Compared to pooling operations, depthwise convolution is learnable and structure-aware. Unlike regular convolution, depthwise convolution operates on a channel-by-channel basis, so it has less computational overhead.

[0115] At layer l, the effective receptive field of the deep convolutional layer is Its effective receptive field size is much larger than the convolution kernel size k l In order to capture the global context of the entire input, especially the high-resolution input, a global average pooling operation is applied, which jointly captures both short-term and long-term context at different levels of granularity.

[0116] like Figure 3 As shown, all extracted feature maps are aggregated based on the weight matrix to obtain an aggregated feature map;

[0117] Preferably, the method for obtaining the aggregate feature map is:

[0118] The weights corresponding to the number of channels in the weight matrix that is consistent with the number of layers of the extracted feature map are used as the corresponding layer perception weights;

[0119] The aggregated feature map is obtained by performing element-by-element weighted summation based on all layer perception weights and the extracted feature maps of the corresponding layers.

[0120] Preferably, based on the weight matrix ▽ containing the number of channels corresponding to the number of layers of the extracted feature map, each channel has a corresponding weight, and the weight corresponding to the number of channels l consistent with the number of layers l of the extracted feature map in the weight matrix ▽ is used as the corresponding layer perception weight ▽ l .

[0121] Preferably, the aggregate feature map is Z out :

[0122]

[0123] Preferably, for markers located on small objects, the module pays more attention to the fine-grained local structure of the low focus level; while for markers located in a uniform background, it needs to pay attention to a wider range of context from a higher level. So far, all aggregation operations are spatial level aggregation.

[0124] Based on the aggregated feature map, it is input into the segmentation module to obtain the aircraft segmentation result of the remote sensing image.

[0125] Preferably, the segmentation module includes: a second convolution layer, a deconvolution layer and an activation function layer;

[0126] The aggregated feature map is input to the second convolutional layer to obtain the processed feature map;

[0127] The processed feature map is input to the deconvolution layer for upsampling to obtain an upsampled feature map;

[0128] The upsampled feature map is input into the activation function layer to obtain the classification probability of each pixel in the remote sensing image;

[0129] The aircraft segmentation result is obtained based on the classification probability.

[0130] Preferably, the aggregate feature map Input to the second convolutional layer, the processed feature map Z′=Conv(Z out ), where Conv represents the convolution operation. Z' is a feature map of size H×W×C', where C' is the number of channels after convolution.

[0131] Preferably, in this embodiment, the convolution kernel size of the second convolution layer is K×K, and the stride is 1.

[0132] Preferably, deconvolution can gradually enlarge the size of the processed feature map Z', and finally obtain an upsampled feature map with the same size as the original input image: in, C out Indicates the number of output channels, usually for multi-category segmentation, C out is equal to the number of categories N, and for the binary classification problem, C out =1.

[0133] Preferably, the upsampled feature map is input to the activation function layer for processing to obtain the classification probability of each pixel. For multi-category segmentation, the activation function layer of this embodiment adopts the Softmax function:

[0134]

[0135] in, It represents the value of the kth class at position (i, j) in the upsampled feature map, and N represents the number of categories.

[0136] Preferably, obtaining the aircraft segmentation result based on the classification probability specifically includes:

[0137] Determine the segmentation result at position (i, j) in the remote sensing image based on the classification probability

[0138]

[0139] Among them, P ij represents the classification probability at position (i, j) in the remote sensing image, Indicates setting threshold;

[0140] Based on all the segmentation results Get the final aircraft segmentation result R seg .

[0141] Preferably, in the embodiment, a threshold is set is 0.5.

[0142] Preferably, through the above steps, the aircraft target in the remote sensing image can be accurately segmented, and the model can not only identify the boundary of the aircraft target, but also handle complex backgrounds and multi-scale targets, thereby providing efficient remote sensing image segmentation results.

[0143] Preferably, the model training process is also included:

[0144] like Figure 4 As shown, the aircraft segmentation model is composed of a preprocessing module, a first convolutional layer, an aggregation control module and a segmentation module;

[0145] Obtain multiple remote sensing images including aircraft and their corresponding category labels as training sets;

[0146] The aircraft segmentation model is optimized and trained based on the training set and the total loss function to obtain a trained aircraft segmentation model.

[0147] Preferably, each sample in the training set consists of an input original image I and a corresponding category label Y, where I is the input image and Y is the true label of the image (ie, the category label at the pixel level).

[0148] Preferably, for multi-class segmentation tasks, cross entropy loss is one of the most commonly used loss functions. Cross entropy loss measures the difference between the class probability distribution output by the model and the true label distribution. The predicted value of the airplane segmentation model is P, which represents the predicted probability that each pixel in the i-th image belongs to each class, and the true label is Y, where the label y of each pixel is ij∈{0, 1, ..., N-1} indicates that the pixel (i, j) belongs to a certain category (N is the number of categories). The cross entropy loss function is:

[0149]

[0150] Among them, H and W represent the height and width of the remote sensing image respectively, and N represents the number of categories. Indicates whether the remote sensing image position (i, j) belongs to category k, Represents the predicted probability of belonging to category k at the remote sensing image location (i, j).

[0151] Preferably, the Dice coefficient is used to measure the similarity between two samples, and the Dice coefficient loss function can be used to optimize the segmentation accuracy of the target area. d Specifically:

[0152]

[0153] Among them, y ij Represents the category label at the remote sensing image location (i, j).

[0154] Preferably, the present invention uses cross entropy loss and Dice loss as the total loss function:

[0155] L total =αL ce +βL d

[0156] Among them, L ce represents the cross entropy loss function, α represents the weight coefficient of the cross entropy loss function, L d represents the Dice coefficient loss function, and β represents the weight coefficient of the Dice coefficient loss function. α and β can be adjusted according to actual needs. Through this weighting method, the impact of different loss functions on training results can be taken into account.

[0157] Example 3

[0158] like Figure 5 As shown, a remote sensing image aircraft segmentation system based on an aggregation control module includes: a data processing module, a feature extraction module, a feature processing module, a multi-layer feature acquisition module, a feature map aggregation module and a segmentation result output module;

[0159] A data processing module, used for acquiring remote sensing images including the aircraft and inputting them into a preprocessing module to obtain an input image;

[0160] A feature extraction module, used to obtain an input feature map based on the input image input to the first convolution layer;

[0161] A feature processing module, which is used to input the input feature map into an aggregation control module including a linear layer, a plurality of convolutional units and a lightweight linear layer; the input features are respectively input into the linear layer and the lightweight linear layer, and an initial feature map and a weight matrix are obtained correspondingly;

[0162] A multi-layer feature acquisition module is used to input the initial feature map into the convolution unit in sequence to obtain multiple extracted feature maps of different layers;

[0163] A feature map aggregation module is used to aggregate all extracted feature maps based on a weight matrix to obtain an aggregated feature map;

[0164] The segmentation result output module is used to input the segmentation module based on the aggregated feature map to obtain the aircraft segmentation result of the remote sensing image.

[0165] Preferably, the functions implemented by each module in this embodiment correspond one-to-one to the above method, and will not be described one by one here.

[0166] Example 4

[0167] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0168] Memory, used to store computer programs;

[0169] The processor, when used to execute the program stored in the memory, can implement a remote sensing image aircraft segmentation method based on an aggregation control module as in Example 1 or 2.

[0170] The electronic device may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor may call the logic instructions in the memory to execute a remote sensing image aircraft segmentation method based on an aggregation control module in embodiment 1 or 2.

[0171] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0172] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image aircraft segmentation method and system based on an aggregation control module, which has the following beneficial effects:

[0173] 1. Improve segmentation accuracy: The present invention introduces multi-level contextual information through an aggregation control module, so that the model can better capture the details and structural information of aircraft targets in remote sensing images, especially in complex backgrounds. It can effectively separate targets from backgrounds and reduce mis-segmentation. While improving the target segmentation accuracy, it enhances the ability to suppress complex backgrounds and improves the reliability of segmentation results.

[0174] 2. The present invention enhances the receptive field of the model by combining deep convolution with global average pooling operations, thereby improving the ability to capture a wide range of contextual information, so that small object targets in remote sensing images, such as airplanes, can be segmented more accurately.

[0175] 3. Through improved convolution and deconvolution operations, the present invention can be efficiently executed in a resource-constrained environment and is suitable for various remote sensing image analysis tasks. Moreover, through efficient parameter fine-tuning and the use of adapters, the resource requirements for model training and deployment are reduced, making it suitable for remote sensing image processing tasks in various computing environments. It has great technical value in target detection and segmentation tasks of aerial remote sensing images.

[0176] 4. The present invention significantly improves the segmentation accuracy by constructing an aggregation control module, combining multi-level context information and structural perception capabilities. By weighted fusion of features of different scales, it can better adapt to the segmentation tasks of complex backgrounds and multi-scale targets. Through the aggregation processing of multi-level context information, this method not only effectively suppresses background noise, but also improves segmentation robustness and improves the segmentation accuracy of small object targets.

[0177] 5. Efficient computing and resource utilization and good generalization ability: By using fine-tuning and adapter technology, the present invention can significantly reduce the computing resources required in the training and reasoning process while maintaining high accuracy, and is suitable for environments with limited computing resources such as embedded systems and edge computing platforms. Through multi-scale and multi-focus level context aggregation, the model has strong adaptability and can process remote sensing images of different resolutions and different environments, which enhances the generalization ability and robustness of the model.

[0178] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0179] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image aircraft segmentation method based on an aggregation control module, characterized in that: include: Acquire a remote sensing image including the aircraft and input it into a preprocessing module to obtain an input image; Inputting the input image into the first convolutional layer to obtain an input feature map; Based on the input feature map, an aggregation control module including a linear layer, a plurality of convolutional units and a lightweight linear layer is input; The input features are respectively input into the linear layer and the light linear layer, and an initial feature map and a weight matrix are obtained correspondingly; The initial feature map is sequentially input into the convolution unit to obtain a plurality of extracted feature maps of different layers; Aggregating all the extracted feature maps based on the weight matrix to obtain an aggregated feature map; The aggregated feature map is input into a segmentation module to obtain an aircraft segmentation result of the remote sensing image.

2. According to claim 1, a remote sensing image aircraft segmentation method based on an aggregation control module is characterized in that: The preprocessing module includes: a denoising unit, an image enhancement unit, a cropping unit, a scaling unit and a normalization unit; The remote sensing image is input into the denoising unit for denoising to obtain a denoised image; The denoised image is input to the image enhancement unit for image enhancement to obtain an enhanced image; The enhanced image is input to the cropping unit for cropping to obtain a cropped image; The cropped image is input to the scaling unit for scaling processing to obtain a scaled image; The scaled image is input to the normalization unit for format conversion to obtain the input image.

3. The remote sensing image aircraft segmentation method based on the aggregation control module according to claim 1 is characterized in that: The multiple convolution units are sequentially connected and have the same structure, and all include: a deep convolution layer and an activation layer; The initial feature map is sequentially input into the deep convolution layer and the activation layer to obtain the extracted feature map of the corresponding layer.

4. The remote sensing image aircraft segmentation method based on the aggregation control module according to claim 1 is characterized in that: The method for obtaining the aggregate feature map is: Based on the weights corresponding to the number of channels in the weight matrix consistent with the number of layers of the extracted feature map as corresponding layer perception weights; The aggregated feature map is obtained by performing a weighted summation based on element-by-element multiplication of all the layer perception weights and the extracted feature maps corresponding to the number of layers.

5. The remote sensing image aircraft segmentation method based on the aggregation control module according to claim 1 is characterized in that: The segmentation module includes: a second convolution layer, a deconvolution layer and an activation function layer; The aggregated feature map is input into the second convolutional layer to obtain a processed feature map; The processed feature map is input into the deconvolution layer for upsampling to obtain an upsampled feature map; The upsampled feature map is input into the activation function layer to obtain the classification probability of each pixel in the remote sensing image; The aircraft segmentation result is obtained based on the classification probability.

6. The remote sensing image aircraft segmentation method based on the aggregation control module according to claim 5 is characterized in that: The aircraft segmentation result is obtained based on the classification probability, specifically including: Determine a segmentation result at position (i, j) in the remote sensing image based on the classification probability Among them, P ij represents the classification probability at position (i, j) in the remote sensing image, Indicates setting threshold; Based on all the segmentation results Get the final aircraft segmentation result R seg .

7. The remote sensing image aircraft segmentation method based on the aggregation control module according to claim 6 is characterized in that: It also includes the model training process: An aircraft segmentation model is formed based on the preprocessing module, the first convolutional layer, the aggregation control module and the segmentation module; Obtain multiple remote sensing images including aircraft and their corresponding category labels as training sets; The aircraft segmentation model is optimized and trained based on the training set and the total loss function to obtain a trained aircraft segmentation model.

8. The remote sensing image aircraft segmentation method based on the aggregation control module according to claim 7 is characterized in that: The total loss function L total Specifically: L total =αL ce +βL d Among them, L ce represents the cross entropy loss function, α represents the weight coefficient of the cross entropy loss function, L d represents the Dice coefficient loss function, and β represents the weight coefficient of the Dice coefficient loss function.

9. The remote sensing image aircraft segmentation method based on the aggregation control module according to claim 8 is characterized in that: The cross entropy loss function L ce Specifically: Among them, H and W represent the height and width of the remote sensing image respectively, and N represents the number of categories. Indicates whether the remote sensing image position (i, j) belongs to category k, Represents the predicted probability of belonging to category k at the remote sensing image location (i, j); The Dice coefficient loss function L d Specifically: Among them, y ij Represents the category label at the remote sensing image location (i, j).

10. A remote sensing image aircraft segmentation system based on an aggregation control module, applied to a remote sensing image aircraft segmentation method based on an aggregation control module as claimed in any one of claims 1 to 9, characterized in that: include: Data processing module, feature extraction module, feature processing module, multi-layer feature acquisition module, feature map aggregation module and segmentation result output module; The data processing module is used to obtain a remote sensing image including an aircraft and input it to a preprocessing module to obtain an input image; The feature extraction module is used to input the input image into the first convolution layer to obtain an input feature map; The feature processing module is used to input the input feature map to an aggregation control module including a linear layer, a plurality of convolutional units and a lightweight linear layer; the input features are respectively input to the linear layer and the lightweight linear layer, and an initial feature map and a weight matrix are correspondingly obtained; The multi-layer feature acquisition module is used to sequentially input the initial feature map into the convolution unit to obtain a plurality of extracted feature maps of different layers; The feature map aggregation module is used to aggregate all the extracted feature maps based on the weight matrix to obtain an aggregated feature map; The segmentation result output module is used to obtain the aircraft segmentation result of the remote sensing image based on the input of the aggregated feature map to the segmentation module.

Images