A method for target detection and recognition in multimodal remote sensing images of microsatellite clusters

Abstract

The present invention discloses a method for detecting and identifying a target in a multimodal remote sensing image of a microsatellite cluster, comprising the following steps: S1. Acquisition of multimodal remote sensing images: shooting multimodal remote sensing images by different cameras carried by a microsatellite cluster; S2. Feature extraction: extracting semantic features of remote sensing images of each modality by using a pseudo twin network; S3. Feature fusion: completing feature-level fusion and pixel-level fusion of multimodal remote sensing information by using spatial and channel attention mechanisms and convolutional sparse representation fusion; S4. Target detection and recognition: inputting feature maps obtained by feature-level fusion and pixel-level fusion into a recognition network to obtain a result of target detection and recognition of a multimodal remote sensing image. The method of the present invention can improve the accuracy of the target detection and recognition algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of satellite remote sensing information processing, and in particular to a method for detecting and identifying multi-modal remote sensing image targets of a micro-satellite cluster. Background Art

[0002] With the rapid development of aerospace technologies such as satellite manufacturing and networking, microsatellite clusters have become an important means of acquiring remote sensing information. A microsatellite cluster refers to a satellite cluster composed of artificial satellites with a single weight of less than 1,000 kilograms. It has the characteristics of low cost, fast response, flexibility and wide coverage. At the same time, many satellites in the cluster can carry different imaging payloads, such as visible light cameras, infrared cameras, synthetic aperture cameras, etc., and have specific natural advantages in multi-modal remote sensing image acquisition, and have great application potential in many fields such as environmental monitoring, urban planning, agricultural resource management, and military reconnaissance.

[0003] At present, in the task of remote sensing target detection and recognition, representative deep learning network frameworks include Region-Convolutional Neural Network (R-CNN), Spatial Pyramid Pooling Net (SPP-Net), Fast R-CNN, RetinaNet, SSD (Single Shot MultiBox Detector) series network frameworks and YOLO (You Only Look Once) series network frameworks, etc. The above algorithms process complex and changeable image data by automatically learning image features, and have achieved remarkable results in the field of target detection and recognition.

[0004] However, the above algorithms face many challenges in the target detection and recognition tasks of multimodal remote sensing images. First, visible light, infrared and SAR remote sensing images have different characteristics. How to make full use of the advantages of remote sensing images of different modalities to improve the performance of algorithms is one of the current research hotspots. Secondly, remote sensing images usually contain complex and diverse background information. How to reduce the impact of irrelevant information in the background on target detection and recognition results is one of the difficulties in current research. In addition, images of different modalities differ in spectral characteristics, spatial resolution, etc. How to effectively fuse this information to improve the accuracy of target detection is a difficult and hot issue in current research. Summary of the invention

[0005] In view of the problems existing in the prior art, the purpose of the present invention is to provide a multimodal remote sensing image target detection and recognition method that meets the data processing requirements in the context of multi-satellite networking and the multimodal data characteristics of multi-source remote sensing images of a constellation, so as to improve the accuracy of the target detection and recognition algorithm.

[0006] To achieve the above object, the present invention proposes a method for detecting and identifying targets in multimodal remote sensing images of a microsatellite cluster, which specifically comprises the following steps:

[0007] S1. Multimodal remote sensing image acquisition: Multimodal remote sensing images are captured by different cameras carried by microsatellite clusters;

[0008] S2. Feature extraction: Semantic features of remote sensing images of each modality are extracted by using the pseudo-twin network;

[0009] S3. Feature fusion: The feature-level fusion and pixel-level fusion of multimodal remote sensing information are completed through multimodal attention mechanism and convolutional sparse representation respectively;

[0010] S4. Target detection and recognition: The feature maps obtained by feature-level fusion and pixel-level fusion are respectively input into the recognition network to obtain the multimodal remote sensing image target detection and recognition results.

[0011] Further, in step S1, the multimodal remote sensing image includes a visible light remote sensing image, an infrared remote sensing image and a SAR remote sensing image.

[0012] Further, in step S2, the specific method of feature extraction is:

[0013] The pseudo-twin network is used as the feature extraction network to extract features from multimodal remote sensing images. The pseudo-twin network contains three parallel branches, whose inputs are visible light, infrared and SAR remote sensing images, and whose outputs are feature maps of the three modalities.

[0014] Furthermore, the pseudo-twin network of the parallel branch can obtain essential features with generalization ability and avoid the interference caused by the difference in the expression of multimodal remote sensing images on scenes and ground targets; the three branches of the pseudo-twin network are multi-scale feature extraction networks Res2net with the same structure, which is an improved Res2net based on dense connection and channel shuffling. After the feature extraction performs a 1×1 convolution operation on the input features, the input features are split into s groups according to the channels, and each group of features is recorded as x i .x i Capture image features at different spatial scales and output y i , the formula can be expressed as:

[0015]

[0016] Among them, Ki () represents a convolution operation with a convolution kernel size of 3×3, and ∑ is a summation operation.

[0017] Then, the s groups of features from the s branches are concatenated by channel and sent to the channel shuffle module, and finally output after a 1×1 convolution operation. The formula can be expressed as:

[0018]

[0019] Among them, y i represents the output feature of the i-th scale, [,] represents channel concatenation, Indicates random shuffling by feature channel. Represents a convolution operation with a kernel size of 1×1.

[0020] Further, in step S3, feature fusion includes:

[0021] S3.1. Feature-level fusion via multimodal attention mechanism;

[0022] S3.2. Pixel-level fusion is achieved through convolution coefficient representation.

[0023] Furthermore, in step S3.1, the steps of feature-level fusion are as follows: first, the parallel input visible light feature map, infrared feature map and SAR feature map are spliced ​​by channel through the concat operation; then, a multimodal attention mechanism is used to assign different weights to different parts of the input features to enhance the model's attention to key information and obtain a multimodal fusion feature map.

[0024] Furthermore, the multimodal attention mechanism of feature-level fusion includes spatial attention and channel attention, which assign different weights to the input data from the spatial and channel dimensions respectively, thereby enhancing the network's attention to key information.

[0025] Further, spatial attention is defined as:

[0026] Spatial attention: First, perform global pooling and average pooling on the input feature map along the channel dimension to compress the channel size; then, concatenate the global pooling and average pooling results by channel; finally, convolve the concatenated results and use Sigmoid for activation processing; the spatial attention calculation formula is:

[0027]

[0028] in, represents the spatial attention operation, F is the input feature map, AvgPool(.) is the average pooling operation, MaxPool(.) is the global pooling operation, [;] represents channel concatenation, K(.) represents the convolution operation, Represents the activation function.

[0029] Further, channel attention is defined as:

[0030] Channel attention: First, global pooling and average pooling are performed from the spatial dimension; then, they are sent to the multi-layer perceptron to learn the importance of each channel; finally, the output of the multi-layer perceptron is superimposed and activated using the sigmoid function; the channel attention calculation formula is:

[0031]

[0032] in, represents the channel attention operation, F is the input feature map, AvgPool(.) is the average pooling operation, MaxPool(.) is the global pooling operation, MLP(.) represents the multi-layer perceptron, Represents the activation function.

[0033] Furthermore, in step S4, the SCFNet network structure is used for target detection and recognition. This network structure designs three recognition heads of different sizes to meet the needs of multi-size target extraction, and uses a local feature correction module to correct local semantic features and a non-local feature module to obtain long-distance semantic features. The feature map output by the feature fusion in step S3 is input into the SCFNet network structure to obtain the results of target detection and recognition.

[0034] The beneficial effects of the present invention are as follows:

[0035] (1) The present invention uses multimodal remote sensing images (visible light, infrared and SAR remote sensing images) taken by a cluster of microsatellites to perform joint target detection and recognition, fully exploiting the characteristics of multimodal remote sensing images and improving the detection and recognition method's ability to resist interference from light and fog.

[0036] (2) The present invention uses feature-level fusion and pixel-level fusion to fuse multimodal information at different levels. The multimodal attention mechanism of feature-level fusion enhances the relevant semantic information between modalities and weakens the irrelevant semantic information within the modality, thereby enhancing the attention of the multimodal remote sensing image target recognition model to key information and improving the algorithm performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is an algorithm flow chart of the method for detecting and identifying targets in multi-modal remote sensing images of a micro-satellite cluster according to the present invention;

[0038] Figure 2 An improved Res2net network based on dense connection and channel shuffling according to the present invention;

[0039] Figure 3A diagram showing a feature fusion network structure according to the present invention;

[0040] Figure 4 Schematic diagram of multimodal attention according to the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0043] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0044] The following combination Figure 1-Figure 4 The specific embodiments of the present invention are described in detail. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0045] The present invention provides a multimodal remote sensing image target recognition technology. With the development of multi-system imaging payloads and breakthroughs in microsatellite networking technology, research on multimodal image intelligent interpretation and fusion technology based on satellite clusters can further improve the accuracy and reliability of target recognition algorithms, which is of great significance for improving the ability to accurately interpret and quickly respond to disasters in emergency rescue scenarios such as plane crashes or natural disasters.

[0046] The method for target detection and recognition in multimodal remote sensing images of a microsatellite cluster according to the present invention is based on the dependency between multimodal remote sensing image targets and target modalities, utilizes the structural correlation and texture correlation of targets in multimodal remote sensing images, mines the complementary information between multimodal remote sensing images, suppresses a large amount of irrelevant information in the background that is not related to the target, and can effectively improve the algorithm recognition accuracy.

[0047] like Figure 1 As shown, the process of the micro-satellite cluster multi-modal remote sensing image target detection and recognition method according to the present invention specifically includes:

[0048] S1. Multimodal remote sensing image acquisition: Multimodal remote sensing images are captured by different cameras carried by a microsatellite cluster. In the present invention, multimodal refers to images of three different modalities: visible light, infrared and SAR.

[0049] S2. Feature extraction: The semantic features of each modality of remote sensing images are extracted separately through the pseudo-twin network.

[0050] S3. Feature fusion: The feature-level fusion and pixel-level fusion of multimodal remote sensing image information are completed through multimodal attention mechanism and convolutional sparse representation respectively. Remote sensing image information refers to the information contained in or extracted from remote sensing images.

[0051] S4. Target detection and recognition: The feature maps obtained by feature-level fusion and pixel-level fusion are respectively input into the recognition network to obtain the multimodal remote sensing image target detection and recognition results.

[0052] Specifically, in step S1, the multimodal remote sensing image includes a visible light remote sensing image, an infrared remote sensing image and a SAR remote sensing image.

[0053] In step S2, the specific method of feature extraction is:

[0054] The pseudo-twin network is used as the feature extraction network to extract features from multimodal remote sensing images. The pseudo-twin network contains three parallel branches, whose inputs are visible light, infrared and SAR remote sensing images, and whose outputs are feature maps of the three modalities.

[0055] Specifically, the three branches of the pseudo-twin network used in the present invention are multi-scale feature extraction networks with the same structure, an improved Res2net. The pseudo-twin network using parallel branches can obtain essential features with generalization capabilities and avoid interference caused by differences in the expression of scenes and ground targets by multimodal images. For visible light, infrared, and SAR images taken at the same position, the essential features with generalization capabilities include the features of the target or background itself in the shooting field of view, as well as some differences caused by different imaging methods. The present invention uses a pseudo-twin network with three parallel branches to express the features of the target or background itself more clearly.

[0056] The improved Res2net network based on dense connection and channel shuffling proposed in this invention is a new backbone network proposed on the basis of Res2net. Figure 2 As shown in the figure, the improved Res2net further solves the gradient vanishing and gradient exploding problems by building denser jump connections (shotcut) between multi-scale convolutional layers. In addition, the improved Res2net adds a channel shuffle module to improve the information flow across feature channels and increase the generalization of features.

[0057] More specifically, after Res2net performs a 1×1 convolution operation on the input features, it splits the input features into s groups by channel, and each group of features is recorded as x i .x i It can capture image features at different spatial scales and improve the network's ability to understand targets and background features of different sizes. i Denotes the output features. Except for the first group, each group of features undergoes a convolution operation with a convolution kernel size of 3×3, and the convolution operation is denoted by K i (). Except for the first two groups, each set of features must first be superimposed with the previous set of output features, and then pass through K i () operation, the formula can be expressed as:

[0058]

[0059] Finally, the s groups of features from the s branches are concatenated by channel and sent to the channel shuffle module, and finally output after a 1×1 convolution operation. The formula can be expressed as:

[0060]

[0061] Among them, y i represents the output feature of the i-th scale, [,] represents channel concatenation, Indicates random shuffling by feature channel. Represents a convolution operation with a kernel size of 1×1.

[0062] In step S3, feature fusion includes:

[0063] S3.1. Feature-level fusion via multimodal attention mechanism;

[0064] S3.2. Pixel-level fusion is achieved through convolution coefficient representation.

[0065] Specifically, in step S3.1, the feature level fusion steps are as follows: Figure 3As shown in the figure: First, the parallel input visible light feature map, infrared feature map and SAR feature map are spliced ​​by channel through the concat operation. Then, the multimodal attention mechanism is used to assign different weights to different parts of the input features, enhance the model's attention to key information, and obtain a multimodal fusion feature map.

[0066] Specifically, the multimodal attention mechanism is a functional module that enables the network to selectively focus on a part of strongly related information and ignore another part of weakly related information. The multimodal attention mechanism of the present invention includes spatial attention and channel attention, which assign different weights to input data from the two dimensions of space and channel, respectively, so as to enhance the network's attention to key information.

[0067] The feature map after the multimodal attention mechanism is assigned strengthens the relevant semantics of the modality and inter-modality that are dependent on the target, and weakens the irrelevant semantics of the modality and inter-modality that are not dependent on the target. Figure 4 As shown. For multimodal images such as visible light, infrared and SAR, the current target is the aircraft in the middle of the visible light RGB image. Its trans-membrane state related semantic information includes the aircraft and other aircraft in the infrared remote sensing image and SAR remote sensing image, and its trans-membrane state irrelevant semantic information includes information on surface covers such as deserts and vegetation in the infrared remote sensing image and SAR remote sensing image. Within a single modality (taking visible light as an example), the current target is an aircraft in the middle of the image. Other aircraft related to this aircraft in the image are all related semantics and are enhanced under the attention mechanism. The background that has no obvious dependency relationship with this aircraft is irrelevant semantics and is suppressed under the attention mechanism.

[0068] S3.1.1. The steps of extracting spatial attention are as follows: First, perform global pooling and average pooling on the input feature map along the channel dimension to compress the channel size. Then, concatenate the global pooling and average pooling results by channel. Finally, after convolution operation on the concatenated result, Sigmoid is used for activation processing. Its calculation formula is:

[0069]

[0070] in, represents the spatial attention operation, F is the input feature map, AvgPool(.) is the average pooling operation, MaxPool(.) is the global pooling operation, [;] represents channel concatenation, K(.) represents the convolution operation, Represents the activation function.

[0071] S3.1.2. The steps of extracting channel attention are: first, perform global pooling and average pooling from the spatial dimension. Then, send them to the multi-layer perceptron to learn the importance of each channel. Finally, after superimposing the output of the multi-layer perceptron, use the sigmoid function for activation processing. Its calculation formula is:

[0072]

[0073] in, represents the channel attention operation, F is the input feature map, AvgPool(.) is the average pooling operation, MaxPool(.) is the global pooling operation, MLP(.) represents the multi-layer perceptron, Represents the activation function.

[0074] Specifically, in step S3.1, the method for obtaining the output feature map of multimodal attention is: respectively multiplying the spatial attention and channel attention with the input features, and superimposing them, and assigning different weights to the input data from the spatial and channel dimensions respectively. The formula is expressed as:

[0075]

[0076] in, represents the feature map after attention weighting. F is the feature map, and is the weight.

[0077] In step S3.2, the role of pixel-level fusion is to perform information fusion at a low semantic level and increase the diversity of information fusion. The present invention uses convolutional sparse representation for pixel-level fusion, and its formula is expressed as:

[0078]

[0079] in, represents the m-dimensional dictionary filter, f represents the image to be sparsely represented, represents a set of feature maps of f, which has the same size as f, * represents the convolution operation, and denote the 1-norm and 2-norm respectively, is a sparse regularization term, representing the balance between model accuracy and sparsity.

[0080] In step S4, the recognition network can adopt any target recognition network that supports multi-channel output. The present invention adopts the semantic correction and focus network (SCFNet) network structure. The network designs three recognition heads of different sizes to meet the needs of multi-size target extraction, and uses the local feature correction module to correct local semantic features and the non-local feature module to obtain long-distance semantic features. The pixel-level fusion feature map and the feature-level fusion feature map are spliced ​​and input into SCFNet to obtain the target detection and recognition results.

[0081] The present invention has been preliminarily verified on a dataset constructed by using multi-satellite multi-modal remote sensing datasets such as Jilin-1, Sentinel-1, and Sentinel satellites, and simulation and registration techniques. The dataset has 10 scenes in total, and 2,400 experimental images are obtained after cutting, including remote sensing images of visible light, infrared, SAR and other modalities, and the labels include the category names of the locations of time-sensitive targets such as aircraft and vehicles. The detailed information of the constructed multi-modal dataset for target detection and recognition is shown in the following table.

[0082] Table 1 Dataset details

[0083]

[0084] The present invention first crops large-size images into small images with a size of 3442×4680 pixels, and crops them into 256×256 pixels. The data set is expanded by using satellite remote sensing image intelligent enhancement technology and image simulation technology. The final number of samples and sample division are shown in the following table. It can be seen that the number of samples has increased significantly, which basically meets the data volume requirements for joint intelligent interpretation of multi-source remote sensing images using convolutional neural networks.

[0085] Table 2 Dataset expansion

[0086]

[0087] Finally, the data set involved in the experiment includes 6,500 sets of multimodal remote sensing images, a total of 19,500 images, of which 70% are used for training and 30% for testing. The experiment was conducted on a GPU workstation with 24G video memory (NVIDIA RTX3090) configured as Ubuntu 21.0 system, CUDA (10.0) and CUDNN 7.0. The network architecture was built using python 3.7 and pytorch 1.1.0.

[0088] The present invention uses the mean average precision (mAP) of the test set as a quantitative evaluation index of the target recognition model, and compares the method of the present invention with two representative target recognition methods, DERT and yolov5. The results are shown in the following table. The comparison results show that the method of the present invention achieved an average accuracy of 91.7%, 94.2% and 90.4% respectively in the vehicle, aircraft and ship recognition of multimodal remote sensing image joint target recognition tasks, and has the best performance. Taking vehicle recognition as an example, the average recognition accuracy of the method of the present invention is 1.4% and 0.6% higher than that of the yolov5 and DERT algorithms, respectively. In addition, from the perspective of inference time, the thrust time difference of the three recognition methods is only at the millisecond level, and the efficiency is comparable.

[0089] Table 3 Performance comparison of multimodal target recognition network models

[0090]

[0091] Any process or method description in the flowchart of the present invention or described in other ways herein can be understood as a module, segment or part of a code including one or more executable instructions for implementing the steps of a specific logical function or process, which can be implemented in any computer-readable medium for use by an instruction execution system, device or equipment, and the computer-readable medium can be any medium containing storage, communication, propagation or transmission programs for use by execution systems, devices or equipment, including read-only memories, magnetic disks or optical disks, etc.

[0092] In the description of this specification, the description with reference to the terms "embodiment", "example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. In addition, those skilled in the art can combine or combine different embodiments or examples described in this specification and the features therein without causing any contradiction.

[0093] Although the above content has shown and described the embodiments of the present invention, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in the field can change, modify, replace, modify and other update operations on the above embodiments within the scope of the present invention.

Claims

1. A method for detecting and identifying targets in multimodal remote sensing images of a micro-satellite cluster, characterized in that: The specific steps include: S1. Multimodal remote sensing image acquisition: Multimodal remote sensing images are captured by different cameras carried by microsatellite clusters; S2. Feature extraction: Semantic features of remote sensing images of each modality are extracted through the pseudo-twin network; S3. Feature fusion: The feature-level fusion and pixel-level fusion of multimodal remote sensing information are completed through the multimodal attention mechanism and convolutional sparse representation respectively; the feature map after the multimodal attention mechanism is assigned enhances the relevant semantics within and between modalities that have a dependency relationship with the target, and weakens the irrelevant semantics within and between modalities that have no dependency relationship with the target; S4. Target detection and recognition: The feature maps obtained by feature-level fusion and pixel-level fusion are input into the recognition network to obtain the target detection and recognition results of multimodal remote sensing images; In step S2, the specific method of feature extraction is: The pseudo-twin network is used as the feature extraction network to extract features from multi-modal remote sensing images. The pseudo-twin network contains three parallel branches, whose inputs are visible light, infrared and SAR remote sensing images, and whose outputs are feature maps of the three modes. The pseudo-twin network of parallel branches can obtain essential features with generalization ability and avoid the interference caused by the difference in expression of multimodal remote sensing images on scenes and ground targets; the three branches of the pseudo-twin network are multi-scale feature extraction networks Res2net with the same structure; The improved Res2net adds a channel shuffle module to improve the information flow across feature channels and increase the generalization of features. After Res2net performs a 1×1 convolution operation on the input features, it splits the input features into s groups by channel, and each group of features is recorded as x i ;x i Capture image features at different spatial scales to improve the network's ability to understand targets and background features of different sizes; at the same time, use y i Represents the output features. Except for the first group, each group of features must undergo a convolution operation with a convolution kernel size of 3×3. The convolution operation is recorded as K i (); Except for the first two groups, each group of features must first be superimposed with the previous group of output features, and then pass through K i () operation, the formula is expressed as: Finally, the s groups of features from the s branches are concatenated by channel and sent to the channel shuffle module, and finally output after a 1×1 convolution operation; Its formula is expressed as: Among them, y i represents the output feature of the i-th scale, [,] represents channel concatenation, Indicates random shuffling by feature channel. Indicates a convolution operation with a convolution kernel size of 1×1; In step S3, the method for obtaining the output feature map of the multimodal attention is: multiplying the spatial attention and channel attention with the input features respectively, and superimposing them, and assigning different weights to the input data from the spatial and channel dimensions respectively; its formula is expressed as: in, represents the feature map after attention weighting; F is the feature map, and is the weight; Convolution sparse representation is used for pixel-level fusion, and its formula is expressed as: in, represents the m-dimensional dictionary filter, f represents the image to be sparsely represented, represents a set of feature maps of f, which has the same size as f, * represents the convolution operation, and denote the 1-norm and 2-norm respectively, is a sparse regularization term, representing the balance between model accuracy and sparsity.

2. The micro-satellite cluster multi-modal remote sensing image target detection and recognition method according to claim 1, characterized in that: In step S1, the multimodal remote sensing image includes a visible light remote sensing image, an infrared remote sensing image and a SAR remote sensing image.

3. The micro-satellite cluster multi-modal remote sensing image target detection and recognition method according to claim 1, characterized in that: In step S3, feature fusion includes: S3.

1. Feature-level fusion via multimodal attention mechanism; S3.

2. Pixel-level fusion via convolutional sparse representation.

4. The micro-satellite cluster multi-modal remote sensing image target detection and recognition method according to claim 3 is characterized in that: In step S3.1, the steps of feature-level fusion are as follows: first, the parallel input visible light feature map, infrared feature map and SAR feature map are spliced ​​by channel through the concat operation; then, the multimodal attention mechanism is used to assign different weights to different parts of the input features to enhance the model's attention to key information and obtain a multimodal fusion feature map.

5. The micro-satellite cluster multi-modal remote sensing image target detection and recognition method according to claim 4, characterized in that: The multimodal attention mechanism of feature-level fusion includes spatial attention and channel attention, which assign different weights to the input data from the spatial and channel dimensions respectively, thereby enhancing the network's attention to key information.

6. The micro-satellite cluster multi-modal remote sensing image target detection and recognition method according to claim 1, characterized in that: In step S4, the target detection and recognition adopts the SCFNet network structure; the network structure designs three recognition heads of different sizes to meet the needs of multi-size target extraction, and uses the local feature correction module to correct the local semantic features and the non-local feature module to obtain the long-distance semantic features; Inputting the feature map output by feature fusion in step S3 into the SCFNet network structure can obtain the results of target detection and recognition.

Images