A dual-branch remote sensing target intelligent detection method fusing privacy computing

Abstract

The application discloses a kind of dual-branch remote sensing target intelligent detection methods of fusing privacy computing, comprising the following steps: S1, data preprocessing, obtain the image weight of each image;S2, establish dual-branch remote sensing target detection model, including feature extraction module, feature fusion module and three double-branch output module;Feature extraction module adds SimAM mechanism and SPPF_ReLU module in the CSPDarknet53 feature extraction module of YOLOv5l;Feature fusion module uses CSPDarknet53 architecture;S3, according to image weight, image samples in data sample set are sampled, input dual-branch remote sensing target detection model is trained, and the trained model is used to detect remote sensing target.The application can enhance its attention to a small number of class features, and excellent detection results are obtained for the disaster remote sensing image scene with extremely unbalanced sample class distribution.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of remote sensing image target detection, and particularly relates to a disaster remote sensing image target detection method for class distribution imbalance. BACKGROUND

[0002] Remote sensing image target detection is a key technology in the field of remote sensing image interpretation, which can identify and locate key instances in remote sensing images. The key instance information extracted from remote sensing images by target detection models has been widely used in aerospace, change detection, wetland protection, land resource investigation, disaster prevention and reduction, etc. With the breakthrough and innovation of artificial intelligence technology, the target detection method based on supervised learning has made significant progress in the field of remote sensing image processing. However, the occurrence frequency of target instances in nature varies, resulting in generally unbalanced number of instances of each class. That is, the number of instances of different classes varies significantly, and the proportion of samples of minority classes in the total is relatively large, while the proportion of samples of majority classes is relatively small. This imbalance in class distribution will be further aggravated after imaging by special remote sensing equipment. During the training process, the target detection model mainly learns the features of the classes with more samples, which will further aggravate the imbalance in class distribution. Therefore, efficiently solving the problem of class distribution imbalance in remote sensing image target detection task is crucial for the intelligentization of remote sensing technology. At present, resampling and reweighting methods are commonly used to solve the problem of class distribution imbalance. Resampling is to undersample the classes with more samples to reduce the number of instances of such classes, and to oversample the classes with fewer samples to increase the number of instances of such classes, so as to balance the number of different classes; however, resampling may lead to overfitting of the model. The reweighting method gives greater weight to the classes with fewer samples and smaller weight to the classes with more samples. Studies have shown that decoupled training helps to alleviate the problem of class distribution imbalance.

[0003] Since remote sensing images are non-contact collected by sensors, they have the characteristics of wide coverage, large variation in target size, unbalanced number of targets, complex background, etc. Although related research has made certain progress, there are still the following problems:

[0004] 1) The occurrence frequency of target instances in nature varies, and the number of instances of different classes is generally unbalanced. That is, the number of instances of different classes varies greatly, and the proportion of samples of minority classes is large, while the proportion of samples of majority classes is small. Imaging by special remote sensing equipment will further aggravate the imbalance in class distribution, directly leading to insufficient learning of minority class features by the model, resulting in missed detection and serious reduction of the performance of the imaging model.

[0005] 2) When using a single-stage target detection method for remote sensing image target detection, taking the YOLO series method as an example, the model first divides the input image into a grid, and then generates a large number of anchor boxes based on the grid points. Most of these anchor boxes are background samples, and only a small number of them contain target instances, which will lead to an imbalance between positive and negative samples. When different classes of samples are input into the neural network for training, most classes have high feature frequency and contribute more to the gradient, while a small number of classes have low feature frequency and contribute less to the gradient. Therefore, the model tends to learn the features of the class with more samples. Therefore, the imbalance of different class sample distribution, combined with the existing single-stage algorithm, will seriously affect the performance of the single-stage target detection method.

[0006] 3) At present, resampling and reweighting are common methods to solve the problem of class distribution imbalance. Resampling means that the number of instances of the class with more samples is reduced by undersampling, and the number of instances of the class with fewer samples is increased by oversampling, so as to balance the number of classes. However, resampling may also lead to model overfitting. Reweighting gives a larger weight to the class with fewer samples and a smaller weight to the class with more samples, which will lead to an imbalance in gradient distribution and affect the performance of the model. SUMMARY

[0007] The purpose of the present application is to overcome the shortcomings of the prior art and provide a dual-branch remote sensing target intelligent detection method fused with privacy computing. The present application designs a sampling strategy based on positive class weighting, which can enhance its attention to minority class features. Secondly, a simple parameter-free attention mechanism is introduced into the backbone network, which further improves the feature extraction ability of the model for minority classes. Finally, a dual-branch detection output network is designed to enable the model to effectively distinguish between regression and classification subtasks, and excellent detection results are achieved for disaster remote sensing image scenes with extremely imbalanced sample class distribution.

[0008] The purpose of the present application is achieved by the following technical solution: a dual-branch remote sensing target intelligent detection method fused with privacy computing, comprising the following steps:

[0009] S1, data preprocessing, the specific method is as follows:

[0010] S1-1, according to the sample label of the image in the training data set, the number of each target class in all images , , represents the total number of target classes;

[0011] Then, take the reciprocal of the number of classes and normalize it:

[0012] (1);

[0013] (2);

[0014] represents the sum of the reciprocals of all categories, is the number of the th category, represents the category proportion of the th category;

[0015] S1-2, count the number of each category in each image in the training data set , , represents the total number of images; then multiply by the category proportion to obtain the proportion of the category in the image :

[0016] (3);

[0017] S1-3, obtain the image weight of each image through the proportion of each category :

[0018] (4);

[0019] (5);

[0020] represents the average category accuracy of the th category at the end of the th iteration training, represents the image weight of the th image in the training data set, and the image weight is taken as the sampling probability of the image ;

[0021] For background images without target categories, the image weight is directly assigned as 0.5;

[0022] S2, a dual-branch remote sensing target detection model is established, which includes a feature extraction module, a feature fusion module and three dual-branch output modules;

[0023] The feature extraction module comprises a first convolutional layer, a second convolutional layer, a first C3 module, a third convolutional layer, a second C3 module, a fourth convolutional layer, a third C3 module, a fifth convolutional layer, a fourth C3 module, an SPPF_ReLU module and a SimAM module connected in sequence; the features output by the SPPF_ReLU module are given different attention weights by the SimAM module, then each pixel in the feature map output by the SPPF_ReLU module is multiplied by the weight of each pixel calculated by the SimAM, as the output feature map of the feature extraction module;

[0024] The feature fusion module comprises sixth to ninth convolutional layers, two up-sampling modules and three Im_Ghost modules; the features output by the feature extraction module are input into the sixth convolutional layer, the output of the sixth convolutional layer is connected with the output features of the third C3 module after being processed by the first up-sampling module, and the connected features are input into the fifth C3 module;

[0025] The output features of the fifth C3 module are connected with the output features of the second C3 module after being processed by the seventh convolutional layer and the second up-sampling module in sequence, the connected features are input into the first Im_Ghost module for processing, the output features of the first Im_Ghost module are added with the output features of the second C3 module element by element, and then the added features are input into the first double-branch output module;

[0026] The output features of the first Im_Ghost module are connected with the output features of the seventh convolutional layer after being processed by the eighth convolutional layer, the connected features are input into the second Im_Ghost module, the output features of the second Im_Ghost module are added with the output features of the third C3 module element by element, and then the added features are input into the second double-branch output module;

[0027] The output features of the second Im_Ghost module are connected with the output features of the sixth convolutional layer after being processed by the ninth convolutional layer, then the connected features are input into the third Im_Ghost module, the output features of the third Im_Ghost module are added with the output features of the SPPF_ReLU module element by element, and then the added features are input into the third double-branch output module;

[0028] In the double-branch output module, first, a convolution module with a convolution kernel size of 1*1 is used to compress the feature channel dimension to 256; then, it is divided into two branches: one branch uses a Conv module with a convolution kernel size of 3*3 to extract features, and then uses a basic convolution module with a convolution kernel size of 1*1 to output the classification feature information Cls of the target instance; the other branch uses a Conv module with a convolution kernel size of 3*3 to extract features, and then uses two basic convolution modules with a convolution kernel size of 1*1 to output the confidence information Obj and the regression feature information Reg of the target instance; the classification feature information Cls, the regression feature information Reg and the confidence information Obj are fused in the encryption domain to generate the final output;

[0029] S3, sample the image samples in the data sample set according to the image weight obtained in S1, input the double-branch remote sensing target detection model, train the model, and detect the remote sensing target by using the trained model.

[0030] The beneficial effects of the present application are: for the problem of class distribution imbalance in the remote sensing image target detection task, the present application proposes an efficient double-branch remote sensing image target detection method based on a simple parameter-free attention mechanism. First, a sampling strategy based on positive class weighting is designed. This strategy enables the model to first learn the features of minority classes and then gradually transition to the features of majority classes, thereby enhancing its attention to minority class features. Second, a simple parameter-free attention mechanism is introduced into the backbone network to further improve the model's feature extraction ability for minority classes. Finally, a double-branch detection output network is designed to enable the model to effectively distinguish between regression and classification subtasks. Excellent detection results are achieved for disaster remote sensing image scenes with extremely imbalanced sample class distribution. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 Figure 1 is a schematic diagram of the target detection model structure of the present application;

[0032] Figure 2 Figure 2 is a schematic diagram of the Im_Ghost module of the present application;

[0033] Figure 3 Figure 3 is a schematic diagram of the double-branch output module of the present application. DETAILED DESCRIPTION

[0034] The present application takes YOLOv5l as the basic model to solve the problem of class distribution imbalance in the remote sensing image target detection task. By integrating a simple parameter-free attention mechanism (SimAM), designing and improving the Ghost module, and constructing a double-branch structure, the feature extraction, fusion and detection network of YOLOv5l are enhanced, thereby proposing an efficient double-branch target detection method. The present application is summarized as follows:

[0035] (1) A remote sensing image sampling strategy based on positive class weight is proposed. Through different sampling probabilities, the model first learns the features of the minority class samples, and then gradually transitions to learning the features of the majority class samples, thereby enhancing the feature information of the small number of samples and avoiding the model training always being biased towards the majority class; therefore, the model can reduce the minority class error and improve the overall performance of the model.

[0036] (2) For the basic YOLOv5l model, each position of each neural network is equally important. During the training process, due to insufficient feature information, the minority class features will be "overwhelmed" by the majority class features, thereby affecting the overall accuracy of the model. The SimAM mechanism is introduced to weight the three-dimensional information of the channel and the spatial dimension of the feature map, enhance the ability of the model to express features, and further improve its feature extraction capability under a small number of samples.

[0037] (3) The basic YOLOv5l model uses a coupling method to directly output the classification and regression information of the target instance through the same feature map. However, in a class distribution unbalanced dataset, the effective feature information of the minority class target is less, which will affect the overall accuracy of the model when directly outputting using the coupling method. Therefore, a double-branch decoupling head is designed to perform classification and regression processing respectively to improve the performance of the model.

[0038] The technical solutions of the present application will be further described below in conjunction with the drawings.

[0039] Target detection in remote sensing images has always been the focus of research by scholars in the field of earth science. The complex background of remote sensing images poses higher requirements for target detectors. Class distribution imbalance of target instances is a common problem in target detection. This imbalance can lead to a decrease in the performance of neural network training. The present application proposes a double-branch detection model based on a simple parameter attention-free mechanism for the imbalance problem in remote sensing image target detection, as shown in the structure of Figure 1 .

[0040] The double-branch remote sensing target intelligent detection method of the present application fuses privacy computing, in order to realize high-precision target detection under privacy protection, the present application combines privacy computing technology to construct a secure feature fusion mechanism in the designed double-branch output structure. The entire double-branch output first extracts global features through a shared backbone network, and then enters two independent functional branches: the classification / confidence branch is responsible for evaluating the probability of candidate regions belonging to each class or the existence of targets, and the regression branch is responsible for predicting the accurate position and scale of the target. Although the network structures of the two branches are the same (both composed of 3x3 convolution modules and 1x1 basic convolution modules), the parameter weights are independent, so they can learn different feature patterns for classification and positioning tasks.

[0041] The detection method specifically comprises the following steps:

[0042] S1, data preprocessing: first, since class imbalance is a common phenomenon in remote sensing image target detection. When a data set with class imbalance is input into a neural network for training, due to the large difference in feature frequency between different classes, the model tends to focus on high-frequency features and selectively ignore the features of the minority class. Therefore, after training, the neural network model tends to predict the majority class and ignore the minority class in the test stage, thereby affecting its overall performance. In order to solve this problem, the present application designs a sampling strategy based on positive class weight, which assigns different weights to different target classes, assigns a corresponding weight to each image according to the class weight, and then calculates the sampling probability of the image based on the weight of the image to alleviate the class imbalance problem; the specific method steps are as follows:

[0043] S1-1, according to the sample label of the image in the training data set, the number of each target class in all images is counted , wherein represents the total number of target classes. The name of the class is the name of the target actually existing in each image. The sample label labels the name and number of each class, so that the information and number of each class can be obtained from the label of the class. The image in which the target class does not appear is called a background image. For example, an image has three classes: car, stadium and ship, then =3; the total number of classes refers to the number of various targets in the image, for example: there are 5 cars, 1 stadium and 3 ships in an image, then

[0044] Then, take the reciprocal of the number of classes and normalize it, as shown in formula (1):

[0045] (1)

[0046] (2)

[0047] , wherein represents the sum of the reciprocals of all classes, is the number of the th class, represents the proportion of the th class;

[0048] S1-2, count the number of each class in each image in the training data set , wherein represents the total number of images; then use​​​​​​​​ Multiply by the category proportion to get the proportion of this category in this image :

[0049] (3);

[0050] S1-3, get the image weight of each image by the proportion of each category , according to the image weight, get the sampling probability of each image in each batch (batch refers to a group of samples that participate in training together in a neural network at a time, used to speed up training and stabilize the gradient. For example, batch=32, which means 32 images are sent into the neural network at a time) ; Then sample in each batch according to the sampling probability, the larger the probability, the larger the sampling probability, and the smaller the probability, the smaller the sampling probability, which can ensure that the minority class with higher weight is sampled first for learning, and then gradually transition to sampling the majority class; As formulas (4) and (5) show:

[0051] (4)

[0052] (5)

[0053] indicates the average class accuracy of the i-th class at the end of the j-th iteration training, indicates the image weight of the i-th image in the training data set, and the image weight is taken as the sampling probability of the image ; For background images without target categories, directly assign their image weights to 0.5.

[0054] For background images without target categories, directly assign their image weights to 0.5.

[0055] S2, establish a dual-branch remote sensing target detection model, which includes a feature extraction module (Feature Extraction Module), a feature fusion module (Feature Fusion Module) and three dual-branch output modules (Dual-branch detect small objects, Dual-branch detect medium objects, Dual-branch detect large objects), the model structure is as shown in Figure 1 ;

[0056] ​​The feature extraction module adopts YOLOv5l as a basic model, and a SiAM mechanism is added at the end of the feature extraction backbone network of YOLOv5l network; the SimAM mechanism and the SPPF_ReLU module are added to the CSPDarknet53 feature extraction module of the YOLOv5l network, and the CSPDarknet53 feature extraction module is improved. The improved feature extraction module includes a first convolutional layer (Conv), a second convolutional layer, a first C3 module, a third convolutional layer, a second C3 module, a fourth convolutional layer, a third C3 module, a fifth convolutional layer, a fourth C3 module, a SPPF_ReLU module and a SimAM module connected in sequence.

[0057] The C3 module (Cross Stage Partial, cross-stage partial connection module) is composed of three basic convolutional blocks connected in sequence, and the data input into the C3 module is divided into two parts, one part of the input data is sequentially input into the three basic convolutional blocks, thereby performing deep transformation on a part of the feature map; the output data of the third basic convolutional block is spliced with the other part of the input data as the output of the C3 module. A part of the input feature map x is cross-stage connected with the output of the third basic convolutional block, thereby realizing gradient flow enhancement and feature fusion of different levels, which can balance the network performance to a certain extent while ensuring the accuracy and improving the network representation ability and computational efficiency.

[0058] The SPPF_ReLU module uses multi-scale pooling to realize efficient context feature fusion, and uses ReLU activation instead of the original SiLU activation to improve the computational efficiency and gradient stability; the features output by the SPPF_ReLU module are input into the SimAM module to assign different attention weights, and then each pixel in the feature map output by the SPPF_ReLU module is multiplied (MUL) with the weight of each pixel calculated by the SimAM, as the output feature map of the feature extraction module, which uses the weight to adjust the response intensity of the feature map.

[0059] The SimAM module is a kind of module that evaluates the importance of each pixel through a neuron energy function, which enhances the attention of the spatial dimension and the channel dimension without increasing the number of additional parameters, thereby significantly improving the feature extraction ability of a small number of classes. The SimAM module mainly calculates the energy of each pixel of the input feature map to generate an attention map. Since the pixels of a small number of classes deviate from the mean value by a small amount, the attention weight needs to be increased, while the pixels of a large number of classes are concentrated, and the energy is high, so the attention weight needs to be reduced. The SimAM module weights the full three-dimensional information of the channel dimension and the spatial dimension of the feature map; the definition of the weight is as follows:

[0060] (6)

[0061] (7)

[0062] (8)

[0063] (9)

[0064] denotes the feature map input into the SimAM module The pixel value of the first pixel point is represented by formula (6) and (7), respectively, and the mean value and variance are calculated. ; denotes the minimum energy of the neuron, i.e., the importance; the neuron is the response value of each pixel position in a certain channel on the feature map input into the SimAM module; the reciprocal of the minimum energy value is taken to obtain the weight value of the neuron. denotes the feature map input into the SimAM module; is a set in the channel dimension and the spatial dimension, i.e., the weight value of each neuron, which is obtained by formula (9) after the value .

[0065] The feature fusion module plays a role of a “bridge” in the target detection network, which firstly effectively fuses multi-scale semantic features from different network layers, and then transmits the effectively fused semantic features to the detection network for outputting classification and regression results. The feature fusion module adopts the CSPDarknet53 architecture in the YOLOv5l basic model, and the improvement of the feature fusion module in the present application mainly replaces the original C3 module with the Im_Ghost module (Improved Ghost Module) designed in the present application.

[0066] The feature fusion module includes sixth-ninth convolution layers (Conv), two up-sampling modules (Upsample), and three Im_Ghost modules; the features output by the feature extraction module are input into the sixth convolution layer, and the output of the sixth convolution layer is connected (Concat) with the output features of the third C3 module after passing through the first up-sampling module, and the connected features are input into the fifth C3 module.

[0067] ​​The output feature of the fifth C3 module is sequentially input into a seventh convolutional layer and a second upsampling module, and then connected with the output feature of the second C3 module, and the connected feature is input into a first Im_Ghost module for processing, and the output feature of the first Im_Ghost module is element-wise added (Add) with the output feature of the second C3 module, and then the added feature is input into a first dual-branch output module (Dual-branch detect small objects);

[0068] The output feature of the first Im_Ghost module is processed by an eighth convolutional layer, and then connected with the output feature of the seventh convolutional layer, and the connected feature is input into a second Im_Ghost module, and the output feature of the second Im_Ghost module is element-wise added with the output feature of the third C3 module, and then the added feature is input into a second dual-branch output module (Dual-branch detect medium objects).

[0069] The output feature of the second Im_Ghost module is processed by a ninth convolutional layer, and then connected with the output feature of the sixth convolutional layer, and then the connected feature is input into a third Im_Ghost module, and the output feature of the third Im_Ghost module is element-wise added with the output feature of the SPPF_ReLU module, and then the added feature is input into a third dual-branch output module (Dual-branch detect large objects).

[0070] As shown in Figure 2 The Im_Ghost module includes two branches, one of which includes a GhostConv structure, a DWConv structure and a GhostConv structure connected in sequence, and the other branch includes a DWConv structure and a normal convolution structure connected in sequence; the output features of the two branches are element-wise added, and then input into a GhostConv structure for processing.

[0071] The dual-branch output network continues the multi-scale output mode of the YOLO series model, in which Figure 1 The detection head with high resolution and large scale corresponds to the small receptive field of the original image, and detects small-scale target instances through the first dual-branch output module; the middle detection head (the second dual-branch output module) is used for detecting medium-scale targets; the bottom detection head (the third dual-branch output module) has the lowest resolution and the highest semantic abstraction degree, and corresponds to the large receptive field of the original image, and is used for detecting large-scale target instances.

[0072] Finally, the object detection task needs to classify and locate instances; the classification task pays more attention to the similarity of features, and the location task pays more attention to the position information. Research shows that if the same feature map is used to perform classification and regression tasks at the same time, the model performance may be affected by the problem of imbalance. In this paper, a double-branch structure is designed in the YOLOv5l base model, and different branches are used to complete the classification and positioning tasks respectively, as shown in the double-branch output module of Figure 3 The kernel size in the figure is the convolution kernel size, the anchor is the anchor box of the model, the Cls is the category, the Reg is the regression information, and the Obj is the confidence.

[0073] In the double-branch output module, a convolution module with a convolution kernel size of 1x1 is first used to compress the feature channel dimension to 256; then it is divided into two branches: one branch uses a Conv module with a convolution kernel size of 3x3 to extract features, and then uses a basic convolution module with a convolution kernel size of 1x1 to output the classification feature information Cls of the target instance; the other branch uses a Conv module with a convolution kernel size of 3x3 to extract features, and then uses two basic convolution modules with a convolution kernel size of 1x1 to output the confidence information Obj and the regression feature information Reg of the target instance; the classification feature information Cls, the regression feature information Reg and the confidence information Obj are fused in the encryption domain to generate the final output.

[0074] Under the multi-scale detection framework, the double-branch structure of the classification feature information Cls, the regression feature information Reg and the confidence information Obj is constructed. The output channel of the classification branch is "anchor box number x category number"; the output channel of the regression branch is "anchor box number x 4" (center point coordinates and height); the output channel of the confidence branch is "anchor box number x 1", indicating the probability of the existence of a target in each anchor box. By using different branches to complete the classification and positioning tasks, the classification branch pays more attention to the semantic similarity of features, and the positioning branch focuses on the offset prediction of the spatial geometric position, thereby effectively alleviating the problem of missing detection of minority class targets caused by class imbalance. On this basis, the invention further integrates the idea of privacy computing to construct a secure branch output fusion mechanism. The intermediate features of the classification branch, the regression branch and the confidence branch and the final output are not directly exposed to the external system, but are processed in the encryption or secure computing domain. For the classification feature information Cls, the regression feature information Reg and the confidence information Obj, the encrypted representation Encrypt(Cls), Encrypt(Reg) and Encrypt(Obj) is generated through homomorphic encryption or secure multi-party computation; the fusion process can be represented as: Encrypt(output) i = Encrypt(obj i )×Encrypt(Cls i)+Encrypt(Reg i ); Encrypt(obj i ), Encrypt(Cls i ), and Encrypt(Reg i ) represent the encrypted confidence, classification score, and regression box, respectively; Encrypt(output) i represents the fused output; i represents the i-th candidate box or target. The fusion process is completed in the encrypted domain, ensuring that the original feature information is not leaked.

[0075] The fused output is decrypted using the private key corresponding to the homomorphic encryption: output i = Dec(Encrypt(output) i ); After decryption, the plaintext target score and regression box coordinates of the i-th candidate box are obtained. In the fusion formula, the confidence obj i is multiplied by the classification score Cls i , so the confidence is already fused into the output target score and does not exist separately, but its information is still reflected in the final score. The fused encrypted output contains classification, confidence, and regression information, and after decryption, the fusion result (target score + box coordinates) is obtained, in which the confidence information is already reflected in the target score; homomorphic encryption allows the calculation to be completed in the encrypted domain, and decryption uses the private key to restore the ciphertext to plaintext, achieving a balance between data security and usability.

[0076] S3, sample the image samples in the data sample set according to the image weights obtained in S1, input the dual-branch remote sensing target detection model, train the model, and use the trained model to detect remote sensing targets.

[0077] Those skilled in the art will realize that the embodiments described herein are for the purpose of helping the reader understand the principles of the present application and should be understood as not limiting the scope of protection of the present application to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the essence of the present application, and these modifications and combinations are still within the scope of protection of the present application.

Claims

1. A dual-branch remote sensing target intelligent detection method integrating privacy computing, characterized in that, Includes the following steps: S1. Data preprocessing, the specific methods are as follows: S1-1. Based on the sample labels of the images in the training dataset, count the target category for each image. quantity , , Indicates the total number of target categories; Then, take the reciprocal of the number of times the category appears and normalize it: （1）； （2）； This represents the sum of the reciprocals of all categories. It is the first The number of categories Indicates the first The percentage of each category; S1-2, Count the number of each category in each image of the training dataset. , , Represent the total number of images; then use Multiplying the percentage by the category percentage gives the percentage of that category in the image. : （3）； S1-3. Obtain the image weight of each image based on the proportion of each category. : （4）； （5）； Indicates the first At the end of the first round of iterative training Average category precision for each category Indicates the first The image weights of each image in the training dataset are used as the sampling probabilities of the images. ; For background images without a target category, assign an image weight of 0.5; S2. Establish a dual-branch remote sensing target detection model, which includes a feature extraction module, a feature fusion module, and three dual-branch output modules; The feature extraction module comprises a first convolutional layer, a second convolutional layer, a first C3 module, a third convolutional layer, a second C3 module, a fourth convolutional layer, a third C3 module, a fifth convolutional layer, a fourth C3 module, an SPPF_ReLU module, and a SimAM module, all connected in sequence. The features output by the SPPF_ReLU module are assigned different attention weights by the SimAM module. Then, each pixel in the feature map output by the SPPF_ReLU module is multiplied by the weight of each pixel calculated by simAM to obtain the output feature map of the feature extraction module. The feature fusion module includes the sixth to ninth convolutional layers, two upsampling modules, and three Im_Ghost modules. The features output from the feature extraction module are input into the sixth convolutional layer. The output of the sixth convolutional layer is passed through the first upsampling module and then concatenated with the output features of the third C3 module. The concatenated features are then input into the fifth C3 module. The output features of the fifth C3 module pass through the seventh convolutional layer and the second upsampling module in sequence, and are then connected to the output features of the second C3 module. The connected features are then input into the first Im_Ghost module for processing. The output features of the first Im_Ghost module and the output features of the second C3 module are added element by element, and the resulting features are then input into the first dual-branch output module. The output features of the first Im_Ghost module are processed by the eighth convolutional layer and then connected with the output features of the seventh convolutional layer. The connected features are then input into the second Im_Ghost module. The output features of the second Im_Ghost module are added element-wise with the output features of the third C3 module, and the added features are then input into the second dual-branch output module. The output features of the second Im_Ghost module are processed by the ninth convolutional layer and then connected with the output features of the sixth convolutional layer. The connected features are then input into the third Im_Ghost module. The output features of the third Im_Ghost module are added element-wise with the output features of the SPPF_ReLU module, and the added features are then input into the third dual-branch output module. The Im_Ghost module includes two branches. One branch includes a GhostConv structure, a DWConv structure, and a GhostConv structure connected in sequence. The other branch includes a DWConv structure and a regular convolutional structure connected in sequence. The output features of the two branches are added element-wise and then input into a GhostConv structure for processing. The dual-branch output module first uses a 1×1 convolutional module to compress the feature channel dimension to 256. Then, it splits into two branches: one branch uses a 3×3 Conv module for feature extraction, and then uses a 1×1 basic convolutional module to output the classification feature information Cls of the target instance; the other branch uses a 3×3 Conv module for feature extraction, and then uses two 1×1 basic convolutional modules to output the confidence information Obj and regression feature information Reg of the target instance. The classification feature information Cls, regression feature information Reg, and confidence information Obj are then fused in the cryptographic domain to generate the final output. The method is as follows: encrypted representations are generated using homomorphic encryption or secure multi-party computation: Encrypt(Cls), Encrypt(Reg), and Encrypt(Obj). The fusion process is represented as: Encrypt(output). i = Encrypt(obj i )×Encrypt(Cls i )+Encrypt(Reg i Encrypt(obj) i ), Encrypt(Cls i ) and Encrypt(Reg i ) represent the encrypted confidence level, classification score, and regression box, respectively; Encrypt(output) i This represents the fused output; i represents the i-th candidate box or target. S3. Based on the image weights obtained in S1, sample the image samples in the data sample set, input them into the dual-branch remote sensing target detection model, train the model, and use the trained model to detect remote sensing targets.

Images