Remote sensing target detection method and system based on multi-scale cross-instance clustering mutual exclusion information contrast self-supervision

Abstract

The application discloses a remote sensing target detection method and system based on multi-scale cross-instance clustering mutual exclusion information contrast self-supervision, adopts multi-scale feature extraction of image features, learns through the confrontation of a negative sample queue and a self-supervision network, learns through instance clustering group discrimination, trains a self-supervision model through mutual exclusion information contrast loss and clustering information contrast loss, and then adopts a downstream remote sensing target detection network suitable for an upstream self-supervision task to perform transfer learning, extracts feature information of a to-be-detected image, obtains a predicted label category, confidence and a prediction box, and is suitable for remote sensing target detection with large scale changes and complex backgrounds.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a remote sensing target detection method and system based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision. Background Art

[0002] With the continuous advancement of modern technology, China's satellite technology is developing rapidly. With the continuous development of satellite technology, Earth observation via satellite has gradually become the mainstream method for observing regional changes on Earth. Remote sensing target detection involves classifying, identifying, and locating objects of interest in remote sensing imagery through manually designed features or deep learning algorithms. This provides significant assistance for better observing changes in Earth's land and water, urban and rural planning, mapping, and smart city development. However, remote sensing images present distinct challenges compared to ordinary natural images.

[0003] First, remote sensing images are large in size and contain a wealth of background information, resulting in a small proportion of the target. Remote sensing images have a wide resolution and are several times larger than natural images. The scene information is extremely complex, and the target only accounts for a small proportion of the image. Furthermore, most scene information contains redundant features, which introduces significant noise interference to feature extraction algorithms, hindering feature extraction and detection of remote sensing targets. Object detection in large images often requires higher experimental costs.

[0004] Secondly, objects in remote sensing images vary widely in size, their shapes are fluid, and they exhibit directional characteristics. The significant size disparity between remote sensing objects within and across different categories requires algorithm design for both small- and large-scale targets, placing high demands on algorithm scale adaptability. Unlike objects in natural images, remote sensing objects often have diverse shapes. For example, ships and roads have widely varying morphologies, resulting in poor positioning algorithms and a tendency to deviate from the target.

[0005] In addition, due to the influence of the external environment, remote sensing images are blurred, the outline of the remote sensing target is not obvious, and the distinction from the surrounding background features is small, which increases the difficulty of classifying and identifying the remote sensing targets. It is necessary to combine defogging, declouding or filtering algorithms to preprocess the remote sensing images.

[0006] In recent years, despite numerous studies on remote sensing target detection, object detection in natural images is not suitable for high-resolution remote sensing images due to the characteristics of remote sensing images and the influence of the external environment. This makes direct application in the remote sensing field difficult, and requires targeted improvements based on the characteristics of remote sensing images. Therefore, for remote sensing target detection, it is necessary to design a target detection method that can exploit the deep features of remote sensing images and maintain good detection results even in complex and changing high-resolution remote sensing images. Summary of the Invention

[0007] The purpose of the present invention is to provide a remote sensing target detection method and system based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision to solve the problems existing in the background technology.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: a remote sensing target detection method based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision, comprising:

[0009] Generate three different global views through the data augmentation module;

[0010] A multi-scale feature extraction module is used to extract features of the global view. The self-supervised model is trained using the mutually exclusive information contrast loss and the clustering information contrast loss to obtain the upstream self-supervised model.

[0011] Use the trained upstream self-supervised model for transfer learning and convert it into a pre-trained model that matches the downstream remote sensing target detection network;

[0012] Use the trained remote sensing target detection model to detect the image to be detected and obtain the predicted label category, confidence and prediction box.

[0013] Furthermore, the process of data augmentation for a global view is:

[0014] A high-resolution remote sensing image is input, and the given image is randomly cropped according to different sizes and aspect ratios, and scaled to a preset size. Random grayscale, Gaussian filtering, color jittering, random flipping, etc. are applied to the cropped image to generate three different global views based on the given image.

[0015] Furthermore, a multi-scale feature extraction module is used to extract features of the global view. Through adversarial learning between negative sample queues and self-supervised networks, instance clustering group discrimination learning is performed, and the self-supervised model is trained by mutually exclusive information contrast loss and clustering information contrast loss to obtain a self-supervised model that can be used for transfer learning.

[0016] Furthermore, the loss function L of the upstream self-supervisory model is,

[0017] L=L me +L cin

[0018]

[0019]

[0020]

[0021]

[0022]

[0023]

[0024]

[0025] Among them, L me Calculate the value of the mutually exclusive information contrast loss function;

[0026] For global view I q1 , I k The contrast loss function calculation value between For global view I q2 , I k The contrast loss function calculation value between Indicates I q1 , I q2 The minimum scale feature after the encoder, Indicates I q1 , I q2 For the I k The minimum scale feature after the encoder, Indicates I q1 , I q2 I for the negative pair k The minimum scale feature of the encoder, n∈[1,K], K is the length of the negative sample queue, and T represents the temperature hyperparameter;

[0027] L cin Calculate the value of the clustering information contrast loss function, For L cin component parts;

[0028] kmeans represents a clustering algorithm based on Euclidean distance, C represents the number of aggregation centers, and iters represents the number of iterations. Respectively represent I q1 , I q2 The other two scale features, Respectively represent I q1 , I q2 The label representation corresponding to the feature representation, Respectively represent I q1 , I q2 The cluster center corresponding to the feature representation, CELoss is the cross entropy loss function CrossEntropyLoss, and t is the transpose.

[0029] Furthermore, the conversion rules for pre-trained model conversion are:

[0030] The upstream self-supervised model M u and downstream remote sensing target detection model Md The model parameter key set k u ∈C uk 、k d ∈C dk , value set v u ∈C uv 、v d ∈C dv The conversion is performed according to the fuzzy matching principle σ. The fuzzy matching principle σ is to convert C dk As a standard key set, C uk As a set of matching keys, remove C uk Middle K u The starting structure name, the remaining key names and k d Perform full character matching and match the k d and k u The corresponding v u Save them to the pre-trained model Mn in the form of key-value pairs, and skip those that do not match.

[0031] Furthermore, the trained self-supervised model is converted into a pre-trained model and loaded into the remote sensing target detection network for training. The trained remote sensing target detection model is used to detect the remote sensing image to be detected, and the predicted label category, confidence and prediction box are obtained.

[0032] The present invention also provides a remote sensing target detection system based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision, comprising:

[0033] Data augmentation module: Generates three different global views using multiple data augmentation methods;

[0034] Self-supervised learning module: A multi-scale feature extraction module is used to extract features of the global view. The self-supervised model is trained using mutually exclusive information contrast loss and cluster information contrast loss to obtain the upstream self-supervised model.

[0035] Model conversion module: Use the trained upstream self-supervised model for transfer learning and convert it into a pre-trained model that matches the downstream remote sensing target detection network;

[0036] Remote sensing target detection module: Use the trained remote sensing target detection model to detect the image to be detected and obtain the predicted label category, confidence level and prediction box.

[0037] Furthermore, the data augmentation module includes: using multiple data augmentation methods such as random cropping, random grayscale, Gaussian filtering, color jittering, random flipping, etc. to generate three different global views based on a given image.

[0038] Furthermore, the self-supervised learning module includes:

[0039] Multi-scale feature extraction module: obtains the global view of high-resolution remote sensing images and extracts features of the global view through the multi-scale feature extraction module;

[0040] Total contrast loss module: Through adversarial learning between negative sample queues and self-supervised networks, through instance clustering group discrimination learning, the self-supervised model is trained by mutually exclusive information contrast loss and clustering information contrast loss, resulting in a self-supervised model that can be used for transfer learning.

[0041] Furthermore, the loss function L of the self-supervised learning module is:

[0042] L=L me +L cin

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] Among them, L me Calculate the value of the mutually exclusive information contrast loss function;

[0051] For global view I q1 , I k The contrast loss function calculation value between For global view I q2 , I k The contrast loss function calculation value between Indicates I q1 , I q2 The minimum scale feature after the encoder, Indicates I q1 , I q2 For the I k The minimum scale feature after the encoder, Indicates I q1 , I q2 I for the negative pair k The minimum scale feature of the encoder, n∈[1,K], K is the length of the negative sample queue, and T represents the temperature hyperparameter;

[0052] Lcin Calculate the value of the clustering information contrast loss function, For L cin component parts;

[0053] kmeans represents a clustering algorithm based on Euclidean distance, C represents the number of aggregation centers, and iters represents the number of iterations. Respectively represent I q1 , I q2 The other two scale features, Respectively represent l q1 、l q2 The label representation corresponding to the feature representation, Respectively represent I q1 , I q2 The cluster center corresponding to the feature representation, CELoss is the cross entropy loss function CrossEntropyLoss, and t is the transpose.

[0054] Furthermore, the conversion rules of the model conversion module are:

[0055] The upstream self-supervised model M u and downstream remote sensing target detection model M d The model parameter key set k u ∈C uk 、k d ∈C dk , value set v u ∈C uv 、v d ∈C dv The conversion is performed according to the fuzzy matching principle σ. The fuzzy matching principle σ is to convert C dk As a standard key set, C uk As a set of matching keys, remove C uk Middle K u The starting structure name, the remaining key names and k d Perform full character matching and match the k d and k u The corresponding v u Save to the pre-trained model M in the form of key-value pairs n Those that do not match are skipped.

[0056] Furthermore, the trained self-supervised model is converted into a pre-trained model and loaded into the remote sensing target detection network for training. The trained remote sensing target detection model is used to detect the remote sensing image to be detected, and the predicted label category, confidence and prediction box are obtained.

[0057] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention adopts multiple data enhancement methods to generate a global view of a given image, and the multi-scale feature extraction module extracts features of the global view. Through adversarial learning between negative sample queues and self-supervised networks, and through instance clustering group discrimination learning, the self-supervised model is trained by mutually exclusive information contrast loss and clustering information contrast loss. Then, the trained upstream self-supervised model is used for transfer learning, and converted into a pre-trained model that matches the downstream target detection network. The trained remote sensing target detection model is used to detect the image to be detected, and the feature information of the image to be detected is extracted to obtain the predicted label category, confidence and prediction box, and obtain high-precision predicted defect category, confidence and prediction box. It is suitable for remote sensing image detection of multiple targets and small targets with large scale changes and complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0060] like Figure 1 As shown, a remote sensing target detection method based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision includes the following steps:

[0061] Step 1: Generate three different global views through the data augmentation module; including:

[0062] 11) Obtain high-resolution remote sensing unlabeled images, and use the random cropping enhancement method to randomly crop the given image according to different sizes and aspect ratios, and cover at least 80% of the original image area, and at most remain unchanged, and scale the cropped image to a preset size of 702*702.

[0063] 12) Using the color dithering enhancement method, the properties of the scaled image are changed, and the brightness, contrast, and saturation are randomly changed to 60% to 140% of the original image brightness, contrast, and saturation, and the hue shift amplitude is 0 to 0.1.

[0064] 13) The image is converted to grayscale using random grayscale enhancement with probability p = 0.2, and Gaussian filtering is performed on the image using a Gaussian kernel with a size of 1:1 and a standard deviation of 0. The image is horizontally flipped using random flip enhancement with probability p = 0.5. Finally, the image is normalized and standardized by channel to generate three different global views. Figure 1 , 2, 3——I q1 , I q2 , Ik .

[0065] Step 2: Train the self-supervised model using mutually exclusive information contrast loss and clustering information contrast loss to obtain the upstream self-supervised model.

[0066] The self-supervised model is trained through the self-supervised learning module as follows:

[0067] A multi-scale feature extraction module is used to extract features from the global view. Through adversarial learning between negative sample queues and a self-supervised network, instance clustering and group discrimination learning are performed. The self-supervised model is trained using mutually exclusive information contrast loss and cluster information contrast loss to obtain a self-supervised model that can be used for transfer learning. The following steps are included:

[0068] 21) Initialize the memory bank MB containing the learnable weights as the negative sample queue. The initialization design of the negative sample queue weight MB_W and bias MB_V is as follows:

[0069] MB_W=ε~N(dim,K)

[0070] MB_V=Zeros(dim, K)

[0071] Among them, ε~N(dim, K) means drawing a set of random numbers from a normal distribution with mean equal to the feature dimension value dim and variance equal to the negative sample queue length K; Zeros(dim, K) means creating a tensor of all zeros with shape (dim, K);

[0072] 22) Global View I q1 , I q2 As the input of the multi-scale feature extraction module Encoder_q in the self-supervised learning module, the global view I k As the input of the multi-scale feature extraction module Encoder_k in the self-supervised learning module, the global image I is obtained. q1 , I q2 with I k The mutually exclusive information contrast loss between them includes the following:

[0073] A1) will I q1 , I q2 Send it to the 53-layer backbone network Encoder_q as the encoder, and I k Send to Encoder_k and get N m = 3 feature maps of different scales, send the feature maps of different scales into three multi-layer perceptrons MLP whose parameters are not shared with each other, perform dimensionality reduction processing on the feature maps to obtain feature representation, and calculate the P norm of the three feature representations through P-Norm operation to obtain I q1 , I q2 Feature representation and Get I k The minimum scale feature representation of

[0074] A2) The same batch of the same image Put it into the query, and put different images When the negative sample is sent to the queue, the parameters of Encoder_k are initialized to be consistent with those of Encoder_q, and its parameters do not participate in back propagation;

[0075] A3) As a query, if I q1 , I q2 and I k From the same image, they are represented as opposite to each other. As Similar sample representation Otherwise they are represented as negative pairs. As Different sample representations Will and The similarity between them is as large as possible, and the similarity with other The similarity between them is as small as possible as the constraint condition for model training, and the contrast learning between the global view features is constructed. q1 , I q2 and I k The loss function between The design is as follows:

[0076]

[0077]

[0078] in, Indicates I q1 , I q2 The minimum scale feature after the encoder, Indicates I q1 , I q2 For the I k The minimum scale feature after the encoder, Indicates I q1 , I q2 I for the negative pair k The minimum scale feature of the encoder, n∈[1,K], K is the length of the negative sample queue, T represents the temperature hyperparameter, which makes the negative samples with higher similarity stay away and makes the representation space more uniform;

[0079] Mutually exclusive information contrast loss function L me The design is as follows:

[0080]

[0081] A4) The feature representation is sent to the negative sample queue to update the weight MB_W and bias MB_V, and the contrastive learning loss is maximized to update the negative sample queue, so that the negative sample queue and the training network can perform mutual exclusion and adversarial learning. The update formulas of MB_W and MB_V are designed as follows:

[0082] MB_V=m*MB_V+g+W*MB_W

[0083] MB W =MB W -lr*MB V

[0084] Among them, m is momentum, g is weight correction value, W is weight decay value, and lr represents negative sample weight learning rate;

[0085] 22) and They are sent to the K-means clustering module for clustering. Instance clustering groups similar instances into the same cluster center. Given the feature representation, different views of the same instance can obtain the same clustering result, making the cluster centers as close as possible and far away from other cluster centers, thus realizing group discrimination learning. The clustering information contrast loss function L cin The design is as follows:

[0086]

[0087]

[0088]

[0089]

[0090] Among them, kmeans represents the clustering algorithm based on Euclidean distance, C represents the number of aggregation centers, and iters represents the number of iterations. Respectively represent I q1 , I q2 The other two scale features, Respectively represent I q1 , I q2 The label representation corresponding to the feature representation, Respectively represent I q1 , I q2 The cluster center corresponding to the feature representation, CELoss is the cross entropy loss function CrossEntropyLoss, and t is the transpose;

[0091] 23) Compare the mutually exclusive information loss function L me Compared with clustering information loss function L cin Combined, we obtain the loss function L of the total contrast loss module, and after iteration, we obtain a self-supervised model that can be used for transfer learning. The total loss function L of the multi-scale cross-instance clustering mutually exclusive information contrast self-supervised model is designed as follows:

[0092] L=L me +L cin

[0093] Encoder_k is updated with momentum by Encoder_q, and the momentum update formula is designed as follows:

[0094] m=(1-θ q )+mθ k

[0095] Step 3: Use the trained upstream self-supervised model for transfer learning and convert it into a pre-trained model that matches the downstream remote sensing target detection network; including:

[0096] The conversion rules of the model conversion module are:

[0097] The upstream self-supervised model M u and downstream remote sensing target detection model M d The model parameter key set k u ∈C uk 、k d ∈C dk , value set v u ∈C uv 、v d ∈C dv The conversion is performed according to the fuzzy matching principle σ. The fuzzy matching principle σ is to convert C dk As a standard key set, C uk As a set of matching keys, remove C uk Middle K u The starting structure name, the remaining key names and k d Perform full character matching and match the k d and k u The corresponding v u Save to the pre-trained model M in the form of key-value pairs n Those that do not match are skipped.

[0098] Specifically, k u ∈C uk Remove the layer structure name starting with 'encoder_k' and change k u ∈C uk Remove the layer structure name containing 'mlp' and change k u∈C uk The layer structure name starting with 'encoder_q' is the same as k d ∈C dk Perform full character matching on the layer structure name in the u Change to k d , and k u The corresponding v u Save to the pre-trained model M in the form of key-value pairs n Those that do not match are skipped.

[0099] The pre-trained model needs to be tested for effectiveness through downstream linear evaluation. The output layer of the self-supervised learning module is changed to a classification layer. The structural layers of the pre-trained model except the classification layer are frozen. The simplified contact network image training set is sent to the training, and the validation set is used as validation to test the classification effect of the validation set.

[0100] Step 4: Use the trained remote sensing target detection model to detect the image to be detected and obtain the predicted label category, confidence level, and prediction box;

[0101] 41) Create remote sensing image training sets, validation sets, and test sets.

[0102] 42) After adjusting the remote sensing image, send it to the remote sensing target detection module and train the remote sensing target detection model through iteration. The loss function L of the remote sensing target detection module is s The design is as follows:

[0103] L s =BCE(obj y , obj s )+BCE(cls y ,cls s )+IOU(box y , box s )

[0104] Among them, BCE represents BCEWithLogitsLoss binary cross entropy loss function, obj y Indicates whether the remote sensing target detection model predicts whether the image has defects, obj s Indicates whether the image actually has defects, cla y ,cls s They represent the defect category predicted by the remote sensing target detection model and the actual category, IOU represents the IOU Loss loss function, and the detection prediction box y With the real box s The overlapping area between them.

[0105] 43) Based on the obtained remote sensing target detection model, test the remote sensing image to be detected to obtain the predicted label category, confidence and prediction box.

[0106] The above method uses multiple data enhancement methods to generate a global view of a given image. The multi-scale feature extraction module extracts features of the global view. Through adversarial learning between negative sample queues and self-supervised networks, instance clustering group discrimination learning is performed, and the self-supervised model is trained by mutually exclusive information contrast loss and clustering information contrast loss. The trained upstream self-supervised model is then used for transfer learning and converted into a pre-trained model that matches the downstream target detection network. The trained remote sensing target detection model is used to detect the image to be detected, extract feature information of the image to be detected, and obtain the predicted label category, confidence and prediction box. The above method is suitable for remote sensing image detection of multiple small targets with large scale changes and complex backgrounds.

[0107] Based on the same inventive concept, the present invention also provides a remote sensing target detection system based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision, comprising:

[0108] Data augmentation module: Generates three different global views using multiple data augmentation methods;

[0109] The data augmentation module includes,

[0110] Various data enhancement methods such as random cropping, random grayscale, Gaussian filtering, color jittering, random flipping, etc. are used to generate three different global views based on a given image.

[0111] Self-supervised learning module: A multi-scale feature extraction module is used to extract features of the global view. The self-supervised model is trained using mutually exclusive information contrast loss and cluster information contrast loss to obtain the upstream self-supervised model.

[0112] The self-supervised learning module includes,

[0113] Multi-scale feature extraction module: obtains the global view of high-resolution remote sensing images and extracts features of the global view through the multi-scale feature extraction module;

[0114] Total contrast loss module: Through adversarial learning between negative sample queues and self-supervised networks, through instance clustering group discrimination learning, the self-supervised model is trained by mutually exclusive information contrast loss and clustering information contrast loss, resulting in a self-supervised model that can be used for transfer learning.

[0115] The loss function L of the self-supervised learning module is,

[0116] L=L me +L cin

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124] Among them, L mfc Calculate the value of the mutually exclusive information contrast loss function;

[0125] For global view I q1 , I k The contrast loss function calculation value between For global view I q2 , I k The contrast loss function calculation value between Indicates I q1 , I q2 The minimum scale feature after the encoder, Indicates I q1 , I q2 For the I k The minimum scale feature after the encoder, Indicates I q1 , I q2 I for the negative pair k The minimum scale feature of the encoder, n∈[1,K], K is the length of the negative sample queue, and T represents the temperature hyperparameter;

[0126] L cin Calculate the value of the clustering information contrast loss function, For L cin component parts;

[0127] kmeans represents a clustering algorithm based on Euclidean distance, C represents the number of aggregation centers, and iters represents the number of iterations. Respectively represent I q1 , I q2 The other two scale features, Respectively represent I q1 , I q2 The label representation corresponding to the feature representation, Respectively represent I q1 , I q2The cluster center corresponding to the feature representation, CELoss is the cross entropy loss function CrossEntropyLoss, and t is the transpose.

[0128] Model conversion module: Use the trained upstream self-supervised model for transfer learning and convert it into a pre-trained model that matches the downstream defect detection network; the model conversion rules are:

[0129] The upstream self-supervised model M u and downstream remote sensing target detection model M d The model parameter key set k u ∈C uk 、k d ∈C dk , value set v u ∈C uv 、v d ∈C dv The conversion is performed according to the fuzzy matching principle σ. The fuzzy matching principle σ is to convert C dk As a standard key set, C uk As a set of matching keys, remove C uk Middle K u The starting structure name, the remaining key names and k d Perform full character matching and match the k d and k u The corresponding v u Save to the pre-trained model M in the form of key-value pairs n Those that do not match are skipped.

[0130] Remote sensing target detection module: Use the trained remote sensing target detection model to detect the image to be detected and obtain the predicted label category, confidence and prediction box. The detection process of the remote sensing target detection module is as follows:

[0131] The trained self-supervised model is converted into a pre-trained model and loaded into the remote sensing target detection network for training. The trained remote sensing target detection model is used to detect the remote sensing image to be detected to obtain the predicted label category, confidence and prediction box.

[0132] A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, enable the computing device to perform a simplified contact network defect detection method based on multi-scale dense contrast enhanced self-supervision.

[0133] A computing device includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing a remote sensing target detection method based on multi-scale cross-instance clustering mutually exclusive information contrast self-supervision.

[0134] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0135] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0138] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A remote sensing target detection method based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision, characterized by: include: Generate three different global views through the data augmentation module; The process of data augmentation for the global view is as follows: input a high-resolution remote sensing image, randomly crop the given image according to different sizes and aspect ratios, and scale it to a preset size. Random grayscale, Gaussian filtering, color dithering, and random flipping are applied to the cropped image to generate three different global views based on the given image. A multi-scale feature extraction module is used to extract features from the global view. A self-supervised model is trained using mutually exclusive information contrast loss and cluster information contrast loss to obtain an upstream self-supervised model. Specifically, the multi-scale feature extraction module is used to extract features from the global view. Through adversarial learning between a negative sample queue and a self-supervised network, and through instance clustering group discriminative learning, the self-supervised model is trained using mutually exclusive information contrast loss and cluster information contrast loss to obtain a self-supervised model that can be used for transfer learning. The loss function L of the upstream self-supervised model is: L=L me +L cin Among them, L me Calculate the value of the mutually exclusive information contrast loss function; For global view I q1 , I k The contrast loss function calculation value between For global view I q2 , I k The contrast loss function calculation value between Indicates I q1 , I q2 The minimum scale feature after the encoder, Indicates I q1 , I q2 For the I k The minimum scale feature after the encoder, Indicates I q1 , I q2 I for the negative pair k The minimum scale feature of the encoder, n∈[1,K], K is the length of the negative sample queue, and T represents the temperature hyperparameter; L cin Calculate the value of the clustering information contrast loss function, For L cin component parts; kmeans represents a clustering algorithm based on Euclidean distance, C represents the number of aggregation centers, and iters represents the number of iterations. Respectively represent I q1 , I q2 The other two scale features, Respectively represent I q1 , I q2 The label representation corresponding to the feature representation, Respectively represent I q1 , I q2 The cluster center corresponding to the feature representation, CELoss is the cross entropy loss function CrossEntropyLoss, and t is the transpose; Use the trained upstream self-supervised model for transfer learning and convert it into a pre-trained model that matches the downstream remote sensing target detection network; Use the trained remote sensing target detection model to detect the image to be detected and obtain the predicted label category, confidence and prediction box.

2. The remote sensing target detection method based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision according to claim 1 is characterized in that: The conversion rules for pre-trained model conversion are: The upstream self-supervised model M u and downstream remote sensing target detection model M d The model parameter key set k u ∈C uk 、k d ∈C dk , value set v u ∈C uv 、v d ∈C dv According to the fuzzy matching principle σ, the conversion is performed; the fuzzy matching principle σ is to convert C dk As a standard key set, C uk As a set of matching keys, remove C uk Middle K u The starting structure name, the remaining key names and k d Perform full character matching and match the k d and k u The corresponding v u Save to the pre-trained model M in the form of key-value pairs n Those that do not match are skipped; The trained self-supervised model is converted into a pre-trained model and loaded into the remote sensing target detection network for training. The trained remote sensing target detection model is used to detect the remote sensing image to be detected to obtain the predicted label category, confidence and prediction box.

3. A remote sensing target detection system based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision, characterized by: include: Data augmentation module: Generates three different global views using multiple data augmentation methods; The data augmentation module uses random cropping, random grayscale, Gaussian filtering, color jittering, and random flipping methods to generate three different global views based on a given image; Self-supervised learning module: A multi-scale feature extraction module is used to extract features of the global view. The self-supervised model is trained using mutually exclusive information contrast loss and cluster information contrast loss to obtain the upstream self-supervised model. The self-supervised learning module includes: Multi-scale feature extraction module: obtains the global view of high-resolution remote sensing images and extracts features of the global view through the multi-scale feature extraction module; Total Contrastive Loss Module: Through adversarial learning between negative sample queues and self-supervised networks, and through instance clustering and group discrimination learning, the self-supervised model is trained using mutually exclusive information contrastive loss and cluster information contrastive loss, resulting in a self-supervised model that can be used for transfer learning. The loss function L of the self-supervised learning module is: L=L me +L cin Among them, L me Calculate the value of the mutually exclusive information contrast loss function; For global view I q1 , I k The contrast loss function calculation value between For global view I q2 , I k The contrast loss function calculation value between Indicates I q1 , I q2 The minimum scale feature after the encoder, Indicates I q1 , I q2 For the I k The minimum scale feature after the encoder, Indicates I q1 , I q2 I for the negative pair k The minimum scale feature of the encoder, n∈[1,K], K is the length of the negative sample queue, and T represents the temperature hyperparameter; L cin Calculate the value of the clustering information contrast loss function, For L cin component parts; kmeans represents a clustering algorithm based on Euclidean distance, C represents the number of aggregation centers, and iters represents the number of iterations. Respectively represent I q1 , I q2 The other two scale features, Respectively represent I q1 , I q2 The label representation corresponding to the feature representation, Respectively represent I q1 , I q2 The cluster center corresponding to the feature representation, CELoss is the cross entropy loss function CrossEntropyLoss, and t is the transpose; Model conversion module: Use the trained upstream self-supervised model for transfer learning and convert it into a pre-trained model that matches the downstream remote sensing target detection network; Remote sensing target detection module: Use the trained remote sensing target detection model to detect the image to be detected and obtain the predicted label category, confidence level and prediction box.

4. The remote sensing target detection system based on multi-scale cross-instance clustering mutually exclusive information comparison self-supervision according to claim 3 is characterized in that: The conversion rules of the model conversion module are: The upstream self-supervised model M u and downstream remote sensing target detection model M d The model parameter key set k u ∈C uk 、k d ∈C dk , value set v u ∈C uv 、v d ∈C dv According to the fuzzy matching principle σ, the conversion is performed; the fuzzy matching principle σ is to convert C dk As a standard key set, C uk As a set of matching keys, remove C uk Middle K u The starting structure name, the remaining key names and k d Perform full character matching and match the k d and k u The corresponding v u Save to the pre-trained model M in the form of key-value pairs n Those that do not match are skipped; The trained self-supervised model is converted into a pre-trained model and loaded into the remote sensing target detection network for training. The trained remote sensing target detection model is used to detect the remote sensing image to be detected to obtain the predicted label category, confidence and prediction box.

Images