Unknown ship target type recognition method in remote sensing images under cross-domain scenarios

Through DenseNet network and semi-supervised density clustering technology, the problem of ship target type recognition model degradation in cross-domain scenarios and unknown type recognition is solved, cross-domain feature alignment and pseudo-label annotation are realized, and the application effect of the recognition model is improved. Comprehensive ability.

CN115797779BActive Publication Date: 2024-11-08NAVAL AVIATION UNIV
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202211667506.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-24
Publication Date
2024-11-08
Estimated Expiration
2042-12-24

AI Technical Summary

Technical Problem

The ship target type recognition model has reduced accuracy and difficulty in identifying unknown types in cross-domain scenarios, which affects the actual application effect.

Method used

The feature vector is extracted by DenseNet network, and the mapping relationship between pseudo-label space and real tag space is established through semi-supervised density clustering, and the joint optimization function of cluster cluster loss and intra-domain and inter-domain loss are constructed to realize cross-domain feature alignment and pseudo-domain feature alignment Labeling improves the recognition ability of the model in different scenarios.

Benefits of technology

It effectively improves the recognition accuracy of the model in cross-domain scenarios, realizes accurate identification of unknown types of ships, and enhances the comprehensive application ability of the recognition model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004015444240000041
    Figure BDA0004015444240000041
  • Figure BDA0004015444240000045
    Figure BDA0004015444240000045
  • Figure BDA0004015444240000046
    Figure BDA0004015444240000046
Patent Text Reader

Abstract

The present invention discloses a method for identifying unknown ship target types in remote sensing images in cross-domain scenarios. The method maps sample features in cross-domain scenarios to a common feature space, and constructs intra-domain and inter-domain losses of sample features of the same type of ships and different types of ships, thereby achieving intra-domain and inter-domain alignment of ship sample features, thereby improving the cross-domain application effect of the model; the method further establishes an extensible pseudo-label space based on the feature space and a mapping relationship between the pseudo-label space and the real label space, and achieves the aggregation of samples of the same type in the feature space and the dispersion of samples of different types in the feature space by constructing a cluster loss, thereby achieving effective distinction and identification of unknown target types. The present invention can identify unknown ship targets in different remote sensing image scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an identification method, in particular to a method for identifying ship target types. Background Art

[0002] For a long time, ship target type identification has been a cutting-edge topic in the field of maritime situational awareness, and it is also a technical problem that the academic and engineering communities pay close attention to and are committed to solving.

[0003] With the vigorous development of deep learning technology, target type classification methods have achieved certain breakthroughs. In recent years, related research on ship target type recognition has made many breakthroughs in small sample limitations, insufficient single-source information, and sample imbalance, but the following problems still exist:

[0004] First, the domain dependence of the ship target type recognition model will lead to a decrease in model accuracy in cross-domain scenarios, affecting the application effect. To address this problem, Xu et al. proposed a domain adaptive support vector machine algorithm (Xu Yongjie, Lang Haitao, Niu Lihui, et al. Discriminative adaptation regularization framework-based transfer learning for ship classification in SAR images [J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16 (11): 1786-1790.), which achieves feature distribution alignment between the target domain and the source domain through decision boundary alignment. However, this method requires the extraction of simple geometric features such as the length and width of the ship, and the accuracy is still low in complex scenarios.

[0005] Second, the type of unknown ship target cannot be identified. To address this problem, Ward et al. proposed using the type spacing metric statistic to identify unknown types (Ward M, Samuel B, Katie R, Stephen H. Identifying novel vessel classes with OOD methods [C]. SPIE Defense + Commercial Sensing, Online Only, 2021.), but the threshold of the HP statistic is not clear.

[0006] In summary, although relevant research in the existing technology has made some progress, the model domain dependency and the recognition of unknown ship target types still restrict the practical application of ship target type recognition. Summary of the invention

[0007] The present invention proposes a method for identifying the type of unknown ship targets in remote sensing images in a cross-domain scenario, the purposes of which are: (1) to solve the problem of decreased accuracy of the ship target type recognition model when it is applied in a cross-domain scenario, and to improve the application effect of the same recognition model in different scenarios; (2) to solve the problem of unknown ship type recognition and to improve the comprehensive ability of ship recognition.

[0008] The technical solution of the present invention is as follows:

[0009] A method for identifying unknown ship target types in remote sensing images in a cross-domain scenario comprises the following steps:

[0010] Step 1. Given ship training data from different domains, the training data includes both training samples of known types and training samples of unknown types; samples of known types are labeled with their types by true labels;

[0011] Step 2. Construct a feature extraction network based on the DenseNet network to extract feature vectors for each training sample;

[0012] Step 3. Cluster all extracted feature vectors to obtain multiple clusters and corresponding cluster center vectors; assign pseudo labels to each cluster;

[0013] Step 4. Obtain the optimal network parameters in the feature extraction network;

[0014] Step 5. Based on the true labels of the samples in the clusters and the pseudo labels of the clusters, a mapping relationship between the pseudo label space and the true label space is constructed to obtain the true labels of each cluster;

[0015] Step 6. Given a test sample, use the optimal network parameters obtained in step 4 to calculate the feature vector of the test sample, and then find the closest cluster based on its feature vector, and use the type represented by the true label of the cluster as the type recognition result of the test sample.

[0016] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario: In step 1, assuming that the training data comes from M different domains, all training data are represented as M training data sets corresponding to the domains one by one: X = {X m |m=1,…,M},

[0017] Among them, N m is the number of samples in the mth domain, represents the i-th training sample from the m-th domain; For sample The true label of , 1~C represents the known C types, and u represents the unknown type.

[0018] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario: in step 2, the feature extraction network adopts the DenseNet network, and the sample The eigenvector of Where m = 1, ..., M, i = 1, ..., N m , ω is the network parameter of DenseNet.

[0019] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario: in step 3, a semi-supervised density clustering algorithm is used to cluster all Perform clustering.

[0020] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario: In step 3, assume that a total of α clusters S are obtained k , k = 1,…,α; let cluster S k The number of samples included is s k , cluster cluster S k The pseudo label is recorded as The cluster S k Samples in The pseudo label is recorded as

[0021] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario: for cluster S k , its cluster center vector q k for:

[0022]

[0023] in, is a bool function. If but otherwise

[0024] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario: in step 4, a joint optimization function J(ω) is constructed, and the stochastic gradient descent method is used for optimization to obtain the optimal network parameter ω * :

[0025] J(ω)=L SA +L SS +L SC ;

[0026]

[0027]

[0028]

[0029] Where Υ is a bool function. but otherwise q + For the current sample The cluster center vector of the cluster; τ is the preset hyperparameter; q k is the cluster center vector of the kth cluster;

[0030] Then the stochastic gradient descent method is used to solve the parameter ω that minimizes J(ω) * ,ω * is the optimal network parameter.

[0031] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario, the specific steps of step 5 are:

[0032] Step 5.1. Set h = 1 and start traversing all clusters;

[0033] Step 5.2. Count the current cluster S k The number of samples of each known type label in the cluster S k The number of samples with the true label c is recorded as

[0034] Step 5.3. Traverse the current cluster S k All true labels in , if for a true label c, the following two conditions are met at the same time:

[0035]

[0036]

[0037] Then cluster S k The true label is recorded as Otherwise, cluster S k The true label is recorded as Then execute h=h+1;

[0038] The value range of λ is [0.5, 1], and the value range of μ is [0.5, 1];

[0039] Step 5.4. Return to step 5.2 and continue to traverse the clusters until the true labels of all clusters are obtained.

[0040] As a further improvement of the remote sensing image unknown ship target type recognition method in the cross-domain scenario: In step 6, for the test sample x, the closest cluster is found by calculating the Euclidean distance between the test sample x and the cluster center vector of each cluster: for cluster S k and its cluster center vector q k , the Euclidean distance is: ω * is the optimal network parameter obtained in step 4; if d k′ =min(d k |k=1,…,α), then the test sample x∈S k′ , the true label corresponding to x

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) This method maps sample features in cross-domain scenarios to a common feature space and constructs intra-domain and inter-domain losses of sample features of the same type of ships and different types of ships, thereby achieving intra-domain and inter-domain alignment of ship sample features, thereby improving the cross-domain application effect of the model. It can identify unknown ship targets in different remote sensing image scenarios (such as offshore scenarios, offshore scenarios, and scenarios with different resolutions), and solve the problem of decreased accuracy of the ship target type recognition model when applied in cross-domain scenarios, thereby improving the application effect of the same recognition model in different scenarios.

[0043] (2) Based on the feature space, this method uses semi-supervised density clustering to establish a scalable pseudo-label space and a mapping relationship between the pseudo-label space and the true label space. By constructing a clustering loss, it realizes the aggregation of samples of the same type in the feature space and the dispersion of samples of different types in the feature space, thereby effectively describing the distribution of known types and unknown types in the feature space and achieving effective distinction and identification of unknown target types.

[0044] (3) This method realizes an integrated multi-objective solution for cross-domain scene target recognition and unknown target type recognition by constructing a joint objective optimization function that includes clustering loss, sample feature intra-domain loss and inter-domain loss. DETAILED DESCRIPTION

[0045] The technical solution of the present invention is described in detail below:

[0046] A method for identifying unknown ship target types in remote sensing images in a cross-domain scenario, characterized by comprising the following steps:

[0047] Step 1. Given ship training data from different domains, the training data includes both training samples of known types and training samples of unknown types; the true labels of samples of known types are marked with their types.

[0048] Specifically, suppose the training data comes from M different domains, and all training data are represented as M training data sets corresponding to the domains one by one: X({X m |m=1,…,M},

[0049] Among them, N m is the number of samples in the mth domain, represents the i-th training sample from the m-th domain; For sample The true label of , 1~C represents the known C types, and u represents the unknown type.

[0050] Step 2. Build a feature extraction network based on the DenseNet network to extract feature vectors for each training sample.

[0051] sample The extracted feature vector is denoted as Where m = 1, ..., M, i = 1, ..., N m , ω is the network parameter of DenseNet.

[0052] Step 3. Cluster all extracted feature vectors to obtain multiple clusters and corresponding cluster center vectors; assign pseudo labels to each cluster.

[0053] Preferably, a semi-supervised density clustering algorithm is used for all Perform clustering.

[0054] Assume that a total of α clusters S are obtained k , k = 1,…,α. Let cluster S k The number of samples included is s k , cluster cluster S k The pseudo label is recorded as At the same time, cluster S k Samples in The pseudo label is recorded as

[0055] For cluster S k , its cluster center vector q k for:

[0056]

[0057] in, is a bool function. If but otherwise

[0058] Step 4. Obtain the optimal network parameters in the feature extraction network.

[0059] First, construct the joint optimization function J(ω):

[0060] J(ω)=L SA +L SS +L SC ;

[0061]

[0062]

[0063]

[0064]

[0065] Where Υ is a bool function. but otherwise q + For the current sample The cluster center vector of the cluster; τ is the preset hyperparameter; q k is the cluster center vector of the kth cluster.

[0066] Then the stochastic gradient descent method is used to solve the parameter ω that minimizes J(ω) * ,ω * is the optimal network parameter.

[0067] Step 5. Based on the true labels of the samples in the clusters and the pseudo labels of the clusters, a mapping relationship between the pseudo label space and the true label space is constructed to obtain the true labels of each cluster.

[0068] The specific steps are:

[0069] Step 5.1. Let h = 1 and start traversing all clusters.

[0070] Step 5.2. Count the current cluster S k The number of samples of each known type label in the cluster S k The number of samples with the true label c is recorded as

[0071] Step 5.3. Traverse the current cluster S k All true labels in , if for a true label c, the following two conditions are met at the same time:

[0072]

[0073]

[0074] Then cluster S k The true label is recorded as Otherwise, cluster S k The true label is recorded as Then execute h=h+1;

[0075] The value range of λ is [0.5, 1], and the value range of μ is [0.5, 1].

[0076] Step 5.4. Return to step 5.2 and continue to traverse the clusters until the true labels of all clusters are obtained.

[0077] Step 6. Given a test sample, use the optimal network parameters obtained in step 4 to calculate the feature vector of the test sample, and then find the closest cluster based on its feature vector, and use the type represented by the true label of the cluster as the type recognition result of the test sample.

[0078] Specifically, for the test sample x, first calculate its feature vector f(x, ω * ), and then calculate the Euclidean distance between the test sample x and the cluster center vector of each cluster. For cluster S k and its cluster center vector q k , the Euclidean distance is: ω * is the optimal network parameter obtained in step 4. If d k′ =min(d k |k=1,…,α), then the test sample x∈S k′ , the true label corresponding to x

Claims

1. A method for identifying unknown ship target types in remote sensing images in a cross-domain scenario. Features The following steps are involved: Step 1. Given ship training data from different domains, the training data includes both training samples of known types and training samples of unknown types; Examples of known types are labeled with their types using true labels; In step 1, assume that the training data comes from M different domains, and represent all the training data as M training data sets corresponding to the domains one by one: X = {X m |m=1,…,M}, Among them, N m is the number of samples in the mth domain, represents the i-th training sample from the m-th domain; For sample The true label, 1~C represents the known C types, and u represents the unknown type; Step 2. Construct a feature extraction network based on the DenseNet network to extract feature vectors for each training sample; In step 2, the feature extraction network uses a DenseNet network, and the sample The eigenvector of Where m = 1, ..., M, i = 1, ..., N m ,ω is the network parameter of DenseNet; Step 3. Cluster all extracted feature vectors to obtain multiple clusters and corresponding cluster center vectors; assign pseudo labels to each cluster; In step 3, assume that a total of α clusters S are obtained k , k = 1,…,α; let cluster S k The number of samples included is s k , cluster cluster S k The pseudo label is recorded as The cluster S k Samples in The pseudo label is recorded as Step 4. Obtain the optimal network parameters in the feature extraction network; In step 4, a joint optimization function J(ω) is constructed and the stochastic gradient descent method is used for optimization to obtain the optimal network parameter ω * : J(ω)=L SA +L SS +L SC ; Among them, γ is a bool function, if but otherwise q + For the current sample The cluster center vector of the cluster; τ is the preset hyperparameter; q k is the cluster center vector of the kth cluster; Then the stochastic gradient descent method is used to solve the parameter ω that minimizes J(ω) * ,ω * That is the optimal network parameter; Step 5. Based on the true labels of the samples in the clusters and the pseudo labels of the clusters, a mapping relationship between the pseudo label space and the true label space is constructed to obtain the true labels of each cluster; Step 6. Given a test sample, use the optimal network parameters obtained in step 4 to calculate the feature vector of the test sample, and then find the closest cluster based on its feature vector, and use the type represented by the true label of the cluster as the type recognition result of the test sample.

2. The method for identifying unknown ship target types in remote sensing images in a cross-domain scenario according to claim 1, Features: In step 3, a semi-supervised density clustering algorithm is used to cluster all Perform clustering.

3. The method for identifying unknown ship target types in remote sensing images in a cross-domain scenario according to claim 1, Features: For cluster S k , its cluster center vector q k for: in, is a bool function. If but otherwise 4. The method for identifying unknown ship target types in remote sensing images in a cross-domain scenario according to claim 1, Features The specific steps of step 5 are: Step 5.

1. Set h = 1 and start traversing all clusters; Step 5.

2. Count the current cluster S k The number of samples of each known type label in the cluster S k The number of samples with the true label c is recorded as Step 5.

3. Traverse the current cluster S k All true labels in , if for a true label c, the following two conditions are met at the same time: Then cluster S k The true label is recorded as Otherwise, cluster S k The true label is recorded as Then execute h=h+1; The value range of λ is [0.5, 1], and the value range of μ is [0.5, 1]; Step 5.

4. Return to step 5.2 and continue to traverse the clusters until the true labels of all clusters are obtained.

5. The method for identifying unknown ship target types in remote sensing images in a cross-domain scenario according to any one of claims 2 to 4, Features: In step 6, for the test sample x, the closest cluster is found by calculating the Euclidean distance between the test sample x and the cluster center vector of each cluster: k and its cluster center vector q k , the Euclidean distance is: ω * is the optimal network parameter obtained in step 4; if d k′ =min(d k |k=1,…,α), then the test sample x∈S k′ , the true label corresponding to x

Citation Information

Patent Citations

  • Inland ship re-identification method and device based on transfer learning, and storage medium

    CN111259812A

  • Remote sensing image ship target integrated detection and fine-grained identification method

    CN111401302A

  • Semi-supervised image classification method and system for image embedded with LBP features

    CN112801153A

  • Online label updating method, system and device and storage medium

    CN114692732A

  • Learning method and learning device for adjusting parameters of CNN in which residual networks are provided for meta learning, and testing method and testing device using the same

    US10496899B1