An unsupervised ship re-identification method and system based on multi-level contrastive learning

Abstract

The present invention belongs to the technical field of ship re-identification, and in particular, to an unsupervised ship re-identification method and system based on multi-level contrastive learning. The method comprises: obtaining a training ship image, generating a pseudo-label of the training ship image, and enhancing the training ship image to obtain an enhanced image; extracting enhanced image features of the enhanced image using a feature extraction network, and training the feature extraction network based on the enhanced image features and the pseudo-label; obtaining a query ship image, extracting query ship image features using a trained feature extraction network, and retrieving images belonging to the same ship as the query ship image in a candidate ship image set based on the query ship image features. The present invention combines instance-level contrastive learning and cluster-level contrastive learning to learn identifiable ship features in an unsupervised manner without any memory blocks, thereby improving the accuracy of ship re-identification tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of ship re-identification, and in particular relates to an unsupervised ship re-identification method and system based on multi-level contrastive learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] As the most representative unmanned equipment at sea, surface unmanned boats play an important role in offshore monitoring and reconnaissance. Therefore, the re-identification of different surface ships is a basic capability of surface unmanned boats. However, due to the lack of large-scale annotated data sets, there are relatively few studies on ship re-identification. Compared with pedestrian re-identification data that can be captured from surveillance videos, the collection of ship re-identification data often requires more manpower and material resources, and even has some safety hazards. In this case, using an unsupervised method to train a ship re-identification model is a more appropriate solution.

[0004] Contrastive unsupervised learning methods have been widely studied in the task of pedestrian re-identification. The usual solution is to use clustering algorithms to generate pseudo-labels to reflect the association between unlabeled samples, and then perform contrastive learning based on the pseudo-labels. From the perspective of loss function, the MMCL method proposed by Zhang Shiliang's team at Peking University also belongs to a method for pedestrian re-identification based on contrastive learning. In the early stage of training, this method uses the index of each sample as a category label for conventional instance-level contrastive learning, which makes it difficult for ship images with the same cluster pseudo-label to participate in training as positive samples during the training process. This is called the difficult positive sample problem in the present invention. If this problem is not solved, the feature space of the data set will gradually approach a uniform distribution, causing the trained model to tend to distinguish each image rather than classify according to the ship ID. The SpCL method proposed by the Multimedia Laboratory of the Chinese University of Hong Kong achieves good unsupervised pedestrian re-identification performance with the help of a hybrid memory block, but due to the low accuracy of pseudo-labels generated by clustering in the early stage of training, a large number of samples cannot be clustered into any cluster due to low clustering confidence, and become free sample points each into one category, resulting in the method also facing the difficult positive sample problem in the early stage of training. Due to the existence of the difficult positive sample problem, existing technical methods are difficult to produce good application results in the more challenging ship re-identification tasks.

[0005] Cluster-level contrastive learning compares the features of each sample with the features of each cluster during training. The features of each cluster are usually obtained by averaging the features of all samples in the cluster. Therefore, the cluster features are a concentrated reflection of all features in the cluster, thereby avoiding positive samples with the same pseudo-label from being pushed away during training. The ICE method recently published in ICCV2021 uses both cluster-level contrastive learning and instance-level difficult sample mining to achieve the best unsupervised person re-identification performance. However, the ICE method only performs instance-level contrastive learning on the most difficult positive sample, while ignoring other positive samples, weakening the impact of instance-level contrastive learning on model training. In fact, for the task of ship re-identification, due to the higher similarity between ships with different IDs, more detailed information is often required to distinguish them. Therefore, the role of instance-level contrastive learning in the task of unsupervised ship re-identification should be fully utilized.

[0006] In addition, most existing comparative re-identification methods rely on instance-level or cluster-level memory blocks, that is, the features of all images in the training set or the features of each cluster after clustering are stored in the memory block in advance, and the current sample features are continuously optimized by comparing them with the features stored in the memory block during the training process. The global memory block for the entire training set often requires a large amount of memory consumption and has high hardware requirements for computing devices, which limits the application of the algorithm on large-scale ship re-identification datasets. Summary of the invention

[0007] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides an unsupervised ship re-identification method and system based on multi-level contrastive learning, which combines instance-level contrastive learning (ICL) and cluster-level contrastive learning (CCL) to learn identifiable ship features in an unsupervised manner without any memory blocks, thereby improving the accuracy of the ship re-identification task.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] A first aspect of the present invention provides an unsupervised ship re-identification method based on multi-level contrastive learning.

[0010] An unsupervised ship re-identification method based on multi-level contrastive learning, comprising:

[0011] Acquire a training ship image, generate a pseudo label of the training ship image, and enhance the training ship image to obtain an enhanced image;

[0012] A feature extraction network is used to extract enhanced image features of the enhanced image, and the feature extraction network is trained based on the enhanced image features and pseudo labels;

[0013] The process of training the feature extraction network includes: calculating the similarity between all enhanced image features based on the enhanced image features; calculating the instance-level contrast loss based on the similarity between the enhanced image features and the principle that samples with the same pseudo-label are positive samples of each other; calculating the cluster features based on the enhanced image features and the pseudo-labels and determining the similarity between the enhanced image feature samples and the cluster features, thereby obtaining the cluster-level contrast loss; obtaining the multi-level contrast learning loss based on the instance-level contrast loss and the cluster-level contrast loss, so as to iteratively optimize the feature extraction network until the iteration stop condition is met, thereby obtaining a trained feature extraction network;

[0014] A query ship image is obtained, features of the query ship image are extracted using a trained feature extraction network, and based on the features of the query ship image, images belonging to the same ship as the query ship image are retrieved from a candidate ship image set.

[0015] Furthermore, the feature extraction network adopts an improved ResNet-50 network, the improved ResNet-50 network removes the classifier, and adds a batch normalization operation layer at the end of the network.

[0016] Furthermore, the enhancing of the ship image includes: performing an enhancement operation on the ship image, generating two different enhanced images for each ship image, and the enhanced images of the same ship image constitute a positive sample pair and share a pseudo label.

[0017] Furthermore, the process of calculating the instance-level contrast loss is implemented using an instance-level contrast loss function. for:

[0018]

[0019] in, τ1 is the hyperparameter of instance-level contrastive learning, i and j represent the index of samples in each batch, and y i and j Represent the pseudo labels of the i-th sample and the j-th sample, s i,j represents the similarity of the features of two samples; N is the batch size, that is, in each training iteration, N images are sampled from the training set, and each image is enhanced twice, resulting in a total of 2N enhanced images, that is, 2N samples. When calculating the instance-level loss, all 2N samples are involved in the calculation. i，j When k represents the index of the kth sample among 2N samples, y k represents its pseudo label.

[0020] Furthermore, the specific process of determining the similarity between the enhanced image feature samples and the cluster features based on the enhanced image features and the pseudo-labels to obtain the cluster-level contrast loss includes: based on the enhanced image features and the pseudo-labels, calculating the mean of several enhanced image feature samples with the same pseudo-labels within each batch to obtain the cluster features; calculating the similarity between the enhanced image feature samples and several cluster features; determining the cluster to which each enhanced image feature sample belongs according to its pseudo-label, in the cluster contrast learning process, the cluster feature of the cluster to which the enhanced image feature sample belongs is a positive sample, and the remaining cluster features are negative samples, and calculating the cluster-level contrast loss.

[0021] Furthermore, the cluster-level contrast loss is calculated using a cluster-level contrast loss function, and the cluster-level contrast loss function is:

[0022]

[0023] Among them, τ2 is the hyperparameter of cluster-level contrastive learning. For the i-th sample, the cluster it belongs to is the c-th cluster (there are C clusters in each batch of samples during training), s i,c Refers to the similarity between the sample and the cluster features of the cluster to which it belongs, s i,k Refers to the similarity between it and the cluster features of the kth cluster in the batch of samples.

[0024] Furthermore, the multi-level contrastive learning loss is obtained according to a multi-level contrastive learning loss function, and the multi-level contrastive learning loss function is:

[0025]

[0026] Among them, λ is the weight coefficient, is the instance-level contrast loss function, is the cluster-level contrast loss function, N is the batch size, that is, in each training iteration, N pictures are sampled from the training set, and each picture is subjected to two image enhancement operations, resulting in a total of 2N enhanced images, i.e., 2N samples. When calculating the instance-level loss, all 2N samples are involved in the calculation, and i represents the index of the i-th sample; when calculating the instance-level contrast loss, only N samples are used (i.e., one enhanced image is taken for each original picture), and j represents the index of the j-th sample.

[0027] A second aspect of the present invention provides an unsupervised ship re-identification system based on multi-level contrastive learning.

[0028] An unsupervised ship re-identification system based on multi-level contrastive learning, comprising:

[0029] A preprocessing module is configured to: acquire a training ship image, generate a pseudo label of the training ship image, and enhance the training ship image to obtain an enhanced image;

[0030] A feature extraction module is configured to: extract enhanced image features of the enhanced image using a feature extraction network, and train the feature extraction network based on the enhanced image features and the pseudo labels;

[0031] The network training module is configured as follows: the process of training the feature extraction network includes: based on the enhanced image features, calculating the similarity between all enhanced image features; based on the similarity between the enhanced image features, according to the principle that samples with the same pseudo-label are mutually positive samples, calculating the instance-level contrast loss; based on the enhanced image features and the pseudo-label, calculating the cluster features and determining the similarity between the enhanced image feature samples and the cluster features, thereby obtaining the cluster-level contrast loss; based on the instance-level contrast loss and the cluster-level contrast loss, obtaining the multi-level contrast learning loss, so as to iteratively optimize the feature extraction network until the iteration stop condition is met, thereby obtaining the trained feature extraction network;

[0032] The re-identification module is configured to: obtain a query ship image, extract features of the query ship image using a trained feature extraction network, and retrieve images in a candidate ship image set that belong to the same ship as the query ship image based on the query ship image features.

[0033] A third aspect of the present invention provides a computer-readable storage medium.

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the unsupervised ship re-identification method based on multi-level contrastive learning as described in the first aspect above.

[0035] A fourth aspect of the present invention provides a computer device.

[0036] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the unsupervised ship re-identification method based on multi-level contrastive learning as described in the first aspect above are implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) The present invention designs a new instance-level contrastive learning loss function. Compared with the most commonly used InfoNCE contrastive loss function, the instance-level contrastive loss function in the present invention regards samples with the same pseudo-label as positive samples. When calculating the loss function of a certain positive sample pair, other positive samples are removed from the denominator of the loss function (i.e., the negative sample term) to avoid these positive samples being pushed away as negative samples, thus solving the problem of difficult positive samples and improving training efficiency and model performance.

[0039] (2) The present invention establishes a multi-level contrastive learning framework that does not require feature storage blocks. The instance-level contrastive learning and cluster-level contrastive loss function in the present invention work together. The former focuses on capturing the detailed information of the image, and the latter can improve the model's anti-interference ability to noise samples. At the same time, the cluster-level loss function can also solve the problem of difficult positive samples to a certain extent (because the cluster-level contrastive loss function uses the mean of all samples with the same pseudo-label in the current batch as cluster features, and the cluster features of the cluster to which it belongs are used as positive samples during the training process, and the cluster features of other clusters are used as negative samples). Most importantly, the present invention does not need to pre-calculate the features of all samples in the entire training set and store them in the memory block, which greatly saves memory space and makes it possible to train a ship re-identification model on a large-scale data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0041] Figure 1 is a flow chart of an unsupervised ship re-identification method based on multi-level contrastive learning shown in Embodiment 1 of the present invention;

[0042] Figure 2 4 is a framework diagram of an unsupervised ship re-identification system based on multi-level contrastive learning shown in the second embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0044] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0046] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0047] Terminology explanation: Ship re-identification means that given a query image (query) of a certain ship ID, other images that belong to the same ship ID as the query image are retrieved from a candidate ship image set (gallery) containing many images.

[0048] Embodiment 1

[0049] like Figure 1As shown, this embodiment provides an unsupervised ship re-identification method based on multi-level contrastive learning. This embodiment uses the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a terminal, a server, and a system, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application is not limited here. In this embodiment, the method includes the following steps:

[0050] Acquire a training ship image, generate a pseudo label of the training ship image, and enhance the training ship image to obtain an enhanced image;

[0051] A feature extraction network is used to extract enhanced image features of the enhanced image, and the feature extraction network is trained based on the enhanced image features and pseudo labels;

[0052] The process of training the feature extraction network includes: calculating the similarity between all enhanced image features based on the enhanced image features; calculating the instance-level contrast loss based on the similarity between the enhanced image features and the principle that samples with the same pseudo-label are positive samples of each other; calculating the cluster features based on the enhanced image features and the pseudo-labels and determining the similarity between the enhanced image feature samples and the cluster features, thereby obtaining the cluster-level contrast loss; obtaining the multi-level contrast learning loss based on the instance-level contrast loss and the cluster-level contrast loss, so as to iteratively optimize the feature extraction network until the iteration stop condition is met, thereby obtaining a trained feature extraction network;

[0053] A query ship image is obtained, features of the query ship image are extracted using a trained feature extraction network, and based on the features of the query ship image, images belonging to the same ship as the query ship image are retrieved from a candidate ship image set.

[0054] Given an unlabeled training set χ, the number of training epochs E, the number of iterations per epoch I, the batch size N for each iteration, and the feature extraction network f θ , pseudo label generator G, image enhancement module T, the specific solution of this embodiment can be implemented by referring to the following contents:

[0055] (a) For the original image x∈χ, generate a pseudo label y=G(f θ (x));

[0056] As a feasible implementation, the pseudo-label generator G in this embodiment is used to generate a pseudo-label y for each ship image x in the training set x, that is, The pseudo-label generator G of this embodiment adopts the DBSCAN clustering algorithm based on Jaccard distance, which is implemented with the help of the scikit-learn library. In the clustering process, each cluster is set to contain at least 4 samples, the maximum distance ∈ between positive samples is set to 0.4, and k1 and k2 are set to 30 and 6 respectively when calculating the Jaccard distance.

[0057] (b) Randomly sample N samples from χ, and the N samples belong to C pseudo labels respectively;

[0058] (c) Perform two image enhancement operations on N samples to obtain 2N enhanced images:

[0059]

[0060] As a feasible implementation, the image enhancement module T in this embodiment is used to generate two different enhanced images x for each ship image x in the training set x. i and x j , the enhanced images from the same ship image constitute a positive sample pair and share pseudo labels. In the training phase, the image enhancement module T in this embodiment mainly includes operations such as Resize, RandomHorizontalFlip, RandomCrop, ColorJitter, and RandomErasing; in the testing phase, only the input image is resized. The RandomErasing operation in this embodiment is implemented using the existing method, and the remaining image enhancement operations are implemented based on PyTorch. The specific parameter settings are as follows:

[0061] Resize: height=180, width=256, interpolation=3;

[0062] RandomHorizontalFlip:p=0.5;

[0063] RandomCrop: height=180, width=256;

[0064] ColorJitter: brightness=0.2, contrast=0.2, saturation=0.2, hue=0.2;

[0065] RandomErasing: probability=0.5, mean=(0.4914,0.4822,0.4465).

[0066] (d) Using feature extraction network f on 2N enhanced images θ Extract features:

[0067]

[0068] As a feasible implementation method, the feature extraction network f in this embodiment θ Used to extract the features of the input ship image x, i.e. z = f θ (x) This part uses a slightly modified ResNet-50 network, deletes the classifier in the original network (that is, the last fully connected layer), and adds a batch normalization (BN) operation at the end of the network. Before training begins, the model parameters pre-trained on the ImageNet dataset are used to adjust f θ Initialize.

[0069] (e) Calculate the similarity between samples based on formula (1) for the 2N enhanced image features;

[0070] As a feasible implementation method, the instance-level contrastive learning module ICL in this embodiment takes all enhanced image features as input and first calculates the L2 similarity between features:

[0071]

[0072] (f) Calculate instance-level contrast loss based on formula (2) and formula (3)

[0073] As a feasible implementation method, the comparison loss function is calculated for the current batch (the batch size is denoted as N) according to the principle that samples with the same pseudo-label are positive samples. The loss function is:

[0074]

[0075] in

[0076]

[0077] in

[0078] Wherein, τ1 is a hyperparameter of instance-level contrastive learning, which is set to 0.05 in the implementation process of this embodiment, i and j represent the index of samples in each batch, y i and jRepresent the pseudo labels of the i-th sample and the j-th sample, s i,j represents the similarity of the features of two samples; N is the batch size, that is, in each training iteration, N images are sampled from the training set, and each image is enhanced twice, resulting in a total of 2N enhanced images, that is, 2N samples. When calculating the instance-level loss, all 2N samples are involved in the calculation. i，j When k represents the index of the kth sample among 2N samples, y k represents its pseudo label.

[0079] As shown in formula (2), the ICL module in this embodiment takes samples with the same pseudo-label as positive samples, unlike the conventional contrastive learning method which only takes enhanced images from the same original ship image as positive samples; in addition, when calculating the positive sample pair (x i ,x j ), the ICL module in this embodiment removes other positive samples from the denominator of formula (3) to avoid other positive samples from being mixed with the current sample x i The distance between them is enlarged, resulting in reduced training efficiency, which is the main technical point of this embodiment to solve the problem of difficult positive samples. At the same time, the ICL module of this embodiment does not need a memory block to pre-store the features of all samples in the training set, and only performs instance-level comparative learning in each batch.

[0080] (g) Calculate the cluster features of C clusters in the current batch based on formula (4);

[0081] As a feasible implementation method, the cluster-level contrastive learning module CCL in this embodiment takes an enhanced image feature corresponding to each original image as input, firstly, according to the pseudo-label, the cluster-level contrastive learning module CCL in the batch has the same pseudo-label y c (i.e., belonging to the cth cluster) c Take the average of sample features to get cluster features:

[0082]

[0083] (h) Calculate the similarity between sample features and cluster features based on formula (5);

[0084] As a feasible implementation, calculate the sample x i With each cluster feature φ k The similarity between:

[0085]

[0086] (i) Calculate the cluster-level contrast loss based on formula (6)

[0087] As a feasible implementation, if xi The pseudo label is y c , then in the cluster-level contrastive learning process φ c is a positive sample, and other cluster features are negative samples, and then the cluster-level contrast loss is calculated:

[0088]

[0089] Among them, τ2 is a hyperparameter of cluster-level contrastive learning, which is set to 0.05 in the implementation process of this embodiment. For the i-th sample, the cluster it belongs to is the c-th cluster (there are C clusters in each batch of samples during training), s i,c Refers to the similarity between the sample and the cluster features of the cluster to which it belongs, s i,k Refers to the similarity between it and the cluster features of the kth cluster in the batch of samples.

[0090] The cluster feature in the cluster-level contrastive learning module CCL in this embodiment is the mean of all sample features belonging to the same cluster in the current batch, taking all positive samples in the current batch into account, avoiding the situation where positive samples with the same pseudo-label are regarded as negative samples and pushed away, thus also solving the problem of difficult positive samples. At the same time, the CCL module in this embodiment only considers the samples in the current batch when calculating the cluster features, so there is no need to pre-calculate the features of all samples in the entire training set and store them in the memory block.

[0091] (j) Calculate the multi-level contrastive learning loss:

[0092]

[0093] Among them, λ is the weight coefficient; N is the batch size, that is, each training iteration, N pictures are sampled from the training set, and each picture is subjected to two image enhancement operations, resulting in a total of 2N enhanced images, that is, 2N samples. When calculating the instance-level loss, all 2N samples are involved in the calculation, and i represents the index of the i-th sample; when calculating the instance-level contrast loss, only N samples are used (that is, one enhanced image is taken for each original picture), and j represents the index of the j-th sample.

[0094] (k) Update the feature extraction network f θ To minimize the multi-class contrastive learning loss L MCL ;

[0095] (l) Repeat (b) to (k) once;

[0096] (m) Repeat (a) to (l) E times to obtain the trained feature extraction network f θ .

[0097] In this embodiment, the trained feature extraction network f θBased on the characterization ability of ship images in the ship re-identification test data, based on f θ The extracted features are used to measure the similarity between the query ship image (query) and the ship images in the alternative propagation image set (gallery), and then the images in the gallery that belong to the same ship ID as the query are retrieved. The advantages and disadvantages of the technical solution are reflected in the retrieval accuracy, including the mean average precision mAP and the top 1 / 5 / 10 accuracy based on the cumulative matching curve (Cumulative Match Characteristic, CMC).

[0098] The effect evaluation of this embodiment is carried out on a self-built ship re-identification dataset, which includes 30,587 pictures of 1,248 ships, of which 624 ship IDs are used for training and the remaining 624 ship IDs are used for testing.

[0099] In order to verify the effect of the technical solution of this embodiment, the specific implementation details are set as follows:

[0100] The number of epochs E in the training phase is set to 50; the number of iterations I of each epoch is set to 200; the batch size N of each iteration is set to 128 / 256, and the number of samples in each cluster is kept at 4 during sampling, that is, the number of clusters C in each batch is 32 / 64; the weight coefficient λ of the multi-level contrastive learning loss function is set to 1; the feature extraction network f is updated θ The Adam optimization algorithm is used, and the basic learning rate is 0.00035. With every 20 epochs of training, the learning rate drops to 1 / 10 of the current value.

[0101] The comparison with other re-identification methods is as follows:

[0102] Table 1 - Comparison of the results without re-ranking in the test phase

[0103]

[0104] Table 2 - Comparison of the effects when re-ranking is performed during the test phase

[0105]

[0106] It can be seen from Table 1 and Table 2 that the method of this embodiment has higher accuracy in ship re-identification and is superior to the existing methods.

[0107] Embodiment 2

[0108] This embodiment provides an unsupervised ship re-identification system based on multi-level contrastive learning.

[0109] An unsupervised ship re-identification system based on multi-level contrastive learning, comprising:

[0110] A preprocessing module is configured to: acquire a training ship image, generate a pseudo label of the training ship image, and enhance the training ship image to obtain an enhanced image;

[0111] A feature extraction module is configured to: extract enhanced image features of the enhanced image using a feature extraction network, and train the feature extraction network based on the enhanced image features and the pseudo labels;

[0112] The network training module is configured as follows: the process of training the feature extraction network includes: based on the enhanced image features, calculating the similarity between all enhanced image features; based on the similarity between the enhanced image features, according to the principle that samples with the same pseudo-label are mutually positive samples, calculating the instance-level contrast loss; based on the enhanced image features and the pseudo-label, calculating the cluster features and determining the similarity between the enhanced image feature samples and the cluster features, thereby obtaining the cluster-level contrast loss; based on the instance-level contrast loss and the cluster-level contrast loss, obtaining the multi-level contrast learning loss, so as to iteratively optimize the feature extraction network until the iteration stop condition is met, thereby obtaining a trained feature extraction network.

[0113] The re-identification module is configured to: obtain a query ship image, extract features of the query ship image using a trained feature extraction network, and retrieve images in a candidate ship image set that belong to the same ship as the query ship image based on the query ship image features.

[0114] The preprocessing module includes: a pseudo-label generator G and an image enhancement module T, and the network training module includes an instance-level contrastive learning module ICL and a cluster-level contrastive learning module CCL. Figure 2 shown.

[0115] It should be noted that the above-mentioned preprocessing module, feature extraction module, network training module and re-identification module are the same as the examples and application scenarios implemented by the steps in Embodiment 1, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the above-mentioned modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0116] Embodiment 3

[0117] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the unsupervised ship re-identification method based on multi-level contrastive learning as described in the first embodiment are implemented.

[0118] Embodiment 4

[0119] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the unsupervised ship re-identification method based on multi-level contrastive learning as described in the first embodiment are implemented.

[0120] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0121] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0124] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An unsupervised ship re-identification method based on multi-level contrastive learning, characterized in that: include: Acquire a training ship image, generate a pseudo label of the training ship image, and enhance the training ship image to obtain an enhanced image; A feature extraction network is used to extract enhanced image features of the enhanced image, and the feature extraction network is trained based on the enhanced image features and pseudo labels; The process of training the feature extraction network includes: calculating the similarity between all enhanced image features based on the enhanced image features; calculating the instance-level contrast loss based on the similarity between the enhanced image features and the principle that samples with the same pseudo-label are positive samples of each other; calculating the cluster features based on the enhanced image features and the pseudo-labels and determining the similarity between the enhanced image feature samples and the cluster features, thereby obtaining the cluster-level contrast loss; obtaining the multi-level contrast learning loss based on the instance-level contrast loss and the cluster-level contrast loss, so as to iteratively optimize the feature extraction network until the iteration stop condition is met, thereby obtaining a trained feature extraction network; Acquire a query ship image, extract features of the query ship image using a trained feature extraction network, and retrieve images in a candidate ship image collection that belong to the same ship as the query ship image based on the features of the query ship image; The process of calculating the instance-level contrast loss is implemented using an instance-level contrast loss function. for: in, τ1 is the hyperparameter of instance-level contrastive learning, i and j represent the index of samples in each batch, and y i and j Represent the pseudo labels of the i-th sample and the j-th sample, s i,j represents the similarity of the features of two samples; N is the batch size, that is, in each training iteration, N images are sampled from the training set, and each image is enhanced twice, resulting in a total of 2N enhanced images, that is, 2N samples. When calculating the instance-level loss, all 2N samples are involved in the calculation. i,j When k represents the index of the kth sample among 2N samples, y k represents its pseudo label; The cluster-level contrast loss is calculated using a cluster-level contrast loss function, which is: Among them, τ2 is the hyperparameter of cluster-level contrastive learning. For the i-th sample, the cluster it belongs to is the c-th cluster, and s i,c Refers to the similarity between the sample and the cluster features of the cluster to which it belongs, s i,k Refers to the similarity between it and the cluster features of the kth cluster in the batch of samples.

2. The unsupervised ship re-identification method based on multi-level contrastive learning according to claim 1 is characterized in that: The feature extraction network adopts an improved ResNet-50 network, the improved ResNet-50 network removes the classifier, and adds a batch normalization operation layer at the end of the network.

3. The unsupervised ship re-identification method based on multi-level contrastive learning according to claim 1 is characterized in that: The enhancing of the ship image comprises: performing an enhancing operation on the ship image, generating two different enhanced images for each ship image, and the enhanced images of the same ship image constitute a positive sample pair and share a pseudo label.

4. The unsupervised ship re-identification method based on multi-level contrastive learning according to claim 1 is characterized in that: The specific process of determining the similarity between the enhanced image feature samples and the cluster features based on the enhanced image features and the pseudo-labels to obtain the cluster-level contrast loss includes: calculating the mean of a number of enhanced image feature samples with the same pseudo-label within each batch to obtain the cluster features; calculating the similarity between the enhanced image feature samples and the number of cluster features; determining the cluster to which each enhanced image feature sample belongs according to its pseudo-label, in the cluster contrast learning process, the cluster feature to which the enhanced image feature sample belongs is a positive sample, and the other cluster features are negative samples, and calculating the cluster-level contrast loss.

5. The unsupervised ship re-identification method based on multi-level contrastive learning according to claim 1 is characterized in that: The multi-level contrastive learning loss is obtained according to a multi-level contrastive learning loss function, and the multi-level contrastive learning loss function is: Among them, λ is the weight coefficient, is the instance-level contrast loss function, is the cluster-level contrast loss function, and N is the batch size.

6. An unsupervised ship re-identification system based on multi-level contrastive learning, characterized in that: include: A preprocessing module is configured to: acquire a training ship image, generate a pseudo label of the training ship image, and enhance the training ship image to obtain an enhanced image; A feature extraction module is configured to: extract enhanced image features of the enhanced image using a feature extraction network, and train the feature extraction network based on the enhanced image features and the pseudo labels; The network training module is configured as follows: the process of training the feature extraction network includes: based on the enhanced image features, calculating the similarity between all enhanced image features; based on the similarity between the enhanced image features, according to the principle that samples with the same pseudo-label are mutually positive samples, calculating the instance-level contrast loss; based on the enhanced image features and the pseudo-label, calculating the cluster features and determining the similarity between the enhanced image feature samples and the cluster features, thereby obtaining the cluster-level contrast loss; based on the instance-level contrast loss and the cluster-level contrast loss, obtaining the multi-level contrast learning loss, so as to iteratively optimize the feature extraction network until the iteration stop condition is met, thereby obtaining the trained feature extraction network; The re-identification module is configured to: obtain a query ship image, extract features of the query ship image using a trained feature extraction network, and retrieve images in a candidate ship image set that belong to the same ship as the query ship image based on the query ship image features; The process of calculating the instance-level contrast loss is implemented using an instance-level contrast loss function. for: in, τ1 is the hyperparameter of instance-level contrastive learning, i and j represent the index of samples in each batch, and y i and j Represent the pseudo labels of the i-th sample and the j-th sample, s i,j represents the similarity of the features of two samples; N is the batch size, that is, in each training iteration, N images are sampled from the training set, and each image is enhanced twice, resulting in a total of 2N enhanced images, that is, 2N samples. When calculating the instance-level loss, all 2N samples are involved in the calculation. i,j When k represents the index of the kth sample among 2N samples, y k represents its pseudo label; The cluster-level contrast loss is calculated using a cluster-level contrast loss function, which is: Among them, τ2 is the hyperparameter of cluster-level contrastive learning. For the i-th sample, the cluster it belongs to is the c-th cluster, and s i,c Refers to the similarity between the sample and the cluster features of the cluster to which it belongs, s i,k Refers to the similarity between it and the cluster features of the kth cluster in the batch of samples.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the unsupervised ship re-identification method based on multi-level contrastive learning as described in any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the unsupervised ship re-identification method based on multi-level contrastive learning as described in any one of claims 1 to 5 are implemented.

Images