Training method of sky-based ship retrieval model and ship retrieval method

Abstract

The application provides a sky-based ship retrieval model training method and a ship retrieval method, and relates to the technical field of artificial intelligence, which comprises the following steps: obtaining a training data set, wherein the training data set contains ship training images from different imaging platforms and different imaging modalities; extracting identity factor features, platform factor features and modal factor features from the ship training images to determine a classification loss and an orthogonal regularization loss; dynamically combining the identity factor features, the platform factor features and the modal factor features of each sample in a sample pair of any triplet to form a combined feature representation, and determining a triplet loss based on the combined feature representation; and updating the model parameters of an initial ship retrieval model based on the classification loss, the orthogonal regularization loss and the triplet loss to obtain a ship retrieval model. The application can improve the cross-platform and cross-modal ship retrieval accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a training method for a space-based ship retrieval model and a ship retrieval method. Background Technology

[0002] Marine surveillance is the cornerstone of national security, traffic management, and economic stability. To achieve all-weather, wide-area situational awareness, modern surveillance systems are gradually shifting from single observation sources to collaborative networks utilizing heterogeneous remote sensing platforms. This collaborative model leverages the complementary advantages of different acquisition mechanisms: space-based platforms provide macroscopic coverage, while airborne platforms offer high-resolution, flexible, and targeted observations. Furthermore, the combination of optical and synthetic aperture radar (SAR) imagery improves interpretation accuracy while enhancing robustness under varying lighting conditions and adverse weather conditions.

[0003] Due to differences in imaging mechanisms across different modalities and geometric variations in image acquisition across different platforms, features extracted from space-based and airborne multi-platform multi-modal data often contain domain biases stemming from platform and modal differences. In the feature space, samples tend to cluster according to modality or spaceborne platform origin, rather than ship category, leading to low retrieval accuracy in existing cross-platform, cross-modal ship retrieval technologies. Summary of the Invention

[0004] This invention provides a training method for a space-based ship retrieval model and a ship retrieval method to address the shortcomings of low retrieval accuracy in existing technologies for cross-platform and cross-modal ship retrieval, thereby improving the accuracy of cross-platform and cross-modal ship retrieval.

[0005] This invention provides a training method for a space-based ship retrieval model, comprising: Obtain a training dataset containing ship training images from different imaging platforms and different imaging modalities, including space-based and air-based imaging platforms; For each ship training image in the training dataset, an initial ship retrieval model is used to extract identity factor features, platform factor features, and modality factor features from the ship training images, and a classification loss is determined based on the identity factor features, platform factor features, and modality factor features. Based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities, an orthogonal regularization loss is determined. The orthogonal regularization loss is used to constrain the similarity between identity factor features of the same ship identity, the similarity between platform factor features of the same platform, and the similarity between modal factor features of the same modality, and the identity factor features, platform factor features, and modal factor features are mutually orthogonal in the vector space. For any triplet in the training batch, a sample pair consisting of an anchor point and a positive ship training image and a sample pair consisting of an anchor point and a negative ship training image are dynamically combined according to the similarity and difference between the platform label and modality label of the two samples in the sample pair. This combination of identity factor features, platform factor features, and modality factor features of each sample in the sample pair is used to form a combined feature representation for calculating the similarity of the sample pair. The distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss. Based on the classification loss, the orthogonal regularization loss, and the triplet loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

[0006] According to the training method of a space-based ship retrieval model provided by the present invention, the step of extracting identity factor features, platform factor features, and modality factor features from the ship training image through an initial ship retrieval model includes: The ship training images are divided into multiple image blocks, and each image block is mapped to a feature vector to obtain an image token sequence; The sequence of identity token, platform token, modality token and image token is concatenated, and a position code is added to the concatenated sequence to obtain the input sequence. The position code is used to characterize the spatial position information of each token in the ship training image. The input sequence is input into the initial ship retrieval model to obtain the identity factor features, platform factor features and modality factor features output by the initial ship retrieval model.

[0007] According to the present invention, a training method for a space-based ship retrieval model is provided, wherein the ship training images have identity labels, platform labels, and modality labels; The determination of classification loss based on the identity factor features, the platform factor features, and the modality factor features includes: The identity factor features, platform factor features, and modality factor features are respectively input into the corresponding identity classifier, platform classifier, and modality classifier to obtain the identity prediction result, platform prediction result, and modality prediction result; Based on the identity prediction result and the identity label, the platform prediction result and the platform label, and the modality prediction result and the modality label, the cross-entropy loss is calculated and weighted summed to obtain the classification loss.

[0008] According to a training method for a space-based ship retrieval model provided by the present invention, the classification loss is obtained by calculating cross-entropy loss and performing weighted summation based on the identity prediction result and the identity label, the platform prediction result and the platform label, and the modality prediction result and the modality label, respectively. The method includes: The classification loss is determined according to formula (1): (1) in, This represents the classification loss. This represents the identity prediction result for the i-th ship training image. This represents the platform prediction result for the i-th ship training image. This represents the modal prediction result for the i-th ship training image. This represents the identity label of the i-th ship training image. This represents the platform label of the i-th ship training image. This represents the modal label of the i-th ship training image. This represents the identity factor cross-entropy loss function. This represents the platform factor cross-entropy loss function. Represents the modality factor cross-entropy loss function. This represents the weighting coefficient corresponding to the identity factor. This represents the weighting coefficient of the platform factor. This represents the weighting coefficient of the modal factor.

[0009] According to the training method of the space-based ship retrieval model provided by the present invention, the step of determining the orthogonal regularization loss based on the identity factor features, platform factor features, and modality factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities includes: Extract the identity factor features, platform factor features, and modal factor features corresponding to the ship training images belonging to the same ship identity and respectively from space-based optics, space-based synthetic aperture radar SAR, air-based optics, and air-based SAR imaging conditions, and form a feature vector set; Calculate the cosine similarity between every two feature vectors in the feature vector set to obtain a similarity matrix; Construct an orthogonal constraint target matrix with the same size as the similarity matrix, wherein the elements in the orthogonal constraint target matrix are assigned values ​​according to whether the factor types and the platforms or modes to which the corresponding two feature vectors are the same; The orthogonal regularization loss is determined based on the similarity matrix and the orthogonal constraint target matrix.

[0010] According to the training method of the space-based ship retrieval model provided by the present invention, the orthogonal constraint target matrix is ​​determined as follows: If both feature vectors are identity factor features, the corresponding element value is 1. If both feature vectors are platform factor features and come from the same imaging platform, the corresponding element value is 1. If both feature vectors are modality factor features and come from the same imaging modality, the corresponding element value is 1. In other cases, the corresponding element value is 0.

[0011] According to a training method for a space-based ship retrieval model provided by the present invention, the step of determining the orthogonal regularization loss based on the similarity matrix and the orthogonal constraint target matrix includes: The orthogonal regularization loss is determined according to formula (2): (2) in, This represents the orthogonal regularization loss. This represents a ship training image under space-based optical imaging conditions. This represents a ship training image under space-based SAR imaging conditions. This represents a ship training image under airborne optical imaging conditions. This represents a ship training image under airborne SAR imaging conditions. This represents the value of the element in the i-th row and j-th column of the similarity matrix. This represents the value of the element in the i-th row and j-th column of the orthogonal constraint target matrix.

[0012] According to the training method of the space-based ship retrieval model provided by the present invention, the method dynamically combines one or more of the identity factor features, platform factor features, and modality factor features of each sample in the sample pair based on the similarity and difference of the platform labels and modality labels of the two samples in the sample pair, to form a combined feature representation for calculating the similarity of the sample pair, including: The combined feature representation is determined according to formula (3): in, This represents the combined feature representation. This represents the identity factor characteristics of sample i. This represents the platform factor characteristic of sample i. This represents the modal factor characteristic of sample i. This represents the platform label for sample i. This represents the platform label for sample j. This represents the modal label of sample i. This represents the modal label of sample j.

[0013] According to the training method of the space-based ship retrieval model provided by the present invention, the step of calculating the distance constraint loss based on the combined feature representation to obtain the triplet loss includes: The triplet loss is determined according to formula (4): (4) in, This represents the loss of the triplet. Indicates the anchor point in a triple. This represents the positive ship training image in the triplet. This represents the negative ship training image in the triplet. Indicates the distance threshold. for operate.

[0014] According to the present invention, a training method for a space-based ship retrieval model includes updating the model parameters of the initial ship retrieval model based on the classification loss, the orthogonal regularization loss, and the triplet loss to obtain a ship retrieval model, comprising: The target loss is obtained by weighted summing of the classification loss, the orthogonal regularization loss, and the triplet loss. Based on the target loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

[0015] According to the training method of the space-based ship retrieval model provided by the present invention, the identity factor feature is used to characterize the physical attribute features of the ship, the platform factor feature is used to characterize the features introduced by the imaging platform, and the modality factor feature is used to characterize the features of the sensor imaging mode.

[0016] The present invention also provides a ship retrieval method, comprising: Acquire target images of the target ship; The target image is input into the ship retrieval model to obtain the target identity factor features output by the ship retrieval model. The target identity factor features are used for ship retrieval. The ship retrieval model is trained based on the method described in any of the above embodiments.

[0017] According to a ship retrieval method provided by the present invention, the method further includes: The target identity factor features are matched with each candidate identity factor feature in the feature library; The identity identifier corresponding to the successfully matched candidate identity factor features is determined as the identity identifier of the target ship.

[0018] The present invention also provides a training device for a space-based ship retrieval model, comprising: The first acquisition module is used to acquire a training dataset, which contains ship training images from different imaging platforms and different imaging modalities. The imaging platforms include space-based and air-based systems. The extraction module is used to extract identity factor features, platform factor features, and modality factor features from the ship training images in the training dataset using an initial ship retrieval model. The determination module is used to determine the classification loss based on the identity factor features, the platform factor features, and the modality factor features; The determining module is further configured to determine an orthogonal regularization loss based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities. The orthogonal regularization loss is used to constrain the similarity between identity factor features of the same ship identity, the similarity between platform factor features of the same platform, and the similarity between modal factor features of the same modality, and the identity factor features, platform factor features, and modal factor features are mutually orthogonal in the vector space. The calculation module is used to dynamically combine one or more of the identity factor features, platform factor features, and modality factor features of each sample in any triplet of the training batch, based on the similarity and difference between the platform labels and modality labels of the two samples in the sample pair, to form a combined feature representation for calculating the similarity of the sample pair, and calculate the distance constraint loss based on the combined feature representation to obtain the triplet loss; The update module is used to update the model parameters of the initial ship retrieval model based on the classification loss, the orthogonal regularization loss, and the triplet loss, so as to obtain the ship retrieval model.

[0019] The present invention also provides a ship retrieval device, comprising: The second acquisition module is used to acquire target images of the target ship; The input module is used to input the target image into the ship retrieval model to obtain the target identity factor features output by the ship retrieval model. The target identity factor features are used for ship retrieval. The ship retrieval model is trained based on the method described in any of the above embodiments.

[0020] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a training method or a ship retrieval method for a space-based ship retrieval model as described above.

[0021] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method or a ship retrieval method for a space-based ship retrieval model as described above.

[0022] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a training method or a ship retrieval method for a space-based ship retrieval model as described above.

[0023] The present invention provides a training method and a ship retrieval method for a space-based ship retrieval model. The method acquires a training dataset containing ship training images from different imaging platforms and modalities. The imaging platforms include space-based and air-based platforms. For each ship training image in the training dataset, an initial ship retrieval model extracts identity factor features, platform factor features, and modal factor features from the ship training images. Based on these features, a classification loss is determined. Orthogonal regularization loss is determined based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and modalities. This orthogonal regularization loss is used to constrain the similarity between identity factor features of the same ship identity. Platform factor features of a platform are similar to each other, modality factor features of the same modality are similar to each other, and identity factor features, platform factor features and modality factor features are orthogonal to each other in the vector space. For any triplet in the training batch, a sample pair consisting of an anchor point and a positive ship training image and a sample pair consisting of an anchor point and a negative ship training image, one or more of the identity factor features, platform factor features and modality factor features of each sample in the sample pair are dynamically combined according to the similarity and difference between the platform labels and modality labels of the two samples in the sample pair to form a combined feature representation for calculating the similarity of the sample pairs. The distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss. Based on the classification loss, orthogonal regularization loss and triplet loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model. By decoupling the learning of identity factor features, platform factor features, and modality factor features, and using a relation-aware dynamic measurement method, the system intelligently selects which factor features to concatenate before calculating similarity based on the differences between specific platform and modality labels of sample pairs. This avoids using irrelevant domain noise as an optimization target. The joint training mechanism, which combines feature decoupling learning and relation-aware dynamic measurement, can effectively remove interference from domain-specific information such as platform and modality at the feature representation level. At the measurement learning level, it achieves intelligent feature alignment for different domain relationships. This enables the ship retrieval model to learn a clean ship identity feature representation that is robust to changes in imaging platform and sensor modality, and has strong discriminative power. As a result, the accuracy of cross-platform and cross-modality ship retrieval can be improved. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating the training method for a space-based ship retrieval model provided in an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of a training method for a space-based ship retrieval model provided in an embodiment of the present invention.

[0027] Figure 3 This is a flowchart illustrating the ship retrieval method provided in an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of the structure of a training device for a space-based ship retrieval model provided in an embodiment of the present invention.

[0029] Figure 5 This is a schematic diagram of the structure of the ship retrieval device provided in an embodiment of the present invention.

[0030] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0032] Currently, modern monitoring systems are evolving from relying on single observation platforms to a networked approach that collaboratively utilizes multiple platforms, including space-based (such as satellites) and air-based (such as drones), as well as multimodal remote sensing data such as optical and SAR. This space-based, multi-platform, multimodal collaborative sensing mode can integrate the macroscopic coverage advantages of space-based platforms with the high-resolution and flexible observation advantages of air-based platforms. It also leverages the color and texture information of optical images with the electromagnetic scattering characteristics and all-weather imaging capabilities of SAR images to create a complementary relationship, thereby significantly enhancing the comprehensive monitoring and identification capabilities for maritime targets, especially ship targets.

[0033] Although space-based multi-platform multi-modal ship retrieval has significant application value, it faces considerable technical challenges in practical applications, mainly in the following two aspects: (1) Differences in imaging mechanisms: Optical images reflect the color and texture features of ships, while SAR images reflect the electromagnetic scattering characteristics of ships. There are significant modal differences in visual features between the two.

[0034] (2) Geometric differences in acquisition: Space-based platforms typically use top-view imaging, while air-based platforms often include large-angle oblique-view imaging. This drastic change in perspective leads to severe geometric deformation of ship targets.

[0035] Due to the aforementioned domain differences, features extracted from space-based multi-platform multimodal data often contain domain biases stemming from platform and modality differences. In the feature space, samples tend to cluster according to modality or space-based platform origin, rather than according to ship category. This makes it difficult for existing retrieval algorithms to perform accurate ship retrieval across platforms and modalities.

[0036] In view of the above-mentioned problems, this invention proposes a training method for a space-based ship retrieval model. In this method, a dynamic metric mechanism that combines feature decoupling learning and relation awareness is designed to jointly train the model. Since this mechanism can explicitly decompose image features into mutually orthogonal identity, platform and modal components, and dynamically select features for alignment based on the domain relationship between sample pairs during the metric learning stage, it ensures that the trained ship retrieval model can learn a robust and clean ship identity representation that is robust to changes in imaging platform and sensor modality, thereby improving the accuracy of cross-platform and cross-modal ship retrieval.

[0037] The following is combined Figure 1 and Figure 2 The training method for the space-based ship retrieval model provided in this embodiment of the invention is described. This embodiment of the invention can be applied to marine monitoring and maritime target identification scenarios, such as using satellite (space-based) optical images to retrieve and track specific ships in UAV (air-based) SAR images.

[0038] The subject executing this method can be an electronic device such as a terminal device, computer, server, server cluster, or specially designed training equipment for a space-based ship retrieval model. It can also be a training device for a space-based ship retrieval model set in the electronic device. The training device for the space-based ship retrieval model can be implemented by software, hardware, or a combination of both.

[0039] Figure 1 This is a flowchart illustrating the training method for a space-based ship retrieval model provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: Step 101: Obtain the training dataset, which contains ship training images from different imaging platforms and different imaging modalities. The imaging platforms include space-based and air-based.

[0040] In this step, the imaging platforms include space-based and air-based platforms. Space-based platforms can be, for example, Earth observation satellites, while air-based platforms can be, for example, unmanned aerial vehicles (UAVs) or manned reconnaissance aircraft. These platforms image ships in the same sea area from different altitudes and perspectives. Imaging modes include optical image modes and SAR image modes. Optical images can better reflect color and texture, while SAR images can better reflect electromagnetic scattering characteristics.

[0041] Each ship training image included in the training dataset is labeled with an identity tag, a platform tag, and a modality tag. The identity tag is used to uniquely identify the ship, such as its model or color. The platform tag indicates whether the imaging platform that acquired the ship training image is space-based or air-based. The modality tag indicates whether the imaging mode of the image is optical or SAR.

[0042] Step 102: For each ship training image in the training dataset, extract identity factor features, platform factor features, and modality factor features from the ship training images using the initial ship retrieval model, and determine the classification loss based on the identity factor features, platform factor features, and modality factor features.

[0043] In this step, the initial ship retrieval model can be a visual Transformer backbone network that includes multi-head self-attention and multilayer perceptrons.

[0044] The training images of each ship in the training dataset are input into the initial ship retrieval model. Through information interaction between layers in the initial ship retrieval model, information related to identity, platform, and modal semantics is aggregated from each ship training image, thereby obtaining the identity factor features extracted from each ship training image. Platform Factor Characteristics and modal factor characteristics Among them, identity factor features Platform factor characteristics are used to characterize the inherent physical properties of a ship (such as shape and structure) and do not change with imaging conditions. Characteristics introduced by the imaging platform, such as the altitude viewpoint of the satellite and the tilt viewpoint of the UAV, and modal factor features. Features used to characterize sensor imaging modes, such as the color of visible light and the scattering characteristics of SAR.

[0045] Furthermore, in order to ensure identity factor characteristics Platform Factor Characteristics and modal factor characteristics Since they do not contain redundant information, the classification loss can be determined based on the three factor features of each ship's training image and the corresponding ship's identity label, platform label, and modality label. The classification loss allows for independent supervised training of the identity factor, platform factor, and modality factor, forcing each factor feature to focus on learning discriminative information within its corresponding semantic category, thus initially achieving semantic decoupling of the three types of features.

[0046] Step 103: Based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities, determine the orthogonal regularization loss. The orthogonal regularization loss is used to constrain the similarity between identity factor features of the same ship identity, the similarity between platform factor features of the same platform, and the similarity between modal factor features of the same modality, and the identity factor features, platform factor features, and modal factor features are mutually orthogonal in the vector space.

[0047] In this step, orthogonal regularization loss is used to impose orthogonal constraints among the feature subspaces of identity factors, platform factors, and modality factors, thereby forcing identity factor features to... Platform Factor Characteristics and modal factor characteristics The vectors are orthogonal to each other, which makes the identity factor features of the same ship similar, the platform factor features of the same platform similar, and the modal factor features of the same modality similar. Furthermore, the semantic information carried by the identity factor features, platform factor features, and modal factor features does not overlap, thereby reducing the cross-leakage of information.

[0048] Step 104: For any triplet in the training batch, the sample pair consisting of the anchor point and the positive ship training image and the sample pair consisting of the anchor point and the negative ship training image are dynamically combined according to the similarity and difference between the platform label and modality label of the two samples in the sample pair. This combination of identity factor features, platform factor features and modality factor features of each sample in the sample pair is used to form a combined feature representation for calculating the similarity of the sample pairs. The distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss.

[0049] In this step, for any training batch containing multiple ship training images, these images can be traversed. Each traversed ship training image is used as an anchor point. From the other ship training images in the training batch (excluding the anchor point), samples with the same identity as the anchor point are selected as positive ship training images. If there are multiple samples with the same identity as the anchor point, the sample with the lowest correlation to the anchor point is selected as the positive ship training image, and the anchor point and the positive ship training image are used as one sample pair in a triplet. Additionally, from the other ship training images in the training batch (excluding the anchor point), samples with different identities from the anchor point are selected as negative ship training images. If there are multiple samples with different identities from the anchor point, the sample with the highest similarity to the anchor point is selected as the negative ship training image, and the anchor point and the negative ship training image are used as the other sample pair in a triplet.

[0050] Using the above method, multiple triples (anchor points) in the training batch can be identified. Images of naval training Images of negative ship training ), and the two sample pairs in each triplet.

[0051] For each sample pair, one or more of the identity factor features, platform factor features, and modality factor features of each sample in the sample pair can be dynamically combined based on whether the platform labels and modality labels of the two samples in the sample pair are the same, so as to obtain a combined feature representation.

[0052] After determining the combined feature representation, the similarity between the anchor point and the positive ship training image, and between the anchor point and the negative ship training image, can be calculated based on the combined feature representation. The distance constraint loss is then calculated based on this similarity, resulting in a triplet loss. This triplet loss can be used to achieve relation-aware dynamic metric learning during training. It does not unconditionally bring all samples with the same identity closer together, but rather intelligently selects and aligns the corresponding feature components (identity, platform, and modality) based on the specific platform and modality relationships between sample pairs. This ensures that the ship retrieval model only aligns specific domain features when domain differences exist, effectively avoiding the use of noise introduced by different domains as identity information for learning, reducing optimization conflicts, and making the final learned identity feature space more robust to changes in platform and modality.

[0053] Step 105: Based on classification loss, orthogonal regularization loss and triplet loss, update the model parameters of the initial ship retrieval model to obtain the ship retrieval model.

[0054] In this step, the total loss can be determined based on classification loss, orthogonal regularization loss, and triplet loss. The model parameters of the initial ship retrieval model can then be adjusted using this total loss. By repeating the aforementioned steps until the number of iterations reaches a preset number or the change in the total loss is less than a preset threshold, the final model obtained is determined as the trained ship retrieval model.

[0055] This ship retrieval model can be used to extract pure identity factor features from input ship images and perform ship retrieval across imaging platforms and imaging modalities based on these identity factor features, or it can perform ship retrieval for a specific category.

[0056] The training method for a space-based ship retrieval model provided in this invention involves acquiring a training dataset containing ship training images from different imaging platforms and modalities. The imaging platforms include space-based and air-based models. For each ship training image in the training dataset, an initial ship retrieval model extracts identity factor features, platform factor features, and modal factor features from the ship training images. Based on these features, a classification loss is determined. Furthermore, based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and modalities, an orthogonal regularization loss is determined. This orthogonal regularization loss constrains the similarity between identity factor features of the same ship identity and between the same platform and modal features. Platform factor features are similar to each other, modality factor features of the same modality are similar to each other, and identity factor features, platform factor features and modality factor features are orthogonal to each other in vector space. For any triplet in the training batch, the sample pairs consisting of anchor points and positive ship training images and the sample pairs consisting of anchor points and negative ship training images are dynamically combined according to the similarity and difference between the platform labels and modality labels of the two samples in the sample pair. This is to form a combined feature representation for calculating the similarity of the sample pairs. The distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss. Based on the classification loss, orthogonal regularization loss and triplet loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model. By decoupling the learning of identity factor features, platform factor features, and modality factor features, and using a relation-aware dynamic measurement method, the system intelligently selects which factor features to concatenate before calculating similarity based on the differences between specific platform and modality labels of sample pairs. This avoids using irrelevant domain noise as an optimization target. The joint training mechanism, which combines feature decoupling learning and relation-aware dynamic measurement, can effectively remove interference from domain-specific information such as platform and modality at the feature representation level. At the measurement learning level, it achieves intelligent feature alignment for different domain relationships. This enables the ship retrieval model to learn a clean ship identity feature representation that is robust to changes in imaging platform and sensor modality, and has strong discriminative power. As a result, the accuracy of cross-platform and cross-modality ship retrieval can be improved.

[0057] For example, based on the above embodiments, when extracting identity factor features, platform factor features, and modality factor features from ship training images using the initial ship retrieval model, it can be done in the following way: The ship training images are divided into multiple image patches, and each patch is mapped to a feature vector to obtain an image token sequence. The identity token, platform token, modality token, and image token sequences are concatenated, and a positional code is added to the concatenated sequence to obtain the input sequence. The positional code represents the spatial location information of each token in the ship training images. This input sequence is then fed into the initial ship retrieval model to obtain the identity factor features, platform factor features, and modality factor features output by the initial ship retrieval model.

[0058] Specifically, for any ship training image in the training dataset It can be divided into N image blocks of fixed size, where N is an integer greater than or equal to 2. Each image block is mapped to... 3D vectors yield the image token sequence. .

[0059] Furthermore, three additional learnable tokens, namely identity tokens, can be concatenated before the image token sequence. Platform Token and modal tokens To enable the initial ship retrieval model to perceive the relative positional relationships between image patches, thereby understanding the ship's overall geometry and spatial layout, positional encoding can be added to the stitched sequence. This is used to represent the spatial location information of each token in the ship training image. Therefore, the resulting input sequence It can be represented as: .

[0060] By introducing independent identity tokens, platform tokens, and modality tokens, image information can be decomposed into different feature subspaces, thereby enabling explicit decoupling of domain-related information.

[0061] Furthermore, the input sequence The initial ship retrieval model is input, which includes a visual Transformer backbone network with multi-head self-attention and a multilayer perceptron. Through inter-layer information interaction, identity tokens, platform tokens, and modality tokens aggregate semantically relevant information from image patches. The initial ship retrieval model's first... The layer output is: L represents the total number of floors.

[0062] After extracting the features, feature decoupling can be performed. Specifically, this is done by taking the last layer (the first layer). The three vectors at corresponding positions in the output sequence of the layer are used as the decoupled feature representations: .

[0063] Among them, identity factor features It can characterize the inherent physical properties of a ship, such as shape and structure, which do not change with imaging conditions. Platform factor characteristics It can characterize the properties brought by the imaging platform, such as the altitude and viewing angle of a satellite and the tilt and viewing angle of a UAV. Modal factor characteristics It can characterize the features of sensor imaging modes, such as the color of visible light and the scattering characteristics of SAR.

[0064] In this embodiment, by introducing identity tokens, platform tokens, and modality tokens, the Transformer backbone network can be guided to selectively and selectively aggregate information most relevant to ship identity, imaging platform, and sensor modality from global image patches during subsequent information interaction (self-attention). Furthermore, by inputting the input sequence into the initial ship retrieval model, identity factor features, platform factor features, and modality factor features are obtained. This provides structured and semantically clear intermediate feature representations for subsequent feature decoupling learning and relation-aware dynamic measurement, laying the foundation for effectively separating identity information and domain information from the original image.

[0065] For example, based on the above embodiments, each ship training image has an identity label, a platform label, and a modality label. When determining the classification loss based on the identity factor features, platform factor features, and modality factor features, the identity factor features, platform factor features, and modality factor features can be input into the corresponding identity classifier, platform classifier, and modality classifier, respectively, to obtain the identity prediction result, platform prediction result, and modality prediction result. Based on the identity prediction result and the identity label, the platform prediction result and the platform label, and the modality prediction result and the modality label, the cross-entropy loss is calculated and weighted summed to obtain the classification loss.

[0066] Specifically, to ensure that the identity factor, platform factor, and modality factor do not contain redundant information, a loss function can be introduced. The features of the identity factor, platform factor, and modality factor are respectively connected to their corresponding classifiers, and supervised training is performed using the identity labels, platform labels, and modality labels of the corresponding ship training images to ensure that each factor has corresponding discriminative power.

[0067] In one implementation, the classification loss can be determined according to formula (1): (1) in, Represents classification loss. This represents the identity prediction result for the i-th ship training image. This represents the platform prediction result for the i-th ship training image. This represents the modal prediction result for the i-th ship training image. This represents the identity label of the i-th ship training image. This represents the platform label of the i-th ship training image. This represents the modal label of the i-th ship training image. This represents the identity factor cross-entropy loss function. This represents the platform factor cross-entropy loss function. Represents the modality factor cross-entropy loss function. This represents the weighting coefficient corresponding to the identity factor. This represents the weighting coefficient of the platform factor. This represents the weighting coefficient of the modal factor.

[0068] in, , as well as Used to measure the difference between the distribution of predicted results and the distribution of true labels.

[0069] In this embodiment, by setting a classification loss, the identity factor features, platform factor features, and modality factor features can be trained independently under supervision. By utilizing the identity labels, platform labels, and modality labels already labeled in the ship training images, the identity factor features are forced to focus on identifying the ship's identity, the platform factor features on identifying the imaging platform (space-based / air-based), and the modality factor features on identifying the imaging modality (optical / SAR). This gives the three factors a clear semantic division of labor and basic discriminative power, thereby improving the training accuracy and robustness of the ship retrieval model.

[0070] For example, based on the above embodiments, when determining the orthogonal regularization loss based on the identity factor features, platform factor features, and modality factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities, it can be done in the following way: The identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity and respectively from space-based optical, space-based synthetic aperture radar (SAR), air-based optical, and air-based SAR imaging conditions are extracted to form a feature vector set. The cosine similarity between every two feature vectors in the feature vector set is calculated to obtain a similarity matrix. An orthogonal constraint target matrix with the same size as the similarity matrix is ​​constructed. The elements in the orthogonal constraint target matrix are assigned values ​​according to whether the factor types of the corresponding two feature vectors are the same and whether they belong to the same platform or mode. The orthogonal regularization loss is determined based on the similarity matrix and the orthogonal constraint target matrix.

[0071] Specifically, this refers to space-based optical ship training images belonging to the same ship identity category within the training batch. Space-based SAR ship training images Airborne optical ship training images Training images of airborne SAR ships Each feature is extracted to obtain a set of feature vectors: in, It can be represented as ;in, Indicates the factor type to which the feature belongs. These represent identity, platform, and modality, respectively. Indicates the platform to which the feature belongs. , representing space-based and air-based, respectively; Indicates the mode to which the feature belongs. , representing optical and SAR respectively.

[0072] Calculate the set of feature vectors The similarity matrix is ​​obtained by calculating the cosine similarity between each pair of feature vectors. ,in This means finding the inner product of two eigenvectors.

[0073] Furthermore, a similarity matrix can be constructed. Orthogonal constraint target matrices of the same size Specifically, the orthogonal constraint objective matrix The values ​​of each element in the algorithm can be determined as follows: If both feature vectors are identity factor features, the corresponding element value is 1. If both feature vectors are platform factor features and come from the same imaging platform, the corresponding element value is 1. If both feature vectors are modality factor features and come from the same imaging modality, the corresponding element value is 1. In other cases, the corresponding element value is 0.

[0074] For example, the orthogonal constraint target matrix can be determined according to the following formula (5). : (5) By determining the specific values ​​of each element in the orthogonal constraint objective matrix based on whether the factor types and the platforms or modes to which the two corresponding feature vectors are the same, the content that the model should learn in the feature space can be precisely defined, thus providing a clear optimization objective for achieving feature decoupling.

[0075] After determining the similarity matrix and the orthogonal constraint target matrix, the orthogonal regularization loss can be determined by calculating the mean square error between the two matrices. This applies orthogonal constraints between the feature subspaces of the three factors, forcing the identity factor features, platform factor features, and modality factor features to be orthogonal to each other in the vector space, thereby reducing cross-leakage of information.

[0076] Specifically, the orthogonal regularization loss can be determined according to formula (2): (2) in, This represents the orthogonal regularization loss. This represents a ship training image under space-based optical imaging conditions. This represents a ship training image under space-based SAR imaging conditions. This represents a ship training image under airborne optical imaging conditions. This represents a ship training image under airborne SAR imaging conditions. This represents the value of the element in the i-th row and j-th column of the similarity matrix. This represents the value of the element in the i-th row and j-th column of the orthogonal constraint objective matrix.

[0077] For example, based on the above embodiments, when dynamically combining one or more of the identity factor features, platform factor features, and modality factor features of each sample in a sample pair according to the similarities and differences in the platform labels and modality labels of the two samples in the sample pair to form a combined feature representation for calculating the similarity of the sample pair, for the sample pair It can be determined according to formula (3): in, This represents the combined feature representation. This represents the identity factor characteristics of sample i. This represents the platform factor characteristic of sample i. This represents the modal factor characteristic of sample i. This represents the platform label for sample i. This represents the platform label for sample j. This represents the modal label of sample i. This represents the modal label of sample j.

[0078] By dynamically combining one or more of the identity factor features, platform factor features, and modality factor features of each sample in a sample pair based on the similarity and difference between the platform labels and modality labels of the two samples in the sample pair, a combined feature representation for calculating the similarity of the sample pairs can be formed. This enables relation-aware intelligent metric learning. Instead of unconditionally bringing all features of all samples with the same identity closer together, it selectively aligns relevant feature components based on the actual domain differences between sample pairs. Furthermore, by dynamically excluding irrelevant domain features in the comparison of specific sample pairs, it prevents the model from mistakenly learning domain noise from different domains (platform or modality) as part of the ship's identity. This effectively solves the optimization conflict caused by the forced alignment of complete features containing noise from different domains in existing technologies.

[0079] For example, the triplet loss can be determined according to formula (4): (4) in, Indicates the loss of the triplet. Indicates the anchor point in a triple. This represents a training image of a ship in a triplet. This represents a training image of a negative ship in a triplet. Indicates the distance threshold. for operate.

[0080] For example, when updating the model parameters of the initial ship retrieval model based on classification loss, orthogonal regularization loss, and triplet loss to obtain the ship retrieval model, the classification loss, orthogonal regularization loss, and triplet loss can be weighted and summed to obtain the target loss. Based on the target loss, the model parameters of the initial ship retrieval model can be updated to obtain the ship retrieval model.

[0081] Specifically, the target loss can be determined according to formula (6): (6) in, Indicates target loss. Represents classification loss The weighting coefficients, Describes the orthogonal regularization loss The weighting coefficients, Indicates the loss of the triplet The weighting coefficients.

[0082] After determining the target loss, the model parameters of the initial ship retrieval model are updated based on the target loss, and a new target loss is determined. The above process is repeated until the number of iterations reaches the preset number or the change in the target loss is less than the preset threshold. The final model is then determined as the ship retrieval model.

[0083] In this embodiment, the target loss is obtained by weighted summation of classification loss, orthogonal regularization loss, and triplet loss. The model parameters of the initial ship retrieval model are then updated based on this target loss, which enables multi-objective collaborative optimization. This allows the model to learn feature decoupling, feature orthogonality, and relation-aware metrics simultaneously in a single training iteration, thereby efficiently converging to an optimal state that can extract pure, discriminative, and robust identity features to domain changes.

[0084] Figure 2 This is a schematic diagram of the training method for a space-based ship retrieval model provided in an embodiment of the present invention, as shown below. Figure 2 As shown, in this embodiment, ship training images from different platforms (space-based / air-based) and different modalities (optical / SAR), such as space-based SAR images, space-based optical images, air-based SAR images, and air-based optical images, are processed by a three-factor tokenization module to obtain the input sequence corresponding to each ship training image. Each input sequence includes... And location encoding. For example, for a space-based optical image, after processing by the three-factor tokenization module, the resulting input sequence includes... , , , The location encoding is similar to that used for training images of other ships.

[0085] By inputting the input sequence into a visual Transformer backbone network that includes multi-head self-attention and multi-layer perceptrons, the identity factor features corresponding to each ship's training image can be obtained. Platform Factor Characteristics and modal factor characteristics .

[0086] This allows for the use of identity factor features corresponding to the training images of each ship. Platform Factor Characteristics and modal factor characteristics Determine the target loss to train the initial ship training model.

[0087] The target loss includes classification loss. Orthogonal regularization loss and triplet loss Classification loss This is also known as the three-factor cross-entropy loss, which includes identity cross-entropy loss, platform cross-entropy loss, and modality cross-entropy loss. It imposes independent classification constraints on the identity, platform, and modality factors, forcing each factor to have basic discriminative ability.

[0088] Orthogonal regularization loss That is, the three-factor orthogonalization loss constructs a similarity matrix and an orthogonal constraint target matrix, applies strict orthogonal constraints, and forces the three types of feature vectors to be orthogonal to each other in space, thereby achieving deep feature decoupling.

[0089] By using triplet loss based on a relation-aware dynamic feature combination strategy, similarity can be calculated by dynamically combining identity, platform, or modality factor features according to the similarity or difference between platform and modality labels of sample pairs, and intelligent measurement is achieved through "Pull" and "Push" operations.

[0090] Finally, the target loss is obtained by weighted summation of the classification loss, orthogonal regularization loss, and triplet loss, which guides the model to learn a pure and discriminative identity feature representation.

[0091] Figure 3 This is a flowchart illustrating the ship retrieval method provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the method includes: Step 301: Obtain the target image of the target ship.

[0092] Step 302: Input the target image into the ship retrieval model to obtain the target identity factor features output by the ship retrieval model. These target identity factor features are used for ship retrieval.

[0093] The ship retrieval model is trained based on the method described in any of the above embodiments.

[0094] In this step, the target image can be a ship image to be queried and whose identity needs to be retrieved. It can come from a space-based or air-based platform, and the imaging modality can be either optical or SAR. For example, it can be an optical image taken by a satellite or a SAR image taken by a UAV.

[0095] The target image is input into a pre-trained ship retrieval model, which performs forward propagation to extract and output its target identity factor features. These features can then be used for similarity matching and retrieval within a pre-built feature library.

[0096] The ship retrieval method provided in this invention acquires a target image of the target ship, inputs the target image into a ship retrieval model, and obtains the target identity factor features output by the ship retrieval model. These target identity factor features are used for ship retrieval. During training, the ship retrieval model can decouple and learn identity factor features, platform factor features, and modality factor features. Through a relationship-aware dynamic measurement method, it intelligently selects which factor features to concatenate before calculating similarity based on the differences between platform and modality labels of sample pairs. This avoids using irrelevant domain noise as an optimization target. The joint training mechanism combining feature decoupling learning and relationship-aware dynamic measurement effectively removes interference from platform and modality-specific information at the feature representation level and achieves intelligent feature alignment for different domain relationships at the measurement learning level. This enables the ship retrieval model to learn a robust and discriminative pure ship identity feature representation that is robust to changes in imaging platform and sensor modality. Therefore, when using this ship retrieval model for ship retrieval, the accuracy of cross-platform and cross-modality ship retrieval can be improved.

[0097] For example, based on the above embodiments, the target identity factor features can be matched with each candidate identity factor feature in the feature library, and the identity identifier corresponding to the successfully matched candidate identity factor features can be determined as the identity identifier of the target ship.

[0098] Specifically, the feature library stores candidate identity factor features extracted from multiple known ship images using the same model. By calculating the similarity (such as cosine similarity) between the target identity factor feature and all features in the feature library, and returning the top few similarity results, the identity identifiers corresponding to these top similarity candidate identity factor features are determined as the identity identifiers of the target ship, thus enabling cross-platform and cross-modal ship identity retrieval and matching.

[0099] Since the identity factor feature has been stripped of interference from platform and modality information during training, the accuracy and robustness of the retrieval can be ensured.

[0100] The training apparatus for the space-based ship retrieval model provided by the present invention will be described below. The training apparatus for the space-based ship retrieval model described below and the training method for the space-based ship retrieval model described above can be referred to in correspondence with each other.

[0101] Figure 4 This is a schematic diagram of the structure of the training device for the space-based ship retrieval model provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the training device 400 for the space-based ship retrieval model includes: The first acquisition module 11 is used to acquire a training dataset, which contains ship training images from different imaging platforms and different imaging modalities. The imaging platforms include space-based and air-based systems. Extraction module 12 is used to extract identity factor features, platform factor features and modality factor features from the ship training images in the training dataset using an initial ship retrieval model. The determination module 13 is used to determine the classification loss based on the identity factor features, the platform factor features, and the modality factor features; The determining module 13 is further configured to determine an orthogonal regularization loss based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities. The orthogonal regularization loss is used to constrain the similarity between identity factor features of the same ship identity, the similarity between platform factor features of the same platform, and the similarity between modal factor features of the same modality, and the identity factor features, platform factor features, and modal factor features are mutually orthogonal in the vector space. The calculation module 14 is used to dynamically combine one or more of the identity factor features, platform factor features, and modality factor features of each sample in the sample pair, based on the similarity and difference between the platform labels and modality labels of the two samples in the sample pair, for any triplet in the training batch, to form a combined feature representation for calculating the similarity of the sample pair, and calculate the distance constraint loss based on the combined feature representation to obtain the triplet loss; The update module 15 is used to update the model parameters of the initial ship retrieval model based on the classification loss, the orthogonal regularization loss and the triplet loss, so as to obtain the ship retrieval model.

[0102] In one example embodiment, the extraction module 12 is specifically used for: The ship training images are divided into multiple image blocks, and each image block is mapped to a feature vector to obtain an image token sequence; The sequence of identity token, platform token, modality token and image token is concatenated, and a position code is added to the concatenated sequence to obtain the input sequence. The position code is used to characterize the spatial position information of each token in the ship training image. The input sequence is input into the initial ship retrieval model to obtain the identity factor features, platform factor features and modality factor features output by the initial ship retrieval model.

[0103] In one example embodiment, the ship training image has an identity tag, a platform tag, and a modality tag; Module 13 is specifically used for: The identity factor features, platform factor features, and modality factor features are respectively input into the corresponding identity classifier, platform classifier, and modality classifier to obtain the identity prediction result, platform prediction result, and modality prediction result; Based on the identity prediction result and the identity label, the platform prediction result and the platform label, and the modality prediction result and the modality label, the cross-entropy loss is calculated and weighted summed to obtain the classification loss.

[0104] In one example embodiment, the calculation module 14 is specifically used for: The classification loss is determined according to formula (1): (1) in, This represents the classification loss. This represents the identity prediction result for the i-th ship training image. This represents the platform prediction result for the i-th ship training image. This represents the modal prediction result for the i-th ship training image. This represents the identity label of the i-th ship training image. This represents the platform label of the i-th ship training image. This represents the modal label of the i-th ship training image. This represents the identity factor cross-entropy loss function. This represents the platform factor cross-entropy loss function. Represents the modality factor cross-entropy loss function. This represents the weighting coefficient corresponding to the identity factor. This represents the weighting coefficient of the platform factor. This represents the weighting coefficient of the modal factor.

[0105] In one example embodiment, the determining module 13 is specifically used for: Extract the identity factor features, platform factor features, and modal factor features corresponding to the ship training images belonging to the same ship identity and respectively from space-based optics, space-based synthetic aperture radar SAR, air-based optics, and air-based SAR imaging conditions, and form a feature vector set; Calculate the cosine similarity between every two feature vectors in the feature vector set to obtain a similarity matrix; Construct an orthogonal constraint target matrix with the same size as the similarity matrix, wherein the elements in the orthogonal constraint target matrix are assigned values ​​according to whether the factor types and the platforms or modes to which the corresponding two feature vectors are the same; The orthogonal regularization loss is determined based on the similarity matrix and the orthogonal constraint target matrix.

[0106] In one example embodiment, the orthogonal constraint target matrix is ​​determined as follows: If both feature vectors are identity factor features, the corresponding element value is 1. If both feature vectors are platform factor features and come from the same imaging platform, the corresponding element value is 1. If both feature vectors are modality factor features and come from the same imaging modality, the corresponding element value is 1. In other cases, the corresponding element value is 0.

[0107] In one example embodiment, the determining module 13 is specifically used for: The orthogonal regularization loss is determined according to formula (2): (2) in, This represents the orthogonal regularization loss. This represents a ship training image under space-based optical imaging conditions. This represents a ship training image under space-based SAR imaging conditions. This represents a ship training image under airborne optical imaging conditions. This represents a ship training image under airborne SAR imaging conditions. This represents the value of the element in the i-th row and j-th column of the similarity matrix. This represents the value of the element in the i-th row and j-th column of the orthogonal constraint target matrix.

[0108] In one example embodiment, the calculation module 14 is specifically used for: The combined feature representation is determined according to formula (3): in, This represents the combined feature representation. This represents the identity factor characteristics of sample i. This represents the platform factor characteristic of sample i. This represents the modal factor characteristic of sample i. This represents the platform label for sample i. This represents the platform label for sample j. This represents the modal label of sample i. This represents the modal label of sample j.

[0109] In one example embodiment, the calculation module 14 is specifically used for: The triplet loss is determined according to formula (4): (4) in, This represents the loss of the triplet. Indicates the anchor point in a triple. This represents the positive ship training image in the triplet. This represents the negative ship training image in the triplet. Indicates the distance threshold. for operate.

[0110] In one example embodiment, the update module 15 is specifically used for: The target loss is obtained by weighted summing of the classification loss, the orthogonal regularization loss, and the triplet loss. Based on the target loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

[0111] In one example embodiment, the identity factor feature is used to characterize the physical property characteristics of the ship, the platform factor feature is used to characterize the characteristics introduced by the imaging platform, and the modal factor feature is used to characterize the characteristics of the sensor imaging mode.

[0112] The apparatus of this embodiment can be used in any embodiment of the training method for a space-based ship retrieval model. Its specific implementation process and technical effects are similar to those in the training method for a space-based ship retrieval model. For details, please refer to the detailed description in the training method for a space-based ship retrieval model, which will not be repeated here.

[0113] Figure 5 This is a schematic diagram of the structure of the ship retrieval device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the ship retrieval device 500 includes: The second acquisition module 21 is used to acquire the target image of the target ship; The input module 22 is used to input the target image into the ship retrieval model to obtain the target identity factor features output by the ship retrieval model. The target identity factor features are used for ship retrieval. The ship retrieval model is trained based on the method described in any of the above embodiments.

[0114] In one example embodiment, the apparatus further includes a matching module and a determining module, wherein: The matching module is used to match the target identity factor features with each candidate identity factor feature in the feature library; The determination module is used to determine the identity identifier corresponding to the successfully matched candidate identity factor features as the identity identifier of the target ship.

[0115] The apparatus of this embodiment can be used in any of the methods in the ship retrieval method side embodiment. Its specific implementation process and technical effects are similar to those in the ship retrieval method side embodiment. For details, please refer to the detailed description in the ship retrieval method side embodiment, which will not be repeated here.

[0116] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 6As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logic instructions in the memory 630 to execute a training method for a space-based ship retrieval model. This method includes: acquiring a training dataset containing ship training images from different imaging platforms and different imaging modalities, the imaging platforms including space-based and air-based; for each ship training image in the training dataset, extracting identity factor features, platform factor features, and modality factor features from the ship training images using an initial ship retrieval model, and determining a classification loss based on the identity factor features, platform factor features, and modality factor features; and determining an orthogonal regularization loss based on the identity factor features, platform factor features, and modality factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities, the orthogonal regularization loss being used to constrain the identity factor features of the same ship identity. Sub-features are similar to each other, platform factor features of the same platform are similar to each other, and modality factor features of the same modality are similar to each other. Furthermore, identity factor features, platform factor features, and modality factor features are orthogonal to each other in the vector space. For any triplet in the training batch, a sample pair consisting of an anchor point and a positive ship training image and a sample pair consisting of an anchor point and a negative ship training image are dynamically combined according to the similarity and difference between the platform labels and modality labels of the two samples in the sample pair. This combination is used to form a combined feature representation for calculating the similarity of the sample pair. The distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss. Based on the classification loss, the orthogonal regularization loss, and the triplet loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

[0117] The processor 610 can call logical instructions in the memory 630 to execute a ship retrieval method, which includes: acquiring a target image of a target ship; inputting the target image into a ship retrieval model to obtain target identity factor features output by the ship retrieval model, wherein the target identity factor features are used for ship retrieval, and the ship retrieval model is trained based on the method described in any of the above embodiments.

[0118] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method for the space-based ship retrieval model provided by the above methods. The method includes: acquiring a training dataset containing ship training images from different imaging platforms and different imaging modalities, wherein the imaging platforms include space-based and air-based; for each ship training image in the training dataset, extracting identity factor features, platform factor features, and modal factor features from the ship training images using an initial ship retrieval model, and determining a classification loss based on the identity factor features, platform factor features, and modal factor features; and based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities. An orthogonal regularization loss is determined, which is used to constrain the similarity between identity factor features of the same ship identity, the similarity between platform factor features of the same platform, and the similarity between modality factor features of the same modality, and the identity factor features, platform factor features, and modality factor features are mutually orthogonal in the vector space. For any triplet in the training batch, a sample pair consisting of an anchor point and a positive ship training image and a sample pair consisting of an anchor point and a negative ship training image, one or more of the identity factor features, platform factor features, and modality factor features of each sample in the sample pair are dynamically combined according to the similarity and difference between the platform labels and modality labels of the two samples in the sample pair, so as to form a combined feature representation for calculating the similarity of the sample pair, and the distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss. Based on the classification loss, the orthogonal regularization loss, and the triplet loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

[0120] When the computer program is executed by the processor, the computer is able to execute the ship retrieval method provided by the above methods. The method includes: acquiring a target image of a target ship; inputting the target image into a ship retrieval model to obtain target identity factor features output by the ship retrieval model, wherein the target identity factor features are used for ship retrieval, and the ship retrieval model is trained based on the method described in any of the above embodiments.

[0121] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a training method for a space-based ship retrieval model provided by the methods described above. This method includes: acquiring a training dataset containing ship training images from different imaging platforms and different imaging modalities, the imaging platforms including space-based and air-based models; for each ship training image in the training dataset, extracting identity factor features, platform factor features, and modality factor features from the ship training images using an initial ship retrieval model, and determining a classification loss based on the identity factor features, platform factor features, and modality factor features; and determining an orthogonal regularization loss based on the identity factor features, platform factor features, and modality factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities. The regularization loss is used to constrain the similarity between identity factor features of the same ship identity, platform factor features of the same platform, and modality factor features of the same modality, and the identity factor features, platform factor features, and modality factor features are orthogonal to each other in the vector space. For any triplet in the training batch, a sample pair consisting of an anchor point and a positive ship training image and a sample pair consisting of an anchor point and a negative ship training image, one or more of the identity factor features, platform factor features, and modality factor features of each sample in the sample pair are dynamically combined according to the similarity and difference between the platform labels and modality labels of the two samples in the sample pair to form a combined feature representation for calculating the similarity of the sample pair, and the distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss. Based on the classification loss, the orthogonal regularization loss, and the triplet loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

[0122] When executed by a processor, the computer program implements the ship retrieval method provided by the above methods, the method comprising: acquiring a target image of a target ship; inputting the target image into a ship retrieval model to obtain target identity factor features output by the ship retrieval model, the target identity factor features being used for ship retrieval, the ship retrieval model being trained based on the method described in any of the above embodiments.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a space-based ship retrieval model, characterized in that, include: Obtain a training dataset containing ship training images from different imaging platforms and different imaging modalities, including space-based and air-based imaging platforms; For each ship training image in the training dataset, an initial ship retrieval model is used to extract identity factor features, platform factor features, and modality factor features from the ship training images, and a classification loss is determined based on the identity factor features, platform factor features, and modality factor features. Based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities, an orthogonal regularization loss is determined. The orthogonal regularization loss is used to constrain the similarity between identity factor features of the same ship identity, the similarity between platform factor features of the same platform, and the similarity between modal factor features of the same modality, and the identity factor features, platform factor features, and modal factor features are mutually orthogonal in the vector space. For any triplet in the training batch, a sample pair consisting of an anchor point and a positive ship training image and a sample pair consisting of an anchor point and a negative ship training image are dynamically combined according to the similarity and difference between the platform label and modality label of the two samples in the sample pair. This combination of identity factor features, platform factor features, and modality factor features of each sample in the sample pair is used to form a combined feature representation for calculating the similarity of the sample pair. The distance constraint loss is calculated based on the combined feature representation to obtain the triplet loss. Based on the classification loss, the orthogonal regularization loss, and the triplet loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

2. The training method for the space-based ship retrieval model according to claim 1, characterized in that, The step of extracting identity factor features, platform factor features, and modality factor features from the ship training images using the initial ship retrieval model includes: The ship training images are divided into multiple image blocks, and each image block is mapped to a feature vector to obtain an image token sequence; The sequence of identity token, platform token, modality token and image token is concatenated, and a position code is added to the concatenated sequence to obtain the input sequence. The position code is used to characterize the spatial position information of each token in the ship training image. The input sequence is input into the initial ship retrieval model to obtain the identity factor features, platform factor features and modality factor features output by the initial ship retrieval model.

3. The training method for the space-based ship retrieval model according to claim 1, characterized in that, The ship training images have identity tags, platform tags, and modality tags; The determination of classification loss based on the identity factor features, the platform factor features, and the modality factor features includes: The identity factor features, platform factor features, and modality factor features are respectively input into the corresponding identity classifier, platform classifier, and modality classifier to obtain the identity prediction result, platform prediction result, and modality prediction result; The classification loss is determined according to formula (1): （1） in, This represents the classification loss. This represents the identity prediction result for the i-th ship training image. This represents the platform prediction result for the i-th ship training image. This represents the modal prediction result for the i-th ship training image. This represents the identity label of the i-th ship training image. This represents the platform label of the i-th ship training image. This represents the modal label of the i-th ship training image. This represents the identity factor cross-entropy loss function. This represents the platform factor cross-entropy loss function. Represents the modality factor cross-entropy loss function. This represents the weighting coefficient corresponding to the identity factor. This represents the weighting coefficient of the platform factor. This represents the weighting coefficient of the modal factor.

4. The training method for a space-based ship retrieval model according to any one of claims 1-3, characterized in that, The orthogonal regularization loss is determined based on the identity factor features, platform factor features, and modality factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities. This includes: Extract the identity factor features, platform factor features, and modal factor features corresponding to the ship training images belonging to the same ship identity and respectively from space-based optics, space-based synthetic aperture radar SAR, air-based optics, and air-based SAR imaging conditions, and form a feature vector set; Calculate the cosine similarity between every two feature vectors in the feature vector set to obtain a similarity matrix; Construct an orthogonal constraint target matrix with the same size as the similarity matrix. The elements in the orthogonal constraint target matrix are assigned values ​​based on whether the factor types and platforms or modes of the two corresponding feature vectors are the same. The orthogonal constraint target matrix is ​​determined as follows: if both feature vectors are identity factor features, the corresponding element value is 1; if both feature vectors are platform factor features and come from the same imaging platform, the corresponding element value is 1; if both feature vectors are mode factor features and come from the same imaging mode, the corresponding element value is 1; otherwise, the corresponding element value is 0. The orthogonal regularization loss is determined according to formula (2): （2） in, This represents the orthogonal regularization loss. This represents a ship training image under space-based optical imaging conditions. This represents a ship training image under space-based SAR imaging conditions. This represents a ship training image under airborne optical imaging conditions. This represents a ship training image under airborne SAR imaging conditions. This represents the value of the element in the i-th row and j-th column of the similarity matrix. This represents the value of the element in the i-th row and j-th column of the orthogonal constraint target matrix.

5. The training method for a space-based ship retrieval model according to any one of claims 1-3, characterized in that, The step of dynamically combining one or more of the identity factor features, platform factor features, and modality factor features of each sample in the sample pair based on the similarities and differences in the platform labels and modality labels of the two samples in the sample pair to form a combined feature representation for calculating the similarity of the sample pair includes: The combined feature representation is determined according to formula (3): in, This represents the combined feature representation. This represents the identity factor characteristics of sample i. This represents the platform factor characteristic of sample i. This represents the modal factor characteristic of sample i. This represents the platform label for sample i. This represents the platform label for sample j. This represents the modal label of sample i. This represents the modal label of sample j.

6. The training method for the space-based ship retrieval model according to claim 5, characterized in that, The calculation of distance constraint loss based on the combined feature representation to obtain triplet loss includes: The triplet loss is determined according to formula (4): （4） in, This represents the loss of the triplet. Indicates the anchor point in a triple. This represents the positive ship training image in the triplet. This represents the negative ship training image in the triplet. Indicates the distance threshold. for operate.

7. The training method for a space-based ship retrieval model according to any one of claims 1-3, characterized in that, The process of updating the model parameters of the initial ship retrieval model based on the classification loss, the orthogonal regularization loss, and the triplet loss to obtain the ship retrieval model includes: The target loss is obtained by weighted summing of the classification loss, the orthogonal regularization loss, and the triplet loss. Based on the target loss, the model parameters of the initial ship retrieval model are updated to obtain the ship retrieval model.

8. A ship retrieval method, characterized in that, include: Acquire target images of the target ship; The target image is input into the ship retrieval model to obtain the target identity factor features output by the ship retrieval model. The target identity factor features are used for ship retrieval. The ship retrieval model is trained based on the method described in any one of claims 1-7.

9. A training device for a space-based ship retrieval model, characterized in that, include: The first acquisition module is used to acquire a training dataset, which contains ship training images from different imaging platforms and different imaging modalities. The imaging platforms include space-based and air-based systems. The extraction module is used to extract identity factor features, platform factor features, and modality factor features from the ship training images in the training dataset using an initial ship retrieval model. The determination module is used to determine the classification loss based on the identity factor features, the platform factor features, and the modality factor features; The determining module is further configured to determine an orthogonal regularization loss based on the identity factor features, platform factor features, and modal factor features corresponding to ship training images belonging to the same ship identity but from different imaging platforms and different imaging modalities. The orthogonal regularization loss is used to constrain the similarity between identity factor features of the same ship identity, the similarity between platform factor features of the same platform, and the similarity between modal factor features of the same modality, and the identity factor features, platform factor features, and modal factor features are mutually orthogonal in the vector space. The calculation module is used to dynamically combine one or more of the identity factor features, platform factor features, and modality factor features of each sample in any triplet of the training batch, based on the similarity and difference between the platform labels and modality labels of the two samples in the sample pair, to form a combined feature representation for calculating the similarity of the sample pair, and calculate the distance constraint loss based on the combined feature representation to obtain the triplet loss; The update module is used to update the model parameters of the initial ship retrieval model based on the classification loss, the orthogonal regularization loss, and the triplet loss, so as to obtain the ship retrieval model.

10. A ship retrieval device, characterized in that, include: The second acquisition module is used to acquire target images of the target ship; An input module is used to input the target image into a ship retrieval model to obtain the target identity factor features output by the ship retrieval model. The target identity factor features are used for ship retrieval. The ship retrieval model is trained based on the method described in any one of claims 1-7.

Images