Target association processing method and device of ship image, equipment, medium and product

Abstract

The invention provides a target association processing method and device for a ship image, equipment, a medium and a product, and relates to the technical field of image processing, and the method comprises the steps: obtaining a ship query image and a ship image database; inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is a model for extracting global features and local features of the ship query image and the ship image database and determining a target association processing result corresponding to the ship query image and the ship image database based on fusion processing of the global features and the local features. According to the technical scheme, the global features and the local features are extracted based on the ship target association processing model, the target association processing result is determined based on fusion processing of the global features and the local features, and the association accuracy of the ship query image and the ship image database is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment, medium and product for target association processing of ship images. Background Art

[0002] Ship target association technology is an important component of the intelligent shipping system and an important component of the visual perception task required for ship detection in the sea area.

[0003] At present, in the prior art, for the target association of ships, only the correlation of the global features of the image is established in each small batch, and the association accuracy of the ship images is determined by establishing the correlation of the global features of the image. However, the determination of the association accuracy of the ship images based on the correlation of the global features ignores the relevant information between the images in the complete data set, and most of the target association work only considers the relationship learning between the samples of the global features, and the learned association information is limited, resulting in a decrease in the association accuracy of the ship images.

[0004] Therefore, there is an urgent need for a target association processing method for ship images to improve the association accuracy of ship images. Summary of the invention

[0005] The present invention provides a target association processing method, device, equipment, medium and product for ship images, which are used to solve the defect that the correlation accuracy of ship images is determined only based on global features in the prior art, and the relevant information between images in the complete data set is ignored, resulting in reduced correlation accuracy of the ship images. The present invention realizes global feature and local feature extraction of ship query images and ship image databases based on a ship target association processing model, determines the target association processing results corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features, and improves the correlation accuracy of the ship query image and the ship image database.

[0006] The present invention provides a target association processing method for ship images, comprising the following steps.

[0007] A ship query image and a ship image database are obtained; wherein the ship query image is a ship image of a target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, different platforms and different devices.

[0008] The ship query image and the ship image database are input into the ship target association processing model to obtain the target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0009] According to a target association processing method for ship images provided by the present invention, a ship query image and a ship image database are input into a ship target association processing model to obtain a target association processing result output by the ship target association processing model, including: inputting the ship query image and the ship image database into an initial feature extraction model in the ship target association processing model to obtain initial image global features and initial image local features output by the initial feature extraction model; wherein the initial feature extraction model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global features into a global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model; wherein In the present invention, the global feature modeling model is a model for modeling the feature correlation of the global features of the initial image based on global attention; the local features of the initial image are input into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model; wherein the local feature modeling model is a model for modeling the feature correlation of the local features of the initial image based on the nearest local attention; the global features of the target image and the local features of the target image are input into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model; wherein the feature correlation fusion model is a model for fusion processing of the global features of the target image and the local features of the target image.

[0010] According to a target association processing method for ship images provided by the present invention, before inputting the global features of the initial image into a global feature modeling model in a ship target association processing model to obtain the global features of the target image output by the global feature modeling model, the method also includes: performing a linear transformation on the global features of the initial image to obtain global linear projection features corresponding to the global features of the initial image.

[0011] According to a target association processing method for ship images provided by the present invention, the global features of the initial image are input into the global feature modeling model in the ship target association processing model to obtain the global features of the target image output by the global feature modeling model, including: inputting the global linear projection features into the dimension reduction network in the global feature modeling model to obtain the global features of the reduced dimension image output by the dimension reduction network; wherein the dimension reduction network is a network that performs dimension reduction processing on the global features of the initial image; inputting the global features of the reduced dimension image into the global modeling network in the global feature modeling model to obtain the global weights output by the global modeling network; wherein the global modeling network is a network that performs correlation modeling on the global features of the reduced dimension image; inputting the global linear projection features and the global weights into the global weighted network in the global feature modeling model to obtain the global features of the target image output by the global weighted network; wherein the global weighted network is a network that performs weighted multiplication of the global linear projection features and the global weights.

[0012] According to a target association processing method for ship images provided by the present invention, local features of an initial image are input into a local feature modeling model in a ship target association processing model to obtain local features of a target image output by the local feature modeling model, including: inputting local features of the initial image into a local modeling network in the local feature modeling model to obtain local weights input by the local modeling network; wherein the local modeling network is a network for performing correlation modeling on local features of the initial image; inputting local features of the initial image and local weights into a local weighted network in the local feature modeling model to obtain local features of the target image output by the local weighted network; wherein the local weighted network is a network for performing weighted multiplication of local features of the initial image and local weights.

[0013] According to a target association processing method for ship images provided by the present invention, a ship target association processing model is trained in the following manner: obtaining a ship query sample image and a ship image sample database; inputting the ship query sample image and the ship image sample database into a basic ship target association processing model to obtain a target association sample processing result output by the basic ship target association processing model; calculating a model loss function according to the target association sample processing result; and updating the network parameters of the basic ship target association processing model according to the model loss function to obtain a ship target association processing model.

[0014] The present invention also provides a target association processing device for ship images, comprising the following modules.

[0015] The image acquisition module is used to acquire a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices.

[0016] The association processing module is used to input the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0017] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned target association processing methods for ship images is implemented.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the target association processing method for ship images as described above is implemented.

[0019] The present invention also provides a computer program product, comprising a computer program, which implements any of the above-mentioned target association processing methods for ship images when the computer program is executed by a processor.

[0020] The present invention provides a method, device, equipment, medium and product for target association processing of ship images, which obtains a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices; the ship query image and the ship image database are input into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features. The technical solution of the present invention is used to solve the defect in the prior art that the correlation accuracy of the ship image is determined only based on the global features, which ignores the relevant information between the images of the complete data set, resulting in a decrease in the correlation accuracy of the ship image. It realizes the extraction of global features and local features of the ship query image and the ship image database based on the ship target association processing model, and determines the target association processing results corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features, thereby improving the correlation accuracy of the ship query image and the ship image database. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a flow chart of the target association processing method of the ship image provided by the present invention.

[0023] Figure 2 It is a structural schematic diagram of the target association processing device for ship images provided by the present invention.

[0024] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Combine the following Figure 1 The target association processing method for ship images provided by the present invention is described. The target association processing method for ship images provided by the present invention can be applicable to the case of ship target association processing in which global and local correlation features are fused. The executor of the method can be an electronic device, or a target association processing device for ship images arranged in the electronic device. The target association processing device for ship images can be implemented by software, hardware or a combination of both. Figure 1 FIG. 1 is a flow chart of a target association processing method for ship images provided by the present invention. Figure 1 As shown, the method includes the following steps 101 and 102.

[0027] Step 101: Obtain a ship query image and a ship image database.

[0028] In this step, the ship query image is a ship image of a target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices.

[0029] Specifically, a ship query image of a target ship for association processing and a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices are obtained.

[0030] Step 102: input the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model.

[0031] In this step, the ship target association processing model is trained based on the acquired ship query sample images and ship image sample database. The ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing results corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0032] Specifically, the ship query image and the ship image database are input into the ship target association processing model, and the global features and local features of the ship query image and the ship image database are extracted through the ship target association processing model, and the target association processing results corresponding to the ship query image and the ship image database are determined based on the fusion processing of the global features and the local features. The target association processing results output by the ship target association processing model are used to represent the retrieval of images with the same identity as the ship query image from the ship image database, and to identify the identity of the ship in the ship query image and its activity trajectory.

[0033] In a specific implementation, a ship query image and a ship image database are input into a ship target association processing model to obtain a target association processing result output by the ship target association processing model, including: inputting the ship query image and the ship image database into an initial feature extraction model in the ship target association processing model to obtain initial image global features and initial image local features output by the initial feature extraction model; wherein the initial feature extraction model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global features into a global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model; wherein the global feature modeling model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global features into a global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model; wherein the global feature modeling model is a network model for extracting features from the ship query image and the ship image database; The model is a model that models the feature correlation of the global features of the initial image based on global attention; the local features of the initial image are input into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model; wherein the local feature modeling model is a model that models the feature correlation of the local features of the initial image based on the nearest local attention; the global features of the target image and the local features of the target image are input into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model; wherein the feature correlation fusion model is a model that fuses the global features of the target image and the local features of the target image.

[0034] In this step, the initial feature extraction model refers to a model that can extract global features and local features of the ship query image and the ship image database that have not been subjected to correlation modeling. The initial feature extraction model can be, for example, ViT (Vision Transformer), which is not limited in this embodiment.

[0035] The global features of the initial image include global features corresponding to the ship query image and global features corresponding to the ship image database, and the local features of the initial image include local features corresponding to the ship query image and local features corresponding to the ship image database.

[0036] For example, ViT can be composed of L blocks, where L is set to 3 in the experiment. Each block contains global perception attention, local perception attention and a feedforward network. The global and local perception attention are used to extract global and local features, respectively.

[0037] Specifically, the ship query image and the ship image database are input into the initial feature extraction model in the ship target association processing model, and the ship query image and the ship image database are respectively extracted by the initial feature extraction model to obtain the initial image global features and initial image local features output by the initial feature extraction model.

[0038] For example, in the initial feature extraction model, the input ship query image and the ship image database are segmented by a slice embedding module, for example, the image , Represents an image, represents the shape of the image, Indicates the height of the image. Indicates the width of the image. Indicates the length of the image. Split into N non-overlapping slices, each slice is considered an "image token" In addition, a learnable "class tocken" is introduced and three “part tokens” Connect with all "imagetoken". To extract the initial image global features, use "class tocken" and all "image token" to perform global perception attention, and then send the output to the feedforward network. Repeat this process M times to get an initial image global feature In order to learn local similarities from the data itself, it is necessary to use local-aware attention to extract the initial image local features. For each “part token” , this study used And the “imagetoken” belonging to a specific area to perform local perception attention. That is, the “part token” will only interact with the “imagetoken” in a specific area. Repeat this process L times, and you will get three initial image local features The number of local features of the initial image is at least one, which is not limited in this embodiment.

[0039] In a specific embodiment, before the global features of the initial image are input into the global feature modeling model in the ship target association processing model to obtain the global features of the target image output by the global feature modeling model, it also includes: performing a linear transformation on the global features of the initial image to obtain the global linear projection features corresponding to the global features of the initial image.

[0040] Specifically, before inputting the global features of the initial image into the global feature modeling model in the ship target association processing model to obtain the global features of the target image output by the global feature modeling model, the global features of the initial image are linearly transformed to obtain the global linear projection features corresponding to the global features of the initial image.

[0041] In a specific embodiment, the global features of the initial image are input into the global feature modeling model in the ship target association processing model to obtain the global features of the target image output by the global feature modeling model, including: inputting the global linear projection features into the dimension reduction network in the global feature modeling model to obtain the global features of the reduced dimension image output by the dimension reduction network; wherein the dimension reduction network is a network that performs dimension reduction processing on the global features of the initial image; inputting the global features of the reduced dimension image into the global modeling network in the global feature modeling model to obtain the global weights output by the global modeling network; wherein the global modeling network is a network that performs correlation modeling on the global features of the reduced dimension image; inputting the global linear projection features and the global weights into the global weighted network in the global feature modeling model to obtain the global features of the target image output by the global weighted network; wherein the global weighted network is a network that performs weighted multiplication of the global linear projection features and the global weights.

[0042] Specifically, in order to reduce the complexity of the operation, the global linear projection features are input into the dimension reduction network in the global feature modeling model, and the attention algorithm of the landmark agent is introduced to perform dimension reduction processing to obtain the global features of the dimension reduction image output by the dimension reduction network; then the global features of the dimension reduction image are input into the global modeling network in the global feature modeling model, and the correlation modeling is performed through the affinity matrix in the global modeling network to determine the correlation, and the correlation is converted into weights by introducing the Softmax function to obtain the global weights output by the global modeling network; finally, the global linear projection features and the global weights are input into the global weighted network in the global feature modeling model, and the global linear projection features and the global weights are multiplied by the global weighted network to obtain the global features of the target image output by the global weighted network.

[0043] In a specific embodiment, the local features of the initial image are input into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model, including: inputting the local features of the initial image into the local modeling network in the local feature modeling model to obtain the local weights input by the local modeling network; wherein the local modeling network is a network for correlation modeling of the local features of the initial image; inputting the local features of the initial image and the local weights into the local weighted network in the local feature modeling model to obtain the local features of the target image output by the local weighted network; wherein the local weighted network is a network for weighted multiplication of the local features of the initial image and the local weights.

[0044] Specifically, the local features of the initial image are input into the local modeling network in the local feature modeling model, a dynamic repository is established through the local modeling network to select the corresponding positive samples, and then the correlation modeling is obtained based on all the positive samples and calculated based on the affinity matrix, and the correlation between samples is determined, and then the clustering loss is introduced, and the correlation between samples is converted according to the clustering loss to obtain the local weights input into the local modeling network, and determine whether the local weights meet the preset weights. If the local weights do not meet the preset weights, the update is continued and the execution is returned to establish the dynamic repository through the local modeling network to select the corresponding positive samples. If the local weights meet the preset weights, the local features of the initial image and the local weights are input into the local weighted network in the local feature modeling model, and the local features of the initial image and the local weights are weighted multiplied by the local weighted network to obtain the local features of the target image output by the local weighted network.

[0045] In a specific embodiment, after obtaining the global features and local features of the target image, the global features and local features of the target image are input into a feature correlation fusion model in a ship target association processing model, and the feature correlation fusion model uses a self-distillation method to perform feature fusion processing on the global features and local features of the target image to obtain a target association processing result output by the feature correlation fusion model; wherein the feature correlation fusion model is a model that performs fusion processing on the global features and local features of the target image.

[0046] Specifically, after obtaining the global features and local features of the target image, the global features and local features of the target image are input into the feature correlation fusion model in the ship target association processing model, and the feature correlation fusion model adopts the self-distillation method to perform feature fusion processing on the global features and local features of the target image to obtain the target association processing result output by the feature correlation fusion model. Based on the target association processing result, the correlation between the ship query image and the ship image database can be determined, thereby improving the accuracy and precision of the correlation and solving the problem of local loss of ship association.

[0047] In a specific embodiment, the ship target association processing model is trained in the following manner: obtaining ship query sample images and a ship image sample database; inputting the ship query sample images and the ship image sample database into the basic ship target association processing model to obtain the target association sample processing results output by the basic ship target association processing model; calculating the model loss function according to the target association sample processing results; updating the network parameters of the basic ship target association processing model according to the model loss function to obtain the ship target association processing model.

[0048] Specifically, the ship target association processing model is trained in the following manner: obtaining a ship query sample image and a ship image sample database; inputting the ship query sample image and the ship image sample database into the basic ship target association processing model, extracting sample global features and sample local features of the ship query sample image and the ship image sample database through the basic ship target association processing model, and performing correlation modeling on the sample global features and the sample local features, and at the same time, according to the correlation modeling, obtaining the target association sample processing result output by the basic ship target association processing model, the target association sample processing result includes the target sample global feature result and the target sample local feature result, and then determining the triple loss function according to the target sample global feature, determining the soft label according to the target sample local feature, and determining the identity loss function according to the soft label, and finally, updating the network parameters of the basic ship target association processing model according to the triple loss function and the identity loss function, thereby obtaining the ship target association processing model.

[0049] Exemplarily, during the training of the basic ship target association processing model to obtain the ship target association processing model, for the basic ship target association processing model Perform learning and calculate the features of each input ship query sample image and ship image sample database During testing of the trained base ship-target association processing model, a query set focusing on a specific target , and the gallery set used for retrieval ,Will The query image in The query set is used to compare the images in the gallery to retrieve images of a specific identity. and training set The image identities in do not overlap, that is, In the above, for the function There is no restriction on the form of . Since most studies are usually calculated on a single input image, they ignore the possible relationship between the features of the same individuals across platforms and scenes. In order to explicitly explain this relationship, a function is introduced during the training process to obtain the global feature results of the target sample . That is, the global feature result of the target sample As shown in formula (1).

[0050] (1) in, Represents the sample global linear projection features corresponding to all input ship query sample images and ship image sample databases, It is through the feature extraction function The representation vector obtained. During training, is a large batch of samples sampled from the training set, and during testing, Contains all images from the ship query sample image and ship image sample databases. yes and The learnable weights between is a constant coefficient. Using self-attention, Formula (1) is modified into Formula (2).

[0051] (2) in, is any pair of input representation vectors and The affinity matrix of similarities between is the softmax function that converts affinity into weights, and is the linear projection function.

[0052] Among them, for the affinity matrix As shown in formula (3).

[0053] (3) in, and are two global linear projection features that can represent the input vector Mapping to query and key matrices . is the dimension of the representation vector, Usually it is the inner product function.

[0054] The advantage of this setting to obtain the global feature results of the target sample is that it reduces the computational burden of the affinity matrix through dimensionality reduction processing, and reduces the noise correlation between unrelated samples during weighted multiplication by introducing the Softmax function, thereby improving the efficiency of weighted processing.

[0055] After obtaining the global feature results of the target sample, the model loss function is calculated according to the global feature results of the target sample. The model loss function includes a triple loss function and an identity loss function. The triple loss function is shown in formula (4).

[0056] (4) in, represents the total number of slice samples for the ship query sample image and the ship image sample database, Represents the anchor sample feature in the global feature result of the target sample, Represents the positive sample features in the global feature results of the target sample, which are of the same category as the anchor sample. Represents the negative sample features in the global feature results of the target sample, which are different from the categories of the anchor sample. Represents the interval parameter, which controls the distance between positive and negative samples.

[0057] Furthermore, to obtain the local feature results of the target sample, it is necessary to first maintain a momentum updated repository , the repository can be expressed as shown in formula (5).

[0058] (5) in, represents the training cycle for the basic ship target association processing model training, represents momentum, . Represents the number of all samples of the ship query sample image and the ship image sample database. For each local feature in the current mini-batch , comparing it to the entire memory bank, from Select The closest The local features of Then, the positive samples and The distance between them is minimized by clustering loss. The formula is shown in formula (6).

[0059] (6) in, represents the temperature coefficient, minimizing the clustering loss The basic ship target association processing model is encouraged to separate similar parts in the feature space Pull to near, while pushing dissimilar parts away In this way, the basic ship target association processing model can learn those visually similar parts in different samples, and make the basic ship target association processing model pay attention to the areas where these useful information are located, so as to obtain the final target sample local feature results.

[0060] Furthermore, after obtaining the target sample global feature result and the target sample local feature result, the target sample global feature result and the target sample local feature result are further fused by a self-distillation method. have positive samples, for global features, there are positive samples, which will correspond to this Identity document (ID) of each component Treated as similar ID. Soft label can be constructed as formula (7).

[0061] (7) in, represents the weight of similar categories, express Middle The number of IDs, express That is, the more similar parts there are, the greater the probability that these parts belong to this ID.

[0062] So according to the soft label Get the identity loss function As shown in formula (8).

[0063] (8) in, represents a hard tag, represents the coefficient used to balance the soft labels, Represents a classifier that predicts the probability distribution on the source dataset. Since the similarity at this time is obtained by comparing the local features of the current sample with the entire dataset, the identity loss function It reflects the similarity between IDs better than the results of the classifier using traditional classification loss.

[0064] Further, we get the triple loss function and identity loss function Afterwards, the network parameters of the basic ship target association processing model are further updated according to the triple loss function and the identity loss function to obtain the updated ship target association processing model.

[0065] The present invention provides a target association processing method for ship images, which comprises obtaining a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, different platforms and different devices; the ship query image and the ship image database are input into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is obtained by training based on the obtained ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features. On the basis of the above-mentioned embodiments, the technical solution of the present invention is used to solve the defect in the prior art that the correlation accuracy of the ship image is determined only based on the global features, which ignores the relevant information between the images of the complete data set, resulting in a decrease in the correlation accuracy of the ship image. It realizes the extraction of global features and local features of the ship query image and the ship image database based on the ship target association processing model, determines the target association processing results corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features, and improves the correlation accuracy of the ship query image and the ship image database.

[0066] The target association processing device for ship images provided by the present invention is described below. The target association processing device for ship images described below and the target association processing method for ship images described above can be referred to each other.

[0067] Figure 2 is a schematic diagram of the structure of the target association processing device for ship images provided by the present invention, referring to Figure 2 As shown, the target association processing device 200 for ship images includes: an image acquisition module 201 and an association processing module 202 .

[0068] The image acquisition module 201 is used to acquire a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices.

[0069] The association processing module 202 is used to input the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0070] In an exemplary embodiment, the association processing module 202 is specifically used to: input the ship query image and the ship image database into the initial feature extraction model in the ship target association processing model, and obtain the initial image global features and initial image local features output by the initial feature extraction model; wherein the initial feature extraction model is a network model that extracts features from the ship query image and the ship image database; input the initial image global features into the global feature modeling model in the ship target association processing model, and obtain the target image global features output by the global feature modeling model; wherein the global feature modeling model is a model that performs feature correlation modeling on the initial image global features based on global attention; input the initial image local features into the local feature modeling model in the ship target association processing model, and obtain the target image local features output by the local feature modeling model; wherein the local feature modeling model is a model that performs feature correlation modeling on the initial image local features based on neighboring local attention; input the target image global features and the target image local features into the feature correlation fusion model in the ship target association processing model, and obtain the target association processing result output by the feature correlation fusion model; wherein the feature correlation fusion model is a model that performs fusion processing on the target image global features and the target image local features.

[0071] In an exemplary embodiment, the device further includes a linear transformation module. The linear transformation module is used to: before inputting the initial image global features into the global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model, linearly transform the initial image global features to obtain the global linear projection features corresponding to the initial image global features.

[0072] In an exemplary embodiment, the association processing module 202 inputs the global features of the initial image into the global feature modeling model in the ship target association processing model to obtain the global features of the target image output by the global feature modeling model, and is specifically used for: inputting the global linear projection features into the dimension reduction network in the global feature modeling model to obtain the global features of the reduced dimension image output by the dimension reduction network; wherein the dimension reduction network is a network that performs dimension reduction processing on the global features of the initial image; inputting the global features of the reduced dimension image into the global modeling network in the global feature modeling model to obtain the global weights output by the global modeling network; wherein the global modeling network is a network that performs correlation modeling on the global features of the reduced dimension image; inputting the global linear projection features and the global weights into the global weighted network in the global feature modeling model to obtain the global features of the target image output by the global weighted network; wherein the global weighted network is a network that performs weighted multiplication of the global linear projection features and the global weights.

[0073] In an exemplary embodiment, the association processing module 202 inputs the local features of the initial image into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model, and is specifically used for: inputting the local features of the initial image into the local modeling network in the local feature modeling model to obtain the local weights input by the local modeling network; wherein the local modeling network is a network for correlation modeling of the local features of the initial image; inputting the local features of the initial image and the local weights into the local weighted network in the local feature modeling model to obtain the local features of the target image output by the local weighted network; wherein the local weighted network is a network for weighted multiplication of the local features of the initial image and the local weights.

[0074] In an exemplary embodiment, the device further includes: a model training module. The model training module is used to: obtain a ship query sample image and a ship image sample database; input the ship query sample image and the ship image sample database into a basic ship target association processing model to obtain a target association sample processing result output by the basic ship target association processing model; calculate a model loss function according to the target association sample processing result; and update a network parameter of the basic ship target association processing model according to the model loss function to obtain a ship target association processing model.

[0075] The device of this embodiment can be used to execute the method of any embodiment in the target association processing method side embodiment of the ship image. Its specific implementation process and technical effects are similar to those in the target association processing method side embodiment of the ship image. For details, please refer to the detailed description in the target association processing method side embodiment of the ship image, which will not be repeated here.

[0076] Figure 3 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 3As shown, the electronic device may include: a processor 310 , a communications interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communications interface 320 , and the memory 330 communicate with each other via the communication bus 340 . The processor 310 can call the logic instructions in the memory 330 to execute the target association processing method of the ship image, which includes: obtaining a ship query image and a ship image database; wherein the ship query image is a ship image of the target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices; inputting the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0077] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0078] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target association processing method of the ship image provided by the above methods, the method including: obtaining a ship query image and a ship image database; wherein the ship query image is a ship image of the target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices; inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is obtained by training based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing results corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0079] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the target association processing method of the ship image provided by the above-mentioned methods, the method comprising: obtaining a ship query image and a ship image database; wherein the ship query image is a ship image of the target ship used for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices; inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing results corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for object association processing of ship images, characterized in that: include: Acquire a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms, and different devices; The ship query image and the ship image database are input into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on a fusion processing of the global features and the local features.

2. The target association processing method of ship images according to claim 1 is characterized in that: The step of inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model comprises: Input the ship query image and the ship image database into the initial feature extraction model in the ship target association processing model to obtain the initial image global features and initial image local features output by the initial feature extraction model; wherein the initial feature extraction model is a network model that extracts features from the ship query image and the ship image database; Inputting the global features of the initial image into the global feature modeling model in the ship target association processing model, and obtaining the global features of the target image output by the global feature modeling model; wherein the global feature modeling model is a model for performing feature correlation modeling on the global features of the initial image based on global attention; Inputting the local features of the initial image into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model; wherein the local feature modeling model is a model for performing feature correlation modeling on the local features of the initial image based on neighboring local attention; The target image global features and the target image local features are input into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model; wherein the feature correlation fusion model is a model for fusing the target image global features and the target image local features.

3. The target association processing method of ship images according to claim 2 is characterized in that: Before inputting the initial image global features into the global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model, the method further includes: Performing a linear transformation on the global features of the initial image to obtain global linear projection features corresponding to the global features of the initial image.

4. The target association processing method of ship images according to claim 3 is characterized in that: The step of inputting the initial image global features into the global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model comprises: Inputting the global linear projection features into the dimension reduction network in the global feature modeling model to obtain the dimension reduction image global features output by the dimension reduction network; wherein the dimension reduction network is a network that performs dimension reduction processing on the initial image global features; Inputting the global features of the reduced-dimensional image into a global modeling network in the global feature modeling model to obtain a global weight output by the global modeling network; wherein the global modeling network is a network for performing correlation modeling on the global features of the reduced-dimensional image; The global linear projection features and the global weights are input into the global weighted network in the global feature modeling model to obtain the global features of the target image output by the global weighted network; wherein the global weighted network is a network that performs weighted multiplication of the global linear projection features and the global weights.

5. The target association processing method of ship images according to claim 2 is characterized in that: The step of inputting the local features of the initial image into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model comprises: Inputting the local features of the initial image into the local modeling network in the local feature modeling model to obtain the local weights of the local modeling network input; wherein the local modeling network is a network for performing correlation modeling on the local features of the initial image; The local features of the initial image and the local weights are input into the local weighted network in the local feature modeling model to obtain the local features of the target image output by the local weighted network; wherein the local weighted network is a network that performs weighted multiplication of the local features of the initial image and the local weights.

6. The target association processing method of ship images according to claim 1, characterized in that: The ship target association processing model is trained in the following way: Acquire the ship query sample image and the ship image sample database; Inputting the ship query sample image and the ship image sample database into a basic ship target association processing model to obtain a target association sample processing result output by the basic ship target association processing model; Calculate the model loss function according to the target associated sample processing result; The network parameters of the basic ship target association processing model are updated according to the model loss function to obtain the ship target association processing model.

7. A target association processing device for ship images, characterized in that: include: An image acquisition module, used to acquire a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices; An association processing module is used to input the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is trained based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on a fusion processing of the global features and the local features.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the target association processing method for ship images according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the target association processing method for ship images according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the target association processing method for ship images according to any one of claims 1 to 6 is implemented.

Images