Method, apparatus, device, medium and product for target association processing of ship images

Abstract

The present invention provides a method, device, equipment, medium and product for target association processing of ship images, relating to the technical field of image processing. The method includes: obtaining a ship query image and a ship image database; inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein, the ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features. The technical solution of the present invention extracts global features and local features based on the ship target association processing model, and determines the target association processing result based on the fusion processing of the global features and the local features, thereby improving the association accuracy between the ship query image and the ship image database.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method, device, equipment, medium and product for target association processing of ship images. Background Art

[0002] The ship target association technology is an important part of the intelligent shipping system and also an important part of the visual perception task required for ship detection in sea areas.

[0003] Currently, in the prior art, for the target association of ships, the correlation of the global features of the image is only established within each small batch. By establishing the association of the global features of the image, the association accuracy of the ship image is determined. However, determining the association accuracy of the ship image by establishing the association based on the global features ignores the correlation information between the images in the complete data set, and most of the target association work only considers learning the relationship between samples of the global features, and the learned association information is limited, resulting in a reduction in the association accuracy of the ship image.

[0004] Therefore, there is an urgent need for a method for target association processing of ship images to improve the association accuracy of ship images. Summary of the Invention

[0005] The present invention provides a method, device, equipment, medium and product for target association processing of ship images, which is used to solve the defect that the association accuracy of ship images determined by only establishing the association based on the global features in the prior art ignores the correlation information between the images in the complete data set, resulting in a reduction in the association accuracy of ship images, and realizes the extraction of global features and local features for the ship query image and the ship image database based on the ship target association processing model, determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features, and improves the association accuracy of the ship query image and the ship image database.

[0006] The present invention provides a method for target association processing of ship images, including the following steps.

[0007] Obtain a ship query image and a ship image database; wherein, the ship query image is a ship image of a target ship for which association processing is to be performed, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, different platforms and different devices.

[0008] Input the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model. Among them, the ship target association processing model is trained based on the obtained ship query sample images and the ship image sample database. The ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0009] According to a ship image target association processing method provided by the present invention, inputting the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model includes: inputting the ship query image and the ship image database into the initial feature extraction model in the ship target association processing model to obtain the initial image global feature and the initial image local feature output by the initial feature extraction model. Among them, the initial feature extraction model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model. Among them, the global feature modeling model is a model for performing feature correlation modeling on the initial image global feature based on global attention; inputting the initial image local feature into the local feature modeling model in the ship target association processing model to obtain the target image local feature output by the local feature modeling model. Among them, the local feature modeling model is a model for performing feature correlation modeling on the initial image local feature based on neighbor local attention; inputting the target image global feature and the target image local feature into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model. Among them, the feature correlation fusion model is a model for performing fusion processing on the target image global feature and the target image local feature.

[0010] According to a ship image target association processing method provided by the present invention, before inputting the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model, it further includes: performing a linear transformation on the initial image global feature to obtain the global linear projection feature corresponding to the initial image global feature.

[0011] A method for target association processing of ship images provided by the present invention inputs the global features of the initial image into the global feature modeling model in the ship target association processing model to obtain the global features of the target image output by the global feature modeling model, including: inputting the global linear projection features into the dimensionality reduction network in the global feature modeling model to obtain the dimensionality reduction global features of the image output by the dimensionality reduction network; wherein, the dimensionality reduction network is a network for performing dimensionality reduction processing on the global features of the initial image; inputting the dimensionality reduction global features of the image into the global modeling network in the global feature modeling model to obtain the global weights output by the global modeling network; wherein, the global modeling network is a network for performing correlation modeling on the dimensionality reduction global features of the image; inputting the global linear projection features and the global weights into the global weighted network in the global feature modeling model to obtain the global features of the target image output by the global weighted network; wherein, the global weighted network is a network for performing weighted multiplication on the global linear projection features and the global weights.

[0012] A method for target association processing of ship images provided by the present invention inputs the local features of the initial image into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model, including: inputting the local features of the initial image into the local modeling network in the local feature modeling model to obtain the local weights input by the local modeling network; wherein, the local modeling network is a network for performing correlation modeling on the local features of the initial image; inputting the local features of the initial image and the local weights into the local weighted network in the local feature modeling model to obtain the local features of the target image output by the local weighted network; wherein, the local weighted network is a network for performing weighted multiplication on the local features of the initial image and the local weights.

[0013] A method for training a ship target association processing model provided by the present invention is as follows: obtaining a ship query sample image and a ship image sample database; inputting the ship query sample image and the ship image sample database into the basic ship target association processing model to obtain the target association sample processing result output by the basic ship target association processing model; calculating the model loss function according to the target association sample processing result; updating the network parameters of the basic ship target association processing model according to the model loss function to obtain the ship target association processing model.

[0014] The present invention also provides a device for target association processing of ship images, including the following modules.

[0015] An image acquisition module, configured to acquire a ship query image and a ship image database; wherein, the ship query image is a ship image of a target ship for performing association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms, and different devices.

[0016] An association processing module is configured to input a ship query image and a ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model. The ship target association processing model is trained based on the acquired ship query sample images and the ship image sample database. The ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the target association processing method for any one of the ship images as described above.

[0018] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the target association processing method for any one of the ship images as described above.

[0019] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the target association processing method for any one of the ship images as described above.

[0020] A method, device, equipment, medium and product for target association processing of ship images provided by the present invention, which obtain a ship query image and a ship image database; wherein, the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images obtained by the target ship at different time sequences, different platforms and different devices; input the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein, the ship target association processing model is trained based on the obtained ship query sample images and ship image sample databases, and the ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features. The technical solution of the present invention is used to solve the defect in the prior art that the association accuracy of ship images is determined only based on global features, ignoring the relevant information between images in the complete data set, resulting in a decrease in the association accuracy of ship images, and realizes the extraction of global features and local features from the ship query image and the ship image database based on the ship target association processing model, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features, so as to improve the association accuracy of the ship query image and the ship image database. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 It is a schematic flowchart of the method for target association processing of ship images provided by the present invention.

[0023] Figure 2 It is a schematic structural diagram of the device for target association processing of ship images provided by the present invention.

[0024] Figure 3 It is a schematic structural diagram of the electronic equipment provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0026] The following Figure 1 describes the target association processing method for ship images provided by the present invention. The target association processing method for ship images provided by the present invention is applicable to the situation of ship target association processing with global and local feature correlation fusion. The execution subject of this method can be an electronic device or a target association processing device for ship images set in the electronic device. The target association processing device for ship images can be implemented by software, hardware, or a combination of both. Figure 1 is a schematic flowchart of the target association processing method for ship images provided by the present invention. As Figure 1 shown, the method includes the following steps 101 and 102.

[0027] Step 101: Obtain a ship query image and a ship image database.

[0028] In this step, the ship query image is a ship image of the target ship for which association processing is to be performed, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, on different platforms, and by different devices.

[0029] Specifically, obtain a ship query image of the target ship for which association processing is to be performed and a database containing a large number of ship images of the target ship obtained at different time sequences, on different platforms, and by different devices.

[0030] Step 102: Input the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model.

[0031] In this step, the ship target association processing model is trained based on the obtained ship query sample images and ship image sample databases. The ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and local features.

[0032] Specifically, the ship query image and the ship image database are input into the ship target association processing model. The ship target association processing model extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features. The target association processing result output by the ship target association processing model is used to represent the image retrieved from the ship image database that has the same identity as the ship query image, and to identify the identity of the ship in the ship query image and its activity trajectory.

[0033] In a specific embodiment, the ship query image and the ship image database are input into the ship target association processing model, and the target association processing result output by the ship target association processing model is obtained, including: inputting the ship query image and the ship image database into the initial feature extraction model in the ship target association processing model to obtain the initial image global feature and the initial image local feature output by the initial feature extraction model; wherein, the initial feature extraction model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model; wherein, the global feature modeling model is a model for performing feature correlation modeling on the initial image global feature based on global attention; inputting the initial image local feature into the local feature modeling model in the ship target association processing model to obtain the target image local feature output by the local feature modeling model; wherein, the local feature modeling model is a model for performing feature correlation modeling on the initial image local feature based on neighbor local attention; inputting the target image global feature and the target image local feature into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model; wherein, the feature correlation fusion model is a model for performing fusion processing on the target image global feature and the target image local feature.

[0034] In this step, the initial feature extraction model refers to a model that can extract global features and local features that have not undergone correlation modeling from the ship query image and the ship image database. The initial feature extraction model can be, for example, ViT (Vision Transformer), and this embodiment does not limit this.

[0035] The initial image global feature includes the global feature corresponding to the ship query image and the global feature corresponding to the ship image database, and the initial image local feature includes the local feature corresponding to the ship query image and the local feature corresponding to the ship image database.

[0036] For example, ViT can be composed of L blocks. In the experiment, L is set to 3. Each block contains global perception attention, local perception attention, and a feed-forward network. The global and local perception attentions are used to extract global and local features respectively.

[0037] Specifically, the ship query image and the ship image database are input into the initial feature extraction model in the ship target association processing model. The initial feature extraction model extracts features from the ship query image and the ship image database respectively, and obtains the initial image global feature and the initial image local feature output by the initial feature extraction model.

[0038] Exemplarily, in the initial feature extraction model, the input ship query image and ship image database are segmented by a slicing embedding module. For example, the image , represents a certain image, represents the shape of the image, represents the height of the image, represents the width of the image, represents the length of the image. The image is segmented into N non-overlapping slices, and each slice is regarded as an "image token" . In addition, a learnable "class tocken" and three "part token" are connected to all "imagetoken". To extract the initial image global feature, the "class tocken" and all "image token" are used to perform global perception attention, and then the output is sent to the feed-forward network. Repeat this process M times to obtain an initial image global feature . To learn local similarity from the data itself, local perception attention is needed to extract the initial image local feature. For each "part token" , in this study, and the "image token" belonging to a specific region are used to perform local perception attention. That is, the "part token" will only interact with the "image token" in a specific region. Repeat this process L times to obtain three initial image local features . The number of initial image local features is at least one, and this embodiment does not limit this.

[0039] In a specific embodiment, before inputting the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model, it further includes: performing a linear transformation on the initial image global feature to obtain the global linear projection feature corresponding to the initial image global feature.

[0040] Specifically, before inputting the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model, a linear transformation is performed on the initial image global feature to obtain the global linear projection feature corresponding to the initial image global feature.

[0041] In a specific embodiment, inputting the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model includes: inputting the global linear projection feature into the dimensionality reduction network in the global feature modeling model to obtain the dimensionality-reduced image global feature output by the dimensionality reduction network; wherein, the dimensionality reduction network is a network for performing dimensionality reduction processing on the initial image global feature; inputting the dimensionality-reduced image global feature into the global modeling network in the global feature modeling model to obtain the global weight output by the global modeling network; wherein, the global modeling network is a network for performing correlation modeling on the dimensionality-reduced image global feature; inputting the global linear projection feature and the global weight into the global weighting network in the global feature modeling model to obtain the target image global feature output by the global weighting network; wherein, the global weighting network is a network for performing weighted multiplication on the global linear projection feature and the global weight.

[0042] Specifically, in order to reduce the operation complexity, the global linear projection feature is input into the dimensionality reduction network in the global feature modeling model, and the attention algorithm of the landmark proxy is introduced for dimensionality reduction processing to obtain the dimensionality-reduced image global feature output by the dimensionality reduction network; then the dimensionality-reduced image global feature is input into the global modeling network in the global feature modeling model, and correlation modeling is performed through the affinity matrix in the global modeling network to determine the correlation, and the Softmax function is introduced to convert the correlation into a weight to obtain the global weight output by the global modeling network. Finally, the global linear projection feature and the global weight are input into the global weighting network in the global feature modeling model, and the global linear projection feature and the global weight are multiplied by the global weighting network to obtain the target image global feature output by the global weighting network.

[0043] In a specific embodiment, the local features of the initial image are input into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model, including: inputting the local features of the initial image into the local modeling network in the local feature modeling model to obtain the local weights input into the local modeling network; wherein, the local modeling network is a network for performing correlation modeling on the local features of the initial image; inputting the local features of the initial image and the local weights into the local weighted network in the local feature modeling model to obtain the local features of the target image output by the local weighted network; wherein, the local weighted network is a network for performing weighted multiplication on the local features of the initial image and the local weights.

[0044] Specifically, input the local features of the initial image into the local modeling network in the local feature modeling model, select the corresponding positive samples by establishing a dynamic repository through the local modeling network, then calculate the correlation modeling based on the affinity matrix according to all positive samples, and determine the correlation between samples. Then introduce the clustering loss, convert the correlation between samples according to the clustering loss to obtain the local weights input into the local modeling network, and determine whether the local weights meet the preset weights. In the case that the local weights do not meet the preset weights, continue to update and return to execute the step of selecting the corresponding positive samples by establishing a dynamic repository through the local modeling network. In the case that the local weights meet the preset weights, input the local features of the initial image and the local weights into the local weighted network in the local feature modeling model, and perform weighted multiplication on the local features of the initial image and the local weights through the local weighted network to obtain the local features of the target image output by the local weighted network.

[0045] In a specific embodiment, after obtaining the global features of the target image and the local features of the target image, input the global features of the target image and the local features of the target image into the feature correlation fusion model in the ship target association processing model, and perform feature fusion processing on the global features of the target image and the local features of the target image by using the self-distillation method through the feature correlation fusion model to obtain the target association processing result output by the feature correlation fusion model; wherein, the feature correlation fusion model is a model for performing fusion processing on the global features of the target image and the local features of the target image.

[0046] Specifically, after obtaining the global features of the target image and the local features of the target image, input the global features of the target image and the local features of the target image into the feature correlation fusion model in the ship target association processing model, and perform feature fusion processing on the global features of the target image and the local features of the target image by using the self-distillation method through the feature correlation fusion model to obtain the target association processing result output by the feature correlation fusion model. Based on the target association processing result, the correlation between the ship query image and the ship image database can be determined, the accuracy and precision of the correlation can be improved, and at the same time, the problem of local loss of ship association can be solved.

[0047] In a specific embodiment, the ship target association processing model is trained in the following manner: obtaining a ship query sample image and a ship image sample database; inputting the ship query sample image and the ship image sample database into the basic ship target association processing model to obtain a target association sample processing result output by the basic ship target association processing model; calculating a model loss function according to the target association sample processing result; and updating the network parameters of the basic ship target association processing model according to the model loss function to obtain the ship target association processing model.

[0048] Specifically, the ship target association processing model is trained in the following manner: obtaining a ship query sample image and a ship image sample database; inputting the ship query sample image and the ship image sample database into the basic ship target association processing model, extracting sample global features and sample local features of the ship query sample image and the ship image sample database through the basic ship target association processing model, performing correlation modeling on the sample global features and the sample local features, and simultaneously obtaining, according to the correlation modeling, a target association sample processing result output by the basic ship target association processing model, where the target association sample processing result includes a target sample global feature result and a target sample local feature result. Then, a triple loss function is determined according to the target sample global feature, a soft label is determined through the target sample local feature, and an identity loss function is determined according to the soft label. Finally, the network parameters of the basic ship target association processing model are updated according to the triple loss function and the identity loss function, thereby obtaining the ship target association processing model.

[0049] Exemplarily, during the training of obtaining the ship target association processing model by training the basic ship target association processing model, for the basic ship target association processing model learning is performed to calculate the features of each input ship query sample image and ship image sample database . During the testing of the trained basic ship target association processing model, a query set focusing on a specific target, and a gallery set for retrieval are considered. The query images in are compared with the gallery images in to retrieve images of a specific identity. The identities of the images in the query set and the training set do not overlap, that is, . In the above, there is no restriction on the form of the function . Since most studies are usually calculated on a single input image, it ignores the possible relationships that may occur between the features of the same individual across platforms and scenarios. To clearly explain this relationship, a function is introduced during the training process to obtain the target sample global feature result That is, the global feature result of the target sample As shown in formula (1).

[0050] (1)

[0051] Wherein, represents the global linear projection features corresponding to all input ship query sample images and the ship image sample database, is the characterization vector obtained through the feature extraction function During training, is a large number of samples sampled from the training set, while during testing, contains all images from the ship query sample images and the ship image sample database. is and the learnable weight between, is a constant coefficient. Using self-attention, formula (1) is modified to formula (2).

[0052] (2)

[0053] Wherein, is the affinity matrix containing the similarity between any two pairs of input characterization vectors and is the softmax function that converts the affinity into a weight, and is the linear projection function.

[0054] Wherein, for the affinity matrix As formula (3).

[0055] (3)

[0056] Wherein, and are two global linear projection features, which can map the input representation vector to the query and key matrices . is the dimension of the representation vector, is usually the inner product function.

[0057] The advantage of setting like this to obtain the global feature result of the target sample is that through dimensionality reduction processing, the computational burden of the affinity matrix is reduced, and by introducing the Softmax function, the noise correlation between irrelevant samples during weighted multiplication is reduced, and the efficiency of weighted processing is improved.

[0058] After obtaining the global feature results of the target samples, the model loss function is calculated based on the global feature results of the target samples. The model loss function includes a triple loss function and an identity loss function. The triple loss function is shown in formula (4).

[0059] (4)

[0060] Among them, represents the total number of sliced samples of the ship query sample image and the ship image sample database, represents the anchor sample feature in the global feature results of the target samples, represents the positive sample feature in the global feature results of the target samples, which is of the same category as the anchor sample. represents the negative sample feature in the global feature results of the target samples, which is of a different category from the anchor sample. represents the margin parameter, which controls the distance between positive and negative samples.

[0061] Furthermore, for obtaining the local feature results of the target samples, it is first necessary to maintain a momentum-updated repository , and the repository can be expressed as shown in formula (5).

[0062] (5)

[0063] Among them, represents the training cycle for training the basic ship target association processing model, represents the momentum, . represents the total number of all samples of the ship query sample image and the ship image sample database. For each local feature in the current mini-batch , it is compared with the entire storage body, and is selected from the closest local features to form a set of positive samples . Then, the distance between the positive samples and is minimized through the clustering loss, and the formula for the clustering loss is shown in formula (6).

[0064] (6)

[0065] Among them, represents the temperature coefficient. Minimizing the clustering loss encourages the basic ship target association processing model to pull the similar parts in the feature space closer to , while pushing the dissimilar parts away from . In this way, the basic ship target association processing model can learn those visually similar parts in different samples and make the basic ship target association processing model notice the regions where this useful information is located, so as to obtain the final local feature results of the target sample.

[0066] Furthermore, after obtaining the global feature results of the target sample and the local feature results of the target sample, the global feature results of the target sample and the local feature results of the target sample are further fused by the method of self-distillation. Specifically, the obtained local feature results of the target sample have positive samples. For the global features, there are a total of positive samples. The identity document (ID) numbers corresponding to these components are regarded as similar IDs. The soft labels can be constructed as formula (7).

[0067] (7)

[0068] Among them, represents the weight of the similar category, represents the th number of the ID in represents the true value label of. That is to say, the more similar parts there are, the greater the probability that these parts belong to this ID.

[0069] Thus, according to the soft label the identity loss function is obtained as shown in formula (8).

[0070] (8)

[0071] Among them, represents the hard label, represents the coefficient used to balance the soft label, represents the classifier for the probability distribution on the predicted source dataset. Since the similarity at this time is obtained by comparing the local features of the current sample with the entire dataset, therefore, the identity loss function reflects the similarity between IDs better than the results of the traditional classification loss classifier.

[0072] Furthermore, after obtaining the triple loss function and the identity loss function After that, the network parameters of the basic ship target association processing model are further updated according to the triple loss function and the identity loss function, so as to obtain the updated ship target association processing model.

[0073] A method for target association processing of ship images provided by the present invention includes obtaining a ship query image and a ship image database; wherein, the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, different platforms, and different devices; inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein, the ship target association processing model is trained based on the obtained ship query sample images and ship image sample databases, and the ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and local features. On the basis of the above embodiments, the technical solution of the present invention is used to solve the defect in the prior art that only global features are used to establish relevance to determine the association accuracy of ship images, ignoring the relevant information between images in the complete data set, resulting in a decrease in the association accuracy of ship images. It realizes extracting global features and local features from the ship query image and the ship image database based on the ship target association processing model, and determining the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and local features, thereby improving the association accuracy of the ship query image and the ship image database.

[0074] The ship image target association processing device provided by the present invention will be described below. The ship image target association processing device described below can be correspondingly referred to the ship image target association processing method described above.

[0075] Figure 2 It is a schematic structural diagram of the ship image target association processing device provided by the present invention. Refer to Figure 2 As shown, the ship image target association processing device 200 includes an image acquisition module 201 and an association processing module 202.

[0076] The image acquisition module 201 is used to obtain a ship query image and a ship image database; wherein, the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, different platforms, and different devices.

[0077] The association processing module 202 is configured to input the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model. The ship target association processing model is trained based on the obtained ship query sample images and the ship image sample database. The ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0078] In an exemplary embodiment, the association processing module 202 is specifically configured to: input the ship query image and the ship image database into the initial feature extraction model in the ship target association processing model to obtain the initial image global feature and the initial image local feature output by the initial feature extraction model. The initial feature extraction model is a network model for extracting features from the ship query image and the ship image database. Input the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model. The global feature modeling model is a model for performing feature correlation modeling on the initial image global feature based on global attention. Input the initial image local feature into the local feature modeling model in the ship target association processing model to obtain the target image local feature output by the local feature modeling model. The local feature modeling model is a model for performing feature correlation modeling on the initial image local feature based on neighbor local attention. Input the target image global feature and the target image local feature into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model. The feature correlation fusion model is a model for performing fusion processing on the target image global feature and the target image local feature.

[0079] In an exemplary embodiment, the apparatus further includes: a linear transformation module. The linear transformation module is configured to perform a linear transformation on the initial image global feature to obtain the global linear projection feature corresponding to the initial image global feature before inputting the initial image global feature into the global feature modeling model in the ship target association processing model to obtain the target image global feature output by the global feature modeling model.

[0080] In an exemplary embodiment, the association processing module 202 inputs the initial image global features into the global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model, specifically for: inputting the global linear projection features into the dimensionality reduction network in the global feature modeling model to obtain the dimensionality reduction image global features output by the dimensionality reduction network; wherein, the dimensionality reduction network is a network for performing dimensionality reduction processing on the initial image global features; inputting the dimensionality reduction image global features into the global modeling network in the global feature modeling model to obtain the global weights output by the global modeling network; wherein, the global modeling network is a network for performing correlation modeling on the dimensionality reduction image global features; inputting the global linear projection features and the global weights into the global weighted network in the global feature modeling model to obtain the target image global features output by the global weighted network; wherein, the global weighted network is a network for performing weighted multiplication on the global linear projection features and the global weights.

[0081] In an exemplary embodiment, the association processing module 202 inputs the initial image local features into the local feature modeling model in the ship target association processing model to obtain the target image local features output by the local feature modeling model, specifically for: inputting the initial image local features into the local modeling network in the local feature modeling model to obtain the local weights input by the local modeling network; wherein, the local modeling network is a network for performing correlation modeling on the initial image local features; inputting the initial image local features and the local weights into the local weighted network in the local feature modeling model to obtain the target image local features output by the local weighted network; wherein, the local weighted network is a network for performing weighted multiplication on the initial image local features and the local weights.

[0082] In an exemplary embodiment, the device further includes: a model training module. The model training module is used for: obtaining a ship query sample image and a ship image sample database; inputting the ship query sample image and the ship image sample database into the basic ship target association processing model to obtain the target association sample processing result output by the basic ship target association processing model; calculating a model loss function according to the target association sample processing result; updating the network parameters of the basic ship target association processing model according to the model loss function to obtain the ship target association processing model.

[0083] The device in this embodiment can be used to execute the method in any one of the method embodiments of the ship image target association processing method. The specific implementation process and technical effects are similar to those in the method embodiments of the ship image target association processing method. Specifically, reference can be made to the detailed introduction in the method embodiments of the ship image target association processing method, which will not be elaborated here.

[0084] Figure 3 is a schematic structural diagram of the electronic device provided by the present invention, as Figure 3As shown in the figure, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340. Among them, the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logical instructions in the memory 330 to execute the target association processing method for ship images. The method includes: obtaining a ship query image and a ship image database; where the ship query image is the ship image of the target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, on different platforms, and by different devices; inputting the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model; where the ship target association processing model is trained based on the obtained ship query sample images and the ship image sample database, and the ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0085] In addition, when the logical instructions in the above-mentioned memory 330 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0086] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target association processing method for ship images provided by the above-mentioned various methods. The method includes: obtaining a ship query image and a ship image database; wherein, the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, on different platforms, and by different devices; inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein, the ship target association processing model is trained based on the obtained ship query sample images and ship image sample databases, and the ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0087] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the target association processing method for ship images provided by the above-mentioned various methods. The method includes: obtaining a ship query image and a ship image database; wherein, the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship obtained at different time sequences, on different platforms, and by different devices; inputting the ship query image and the ship image database into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein, the ship target association processing model is trained based on the obtained ship query sample images and ship image sample databases, and the ship target association processing model is a model that extracts global features and local features from the ship query image and the ship image database, and determines the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for object association processing of ship images, characterized in that: include: Acquire a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms, and different devices; The ship query image and the ship image database are input into a ship target association processing model to obtain a target association processing result output by the ship target association processing model; wherein the ship target association processing model is obtained by training based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on a fusion process of the global features and the local features; the ship query image and the ship image database are input into the ship target association processing model to obtain The target association processing result output by the ship target association processing model includes: inputting the ship query image and the ship image database into an initial feature extraction model in the ship target association processing model to obtain the initial image global features and initial image local features output by the initial feature extraction model; wherein the initial feature extraction model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global features into a global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model; wherein the global feature modeling model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global features into a global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model; wherein the global feature modeling model is a network model for extracting features from the initial image global features based on global attention. The invention relates to a model for feature correlation modeling; inputting the local features of the initial image into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model; wherein the local feature modeling model is a model for feature correlation modeling of the local features of the initial image based on the nearest local attention; inputting the global features of the target image and the local features of the target image into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model; wherein the feature correlation fusion model is a model for fusion processing of the global features of the target image and the local features of the target image; Inputting the local features of the initial image into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model includes: inputting the local features of the initial image into the local modeling network in the local feature modeling model to obtain the local weights input by the local modeling network; wherein the local modeling network is a network for correlation modeling of the local features of the initial image; inputting the local features of the initial image and the local weights into the local weighted network in the local feature modeling model to obtain the local features of the target image output by the local weighted network; wherein the local weighted network is a network for weighted multiplication of the local features of the initial image and the local weights.

2. The target association processing method of ship images according to claim 1 is characterized in that: Before inputting the initial image global features into the global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model, the method further includes: Performing a linear transformation on the global features of the initial image to obtain global linear projection features corresponding to the global features of the initial image.

3. The target association processing method of ship images according to claim 2 is characterized in that: The step of inputting the initial image global features into the global feature modeling model in the ship target association processing model to obtain the target image global features output by the global feature modeling model comprises: Inputting the global linear projection features into the dimension reduction network in the global feature modeling model to obtain the dimension reduction image global features output by the dimension reduction network; wherein the dimension reduction network is a network that performs dimension reduction processing on the initial image global features; Inputting the global features of the reduced-dimensional image into a global modeling network in the global feature modeling model to obtain a global weight output by the global modeling network; wherein the global modeling network is a network for performing correlation modeling on the global features of the reduced-dimensional image; The global linear projection features and the global weights are input into the global weighted network in the global feature modeling model to obtain the global features of the target image output by the global weighted network; wherein the global weighted network is a network that performs weighted multiplication of the global linear projection features and the global weights.

4. The target association processing method of ship images according to claim 1, characterized in that: The ship target association processing model is trained in the following way: Acquire the ship query sample image and the ship image sample database; Inputting the ship query sample image and the ship image sample database into a basic ship target association processing model to obtain a target association sample processing result output by the basic ship target association processing model; Calculate the model loss function according to the target associated sample processing result; The network parameters of the basic ship target association processing model are updated according to the model loss function to obtain the ship target association processing model.

5. A target association processing device for ship images, characterized in that: include: An image acquisition module, used to acquire a ship query image and a ship image database; wherein the ship query image is a ship image of a target ship for association processing, and the ship image database is a database containing a large number of ship images of the target ship acquired at different time sequences, different platforms and different devices; The association processing module is used to input the ship query image and the ship image database into the ship target association processing model to obtain the target association processing result output by the ship target association processing model; wherein the ship target association processing model is obtained by training based on the acquired ship query sample image and the ship image sample database, and the ship target association processing model is a model for extracting global features and local features from the ship query image and the ship image database, and determining the target association processing result corresponding to the ship query image and the ship image database based on the fusion processing of the global features and the local features; the ship query image and the ship image database are input into the ship target association processing model. In the processing model, the target association processing result output by the ship target association processing model is obtained, including: inputting the ship query image and the ship image database into the initial feature extraction model in the ship target association processing model, and obtaining the initial image global features and initial image local features output by the initial feature extraction model; wherein the initial feature extraction model is a network model for extracting features from the ship query image and the ship image database; inputting the initial image global features into the global feature modeling model in the ship target association processing model, and obtaining the target image global features output by the global feature modeling model; wherein the global feature modeling model is based on global attention to the global features of the initial image The invention relates to a model for modeling feature correlation of the local features of the initial image; inputting the local features of the initial image into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model; wherein the local feature modeling model is a model for modeling feature correlation of the local features of the initial image based on the nearest local attention; inputting the global features of the target image and the local features of the target image into the feature correlation fusion model in the ship target association processing model to obtain the target association processing result output by the feature correlation fusion model; wherein the feature correlation fusion model is a model for fusion processing of the global features of the target image and the local features of the target image; The local features of the initial image are input into the local feature modeling model in the ship target association processing model to obtain the local features of the target image output by the local feature modeling model, including: inputting the local features of the initial image into the local modeling network in the local feature modeling model to obtain the local weights input by the local modeling network; wherein the local modeling network is a network for performing correlation modeling on the local features of the initial image; inputting the local features of the initial image and the local weights into the local weighted network in the local feature modeling model to obtain the local features of the target image output by the local weighted network; wherein the local weighted network is a network for performing weighted multiplication of the local features of the initial image and the local weights.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the target association processing method for ship images as claimed in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the target association processing method for ship images according to any one of claims 1 to 4 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the target association processing method for ship images according to any one of claims 1 to 4 is implemented.

Images