A ship re-identification method, device, equipment and storage medium

Abstract

The application discloses a ship re-identification method and device, equipment and storage medium, and relates to the technical field of signal processing, and comprises the following steps: acquiring automatic identification system information of a ship and a photographed real-time ship image; searching for a corresponding reference ship image from a database according to the automatic identification system information; inputting the real-time ship image and the reference ship image into a feature extraction network, extracting attention features from feature layers at different depths of a backbone network by using a layered heterogeneous attention branch, and aggregating the attention features into heterogeneous attention features; extracting global features from the output of the backbone network by using a multi-scale feature extraction branch; and outputting image features of the real-time ship image and the reference ship image after fusing the heterogeneous attention features and the global features. The application can verify the identity information of the ship and improve the identification accuracy of the ship information.

Description

Technical Field

[0001] This application relates to the field of signal processing technology, specifically to a ship re-identification method, apparatus, device, and storage medium. Background Technology

[0002] There are many types of ships at sea, each with significant differences in shape, size, and characteristics. However, in practical applications, identity verification is currently mainly based on the self-reported information from the ship's AIS (Automatic Identification System).

[0003] However, the information reported by ships is not entirely reliable. Ships may modify AIS information, which interferes with ship identification and may lead to serious safety incidents.

[0004] Furthermore, ships may use false AIS information to conceal their true identity or intentions, thus increasing the difficulty of target identification. Due to the tamperability of AIS information, relying solely on this method for ship identification has significant limitations. Summary of the Invention

[0005] This application provides a ship re-identification method, device, equipment, and storage medium, which can effectively solve the problems of easy mis-amplification, large noise interference, and unclear sound quality of traditional loudspeakers in noisy environments, and significantly improve the reliability of voice communication in complex scenarios.

[0006] In a first aspect, embodiments of this application provide a ship re-identification method, the ship re-identification method comprising: Acquire information from the vessel's Automatic Identification System (AIS) and real-time images of the vessel. The system retrieves the corresponding reference ship image from the database based on the information from the automatic identification system. The real-time ship image and the reference ship image are input into the feature extraction network. The hierarchical heterogeneous attention branch in the feature extraction network extracts attention features from feature layers of different depths in the backbone network and aggregates them into heterogeneous attention features. The multi-scale feature extraction branch extracts global features from the output of the backbone network. The heterogeneous attention features are fused with the global features to output the image features of the real-time ship image and the reference ship image. The similarity between the image features of the real-time ship image and the reference ship image is calculated, and the information of the automatic recognition system is verified based on the similarity.

[0007] In conjunction with the first aspect, in one implementation, the step of extracting attention features from feature layers of different depths in the backbone network and aggregating them into heterogeneous attention features using the hierarchical heterogeneous attention branches in the feature extraction network includes: After the features output by the first residual unit in the backbone network residual block are processed by the first heterogeneous attention sub-network, the low-level attention features representing the overall macroscopic structure of the ship are output. The features output by the second residual unit in the same residual block are processed by the second heterogeneous attention sub-network to output the mid-level attention features that characterize the uniqueness and differences of the ship. The features output by the third residual unit in the same residual block are processed by the third heterogeneous attention sub-network to output high-level attention features that characterize the fine details of the ship. The heterogeneous attention features are obtained by concatenating the low-level, mid-level, and high-level attention features by channel and then aggregating them through attention weighting.

[0008] In conjunction with the first aspect, in one implementation, the step of concatenating the low-level attention features, mid-level attention features, and high-level attention features by channel and then aggregating them through attention weighting to obtain the heterogeneous attention features includes: According to the formula: Calculate the heterogeneous attention features ; in, These are the low-level attention features, mid-level attention features, or high-level attention features. To and The corresponding attention weights.

[0009] In conjunction with the first aspect, in one implementation, the step of extracting global features from the backbone network output using a multi-scale feature extraction branch includes: By utilizing a multi-scale feature extraction branch, the output features of the backbone network are input into multiple parallel convolutional branches with different sampling rates for processing to generate global features.

[0010] In conjunction with the first aspect, in one implementation, the step of utilizing a multi-scale feature extraction branch to input the output features of the backbone network into multiple parallel convolutional branches with different sampling rates for processing to generate global features includes: The number of channels in the output features of the backbone network is reduced by using convolutional layers; The dimensionality-reduced features are input into multiple parallel convolutional branches; The output features of the multiple parallel convolutional branches are concatenated, and the global features are obtained through global pooling.

[0011] In conjunction with the first aspect, in one embodiment, the plurality of parallel convolutional branches are three parallel branches, and the sampling rates of the convolutional kernels of the three parallel branches are A, B, and C, respectively, wherein A <B<C。

[0012] In conjunction with the first aspect, in one implementation, verifying the automatic identification system information based on the similarity includes: The similarity is compared with a preset similarity threshold; If the feature similarity is lower than the preset similarity threshold, the declared identity information of the ship is determined to be abnormal, and an abnormal reporting information is generated.

[0013] Secondly, embodiments of this application provide a ship re-identification device, the ship re-identification device comprising: The acquisition module is used to acquire information from the ship's Automatic Identification System and real-time images of the ship. The retrieval module retrieves corresponding reference ship images from the database based on the information from the automatic identification system. The extraction module is used to input the real-time ship image and the reference ship image into the feature extraction network, extract attention features from feature layers of different depths of the backbone network using the hierarchical heterogeneous attention branch in the feature extraction network and aggregate them into heterogeneous attention features, extract global features from the output of the backbone network using the multi-scale feature extraction branch, and output the image features of the real-time ship image and the reference ship image after fusing the heterogeneous attention features with the global features. The calculation module is used to calculate the similarity between the image features of the real-time ship image and the reference ship image, and to verify the automatic recognition system information based on the similarity.

[0014] Thirdly, embodiments of this application provide a ship re-identification device, which includes a processor, a memory, and a ship re-identification program stored in the memory and executable by the processor, wherein when the ship re-identification program is executed by the processor, it implements the steps of the ship re-identification method as described above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a ship re-identification program, wherein when the ship re-identification program is executed by a processor, it implements the steps of the ship re-identification method as described above.

[0016] The beneficial effects of the technical solutions provided in this application include: The ship re-identification method in this application acquires automatic identification system (AIS) information and real-time ship images; retrieves corresponding reference ship images from a database based on the AIS information; inputs the real-time ship images and reference ship images into a feature extraction network; utilizes a hierarchical heterogeneous attention branch in the feature extraction network to extract attention features from feature layers of different depths in the backbone network and aggregates them into heterogeneous attention features; utilizes a multi-scale feature extraction branch to extract global features from the output of the backbone network; and fuses the heterogeneous attention features with the global features to output image features of the real-time ship images and reference ship images; calculates the similarity between the image features of the real-time ship images and reference ship images, and verifies the AIS information based on the similarity.

[0017] Therefore, this application constructs a feature extraction network that includes a hierarchical heterogeneous attention branch and a multi-scale feature extraction branch. This network can extract and aggregate attention features from feature layers of different depths in the backbone network, while simultaneously extracting multi-scale global features and fusing the two to form image features. This dual-branch parallel extraction and fusion mechanism ensures that the generated image features retain subtle differences at different levels and encompass multi-scale contextual global information, thereby improving the discriminativeness and robustness of feature representation. Furthermore, based on feature similarity, it can more accurately verify the authenticity of information in the automatic identification system, effectively ensuring the reliability of ship identification. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an embodiment of the ship re-identification method of this application; Figure 2 This is a diagram of the multi-scale feature fusion network structure based on heterogeneous attention in this application; Figure 3 This is a diagram of the heterogeneous attention network structure in this application; Figure 4 This is a diagram of the multi-scale feature extraction network structure for this application; Figure 5 This is a functional module diagram of an embodiment of the ship re-identification device of this application; Figure 6 This is a schematic diagram of the hardware structure of the ship re-identification device involved in the embodiments of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0021] In a first aspect, embodiments of this application provide a method for re-identifying ships.

[0022] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the ship re-identification method of this application. Figure 1 As shown, ship re-identification methods include: S1. Acquire information about the vessel's Automatic Identification System (AIS) and real-time images of the vessel. In step S1, the main task is to receive the Automatic Identification System (AIS) information declared by the vessel, and at the same time, take real-time photos of the vessel to obtain real-time images of the vessel.

[0023] S2. Retrieve the corresponding reference ship image from the database based on the information from the automatic identification system; After obtaining information from the Automatic Identification System (AIS), information can be retrieved from the ship information database based on the AIS information to obtain corresponding reference ship images for comparison.

[0024] S3. Input the real-time ship image and the reference ship image into the feature extraction network. Use the hierarchical heterogeneous attention branch in the feature extraction network to extract attention features from the feature layers of different depths of the backbone network and aggregate them into heterogeneous attention features. Use the multi-scale feature extraction branch to extract global features from the output of the backbone network. After fusing the heterogeneous attention features with the global features, output the image features of the real-time ship image and the reference ship image. It is worth noting that in step S3, both the real-time ship image and the reference ship image need to be input into the feature extraction network, and then processed by the hierarchical heterogeneous attention branch and the multi-scale feature extraction branch to obtain their respective image features.

[0025] The step of extracting attention features from feature layers of different depths in the backbone network and aggregating them into heterogeneous attention features using the hierarchical heterogeneous attention branches in the feature extraction network includes: After processing the features output by the first residual unit in the residual block of the backbone network through the first heterogeneous attention sub-network, output low-level attention features representing the overall macroscopic structure of the ship; after processing the features output by the second residual unit in the same residual block through the second heterogeneous attention sub-network, output middle-level attention features representing the uniqueness and difference of the ship; after processing the features output by the third residual unit in the same residual block through the third heterogeneous attention sub-network, output high-level attention features representing the fine details of the ship; after concatenating the low-level attention features, middle-level attention features and high-level attention features by channels, the heterogeneous attention features are obtained through attention weighted aggregation.

[0026] Furthermore, the multi-scale feature extraction branch is used to extract global features from the output of the backbone network, which includes: Using the multi-scale feature extraction branch, the output features of the backbone network are input into a plurality of parallel convolution branches with different sampling rates for processing to generate global features.

[0027] Specifically, said using the multi-scale feature extraction branch to input the output features of the backbone network into a plurality of parallel convolution branches with different sampling rates for processing to generate global features includes: Reducing the number of channels of the output features of the backbone network through a convolution layer; inputting the dimension-reduced features into a plurality of parallel convolution branches; concatenating the output features of the plurality of parallel convolution branches, and obtaining the global features through global pooling.

[0028] Preferably, the plurality of parallel convolution branches are three parallel branches, and the convolution kernel sampling rates of the three parallel branches are A, B and C respectively, wherein A < B < C. Further, the values of A, B and C are 1, 2 and 4 respectively.

[0029] For ease of understanding, a specific example is given below for illustration: In this embodiment, a trained feature extraction network needs to be obtained first, see Figure 2 As shown, the feature extraction network in this embodiment is a multi-scale feature fusion network based on heterogeneous attention, and its training process is as follows: Step 1, pre-train the ResNet50 network using the ImageNet dataset, and assign initial parameter values to the ResNet50 network; Step 2, construct a heterogeneous attention-based multi-scale feature fusion network based on the pre-trained ResNet50 network; Step 3, train the constructed heterogeneous attention-based multi-scale feature fusion network using the ship image dataset; Subsequently, a multi-scale feature fusion network can be used to extract features from both the real-time ship image and the reference ship image, and then the feature similarity can be calculated, i.e.: Step 4: Extract features from real-time ship images using a trained multi-scale feature fusion network based on heterogeneous attention. Step 5: Use the obtained AIS information to extract the corresponding ship images from the database, and use a multi-scale feature fusion network based on heterogeneous attention to extract image features; Step 6: Calculate the feature similarity between the real-time acquired image and the image extracted from the database. When the similarity exceeds the threshold, it proves that the AIS information is correct and there is no need to update the database. When the similarity is below the threshold, it proves that the AIS information has been modified. The ship information should be marked, registered, and reported.

[0030] It is worth noting that the multi-scale feature fusion network based on heterogeneous attention also includes: features extracted from the first layer (first heterogeneous attention sub-network) of the Layer 4 network in the ResNet50 network are passed through a heterogeneous attention module and then global pooled to output low-level attention features; features extracted from the second layer (second heterogeneous attention sub-network) of the Layer 4 network in the ResNet50 network are passed through a heterogeneous attention module and then global pooled to output mid-level attention features; features extracted from the third layer (third heterogeneous attention sub-network) of the Layer 4 network in the ResNet50 network are passed through a heterogeneous attention module and then global pooled to output high-level attention features; the features output from these three layers are passed through a feature aggregation module to output the heterogeneous attention features of this network; simultaneously, the features output from the ResNet50 network are passed through a multi-scale feature extraction module and global pooled to output the global features of this network; the attention features and global features are concatenated to form the output features of this network, thus obtaining the image features of the real-time ship image and the reference ship image.

[0031] The multi-scale feature fusion network uses the amsgrad algorithm to update network parameters. The amsgrad algorithm formula is as follows:

[0032]

[0033]

[0034]

[0035] in, This represents the current average gradient, initially set to 0. This represents the sum of squared gradients, initially set to 0. This represents the current maximum sum of squared gradients. , Parameters can be set. Indicates the current gradient. Indicates the learning rate. To represent an infinitesimal number, to prevent the denominator from being 0, This indicates the parameter to be updated.

[0036] See Figure 3 As shown, the specific structures of the first heterogeneous attention subnetwork, the second heterogeneous attention subnetwork, and the third heterogeneous attention subnetwork are described below: The first heterogeneous attention subnetwork passes through two 1 After the first convolutional block, the features are input into three separate 1-bit convolutional blocks. In the 1-th convolutional module; the outputs of any two convolutional modules are multiplied together, the result is passed through a softmax classification module, multiplied by the output of a third convolutional module, and the result is passed through a 1-th convolutional module. The output of the convolutional module 1 is a low-level attention feature.

[0037] The second heterogeneous attention subnetwork passes through two parallel 1s One convolutional block, multiplying the output features and then passing them through a 1-bit convolutional block. After the first convolutional block, the features are input into a global cross-channel average pooling layer with a stride of 2. A network consisting of three convolutional layers, upsampling layers, and scaled convolutional layers, and a network consisting of a cross-channel global average pooling layer and two convolutional layers, multiplying the features output by the two networks, and then passing the result through a 1-channel multiplication table. The output of the convolutional module 1 is the mid-level attention feature.

[0038] The third heterogeneous attention subnetwork passes through three parallel 1s After a convolutional block, the output features are multiplied, and then passed through a 1-bit multiplication table. After the first convolutional block, the features are input into one or two 1-bit convolutional blocks respectively. A network consisting of 1 convolutional block; a network consisting of two parallel 1 After feature multiplication, the convolutional block is then processed by a 1-bit multiplication layer. A network consisting of 1 convolutional block; and a network consisting of three parallel 1-to-1 convolutional blocks. After feature multiplication, the convolutional block is then processed by a 1-bit multiplication layer. The output of the first convolutional block is a high-level attention feature.

[0039] The low-level attention features, mid-level attention features, and high-level attention features are aggregated, passed through a Simoid activation function, and then multiplied with the input features to output the final features.

[0040] Preferably, the feature aggregation method adopts the form of feature attention, which connects the low-level heterogeneous attention features, the mid-level heterogeneous attention features and the high-level heterogeneous attention features as the heterogeneous attention feature representation of the network.

[0041] The formula for feature attention is: ; in, These are the low-level attention features, mid-level attention features, or high-level attention features. To and The corresponding attention weights.

[0042] Furthermore, the loss function of the multi-scale feature fusion network based on heterogeneous attention simultaneously employs Softmax loss and triplet loss, calculated as follows:

[0043] in, These represent the Softmax loss and the triplet loss, respectively.

[0044] Preferably, the Softmax classification loss function formula is:

[0045] in, Let the feature vector of the i-th sample be of dimension d, and Belongs to the Ships of various categories; This represents the weight of the j-th column in the classification layer. express The weights corresponding to the samples Indicates the first Column bias parameters, express The paradox parameter corresponding to the sample; m and n represent the number of samples and the number of classes in the batch training, respectively.

[0046] Preferably, a triple is randomly selected from the dataset. As a sample For a fixed sample, As a positive sample, For negative samples, the corresponding features are as follows: , and The formula for the triplet loss function is:

[0047] in, , Indicates margin, This represents the number of images in a batch, where P represents the P randomly selected ship identities in each batch, and K represents the K images randomly selected from each ship identity. It represents Euclidean distance.

[0048] In addition, it should be noted that the ResNet50 network consists of multiple residual blocks, usually divided into 4 main blocks, each corresponding to a different resolution or feature map size; Layer4 refers to the fourth residual block (Block4), which generally includes three residual units, each responsible for extracting features of different complexity.

[0049] The first residual unit receives a lower-level feature map from the output of Block3. After passing through the first layer, it can capture the more basic but overall macroscopic structural features of the hull. After passing through the heterogeneous attention module, it can extract macroscopic features that are easier to distinguish. The second residual unit processes the feature map output from the first layer, further refining and deepening the feature representation, focusing on specific parts of the hull or details at medium scale. After passing through the heterogeneous attention module, these medium-level features can better reflect the uniqueness and differences of the ship. The third residual unit processes the feature map output from the second layer, performing higher-level nonlinear transformations and combinations, focusing on the subtle details of the ship. Through the heterogeneous attention module, these high-level features can further distinguish similar but different ship types.

[0050] Features at different levels contribute differently to the recognition task. Low-level features provide basic overall information, mid-level features enhance details, and high-level features capture subtle details. By applying heterogeneous attention modules to each level, feature enhancement can be performed at different scales and levels of importance, thereby improving the accuracy of re-recognition and enabling the systematic capture of ship feature information from basic to complex.

[0051] Traditional attention mechanisms typically focus on a single scale or level, failing to fully utilize feature information at different levels. The hierarchical approach in this embodiment not only enhances the model's attention to ship details but also improves the overall accuracy and reliability of recognition through feature fusion at different levels. By extracting low, medium, and high-level features hierarchically, the model can simultaneously focus on fine-grained details (such as local texture) and coarse-grained overall structure (such as contour shape) in the image, thereby achieving a multi-dimensional representation of the ship. This means focusing on key areas of the ship from multiple scales and dimensions, amplifying the attention to subtle distinguishing features.

[0052] See Figure 4 As shown, the multi-scale feature extraction module structure is as follows: after a 1 The first convolutional block reduces the number of feature channels to one-third of the original number. Then, the features are input into three parallel branches with sampling rates of 1, 2, and 4. The output features of the three channels are concatenated along the channel dimension and multiplied with the original input features to output the final features. This module is used to capture multi-scale information of ships and fuse multi-scale features along the channel dimension.

[0053] By using a 1x1 convolutional layer, the number of feature channels is reduced to one-third of the original number. This step helps reduce computational complexity and prepares the ground for subsequent multi-scale feature processing.

[0054] Sampling rate 1 (preserves original resolution): This branch directly captures fine details in the image; Sampling rate 2 (reduces feature map size by half): Extracts medium-scale structural information; Sampling rate 4 (reduces feature map size by one-quarter): Focuses on the overall structure and macroscopic features of the entire ship; The feature maps output by the three branches are concatenated along the channel dimension: each branch produces a feature map with different scale information, and the three feature maps are concatenated along the channel axis so that the final feature map contains fine-grained, medium-grained, and coarse-grained information.

[0055] With three branches at different sampling rates, the module can process data from different spatial ranges in the same layer, thereby comprehensively capturing the details and overall structure of ship images. Channel stitching ensures that each channel can carry information at a specific scale, which helps to improve the model's ability to understand complex scenes.

[0056] Understandably, integrating multi-scale information enhances the network's descriptive capabilities, enabling it to accurately identify ships under varying lighting and angle conditions. By fusing information from multiple scales, the system is less susceptible to the effects of missing features or noise at a single scale.

[0057] S4. Calculate the similarity between the image features of the real-time ship image and the reference ship image, and verify the automatic recognition system information based on the similarity.

[0058] In step S4, verifying the automatic recognition system information based on the similarity includes: The similarity is compared with a preset similarity threshold; if the feature similarity is lower than the preset similarity threshold, the declared identity information of the ship is determined to be abnormal, and an abnormal reporting information is generated.

[0059] In summary, the ship re-identification method in this application acquires automatic identification system (AIS) information and real-time ship images; retrieves corresponding reference ship images from a database based on the AIS information; inputs the real-time ship images and reference ship images into a feature extraction network; utilizes a hierarchical heterogeneous attention branch in the feature extraction network to extract attention features from feature layers of different depths in the backbone network and aggregates them into heterogeneous attention features; utilizes a multi-scale feature extraction branch to extract global features from the output of the backbone network; and fuses the heterogeneous attention features with the global features to output image features of the real-time ship images and reference ship images; calculates the similarity between the image features of the real-time ship images and reference ship images; and verifies the AIS information based on the similarity.

[0060] Therefore, this application constructs a feature extraction network that includes a hierarchical heterogeneous attention branch and a multi-scale feature extraction branch. This network can extract and aggregate attention features from feature layers of different depths in the backbone network, while simultaneously extracting multi-scale global features and fusing the two to form image features. This dual-branch parallel extraction and fusion mechanism ensures that the generated image features retain subtle differences at different levels and encompass multi-scale contextual global information, thereby improving the discriminativeness and robustness of feature representation. Furthermore, based on feature similarity, it can more accurately verify the authenticity of information in the automatic identification system, effectively ensuring the reliability of ship identification.

[0061] Secondly, embodiments of this application also provide a ship re-identification device.

[0062] In one embodiment, reference is made to Figure 5 , Figure 5 This is a functional module diagram of an embodiment of the ship re-identification device of this application. Figure 5 As shown, the ship re-identification device includes: The acquisition module is used to acquire information from the ship's Automatic Identification System and real-time images of the ship. The retrieval module retrieves corresponding reference ship images from the database based on the information from the automatic identification system. The extraction module is used to input the real-time ship image and the reference ship image into the feature extraction network, extract attention features from feature layers of different depths of the backbone network using the hierarchical heterogeneous attention branch in the feature extraction network and aggregate them into heterogeneous attention features, extract global features from the output of the backbone network using the multi-scale feature extraction branch, and output the image features of the real-time ship image and the reference ship image after fusing the heterogeneous attention features with the global features. The calculation module is used to calculate the similarity between the image features of the real-time ship image and the reference ship image, and to verify the automatic recognition system information based on the similarity.

[0063] Further, in one embodiment, the extraction module utilizes the hierarchical heterogeneous attention branches in the feature extraction network to extract attention features from feature layers of different depths in the backbone network and aggregates them into heterogeneous attention features, including: After the features output by the first residual unit in the backbone network residual block are processed by the first heterogeneous attention sub-network, the low-level attention features representing the overall macroscopic structure of the ship are output. The features output by the second residual unit in the same residual block are processed by the second heterogeneous attention sub-network to output the mid-level attention features that characterize the uniqueness and differences of the ship. The features output by the third residual unit in the same residual block are processed by the third heterogeneous attention sub-network to output high-level attention features that characterize the fine details of the ship. The heterogeneous attention features are obtained by concatenating the low-level, mid-level, and high-level attention features by channel and then aggregating them through attention weighting.

[0064] Further, in one embodiment, the extraction module concatenates the low-level attention features, mid-level attention features, and high-level attention features by channel, and then obtains the heterogeneous attention features through attention-weighted aggregation, including: According to the formula: Calculate the heterogeneous attention features ; in, These are the low-level attention features, mid-level attention features, or high-level attention features. To and The corresponding attention weights.

[0065] Furthermore, in one embodiment, the extraction module extracts global features from the backbone network output using a multi-scale feature extraction branch, including: By utilizing a multi-scale feature extraction branch, the output features of the backbone network are input into multiple parallel convolutional branches with different sampling rates for processing to generate global features.

[0066] Furthermore, in one embodiment, the extraction module utilizes a multi-scale feature extraction branch to input the output features of the backbone network into multiple parallel convolutional branches with different sampling rates for processing to generate global features, including: The number of channels in the output features of the backbone network is reduced by using convolutional layers; The dimensionality-reduced features are input into multiple parallel convolutional branches; The output features of the multiple parallel convolutional branches are concatenated, and the global features are obtained through global pooling.

[0067] Further, in one embodiment, the plurality of parallel convolutional branches are three parallel branches, and the sampling rates of the convolutional kernels of the three parallel branches are A, B, and C, respectively, wherein A <B<C。

[0068] Furthermore, in one embodiment, the calculation module verifies the automatic recognition system information based on the similarity, including: The similarity is compared with a preset similarity threshold; If the feature similarity is lower than the preset similarity threshold, the declared identity information of the ship is determined to be abnormal, and an abnormal reporting information is generated.

[0069] The functions of each module in the above-mentioned ship re-identification device correspond to the steps in the above-mentioned ship re-identification method embodiment, and their functions and implementation processes will not be described in detail here.

[0070] Thirdly, embodiments of this application provide a ship re-identification device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0071] Reference Figure 6 , Figure 6 This is a schematic diagram of the hardware structure of the ship re-identification device involved in the embodiments of this application. In the embodiments of this application, the ship re-identification device may include a processor, a memory, a communication interface, and a communication bus.

[0072] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0073] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting components within the ship re-identification equipment, as well as interfaces used for interconnecting the ship re-identification equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0074] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0075] The processor can be a general-purpose processor, which can call the ship re-identification program stored in memory and execute the ship re-identification method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the ship re-identification program is called can be referred to in the various embodiments of the ship re-identification method of this application, and will not be repeated here.

[0076] Those skilled in the art will understand that Figure 6 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0077] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0078] The present application has a ship re-identification program stored on a computer-readable storage medium, wherein when the ship re-identification program is executed by a processor, it implements the steps of the ship re-identification method as described above.

[0079] The method implemented when the ship re-identification procedure is executed can be referred to in various embodiments of the ship re-identification method of this application, and will not be repeated here.

[0080] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0081] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0082] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0083] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0084] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0086] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for re-identifying ships, characterized in that, The ship re-identification method includes: Acquire information from the vessel's Automatic Identification System (AIS) and real-time images of the vessel. The system retrieves the corresponding reference ship image from the database based on the information from the automatic identification system. The real-time ship image and the reference ship image are input into the feature extraction network. The hierarchical heterogeneous attention branch in the feature extraction network extracts attention features from feature layers of different depths in the backbone network and aggregates them into heterogeneous attention features. The multi-scale feature extraction branch extracts global features from the output of the backbone network. The heterogeneous attention features are fused with the global features to output the image features of the real-time ship image and the reference ship image. The similarity between the image features of the real-time ship image and the reference ship image is calculated, and the information of the automatic recognition system is verified based on the similarity.

2. The ship re-identification method as described in claim 1, characterized in that, The step of extracting attention features from feature layers of different depths in the backbone network and aggregating them into heterogeneous attention features using the hierarchical heterogeneous attention branches in the feature extraction network includes: After the features output by the first residual unit in the backbone network residual block are processed by the first heterogeneous attention sub-network, the low-level attention features representing the overall macroscopic structure of the ship are output. The features output by the second residual unit in the same residual block are processed by the second heterogeneous attention sub-network to output the mid-level attention features that characterize the uniqueness and differences of the ship. The features output by the third residual unit in the same residual block are processed by the third heterogeneous attention sub-network to output high-level attention features that characterize the fine details of the ship. The heterogeneous attention features are obtained by concatenating the low-level, mid-level, and high-level attention features by channel and then aggregating them through attention weighting.

3. The ship re-identification method as described in claim 2, characterized in that, The process of concatenating the low-level, mid-level, and high-level attention features by channel and then aggregating them through attention weighting to obtain the heterogeneous attention features includes: According to the formula: Calculate the heterogeneous attention features ; in, These are the low-level attention features, mid-level attention features, or high-level attention features. To and The corresponding attention weights.

4. The ship re-identification method as described in claim 1, characterized in that, The method of extracting global features from the backbone network output using a multi-scale feature extraction branch includes: By utilizing a multi-scale feature extraction branch, the output features of the backbone network are input into multiple parallel convolutional branches with different sampling rates for processing to generate global features.

5. The ship re-identification method as described in claim 4, characterized in that, The multi-scale feature extraction branch processes the output features of the backbone network by inputting them into multiple parallel convolutional branches with different sampling rates to generate global features, including: The number of channels in the output features of the backbone network is reduced by using convolutional layers; The dimensionality-reduced features are input into multiple parallel convolutional branches; The output features of the multiple parallel convolutional branches are concatenated, and the global features are obtained through global pooling.

6. The ship re-identification method as described in claim 5, characterized in that: The multiple parallel convolutional branches are divided into three parallel branches, and the sampling rates of the convolutional kernels of the three parallel branches are A, B, and C, respectively, where A... <B<C。 7. The ship re-identification method as described in claim 1, characterized in that, The step of verifying the automatic recognition system information based on the similarity includes: The similarity is compared with a preset similarity threshold; If the feature similarity is lower than the preset similarity threshold, the declared identity information of the ship is determined to be abnormal, and an abnormal reporting information is generated.

8. A ship re-identification device, characterized in that, The ship re-identification device includes: The acquisition module is used to acquire information from the ship's Automatic Identification System and real-time images of the ship. The retrieval module retrieves corresponding reference ship images from the database based on the information from the automatic identification system. The extraction module is used to input the real-time ship image and the reference ship image into the feature extraction network, extract attention features from feature layers of different depths of the backbone network using the hierarchical heterogeneous attention branch in the feature extraction network and aggregate them into heterogeneous attention features, extract global features from the output of the backbone network using the multi-scale feature extraction branch, and output the image features of the real-time ship image and the reference ship image after fusing the heterogeneous attention features with the global features. The calculation module is used to calculate the similarity between the image features of the real-time ship image and the reference ship image, and to verify the automatic recognition system information based on the similarity.

9. A ship re-identification device, characterized in that, The ship re-identification device includes a processor, a memory, and a ship re-identification program stored in the memory and executable by the processor, wherein when the ship re-identification program is executed by the processor, it implements the steps of the ship re-identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a ship re-identification program, wherein when the ship re-identification program is executed by a processor, it implements the steps of the ship re-identification method as described in any one of claims 1 to 7.

Images