MambaVision hybrid architecture-based ship image fine-grained identification method and system

Abstract

The invention provides a ship image fine-grained identification method and system based on a MambaVision hybrid architecture, and aims to realize accurate matching identification of a ship image and a maritime organization serial number, and the method comprises the steps: carrying out the image preprocessing of a data set containing the ship image and a label; a MambaVision image recognition model is constructed, a four-stage pyramid structure is adopted for training, local features are extracted from a bottom layer through convolution, and a Mamba state space model and a window attention mechanism are fused from a high layer; prediction results of a plurality of different architecture models are integrated, and the robustness and accuracy of recognition are improved; an incremental learning strategy is adopted, related parameters of a newly-added ship are initialized by inheriting historical parameter statistical distribution, the learning rate is dynamically adjusted, and a gradient adjustment strategy and a knowledge distillation mechanism are combined, so that the model keeps the recognition capability of the historical ship while learning the newly-added ship.

Description

Technical Field

[0001] This invention proposes a fine-grained recognition method and system for ship images based on the MambaVision hybrid architecture, which relates to the fields of computer vision and artificial intelligence. Background Technology

[0002] Automatic identification of vessels is crucial for port scheduling, maritime supervision, and shipping safety. With the development of global trade, the number of vessels in ports is increasing daily, creating an urgent need for automatic, accurate, and efficient vessel identification. Traditional vessel identification relies heavily on Automatic Identification Systems (AIS), radar systems, and manual inspection. AIS depends on vessels actively reporting their information and is ineffective against objects without AIS installed. Small vessels often lack AIS systems, and some illegal vessels may actively disable, tamper with, or deceive information to evade system detection. Radar systems are weak at detecting small targets and have blind spots such as those at close range or obstructed views, failing to provide rich visual information for refined identification. Manual visual inspection suffers from unreliable data, low efficiency, and susceptibility to environmental interference. With the continuous improvement of computing power and the rapid development of deep learning technology, computer vision-based vessel image recognition technology has received widespread attention in recent years as an effective supplementary method.

[0003] The existing technologies have four main drawbacks: First, while Convolutional Neural Networks (CNNs) are the mainstream model, there is limited application of cutting-edge Transformer and Mamba vision models. These models perform poorly in scenarios involving complex weather, varying lighting conditions, or the identification of small targets like ships. Second, it is difficult to balance accuracy and efficiency in vision models. CNN models are efficient but have weak global modeling capabilities, Transformer models are accurate but computationally inefficient, and pure Mamba state-space models have superior computational efficiency but insufficient recognition accuracy. Third, environmental robustness is insufficient. Single models have limited adaptability to complex environmental changes such as lighting, weather, and occlusion, easily leading to a significant decline in recognition performance. Fourth, the limitations of single models mean that the structure of any single model determines its performance ceiling, making further improvement difficult. Fifth, model scalability is poor. Once a traditional model is trained, the ships it can identify are fixed. To identify new ships, the entire model usually needs to be retrained, which is costly. Summary of the Invention

[0004] In view of this, in order to fill the gaps and deficiencies in the existing technology, this invention proposes a fine-grained recognition method and system for ship images based on the MambaVision hybrid architecture.

[0005] This invention aims to innovatively construct a ship image recognition and International Maritime Organization Number (IMONumber) recognition model based on a Computer Vision (CV) model architecture, achieving high recognition accuracy, high computational efficiency, and strong environmental robustness. Through an ensemble learning strategy, it integrates the advantages of multiple heterogeneous models, overcoming the performance bottleneck of a single model and achieving a balance between high accuracy and high computational efficiency. It enhances the model's adaptability to complex environments, ensuring stable recognition performance. It possesses incremental learning capabilities, enabling it to efficiently learn new ship features without forgetting existing knowledge, demonstrating good scalability and practicality.

[0006] This invention proposes a fine-grained image recognition method and system based on the MambaVision hybrid architecture, comprising the following: This invention proposes a fine-grained ship image recognition method based on the MambaVision hybrid architecture, characterized by the following: Step S1: Perform image preprocessing on the collected dataset containing ship images and their corresponding labels. The preprocessing is divided into training data preprocessing and image preprocessing for the target image, depending on the purpose of the data. The training data preprocessing includes: randomly applying various enhancement transformations to the input image, including multi-scale random scaling and cropping; random horizontal flipping; color jittering; automatic enhancement (AutoAugment, automatic combination of enhancement strategies); random erasure; and mixed enhancement and cut-Mix (cross-sample fusion enhancement methods). The image preprocessing includes: performing image consistency processing, including scaling the image to a fixed resolution, resampling, and normalizing pixel values ​​to ensure input consistency. Step S2: Construct an image recognition model based on MambaVision. The training process of the image recognition model based on MambaVision adopts a four-stage pyramid structure. The bottom layer extracts local features through convolution operations, and the top layer integrates the Mamba state space model and window attention mechanism to achieve efficient modeling of long and short range features, thereby improving recognition accuracy and computational efficiency. Step S3: Integrate the prediction results, including additionally training multiple models with different architectures, and using a dynamic weighting mechanism to integrate the prediction results of different models to improve the robustness and accuracy of recognition; Step S4: Employ an incremental learning strategy to enable the image recognition model to adapt to newly added ship individuals. This includes extending the classification head for new ships during model creation, inheriting the statistical characteristics of the old classification head; dynamically adjusting the learning rates for new and historical ships in the initial training phase, including gradient adjustment and knowledge distillation; initializing weights and biases during image recognition model creation; dynamically adjusting the learning rates for new and historical ships in the initial training phase by dividing training into two stages: learning new knowledge and reviewing old knowledge, thus optimizing learning efficiency; gradient adjustment by finer-grained control over the learning of knowledge about new and old ships based on dynamically differentiated learning rates; and knowledge distillation by introducing distillation loss into the loss function to efficiently learn new ship knowledge while better preserving existing knowledge.

[0007] Further, step S1 includes the following: Step S11: Randomly apply various enhancement transformations to the input image, including for a batch of input training images X∈R B ×C×H×W The following transformations are performed in sequence: multi-scale random scaling and cropping, random horizontal flipping, color dithering, automatic enhancement (AutoAugment, automatic data augmentation strategy), random erasure, mixed enhancement and cut-Mix (Mixup and CutMix, cross-sample mixed enhancement methods), fixed-ratio cropping at the center, bicubic interpolation to adjust image size, and normalization. Where B is the batch size, C is the number of channels, and H and W are the height and width; Step S12: Multi-scale random scaling and cropping includes employing a multi-scale random scaling and cropping strategy to simulate changes in objects of different sizes and viewpoints and to improve the model's robustness to object size and shape. Step S13: Random horizontal flipping utilizes the left-right symmetry of the image to perform horizontal flipping, expanding the training data without changing the semantics; vertical flipping remains off to ensure semantic consistency. Step S14: Color jittering includes simulating lighting changes by adjusting brightness, contrast, saturation, and hue to enhance the model's robustness to color conditions; Step S15: AutoAugment employs a strategy called rand-m9-mstd0.5 (a configuration scheme that randomly combines multiple augmentation operations. For the input image, it is first randomly decided whether to perform augmentation. If it is performed, one group is selected from nine preset "basic operation combinations", and the strength of each operation is randomly adjusted according to a standard deviation of 0.5). By randomly combining multiple augmentation operations, it further enhances data diversity and model generalization ability. Step S16: Random erasure includes filling local image regions with pixel noise with a 25% probability, forcing the model to focus on global contextual features to alleviate local overfitting; Step S17: Mixup and CutMix generate soft labels by fusing cross-sample features, thereby improving the smoothness of the decision boundary; Step S18: Fixed-ratio cropping at the center. By setting the cropping ratio, an image area of ​​a certain size according to the original height and width is cropped from the center of the original image, retaining a fixed percentage of the original image area and removing edge noise to reduce the interference of the background on subsequent processing. Step S19: Adjusting the image size using bicubic interpolation means rescaling the cropped image to a uniform standard input size, so that all images received by the model have a consistent spatial resolution and ensure uniform input dimensions. Step S110: Standardize the differenced images by channel to make the pixel value distribution of different images more consistent, reduce the impact of illumination and color differences on the model, and achieve faster model convergence and improved accuracy.

[0008] Further, step S2 includes the following: Step S21: The local feature extraction capability of CNN is combined with the image recognition model of MambaVision, which quickly extracts local features through the convolution operation of CNN residual blocks at the bottom layer. After the convolution kernel slides and scans on the input image, it performs element-wise multiplication and accumulation on the corresponding elements in the input image to generate each element of the feature map. The stability of training is accelerated by batch normalization layer, and nonlinear transformation is introduced by GELU activation function. Finally, the original input is added to each element of the processed feature map through residual connection to generate the complete feature map.

[0009] Furthermore, step S2 also includes the following: Step S22: The Mamba state space model includes the following: using the Mamba state space model for long sequence modeling to capture long-range dependencies within the sequence; replacing the causal convolutions in the original Mamba with regular 1D convolutions to see contextual information, breaking the limitations of sequential computation and enhancing parallel processing capabilities; and fusing the ability of the state space model to capture long-range dependencies with the efficiency of traditional convolutions in extracting local features through a dual-branch parallel structure design. Step S221: The State Space Model (SSM) branch first reduces the number of channels of the input features by linear projection to reduce computation, then extracts local features using one-dimensional convolution, introduces nonlinearity through the SiLU activation function, and finally performs selective scanning to efficiently fuse global information of the sequence and capture complex long-distance dependencies. Step S222: Adding a symmetric branch involves introducing a non-SSM path, directly processing features through convolution and SiLU activation functions, preserving local detail features after convolution and activation transformation, and providing supplementary near-context information; Step S223: Stitching and projection involves stitching the outputs of the two branches, which contain global context and local details, and projecting them back to the original dimension to fuse local and global information.

[0010] Furthermore, step S2 also includes the following: Step S23: Introduce a multi-head attention mechanism in the low-resolution stage to supplement the limitations of the model, including Mamba sequences, in terms of global modeling capabilities. The multi-head attention mechanism calculates the interaction relationship between queries, keys, and values, and dynamically assigns relevance weights to each position in the sequence with all other positions using the scaling dot product attention formula. This captures long-distance spatial dependencies and achieves deep fusion and modeling of global contextual information.

[0011] Further, step S3 includes the following: A multi-model fusion strategy based on weighted voting is adopted. In terms of the fusion strategy, each model outputs the predicted probability distribution of the input ship image corresponding to each maritime organization number. The fusion weight is assigned to the model according to its recognition accuracy on the validation set. The probability distributions output by different models are weighted and fused, and the maritime organization number with the highest probability after fusion is taken as the final recognition result, so as to achieve stable matching between ship images and unique numbers.

[0012] Further, step S4 includes the following: Step S41: Initialization of the inherited model classification head includes: When the image recognition model expands from recognizing only historical ships to recognizing both historical and newly added ships, a statistical classification head initialization method will be adopted. By sampling the mean and variance of the weight parameters corresponding to historical ships, the weight distribution corresponding to newly added ships will be kept consistent with that of historical ships. At the same time, the mean of the bias parameters corresponding to historical ships will be down-adjusted to reduce the activation level of newly added ships in the initial stage, thereby reducing initialization randomness, accelerating the convergence speed of newly added ships, and protecting the ability to recognize historical ships during training. Step S42: Dynamic warm-up includes the introduction of a dynamic warm-up strategy in the early stage of incremental training. Within a preset number of training rounds, the learning rate of parameters related to newly added ships is gradually increased, while the update rate of parameters related to historical ships is further reduced. The incremental learning stage is divided into two stages: learning new information and reviewing old information. This allows the new information to be gradually integrated into the existing knowledge structure, thereby mitigating the impact on the recognition performance of historical ships.

[0013] Furthermore, step S4 also includes the following: Step S43: Gradient adjustment includes an optimization strategy based on gradient post-processing. Before the optimizer performs parameter updates, the gradients corresponding to the parameters related to historical ships and the parameters related to newly added ships are scaled respectively to more finely balance the learning intensity of the two types of parameters and achieve finer-grained update control. Step S44: Knowledge distillation includes introducing the output probability distribution of the model corresponding to the parameters of the historical model as a soft label to provide the relative probability relationship between different ships; the current model is fine-tuned by distillation using temperature scaling, and optimized by combining recognition loss and distillation loss, so that the image recognition model can retain the ability to recognize historical ships to the greatest extent while learning new ships.

[0014] According to a second aspect of the present invention, the present invention proposes a ship image fine-grained recognition system based on a MambaVision hybrid architecture, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements a ship image fine-grained recognition method based on a MambaVision hybrid architecture as described in any one of the present invention.

[0015] According to a third aspect of the present invention, the present invention proposes a fine-grained ship image recognition system based on a MambaVision hybrid architecture, comprising a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements a fine-grained ship image recognition method based on a MambaVision hybrid architecture as described in any one of the present invention.

[0016] The present invention has the following advantages: By comparing the performance of visual models with different architectures in intelligent ship image recognition, the method of the present invention has the following advantages over traditional methods: (1) The present invention adopts a combined data augmentation strategy to improve the diversity of data and simulate various complex imaging conditions. Compared with traditional methods, it has stronger environmental robustness in the face of interference factors such as changes in illumination, differences in viewing angles and occlusion. (2) The present invention introduces a hybrid architecture MambaVision model with the Mamba state space model with near-linear computational complexity as the core. Compared with the pure Transformer model with quadratic computational complexity, it significantly reduces computational overhead and memory usage, and has higher computational efficiency while maintaining high accuracy. (3) The present invention leverages the advantages of group decision-making through a heterogeneous multi-model integration strategy, effectively integrating the advantages of CNN, Transformer and Mamba architecture models, breaking through the bottlenecks in single model structure design, training process and performance, and improving overall recognition performance. (4) The present invention integrates the ability of incremental learning. The model can continuously and efficiently learn new ships without forgetting existing knowledge, without having to train from scratch. This significantly alleviates the problem of "catastrophic forgetting" in traditional models, reduces the cost of model maintenance, and has strong practical application potential. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0018] Figure 2 This is a schematic diagram of the fine-grained ship image recognition process based on the MambaVision hybrid architecture of the present invention.

[0019] Figure 3 This is a schematic diagram of the hybrid architecture of the MambaVision model of the present invention.

[0020] Figure 4 This is a schematic diagram of the convolutional block portion of the MambaVision model of the present invention.

[0021] Figure 5 This is a schematic diagram of the convolution process of the present invention.

[0022] Figure 6 This is a schematic diagram of the Mamba state space model of the MambaVision model of the present invention.

[0023] Figure 7 This is a schematic diagram of the experiment of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0027] like Figures 1 to 7 As shown, this invention proposes a fine-grained image recognition method and system based on the MambaVision hybrid architecture, including the following: In relevant embodiments of the present invention, the objective is to propose a ship image recognition method with high accuracy, robustness, high computational efficiency, and adaptability to new ships. The technical process is as follows: Figure 1 and Figure 2 As shown, it includes the following: The first step, data augmentation, includes the following: Image preprocessing of the collected dataset containing ship images and their corresponding IMO Number labels mainly includes two parts: (1) Preprocessing of the training data, which involves randomly applying various enhancement transformations to the input images, including multi-scale random scaling and cropping; random horizontal flipping; color jittering; automatic enhancement (AutoAugment, automatic combination of enhancement strategies); random erasure; and mixed enhancement and cut-Mix (cross-sample fusion enhancement methods). (2) Preprocessing of the images to be identified includes: performing image consistency processing, uniformly scaling the images to a fixed resolution, resampling, and normalizing pixel values ​​to ensure input consistency.

[0028] The second step, building the MambaVision image recognition model, includes the following: An image recognition model based on MambaVision is constructed. A four-stage pyramid structure is adopted during training. The bottom layer extracts local features through convolutional operations, while the top layer integrates the Mamba state-space model and window attention mechanism to achieve efficient modeling of both long-range and short-range features, thereby improving recognition accuracy and computational efficiency.

[0029] The third step is to integrate the prediction results, including the following: In addition to the main MambaVision model of this invention, several models with different architectures are trained: (1) ResNet, ConvNeXt and EfficientNet with CNN architecture; (2) EfficientViT and DeiT with Transformer architecture; (3) VisionMamba and MambaOut with Manba architecture; (4) The prediction results of different models are integrated by adopting a dynamic weighting mechanism to further improve the robustness and accuracy of recognition.

[0030] The fourth step is incremental training, which includes the following: To enable the model to adapt to new ships, an incremental learning strategy is adopted: (1) When the model is created, the classification head for new ships will inherit the statistical features of the corresponding classification heads of historical ships and initialize the weights and biases; (2) In the initial stage of training, the learning rate of new ships and historical ships is dynamically adjusted, and the training is divided into two stages: learning new things and reviewing old things, so as to optimize the learning efficiency; (3) Gradient adjustment, on the basis of dynamic differential learning rate, to more finely control the learning process of relevant parameters of new ships and historical ships; (4) Knowledge distillation, distillation loss is introduced into the loss function to learn new ships efficiently while better preserving the existing historical ship recognition ability.

[0031] Among the above steps, data augmentation, image classification model construction, prediction result integration, and incremental training are the key steps in this invention, which will be discussed in detail in the following sections.

[0032] In relevant embodiments of the present invention, data augmentation processing includes the following: The data augmentation strategy of this invention aims to maximize the diversity of training data, simulate complex imaging condition changes in the real environment, and allow the model to learn more fundamentally invariant ship features. For a batch of input training images X∈R B×C×H×W (Where B is the batch size, C is the number of channels, and H and W are the height and width) Perform the following transformations in sequence: (a) Multi-scale random scaling and cropping, including the following: A multi-scale random scaling and cropping strategy is adopted (scaling ratio range of 0.08–1.0, aspect ratio range of 0.75–1.33) to simulate the changes of objects of different sizes and perspectives, thereby improving the robustness of the model to object size and shape.

[0033] ; (1); (2); Where (h) crop w crop) represents the size of the cropping window, ρ represents the scaling ratio, and θ represents the aspect ratio.

[0034] (b) Random horizontal flipping, including the following: Horizontal flipping (with a probability of 0.5) is performed on the images to augment the training data without altering the semantics. For ship recognition tasks, vertical flipping remains off to ensure semantic consistency.

[0035] (3); Where X flipped This is the image after being horizontally flipped.

[0036] (c) Color jitter, including the following: By adjusting brightness, contrast, saturation, and hue (brightness / contrast / saturation perturbation range of 0.4, hue offset range of ±0.1) to simulate lighting changes, the robustness of the model to color conditions is enhanced.

[0037] (4); For example, in terms of brightness, x i 'x' represents the perturbed value of a single pixel. i α is the original pixel value of image X, α is the brightness perturbation amplitude, and β is the offset.

[0038] (d) AutoAugment (automatic combination of enhancement strategies), including the following: The strategy rand-m9-mstd0.5 is adopted to further improve data diversity and model generalization ability by randomly combining various enhancement operations. A sub-strategy is randomly selected from the set of enhancement strategies No. 9, and a series of image enhancement operations (including rotation, shearing, translation, etc.) specified by the sub-strategy are applied. On the basis of the original strength of the strategy, a random perturbation with a standard deviation of 0.5 is introduced.

[0039] (e) Random erasure, including the following: By filling local image regions with pixel noise with a 25% probability (erase probability 0.25, erase at most once per image), the model is forced to focus on global contextual features, which helps to alleviate local overfitting.

[0040] (5); Where R is a rectangular region randomly selected from image X, and ε is the noise value used to reset pixel values.

[0041] (f) Mixup and CutMix (cross-sample fusion enhancement methods), including the following: Soft labels are generated through cross-sample feature fusion (Mixup weight 0.8, CutMix weight 1.0), optimizing the smoothness of the decision boundary. Mixup performs linear interpolation on two random images Xi and Xj in the batch; CutMix pastes a randomly cropped region of Xj into image Xi, and the image classification labels are mixed according to the area ratio.

[0042] (6); (7); in( x mix , y mix (This is the image after Mixup enhancement processing.) x i , y i )and ( x j , y j ) represents two random images in the batch, and λ is the Mixup parameter λ~Beta (0.8, 0.8). (g) Center-fixed proportion cutting, including the following: l By setting the cropping ratio, an image region of size (αH, αW) is cropped from the center of the original image, retaining a fixed percentage of the original image, removing edge noise, and avoiding background interference with subsequent processing: (8) Where I( x, y ) is the original image in coordinates ( x, y The pixel value at position ) is where α is the cropping ratio, and I is the pixel value at position ) C ( x, y ) is the cropped image at coordinates ( x, y The pixel value at ().

[0043] (h) Bicubic interpolation resizing of images, including the following: Adjust the size of the cropped image to the standard size Ht×Wt to ​​maintain a uniform dimension for the model input. Bicubic interpolation uses a weighted average of 16 neighboring pixels to estimate the pixel value of the target point (u, v), producing smoother results and reducing jagged edges and blur.

[0044] (9) Where B(·) is the kernel function for bicubic interpolation, and I r (u,v) is the pixel value at (u,v) of the interpolated image.

[0045] (i) Standardization processes, including the following: Standardize the differenced images by channel, subtract the mean, and divide by the standard deviation to make the pixel value distribution of different images more consistent, reduce the impact of lighting and color differences on the model, and help the model converge faster and improve accuracy.

[0046] (10) I n ( u,v ) is the image after normalization. u,v The pixel value at point is μ, and σ is the mean and standard deviation of each channel of the image.

[0047] In relevant embodiments of the present invention, the construction of the MambaVision image classification model includes the following: MambaVision combines the local feature extraction capabilities of CNNs, the long sequence modeling capabilities of the Mamba State Space Model (SSM), and the global dependency modeling capabilities of self-attention mechanisms. The hybrid model architecture is designed as follows: Figure 2 As shown: (a) A CNN convolutional block, including the following: as follows Figure 4 As shown: The MambaVision model rapidly extracts local features at the bottom layer through convolutional operations on CNN residual blocks. See below. Figure 5 As shown, after the convolutional kernel slides and scans across the input image, it performs element-wise multiplication and accumulation on the corresponding elements in the input image to generate each element of the feature map. The stability of the training is accelerated by a batch normalization layer, and a nonlinear transformation is introduced by using the GELU activation function. Finally, the original input is added to each element of the processed feature map through residual connections (also known as skip connections) to generate the complete feature map.

[0048] (11); (12); (b) The Mamba state-space model, including the following: Long sequence modeling is performed using the Mamba state-space model to capture long-range dependencies within the sequence. The original Mamba causal convolution only sees information preceding the current sequence; replacing it with a regular 1D convolution allows for the inclusion of contextual information, breaking the limitations of sequential computation and enhancing parallel processing capabilities. A dual-branch parallel architecture is designed to fuse the state-space model's (SSM) ability to capture long-range dependencies with the efficiency of traditional convolution in extracting local features. The results are as follows... Figure 6 As shown.

[0049] The SSM branch first reduces the number of channels in the input features through linear projection to decrease computation. Then, it extracts local features using one-dimensional convolution and introduces non-linearity through the SiLU activation function. Finally, it performs a selective scan operation to efficiently fuse global information of the sequence and capture complex long-range dependencies. Let the input be... , The output of the SSM branch is then: (13); Add a symmetric branch: Introduce a non-SSM path (stateless operation) that directly processes features through convolution and SiLU (Sigmoid Linear Unit) activation function, preserving local detail features after convolution and activation transformation, and providing supplementary near-context information.

[0050] (14); Stitching and Projection: The outputs of the two branches, which contain global context and local details, are stitched together and projected back to the original dimension, thus fusing local and global information.

[0051] (15); (c) Self-attention mechanisms, including the following: Introducing a multi-head attention mechanism in the low-resolution stage complements the limitations of sequence models such as Mamba in global modeling capabilities. It calculates the interaction relationships between queries, keys, and values, and dynamically assigns relevance weights to each position in the sequence with all other positions using the scaling dot product attention formula. This efficiently captures long-distance spatial dependencies and achieves deep fusion and modeling of global contextual information.

[0052] (16) In relevant embodiments of the present invention, the prediction results are integrated, including the following: This invention employs a multi-model ensemble strategy based on weighted voting. The design of the ensemble method mainly focuses on two aspects: result selection and strategy optimization. Given the unknown nature of the prediction data, the ensemble process does not rely on model features but uses the prediction score of a single model as the only reliable reference. In the design of the ensemble scheme, this study attempts to select 3-5 models for combination to explore the optimal configuration of the number of models. In terms of the ensemble strategy, a weighted voting mechanism is adopted by combining the prediction results of different models. That is, weights are assigned according to the prediction accuracy of each model on the validation set, and the result with the highest number of votes is taken as the final output.

[0053] Given N heterogeneous models {M1, M2, ..., M... N Its accuracy on the validation set is {acc1, acc2, ..., acc}. N}

[0054] (17) (18) Where X is the input prediction image, P i (X) represents the prediction results of each model for image X, P ensemble The final ensemble prediction probability after weighting multiple models. For the final predicted ship label.

[0055] In relevant embodiments of the present invention, incremental training includes the following: This invention employs a mechanism of inherited model classification head initialization, dynamic preheating, gradient adjustment, and knowledge distillation.

[0056] (a) Initialize the inherited model classification head, including the following: When the model expands from the original Nold class to the Nnew class, it is necessary to intelligently initialize the parameters of the newly added classifier header. Since new ships and historical ships share the same feature space, they should have similar statistical characteristics. A statistically based classifier header initialization method will be adopted. This method samples the weights using the mean and variance of the existing classifier headers to ensure that the weight distribution of new ships is consistent with that of historical ships. The initial activation value of the new ship is reduced by subtracting 2 from the mean bias of historical ships. This reduces the randomness of initialization, accelerates the convergence of new ships, and preserves historical ship knowledge.

[0057] (19) Among them W extend [i] represents the weight of the newly added ship, W old [j] represents the weight of historical ships, N old The number of historical ships.

[0058] ; (20); where b extend [i] is the offset of the newly added ship, b old [j] is the offset of the historical ship, N old is the number of historical ships.

[0059] (b) Dynamic warm-up, including the following: Introduced a dynamic warm-up strategy at the initial stage of incremental training. Gradually increase the learning rate of the newly added ship parameters in the first Ewarmup epochs, while further reducing the historical ship update rate. Divide the incremental learning stage into two stages of learning new knowledge and reviewing old knowledge, and gradually integrate new knowledge into the existing knowledge structure.

[0060] ; (21); ; (22); where η new (e) is the adjusted learning rate of the newly added ship, η old (e) is the adjusted learning rate of the historical ship, e is the current epoch, λ new is the learning rate multiple of the newly added ship, η base is the base learning rate.

[0061] The learning rate adjustment in the warm-up stage follows the following principle: ; (23); When e < Ewarmup, the newly added ship obtains a learning advantage of 20 - 50 times.

[0062] (c) Gradient adjustment, including the following: An optimization strategy based on gradient post-processing. Before the optimizer updates, scale the gradients of the newly added ship and historical ship parameters in the classification head respectively, more precisely balance the learning of new and old ships, and achieve more fine-grained update control.

[0063] ; (24); ; (25); where g old is the updated gradient of the historical ship, g new is the updated gradient of the newly added ship, γ old is the gradient parameter γ of the historical ship old << 1 is used to suppress the historical ship weight update, γ new is the gradient parameter γ of the newly added ship new >> 1 is used to amplify the gradient of the newly added ship.

[0064] (d) Knowledge distillation, including the following: To enable the model to learn new ships Cnew based on historical ships Cold and mitigate knowledge forgetting, the output distribution of the old model (Teacher) is introduced as a soft label to provide relative probability information between ships. Temperature-scaled knowledge distillation is used to fine-tune the new model (Student). By combining classification loss and distillation loss, the model can retain the memory of historical ships to the greatest extent while learning new ships.

[0065] Distillation loss is represented by the KL divergence scaled for temperature: (26); Where L KD For distillation losses on historical ships, The probability output distribution of the new model is only for historical ships. This represents the probability output distribution of the old model, which is only applicable to historical ships. T is the smoothness of the soft label controlled by the distillation temperature parameter.

[0066] The total loss function is a weighted sum of the classification loss (cross-entropy loss between the student model and the true label) and the distillation loss: (27); Where α is the distillation weighting coefficient balancing the two losses, z student This is the predicted probability distribution of the new model for all ships, y ture It is the ship's true label.

[0067] In addition to the above, the present invention also has related embodiments, including the following: This invention utilizes real-world ship image data for thorough experimental validation. The dataset contains 15,776 training samples and 1,752 test samples, covering a total of 508 different ships, used to evaluate the model's performance in terms of ship identification accuracy, computational efficiency, and the feasibility of incremental training. Results examples are provided below. Figure 7 As shown.

[0068] This invention primarily uses Top-1 accuracy as the core evaluation metric, while also recording the training time of a single round to assess the model's training efficiency. For comparative analysis, several advanced models from mainstream architectures (CNN, Transformer, and Mamba) were selected as baseline models for comparison. The comparison results are shown in Table 1 below.

[0069] Table 1 Comparison of accuracy and training time between MambaVision and multi-architecture baseline models. Model categories Model Name Accuracy (%) Training duration (min / epoch) CNN ResNet 88.13 1.1 CNN ConvNexT 88.93 1 CNN EfficientNet 89.29 1.2 Transform EfficientVit 88.86 12 Transform DeiT 92.46 10 Mamba Vision Manba 89.26 9 Mamba MambaOut 92.52 8 Mamba ManbaVision_L2(MambaVision-L2-512-21K) 94.17 10 Mamba ManbaVision_L3(MambaVision-L3-512-21K) 95.71 16 Mamba ManbaVision_L3 (142 epochs) 95.89 16 Mamba ManbaVision_L3 (150 epochs) 96 16 Mamba ManbaVision_L3 (195 epochs) 95.66 16 Mamba ManbaVision_L3 (200 epochs) 95.49 16 Table 1 shows the comparison results of accuracy and training time between MambaVision and multi-architecture baseline models.

[0070] To verify the model's adaptability to the addition of new ships, an incremental learning experiment was designed. Thirty ships were randomly removed from the original training set to simulate future data, and the initial model was trained using the remaining data. Old Class Retention (OCR) and New Class Recognition (NCR) were selected to measure the model's accuracy in recognizing existing and newly added ships, respectively. Ablation experiments involving a warm-up mechanism, gradient adjustment, and knowledge distillation were conducted. The experimental results are shown in Table 2 below.

[0071] Table 2 Ablation experiments of MambaVision incremental learning Processing steps Overall accuracy Old category retention rate New category recognition rate Preheating mechanism 89.14％ 89.56％ 82.20% Preheating mechanism + gradient adjustment 89.36％ 89.38％ 89.00％ Preheating mechanism + gradient adjustment + knowledge distillation 89.78％ 89.83％ 89.00％ Table 2 shows a schematic diagram of the ablation experiment using MambaVision incremental learning.

[0072] The relevant expressions are as follows: (28); (29) Correct old The Correct value represents the total number of correctly predicted ship data points by the model after incremental training. new Total represents the total number of correctly predicted samples for newly added ships in the incremental training model. old To test the total number of existing ship samples, Total new This represents the total number of newly added ship samples in the test set.

[0073] Experimental results show that the MambaVision model improves Top-1 accuracy by 6.11% and 3.5% compared to the highest-accurate CNN and Transformer baseline models, respectively. Furthermore, MambaVision outperforms the Transformer baseline model in training efficiency, achieving a superior balance between accuracy and efficiency, and outperforming multiple baseline models in overall performance. The weighted voting ensemble strategy further improves recognition accuracy by 0.51% on top of the best performance of a single model. After incremental learning, the model maintains an accuracy of 89.83% for recognizing existing ships, effectively mitigating "catastrophic forgetting," and achieves an accuracy of 89.00% for recognizing newly added ships, demonstrating excellent ability to quickly learn new knowledge and continuously learn.

[0074] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A fine-grained image recognition method based on the MambaVision hybrid architecture, characterized in that, Includes the following: Step S1: Perform image preprocessing on the collected dataset containing ship images and their corresponding labels. The preprocessing is divided into training data preprocessing and image preprocessing according to the different uses of the data. The preprocessing process for training data includes: randomly applying various enhancement transformations to the input image, including multi-scale random scaling and cropping; random horizontal flipping; color dithering; automatic enhancement; random erasure; and blending enhancement and cropping blending; the preprocessing process for the image to be recognized includes: performing image consistency processing, including uniformly scaling the image to a fixed resolution, resampling, and normalizing pixel values ​​to ensure input consistency; Step S2: Construct an image recognition model based on MambaVision. The training process of the image recognition model based on MambaVision adopts a four-stage pyramid structure. The bottom layer extracts local features through convolution operations, and the top layer integrates the Mamba state space model and window attention mechanism to achieve efficient modeling of long and short range features, thereby improving recognition accuracy and computational efficiency. Step S3: Integrate the prediction results, including additionally training multiple models with different architectures, and using a dynamic weighting mechanism to integrate the prediction results of different models to improve the robustness and accuracy of recognition; Step S4: Employ an incremental learning strategy to enable the image recognition model to adapt to newly added ship individuals. This includes extending the classification head for new ships during model creation, inheriting the statistical characteristics of the old classification head; dynamically adjusting the learning rates for new and historical ships in the initial training phase, including gradient adjustment and knowledge distillation; initializing weights and biases during image recognition model creation; dynamically adjusting the learning rates for new and historical ships in the initial training phase by dividing training into two stages: learning new knowledge and reviewing old knowledge, thus optimizing learning efficiency; gradient adjustment by finer-grained control over the learning of knowledge about new and old ships based on dynamically differentiated learning rates; and knowledge distillation by introducing distillation loss into the loss function to efficiently learn new ship knowledge while better preserving existing knowledge.

2. The fine-grained ship image recognition method based on the MambaVision hybrid architecture according to claim 1, characterized in that, Step S1 includes the following: Step S11: Randomly apply various enhancement transformations to the input image, including for a batch of input training images X∈R B ×C×H×W The following transformations are performed in sequence: multi-scale random scaling and cropping, random horizontal flipping, color dithering, automatic enhancement, random erasure, blend enhancement and cropping, fixed-ratio cropping at the center, bicubic interpolation to adjust image size, and normalization. Where B is the batch size, C is the number of channels, and H and W are the height and width; Step S12: Multi-scale random scaling and cropping includes employing a multi-scale random scaling and cropping strategy to simulate changes in objects of different sizes and viewpoints and to improve the model's robustness to object size and shape. Step S13: Random horizontal flipping utilizes the left-right symmetry of the image to perform horizontal flipping, thereby expanding the training data without changing the semantics. Vertical flip remains off to ensure semantic consistency; Step S14: Color jittering includes simulating lighting changes by adjusting brightness, contrast, saturation, and hue to enhance the model's robustness to color conditions; Step S15: AutoAugment includes employing a strategy called rand-m9-mstd0.5 to further enhance data diversity and model generalization ability by randomly combining various augmentation operations; Step S16: Random erasure includes filling local regions of the image with pixel noise with a 25% probability, forcing the model to focus on global contextual features to alleviate local overfitting; Step S17: Hybrid enhancement and shearing fusion generate soft labels through cross-sample feature fusion, thereby optimizing the smoothness of the decision boundary; Step S18: Fixed-ratio cropping at the center. By setting the cropping ratio, an image area of ​​a certain size according to the original height and width is cropped from the center of the original image, retaining a fixed percentage of the original image area and removing edge noise to reduce the interference of the background on subsequent processing. Step S19: Adjusting the image size using bicubic interpolation means rescaling the cropped image to a uniform standard input size, so that all images received by the model have a consistent spatial resolution and ensure uniform input dimensions. Step S110: Standardize the differenced images by channel to make the pixel value distribution of different images more consistent, reduce the impact of illumination and color differences on the model, and achieve faster model convergence and improved accuracy.

3. The fine-grained ship image recognition method based on the MambaVision hybrid architecture according to claim 1, characterized in that, Step S2 includes the following: Step S21: Combining the local feature extraction capability of CNN, the image recognition model of MambaVision quickly extracts local features at the bottom layer through the convolution operation of CNN residual blocks. The convolution kernel slides and scans on the input image, performs element-wise multiplication and accumulation on the corresponding elements in the input image to generate each element of the feature map. The batch normalization layer accelerates the training stability, and the GELU activation function is used to introduce nonlinear transformation. Finally, residual connections are used to add each element of the original input and the processed feature map, ultimately generating a complete feature map.

4. The fine-grained ship image recognition method based on the MambaVision hybrid architecture according to claim 3, characterized in that, Step S2 also includes the following: Step S22: The Mamba state space model includes the following: using the Mamba state space model for long sequence modeling to capture long-range dependencies within the sequence; replacing the causal convolutions in the original Mamba with regular 1D convolutions to see contextual information, breaking the limitations of sequential computation and enhancing parallel processing capabilities; and fusing the ability of the state space model to capture long-range dependencies with the efficiency of traditional convolutions in extracting local features through a dual-branch parallel structure design. Step S221: The state space model branch first reduces the number of input feature channels by linear projection to reduce computation, then extracts local features using one-dimensional convolution, introduces nonlinearity through the SiLU activation function, and finally performs selective scanning to efficiently fuse global information of the sequence and capture complex long-distance dependencies. Step S222: Adding a symmetric branch involves introducing a non-SSM path, directly processing features through convolution and SiLU activation functions, preserving local detail features after convolution and activation transformation, and providing supplementary near-context information; Step S223: Stitching and projection involves stitching the outputs of the two branches, which contain global context and local details, and projecting them back to the original dimension to fuse local and global information.

5. The fine-grained ship image recognition method based on the MambaVision hybrid architecture according to claim 4, characterized in that, Step S2 also includes the following: Step S23: Introduce a multi-head attention mechanism in the low-resolution stage to supplement the limitations of the model, including Mamba sequences, in terms of global modeling capabilities. The multi-head attention mechanism calculates the interaction relationship between queries, keys, and values, and dynamically assigns relevance weights to each position in the sequence with all other positions using the scaling dot product attention formula. This captures long-distance spatial dependencies and achieves deep fusion and modeling of global contextual information.

6. The fine-grained ship image recognition method based on the MambaVision hybrid architecture according to claim 1, characterized in that, Step S3 includes the following: A multi-model fusion strategy based on weighted voting is adopted. In terms of the fusion strategy, each model outputs the predicted probability distribution of the input ship image corresponding to each maritime organization number. The fusion weight is assigned to the model according to its recognition accuracy on the validation set. The probability distributions output by different models are weighted and fused, and the maritime organization number with the highest probability after fusion is taken as the final recognition result, so as to achieve stable matching between ship images and unique numbers.

7. The fine-grained ship image recognition method based on the MambaVision hybrid architecture according to claim 1, characterized in that, Step S4 includes the following: Step S41: Initialization of the inherited model classification head includes: When the image recognition model expands from recognizing only historical ships to recognizing both historical and newly added ships, a statistical classification head initialization method will be adopted. By sampling the mean and variance of the weight parameters corresponding to historical ships, the weight distribution corresponding to newly added ships will be kept consistent with that of historical ships. At the same time, the mean of the bias parameters corresponding to historical ships will be down-adjusted to reduce the activation level of newly added ships in the initial stage, thereby reducing initialization randomness, accelerating the convergence speed of newly added ships, and protecting the ability to recognize historical ships during training. Step S42: Dynamic warm-up includes the introduction of a dynamic warm-up strategy in the early stage of incremental training. Within a preset number of training rounds, the learning rate of parameters related to newly added ships is gradually increased, while the update rate of parameters related to historical ships is further reduced. The incremental learning stage is divided into two stages: learning new information and reviewing old information. This allows the new information to be gradually integrated into the existing knowledge structure, thereby mitigating the impact on the recognition performance of historical ships.

8. The fine-grained ship image recognition method based on the MambaVision hybrid architecture according to claim 7, characterized in that, Step S4 also includes the following: Step S43: Gradient adjustment includes an optimization strategy based on gradient post-processing. Before the optimizer performs parameter updates, the gradients corresponding to the parameters related to historical ships and the parameters related to newly added ships are scaled respectively to more finely balance the learning intensity of the two types of parameters and achieve finer-grained update control. Step S44: Knowledge distillation includes introducing the output probability distribution of the model corresponding to the parameters of the historical model as a soft label to provide the relative probability relationship between different ships; the current model is fine-tuned by distillation using temperature scaling, and optimized by combining recognition loss and distillation loss, so that the image recognition model can retain the ability to recognize historical ships to the greatest extent while learning new ships.

9. A fine-grained ship image recognition system based on a MambaVision hybrid architecture, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a fine-grained ship image recognition method based on the MambaVision hybrid architecture as described in any one of claims 1 to 8.

10. A fine-grained ship image recognition system based on a MambaVision hybrid architecture, comprising a computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a fine-grained ship image recognition method based on the MambaVision hybrid architecture as described in any one of claims 1 to 8.

Images