Lightweight real-time identification system for fish body surface abnormalities

Abstract

The application discloses a lightweight real-time identification system for fish body surface abnormalities, comprising: a data processing module for acquiring and preprocessing images containing fish; a backbone network for extracting multi-scale feature maps, the core of which is a local visual state space module containing parallel double paths, which respectively extracts long-distance context and local spatial features based on selective scanning and deep convolution; a lightweight neck network that fuses multi-scale features based on GSConv and VoVGSCSP modules; a detection head module for outputting abnormal categories and positions. The application introduces a parallel fusion mechanism in the backbone network and adopts an extremely lightweight design in the neck, achieving a balance between high detection accuracy and high reasoning speed, significantly reducing computational complexity, having the potential to be deployed to edge devices, and having strong robustness to complex breeding environments.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, and in particular relates to a lightweight real-time recognition system for fish surface abnormalities. Background Technology

[0002] With the continuous expansion of global aquaculture, the industry's demand for refined and intelligent management of the production process is becoming increasingly urgent. Among these, achieving automated, non-contact visual monitoring of the body surface condition of farmed fish is a key technological step in improving aquaculture efficiency and reducing the cost of manual inspections.

[0003] Traditional fish observation relies primarily on visual inspection, often requiring fishing operations, which is inefficient and can disturb or physically damage the fish. Furthermore, this method is highly dependent on human experience and lacks objective, standardized judgment criteria. In recent years, non-contact imaging and analysis methods based on computer vision have become a research hotspot in this field. Automated analysis of underwater image sequences using deep learning models provides a new technological approach for real-time monitoring of fish populations.

[0004] In existing computer vision-based real-time detection methods for fish body surface conditions, the mainstream technical approach still focuses on improving convolutional neural networks (CNNs) and the YOLO (You Only Look Once) series of models. Specifically, while CNN-based methods perform well in terms of detection accuracy, their high computational complexity and inference latency make it difficult to meet the stringent real-time requirements of practical applications. In contrast, models based on improved YOLO series models, with their end-to-end architecture and lower computational overhead, have shown significant advantages in real-time performance. However, these methods often encounter performance bottlenecks in detection accuracy, especially when dealing with complex backgrounds or subtle disease symptoms. Furthermore, Huda et al. proposed a soft-voting fusion strategy that integrates YOLO and the Real-Time Detection Transformer model (RT-DETR) for ornamental fish disease detection. Although this method balances accuracy and speed to some extent, it does not substantially optimize the underlying model structure, and the overall computational complexity remains high, limiting its deployment potential in resource-constrained or high-throughput scenarios.

[0005] RT-DETR is a high-efficiency detector for real-time object detection tasks. Its core innovation lies in the introduction of a hybrid encoder structure and a query selection strategy based on uncertainty minimization, successfully extending the DETR framework to real-time applications. Experiments on the COCO val2017 dataset show that RT-DETR maintains high inference speed while having certain advantages in detection accuracy compared to current mainstream real-time detection frameworks. Thanks to its good balance between performance and efficiency, RT-DETR has been widely used in various practical fields such as agriculture and industry. However, the AIFI (Adaptive Intra-scale Feature Interaction) module used in RT-DETR, as a variant of the Transformer architecture, relies on a multi-head self-attention mechanism for feature modeling. The number of parameters and computational cost of this mechanism increases significantly with the number of input feature channels. Compared with multi-scale feature fusion methods based on lightweight convolutional operations, such as Feature Pyramid Networks (FPN) and Bidirectional Feature Pyramid Networks (BiFPN), AIFI has significantly higher model complexity and parameter scale. Furthermore, the core computational unit of the Transformer self-attention mechanism has O(N) time complexity when the input sequence length is N. 2 The computational complexity is significantly increased. In object detection tasks, input feature maps are typically flattened into token sequences, causing N to increase rapidly with image resolution. In the AIFI module, the need for cross-scale feature interaction further exacerbates the overall computational burden. This characteristic is particularly pronounced when processing high-resolution images, limiting the model's deployment efficiency and scalability in resource-constrained environments.

[0006] To address the high computational complexity of self-attention mechanisms, State Space Models (SSMs) have gradually become an effective alternative in visual representation learning in recent years. As a sparser and structured replacement for self-attention, the core advantage of SSMs lies in their ability to capture long-range contextual dependencies with linear complexity. However, existing SSMs are primarily designed for one-dimensional sequence modeling. When directly applied to two-dimensional image data, simply flattening the image into a one-dimensional sequence ignores spatial structure information. Furthermore, since sequence length increases quadratically with image resolution, it can introduce significant computational burden. In addition, while SSMs excel in sequence modeling, they often lack inherent spatial awareness, making it difficult to effectively utilize two-dimensional relationships in visual applications. To address this, a method called 2D-Selective-Scan (SS2D) has been proposed to resolve the mismatch between one-dimensional scanning and two-dimensional image data. Figure 1 As shown, SS2D traverses each input patch along four different scan paths and processes each sequence independently through the S6 module, finally merging the outputs. While SS2D demonstrates strong long-range contextual information modeling capabilities in tasks requiring large-scale spatial relationships, such as scene parsing, semantic segmentation, and complex object detection, its application in fish surface anomaly detection relies more on local textures, edges, and subtle color differences for discrimination, rather than large-scale semantic relationships spanning the entire image, because anomalies are typically small-scale local details with a very small proportion of the overall image. In underwater imaging, these subtle features are often masked by complex environmental factors, including turbidity, suspended particles, and uneven lighting. Although the global environment helps locate fish within the frame, accurate classification of health status depends more on capturing local textures, edges, and subtle color changes. Therefore, to optimize performance for this specific task, it is necessary to balance SS2D's long-range modeling and enhanced local feature extraction, ensuring the model remains sensitive to local pathological details in potentially noisy underwater backgrounds. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a lightweight real-time identification system for fish surface abnormalities, thereby resolving the issues present in the prior art.

[0008] To achieve the above objectives, the present invention provides a lightweight real-time identification system for fish surface abnormalities, comprising:

[0009] The data processing module is used to acquire and preprocess the input image; the input image is an image containing fish.

[0010] A backbone network is used to extract features from a preprocessed input image to obtain multi-scale feature maps. The backbone includes at least one local visual state space module. The local visual state space module includes a first processing path and a second processing path that are set in parallel and whose outputs are fused. The first processing path extracts long-range contextual information based on a selective scanning mechanism, and the second processing path extracts local spatial features based on depthwise convolution.

[0011] A neck network is used to perform feature fusion on the multi-scale feature maps and output an enhanced feature map.

[0012] The detection head module is used to receive the enhanced feature map and output the category label and bounding box coordinates of the abnormal regions on the fish body surface in the input image based on the Transformer decoder structure.

[0013] The data processing module, the backbone network, the neck network, and the detection head module are connected in sequence to work together to identify abnormalities on the fish's body surface.

[0014] Optionally, the preprocessing performed by the data processing module includes: performing contrast-limited adaptive histogram equalization on the input image; and performing image sharpening on the image after contrast-limited adaptive histogram equalization using a sharpening convolution kernel.

[0015] Optionally, the backbone network further includes a preliminary downsampling module, a channel merging module, and a fast spatial pyramid pooling module;

[0016] The preliminary downsampling module is used to downsample the preprocessed input image twice to generate a first feature map;

[0017] The backbone network includes at least one feature extraction unit consisting of a local visual state space module and a channel merging module connected in sequence; the first feature map is input to the local visual state space module in the first feature extraction unit, and the output of the channel merging module in each feature extraction unit is used as the input of the next feature extraction unit or the input of the fast spatial pyramid pooling module;

[0018] The fast spatial pyramid pooling module is used to perform multi-scale contextual information fusion on the feature map output by the last channel merging module.

[0019] Optionally, the preliminary downsampling module uses the GELU activation function after the first downsampling and the SiLU activation function after the second downsampling.

[0020] Optionally, the local visual state space module further includes a conditional projection layer;

[0021] The conditional projection layer is used to adjust the number of channels in the input feature map to the desired number of channels by point convolution, batch normalization and GELU activation function when the number of channels in the input feature map is inconsistent with the desired number of channels in the module.

[0022] The first processing path sequentially includes the conditional projection layer, a lightweight convolutional block, a normalization layer, and a two-dimensional selective scanning unit; the lightweight convolutional block sequentially includes a depthwise convolutional layer, a first point convolutional layer, and a second point convolutional layer; and the lightweight convolutional block adds its input feature map to the output feature map of the second point convolutional layer through residual connections, which is used as the output of the lightweight convolutional block.

[0023] Optionally, the second processing path is a local convolutional unit; the local convolutional unit sequentially includes a 3×3 depth convolutional layer, a batch normalization layer, and a SiLU activation function;

[0024] The residuals of the output features of the two-dimensional selective scanning unit and the output features of the conditional projection layer are added together, and the result is used as the input of the local convolution unit.

[0025] Optionally, the local visual state space module uses learnable weighting parameters to perform weighted fusion of the output features of the two-dimensional selective scanning unit and the output features of the local convolution unit to obtain the module's final output.

[0026] Optionally, the neck network is a Slimneck structure built based on GSConv and VoVGSCSP modules;

[0027] The neck network receives multi-scale feature maps output by the backbone network; the multi-scale feature maps include a first-scale feature map, a second-scale feature map, and a third-scale feature map; the third-scale feature map is upsampled and then concatenated with the second-scale feature map, and the concatenation result is input to the first VoVGSCSP module; the feature map output by the first VoVGSCSP module is upsampled and then concatenated with the first-scale feature map; the concatenation result is processed sequentially through the second VoVGSCSP module and the first GSConv unit; the feature map output by the second VoVGSCSP module is processed sequentially through the second GSConv unit and the third VoVGSCSP module; the outputs of the first VoVGSCSP module, the second VoVGSCSP module, and the third VoVGSCSP module are used together as the enhanced feature map.

[0028] Optionally, the GSConv unit divides the input feature map into two parts in the channel dimension, performs a depthwise convolution on one part and concatenates it with the other part of the feature map, and performs channel mixing on the concatenated feature map to obtain an output feature map with the same number of channels as the input.

[0029] The VoVGSCSP module compresses and divides the input feature map into two branches along the channel dimension. After feature processing is performed on one branch using two GSConv units, it is concatenated with the other branch. The number of channels in the concatenated result is adjusted to the target number of channels through linear mapping, and the final feature map is output. Only the first GSConv unit uses the SiLU activation function, while the second GSConv unit does not use any activation function.

[0030] Optionally, the detection head module is RTDETRDecoder; the loss function used by the detection head module is a weighted summation classification loss, bounding box L1 regression loss, and distribution focus loss.

[0031] Compared with the prior art, the present invention has the following advantages and technical effects:

[0032] The system provided by this invention significantly improves the ability to perceive subtle anomalies on the fish's body surface while greatly reducing the number of model parameters and computational complexity through a Local Visual State Space (LocalVSS) module that fuses long-range context and local features in parallel within the backbone network, and an ultra-lightweight Slimneck structure used in the neck area. This achieves an excellent balance between high detection accuracy and real-time inference speed. Its lightweight nature makes the model potentially deployable on resource-constrained edge devices, reducing hardware costs and deployment barriers. Furthermore, targeted image preprocessing and model enhancement designs effectively improve the system's robustness and adaptability in complex real-world aquaculture environments such as low light and turbid water, ensuring reliability for practical applications. Attached Figure Description

[0033] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0034] Figure 1 This is a schematic diagram of a two-dimensional selective scanning module, which is part of the background technology of this invention.

[0035] Figure 2 This is a schematic diagram comparing images before and after enhancement according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the RT-GalaDet model structure according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the Stem module structure according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of the LocalVSS module structure according to an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of the test set confusion matrix according to an embodiment of the present invention;

[0040] Figure 7 This is a heatmap of the model attention in an embodiment of the present invention;

[0041] Figure 8 These are simulated images of the real environment in embodiments of the present invention;

[0042] Figure 9 This is a schematic diagram of the VoVGSCSP and GSConv module structures in an embodiment of the present invention. Detailed Implementation

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0045] Example 1

[0046] This embodiment provides a lightweight real-time identification system for fish surface abnormalities, including:

[0047] (1) Data acquisition and preprocessing;

[0048] The dataset used in this embodiment comes from the publicly available dataset fish-project Computer Vision Model. After removing redundant categories from the dataset, 6049 bounding boxes were retained, which include 20 categories of four types of fish: striped rock seabream, black seabream, rockfish, and red seabream. Each of the four types of fish has one healthy category and four abnormal categories. Each fish has at most one abnormal state on its body surface.

[0049] In real-world aquaculture environments, the number of healthy fish is usually far greater than that of abnormal fish, resulting in significant class imbalance in the collected image samples. To address this, the dataset constructed in this embodiment has a higher sample size for each of the four healthy fish species than for their corresponding abnormal fish. Given the imbalanced data distribution, this embodiment employs stratified sampling to divide the images in the dataset into an 80% training set, a 10% validation set, and a 10% test set. Furthermore, to enable the model to learn the features of the minority classes more effectively, reduce its over-reliance on background information, and improve its robustness, this embodiment uses a copy-paste method to augment the data for the classes with fewer samples in the training set. In this method, to simulate target occlusion in real-world environments and improve the model's general robustness while preserving the effectiveness of the original image annotations, the occlusion ratio between annotation boxes is set to no more than 0.2. The paste transparency is set to 0.7 to improve the blending of the paste boundaries. The random seed is set to 42, and the number of randomly augmented original samples is increased by 1-3 times. Using this method, 16 anomalous samples were augmented, resulting in a final training set containing 6762 bounding boxes. The number of bounding boxes for each category is shown in Table 1. In Table 1, Doldom represents Striped Beakfish (Oplegnathus fasciatus); Gamseongdom represents Black Sea Bream (Acanthopagrus schlegelii); Jopi-bollag represents Scorpionfish (Schlegel's Black Rockfish (Sebastes schlegelii); and Chamdom represents Red Sea Bream (Pagrus major).

[0050] Table 1

[0051]

[0052] Because this embodiment uses the CopyPaste method to expand categories with a small number of samples in the training set, background overlay from different shooting environments appears in the enhanced images, resulting in significant differences in grayscale value ranges in different regions of the image. To prevent these differences from causing abnormal shifts in the overall brightness and contrast of the image, and to suppress the amplification of noise during the enhancement process, this embodiment first applies the CLAHE (Contrast Suppression Adaptive Histogram Equalization) method to the entire dataset with a contrast limit threshold of 2 and a grid size of 8*8 to improve the overall contrast of the image. Furthermore, considering the general characteristics of abnormal fish features in the dataset, this embodiment further sharpens the dataset with a sharpening convolution kernel with a center value of 5 and a surrounding value of -1 to highlight local details and texture features in the image. Image comparison before and after enhancement is shown below. Figure 2 As shown, this can improve the model's ability to perceive and distinguish subtle features to a certain extent. Figure 2 In the image, (a) shows a healthy red sea bream, (b) shows an abnormal red sea bream, (c) shows an image processed with CopyPaste, and (d) shows an image processed with CLAHE and sharpening.

[0053] (2) Network architecture;

[0054] The RT-GalaDet network consists of three main parts: the backbone, the neck, and the head. The structure diagram is as follows: Figure 3 As shown in the diagram, the Backbone part is an improvement on the YOLO architecture's Backbone, consisting of the initial downsampling module Stem, the Local Visual State Space module (LocalVSS), the ChannelMerge module, and the Spatial Pyramid Pooling-Fast module (SPPF). It aims to extract features from different dimensions of the input image and extract long-range contextual information while maintaining the model's focus on local information. Furthermore, compared to the original RT-DETR model using a CNN architecture, the YOLO architecture RT-DETR model used in this embodiment is more lightweight and efficient, making it more suitable for real-time detection scenarios and better meeting the requirements of lightweight computing devices in fish surface anomaly detection tasks. The Neck part consists of VoVGSCSP (a lightweight feature fusion module combining VoVNet and GSConv designs) and GSConv (a lightweight convolution method that mixes standard convolution and depthwise convolution), designed to fuse the multi-scale features extracted by the Backbone. The head is ultimately predicted by RTDETRDecoder regression.

[0055] (3) Backbone network;

[0056] (a) Stem;

[0057] Because SSM lacks positional awareness of the image, the model may treat all regions in the image, including the background, equally. For example... Figure 4 The Stem design shown adopts a lightweight structure, aiming to provide local priors for the model and reduce background interference. Stem reduces the input space size of the image to one-quarter of its original size through two downsampling operations, while finally mapping the channels to 128, which facilitates the processing of subsequent modules. In order to better preserve details and gradients in low-level features, this embodiment draws on the experience of VisionTransformer and uses GELU as the activation function after the first downsampling, as shown in Equation (1). GELU (Gaussian Error Linear Unit) can retain some negative values ​​in the calculation results, avoiding information being discarded too early. In order to maintain consistency with the mainstream activation in the detection task, SiLU (Sigmoid Linear Unit) is used as the activation function after the second downsampling to maintain strong signals in the high-dimensional semantic space.

[0058] (1)

[0059] (b) LocalVSS;

[0060] Since abnormal areas on the fish's body surface are mostly small targets, strategies that enhance local features can significantly improve model performance in small target detection tasks. To retain the lightweight computational characteristics of SS2D and supplement the model's effective perception of local features while relying on long-range context modeling, thus balancing the model's capabilities in long-range context perception and local feature perception, this embodiment proposes the LocalVSS module. This module mainly consists of three sub-modules: Projection, LiteConvBlock (LCBlock, lightweight convolutional block), and LocalConv. Module details are as follows... Figure 5 As shown.

[0061] When the feature map is input to the LocalVSS module, it first uses a conditional projection layer to determine the number of channels in the feature map. If the number of channels in the feature map is different from the expected number of channels in the module, it will be converted to the expected number of channels in the module through point convolution, BatchNorm2d and GELU activation function. Otherwise, it will skip the projection layer and directly enter LCBlock. LCBlock is a lightweight residual convolutional block. The feature map first goes through a depthwise convolution to extract the local spatial features of the feature map and reduce the spatial cost of model computation. Then it goes through two point convolutions and restores the number of channels by performing a linear transformation in the channel dimension, realizing the channel interaction of the module. This design can mix and separate spatial convolution and channel convolution, which can significantly reduce the number of parameters and computation of the module compared with the standard 3*3 convolution. Finally, a residual structure is introduced to connect the original input feature map and the new feature map, which plays a role in ensuring information flow and stable training. The calculation process of LCBlock is shown in Equation (2).

[0062] (2)

[0063] Here, DWConv refers to depthwise convolution, and PWConv refers to pointwise convolution.

[0064] After passing through LCBlock, the feature map is normalized using Norm to stabilize the feature distribution and avoid gradient vanishing or exploding. SS2D then extracts long-range contextual information from the feature map, and the result is summed with the residual of the initial projected feature map as input to LocalConv. Related research shows that enhancing local features or adding local enhancement branches can significantly improve the model's performance on small objects and image details. This embodiment designs a lightweight local enhancement module, LocalConv. A 3x3 depthwise convolution is used to extract the local space of each channel, followed by BatchNorm and SiLU. A learnable parameter α controls the weight of this branch. α is initialized to 0, making this branch harmless to the model in the early stages of training while also serving as a warmup. During training, the model can automatically adjust the value of this parameter as it learns the local features of the feature map, which helps maintain model stability and controllability. At the end of the backbone, this embodiment introduces SPPF. As a variant of SPP, SPPF can cascade pooling computations with the same kernel size, resulting in lower computational cost and making it more suitable for real-time object detection tasks. Furthermore, SPPF integrates multi-scale contextual information at low computational cost, effectively improving the model's robustness to targets of different sizes.

[0065] (c) ChannelMerge;

[0066] In RT-GalaDet, each LocalVSS is connected using ChannelMerge. This paper introduces a space-to-depth method to downsample the feature map, placing the four sub-positions of each 2*2 pixel block into different channels of the same pixel. Unlike traditional pooling operations, the space-to-depth downsampling method does not discard pixel information of the feature map; it only rearranges the spatial information onto the channels, and subsequent point convolutions fuse this local information, reducing the number of model parameters and computational overhead.

[0067] (d) Neck network;

[0068] In the original YOLO-style neck, C2f (the feature extraction module in YOLOv8) is often used to aggregate multi-scale information. Its multi-layer residual stacking structure gives it powerful learning capabilities. However, because C2f contains multiple 3*3 convolutional and residual units, the original neck has a large number of parameters and FLOPs, which can easily become a performance bottleneck in lightweight models. To better adapt to the lightweight model requirements in fish surface anomaly detection applications, RT-GalaDet introduces a more lightweight Slimneck based on VoVNet as the model's neck. The Slimneck mainly consists of VoVGSCSP and GSConv.

[0069] The neck network receives multi-scale feature maps output by the backbone network; the multi-scale feature maps include a first-scale feature map, a second-scale feature map, and a third-scale feature map; the third-scale feature map is upsampled and then concatenated with the second-scale feature map, and the concatenation result is input to the first VoVGSCSP module; the feature map output by the first VoVGSCSP module is upsampled and then concatenated with the first-scale feature map; the concatenation result is processed sequentially through the second VoVGSCSP module and the first GSConv unit; the feature map output by the second VoVGSCSP module is processed sequentially through the second GSConv unit and the third VoVGSCSP module; the outputs of the first VoVGSCSP module, the second VoVGSCSP module, and the third VoVGSCSP module are used together as the enhanced feature map.

[0070] The GSConv unit divides the input feature map into two parts along the channel dimension. After performing a depthwise convolution on one part, it concatenates it with the other part of the feature map. The concatenated feature map is then channel-mixed to obtain an output feature map with the same number of channels as the input.

[0071] The VoVGSCSP module compresses and splits the input feature map into two branches along the channel dimension. After performing feature processing on one branch using two GSConv units, it is concatenated with the other branch. The number of channels in the concatenated result is adjusted to the target number of channels through a linear mapping, and the final feature map is output. Only the first GSConv unit uses the SiLU activation function, while the second GSConv unit does not use any activation function.

[0072] Furthermore, the feature map output by SPPF is upsampled and concatenated with the output of LocalVSS in the Backbone. Then, it undergoes feature fusion by VoVGSCSP. After two layers of VoVGSCSP processing, the feature map is channel-mixed by GSConv before subsequent VoVGSCSP modules. Finally, the output feature maps from the last three VoVGSCSP modules are passed to RtedtrDcoder for decoding. GSConv first divides the input feature map's channels into two equal parts. One part undergoes depthwise convolution with a 5x5 kernel and is then concatenated with the other half of the feature map. To prevent feature aggregation along channel partitions and to save computational overhead from memory dimension transformation, GSConv mixes the channels to obtain an output feature map with the same number of channels as the input feature map. Similar to GSConv, VoVGSCSP first compresses the input channels and divides them into two branches. GSConv performs feature aggregation on one branch, which is then concatenated with the other branch and projected back to the target channel. Thus, while maintaining the expressive power of the model, the model's parameters and computational cost are significantly reduced.

[0073] like Figure 9 As shown, GSConv first performs a standard 3×3 convolution on the input feature map to obtain an intermediate feature representation. This intermediate feature is then further transformed using a 5×5 depthwise convolution. Next, the feature maps from the 3×3 and 5×5 convolution branches are fused through a concatenation operation. To avoid feature aggregation being limited to specific channel partitions and to reduce the additional computational and storage overhead caused by channel dimension rearrangement, GSConv introduces a channel rearrangement operation after fusion. This mechanism can fully mix feature information across different channels, thereby generating the final output feature map while maintaining the same number of channels before and after rearrangement.

[0074] As a lightweight feature fusion module, VoVGSCSP first compresses the number of channels in the input features and divides them into two parallel branches, each with a channel width half that of the output channels. One branch aggregates features through two consecutive GSConv layers, while the other branch acts as a shortcut, directly passing the input features. It's important to note that to maintain the integrity of the feature representation, only the first GSConv layer uses the SiLU activation function, while the second GSConv layer does not use any activation function. Subsequently, the features from the two branches are concatenated and restored to the target number of channels through a linear mapping. This design effectively maintains the model's expressive power while significantly reducing the overall parameter size and computational complexity.

[0075] (e) Loss function;

[0076] In terms of loss functions, this embodiment uses the classification loss, bounding box L1 regression loss, and distribution focal loss from RT-DETR. A comprehensive loss score is obtained by weighting the three losses with weights of 0.5, 7.5, and 1.5 respectively. Using y... i ∈0,1 represents the true label of the i-th sample, p i ∈R represents the predicted class output by the model, σ represents the sigmoid function, and the classification loss is calculated as shown in Equation (3). For the i-th predicted sample, pᵢ represents the classification score predicted by the model, and qᵢ represents the corresponding target score. Specifically, for positive samples, qᵢ is assigned the IoU value between the predicted box and the ground truth box; while for negative samples, qᵢ is set to 0. The hyperparameters α and γ are usually set to 0.75 and 2.0, respectively, consistent with the standard implementation of RT-DETR. By introducing an IoU-based target allocation mechanism, high-quality positive samples contribute more weight in the optimization process, while the focus modulation term is used. Suppressing easily classified negative samples, this classification loss can effectively improve the optimization efficiency and stability in multi-class symptom detection tasks; in bounding box regression, it uses... The predicted bounding box is represented by b=(x,y,w,h), which represents the position of the ground truth bounding box. The L1 regression loss of the bounding box is calculated as shown in Equation (4) to constrain the deviation between the predicted bounding box and the ground truth bounding box in terms of position and scale. In addition, to further improve the accuracy of bounding box localization, this embodiment introduces the distributed focus loss, which is calculated as shown in Equation (5). For the continuous ground truth coordinates y∈[0,n−1], we define l=⌊y⌋ and r=l+1, which represent the two discrete integers closest to y, respectively. The model outputs a probability distribution vector through the Softmax layer. In formula (5), p l and p r These represent the components of the probability vector at indices l and r, respectively. This loss function effectively improves the granularity and localization accuracy of bounding box regression by guiding the probability mass towards discrete integers near the true continuous coordinates.

[0077] (3)

[0078] (4)

[0079] (5)

[0080] This embodiment utilizes an improved YOLO architecture-based RT-DETR computation model to achieve real-time detection of fish surface anomalies, enabling model training, validation, and testing. The experimental environment included a computer running Linux, equipped with an NVIDIA GeForce RTX 4090D (24GB) graphics card and an AMD EPYC 9754 CPU; the software environment consisted of CUDA 12.4, Python 3.9, and PyTorch 2.8.

[0081] By combining multiple metrics to evaluate the model, the overall performance of the model can be comprehensively assessed. For the model evaluation metrics, Precision, Recall, mAP50 (mAP, mean Average Precision), and mAP50-95 were used to evaluate the computational accuracy of the model. For the model scale, Parameters and GFLOPs (Giga Floating-point Operations Per Second) were used for evaluation. Precision measures the proportion of truly diseased fish among samples predicted as diseased by the model, while Recall measures the proportion of successfully detected fish among all truly diseased fish. TP represents the number of correctly detected diseased fish, FP represents the number of healthy fish misclassified as diseased fish, and FN represents the number of truly diseased fish that were not detected by the model. Precision and Recall can be calculated using equations (6) and (7).

[0082] (6)

[0083] (7)

[0084] The average precision (AP) for each category is defined as the area under its precision-recall (PR) curve. Following standard practice in the COCO evaluation protocol, this embodiment employs a full-point interpolation strategy to ensure that the interpolated precision does not increase with recall, and to mitigate evaluation bias caused by local ranking fluctuations.

[0085] Specifically, for a given category, the interpolation precision P at a recall of r. interp (r) is defined as the maximum of the measured precisions among all recalls that satisfy r̃≥r. Then, the AP for that class is obtained by integrating the interpolated precision over the interval [0,1], as shown in Equation (8). Here, k represents the index of the sampling point when performing discrete sampling along the PR curve.

[0086] (8)

[0087] Since this embodiment involves C different categories, mAP is used as the overall performance evaluation index, that is, the AP of all categories under a given IoU threshold. c Take the average value. Specifically, mAP50 represents the average AP calculated under the IoU=0.5 threshold condition, and its definition is shown in formula (9), where APc This represents the average precision of class c.

[0088] To conduct a more rigorous evaluation of target localization accuracy and multi-category detection performance, this embodiment further calculates the mAP50-95 index. This index is calculated by taking the average of 10 IoU thresholds selected in the range of 0.50 to 0.95 with a step size of 0.05, and then using the average of the results. This provides a more comprehensive reflection of the model's overall detection performance under different overlap accuracy requirements.

[0089] (9)

[0090] Regarding the computational complexity of the model, this embodiment uses Parameters to measure the total number of all learnable parameters in the model, and GFLOPs to measure the number of floating-point operations required for the model to complete one forward inference, thereby evaluating the model's storage and computing performance requirements on the deployment device.

[0091] In the experimental stage, the input sample size of RT-GalaDet was adjusted to 640*640 pixels. During model training, the batch size was set to 16, the number of training epochs was set to 200, the AdamW optimizer was used, the initial learning rate was set to 0.0001, the warmup was set to 2000 steps, and mixed precision computing (AMP) was enabled for training. The comparison before and after model preprocessing is shown in Table 2. On various benchmark models and RT-GalaDet of this invention, the metrics calculated by the model after data preprocessing show that the data preprocessing method of this invention helps to improve the training effect of the model.

[0092] Table 2

[0093]

[0094] In deep learning model evaluation, the confusion matrix is ​​a key evaluation tool used to visualize a model's performance on classification tasks. The confusion matrix of a model on the test set is calculated by... Figure 6As shown in the diagram, the confusion matrix reveals that the model still exhibits some misclassifications in the classification task, primarily in eye diseases and ulcers. This is presumably due to the small lesion areas in these two types of diseases, resulting in insufficient accuracy in detecting subtle local features. Furthermore, the limited number of diseased fish samples in the dataset used in this embodiment leads to insufficient feature learning by the model, causing it to misclassify these two anomalies as belonging to the healthy category of the same fish species. However, the model achieves high accuracy in most categories, demonstrating excellent overall classification accuracy. To address these issues, future work can supplement the number of diseased fish samples through real-world photography or adversarial neural network generation; more advanced data augmentation methods can be used to expand the limited number of samples, increasing the number of features the model can learn. Finally, the model can be further optimized by introducing a lightweight local attention mechanism to enhance its ability to distinguish fine-grained lesions.

[0095] To better evaluate the performance of RT-GalaDet, this embodiment uses an enhanced fish surface lesion dataset and compares it with other real-time detection models. The comparison results are shown in Table 3. The comparison results in the table show that traditional YOLO series models have a certain advantage in the number of model parameters, but their precision and recall are lower than RT-GalaDet in most cases. This indicates that YOLOv8s has limited ability to identify fine-grained lesions in the fish surface abnormality detection scenario. RTDETR-YOLOv8s, after introducing the Transformer structure, shows improved detection accuracy, but there is still room for optimization in mAP. For the YOLO series models, RT-GalaDet achieves the highest accuracy improvement of 25.3% compared to YOLOX, which is roughly equivalent to YOLO12s, which has the highest accuracy. However, RT-GalaDet's recall, mAP50, and mAP50-95 are improved by 9.8%, 4.3%, and 3.6% respectively compared to YOLO12s. While YOLO-NAS achieves comparable mAP to RT-GalaDet with lower GFLOPs, RT-GalaDet optimizes precision, recall, and model parameter count by 11.1%, 9.3%, and 29.6%, respectively. Similarly, RT-GalaDet achieves comparable mAP to YOLOX while significantly outperforming it in precision and recall. Therefore, RT-GalaDet offers a better balance between lightweight design and high-precision detection. As classic RT-DETR models, RTDETR-Resnet18 and RTDETR-L achieve high precision, but their high parameter count and computational complexity make them less suitable for real-time detection compared to the model in this embodiment. Specifically, compared to the large-scale RTDETR-L, RT-GalaDet reduces the number of parameters by 59.21% and GFLOPs by 76.88%, while maintaining superior precision and achieving higher recall. Even compared to the more lightweight RTDETR-Resnet18, RT-GalaDet achieves significant reductions in parameter count and GFLOPs, by 33.3% and 57.1%, respectively. While the existing lightweight model NanoDet-m is superior in terms of weight reduction, its detection accuracy is insufficient for practical applications.The RT-GalaDet proposed in this embodiment achieves comprehensive performance of 93.3% Precision, 89.7% Recall, 89.0% mAP50, and 79.0% mAP50–95 while maintaining low parameter and computational costs. Compared to RTDETR-YOLOv8s, it improves Precision, Recall, mAP50, and mAP50–95 by 1.2%, 1.9%, 1.5%, and 1.2%, respectively, while reducing GFLOPs by 9.1%. This demonstrates that the model proposed in this embodiment can effectively reduce computational costs while maintaining feature representation capabilities, achieving an effective balance between detection accuracy and computational speed. According to industry consensus in computer vision, a model with an FPS of 30 is generally considered to meet real-time requirements. Experiments showed that the model proposed in this embodiment achieved an FPS (Frames Per Second, used to measure inference speed) of 51.98 with a batch size of 1, which can be considered to meet the real-time requirements for detecting fish surface lesions. These experimental results demonstrate that RT-GalaDet has strong practical application value and potential in real-time detection of fish surface lesions.

[0096] Table 3

[0097]

[0098] To verify the effectiveness of the improved algorithm, five ablation experiments were designed. These experiments used the same dataset and hyperparameters on the same computational apparatus, with fixed detection heads for training and testing to control the determinism and comparability of the experimental process. The experimental groups included the original RTDETR-YOLOv8s model, a model using SS2D with the Neck retained but the Backbone replaced, a model using SS2D and LCBlock, a model using SS2D, LCBlock, and LocalConv, and a model incorporating all Backbone improvements while replacing the Neck. As shown in Table 4, the ablation experiment results show that after introducing SS2D into RTDETR-YOLOv8s to become RTDETR-SS2D, the model's GFLOPs decreased due to its simplified computational nature. In terms of performance, the model's Recall and mAP improved, but Precision decreased. This is because SS2D places greater emphasis on spatial information extraction, increasing the response range of low- and mid-level features. This leads to an enhanced response to blurred boundaries, small targets, or blurred backgrounds during prediction, thus increasing the probability of false detections. After adding LCBlock, the enhanced local spatial modeling leads to a sparser feature distribution. This strengthens discriminativity at the expense of some target coverage, resulting in increased precision but decreased recall and mAP. The performance degradation caused by adding LCBlock is reversed by adding LocalConv, which compensates for the activation loss due to LCBlock's excessive selectivity, improving the model's response to small targets and edges. A balance is achieved between global and local features, significantly improving overall model performance. Finally, this embodiment introduces Slimneck on the Neck. The stable performance of precision and mAP50-95, the increase in recall and mAP50, and the decrease in parameter count and GFLOPs verify that the model improves computational performance while reducing computational complexity. Ablation experiments demonstrate that the final model outperforms the initial RTDETR-YOLOv8s model in balancing computational performance and overhead.

[0099] Table 4

[0100]

[0101] Figure 7 This paper presents a heatmap of class saliency obtained based on the Grad-CAM method, used to visualize the distribution of the model's response to discriminative regions during prediction. This heatmap is computed on the feature map of the last convolutional layer of the network. Figure 7This is the result of calculating the attention heatmap of the model using Grad-cam. As you can see, the model correctly captured... Figure 7 (a) Abnormal areas on the surface of the fish. Figure 7 In model (b), only the long-range contextual attention module of SS2D was added. The model's attention is widely distributed across various regions of the image. Figure 7 (b) to Figure 7 In (e), due to the addition of local feature enhancement and channel blending operations, the model's attention gradually converges to the lesion region, shifting from long-range information to capturing fine-grained features. A high-intensity, highly focused circular activation kernel is formed at the abnormality site, demonstrating the model's sensitivity to local features while acquiring long-range contextual information for modeling. This verifies that the model achieves more cohesive and reliable attention while reducing computational overhead, demonstrating excellent performance in detecting fish surface lesions.

[0102] Compared to ideal laboratory environments, real-world fish farming scenarios typically feature low light levels and turbid water, posing new challenges to underwater imaging quality. To verify the model's generalization ability and test its performance in real-world farming scenarios, this study assumes that image features maintain visual discriminability and performs simulated augmentation on a complete dataset including training, validation, and test sets. Figure 8 As shown, all random augmentations apply a global random seed of 42 to ensure the repeatability of the augmentation process. Figure 8 In (a), the original image is obtained by randomly reducing its brightness by 10%-30% and its contrast by 10%-15%. Figure 8 (b) in the diagram aims to simulate low-light conditions. To simulate turbid water conditions, the original image is Gaussian blurred with a random kernel size of 2-4 pixels to simulate light scattering effects; RGB channel shifting is performed within a random range of ±(3,10) with a 40% probability to simulate the common greenish and yellowish tints in turbid water; random Gamma variations are applied within a range of ±15% with a 30% probability to reflect the brightness differences at different water depths, ultimately resulting in... Figure 8 Image (c) is obtained by superimposing the enhancement methods of images (b) and (c). Figure 8 (d) in the model aims to simulate the superposition of low light and water quality blurring. After processing the images as described above, no further optimization or enhancement was performed. The results of evaluating the model with the original hyperparameters are shown in Table 5.

[0103] Table 5

[0104]

[0105] The results show that the model has strong robustness. In low-light and murky scenes, without optimization using other image enhancement methods, mAP50 only decreased by 3.1% and 3.3%, respectively. Even when the two simulated scenes were mixed and superimposed, mAP50 only decreased by 3.3%. This indicates that the proposed RT-GalaDet model can effectively extract features even when visual details are degraded. Notably, in the mixed simulated scene, recall dropped to 84.6%, while precision unexpectedly increased to 93.6%. This phenomenon can be attributed to the confidence suppression effect caused by severe noise. Strong environmental interference suppressed the confidence scores of blurred targets and potential background false positives below the detection threshold. Therefore, the model adopts a more conservative prediction strategy, only outputting high-confidence targets with obvious features. Although this trade-off leads to a lower recall, it significantly reduces false warnings and ensures high reliability of detected abnormalities on the fish surface.

[0106] To achieve a better balance between detection accuracy and inference efficiency, this embodiment proposes a novel real-time object detection method for fish surface symptom detection tasks—Real-Time DETR with Global-Local Adaptive Enhancement for Efficient Object Detection (RT-GalaDet). Based on the RT-DETR framework, this method significantly improves the model's lightweight nature and inference speed while maintaining high detection accuracy, thus better meeting the practical needs of real-time, efficient, and accurate detection of abnormal states on fish surfaces in aquaculture scenarios. RT-GalaDet effectively enhances the model's ability to perceive subtle anomalies by introducing an adaptive enhancement mechanism that considers both global and local factors, achieving accurate detection of fish surface conditions. The purpose of this embodiment is to provide a lightweight model that is easy to deploy on mobile devices to monitor the fish farming environment in real time. When fish with abnormal body surface conditions are detected, an alarm is promptly issued to the fish farmers. The detected abnormalities also assist professionals in diagnosing fish diseases, allowing fish farmers to intervene in the early stages of fish disease symptoms, reducing the inspection costs of fish farming and minimizing economic losses caused by fish diseases.

[0107] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A lightweight real-time identification system for fish surface abnormalities, characterized in that, include: The data processing module is used to acquire and preprocess the input image; the input image is an image containing fish. A backbone network is used to extract features from a preprocessed input image to obtain multi-scale feature maps. The backbone includes at least one local visual state space module. The local visual state space module includes a first processing path and a second processing path that are set in parallel and whose outputs are fused. The first processing path extracts long-range contextual information based on a selective scanning mechanism, and the second processing path extracts local spatial features based on depthwise convolution. The backbone network also includes a preliminary downsampling module, a channel merging module, and a fast spatial pyramid pooling module; The preliminary downsampling module is used to downsample the preprocessed input image twice to generate a first feature map; The backbone network includes at least one feature extraction unit consisting of a local visual state space module and a channel merging module connected in sequence; the first feature map is input to the local visual state space module in the first feature extraction unit, and the output of the channel merging module in each feature extraction unit is used as the input of the next feature extraction unit or the input of the fast spatial pyramid pooling module; The fast spatial pyramid pooling module is used to perform multi-scale contextual information fusion on the feature map output by the last channel merging module. The local visual state space module also includes a conditional projection layer; The conditional projection layer is used to adjust the number of channels in the input feature map to the desired number of channels by point convolution, batch normalization and GELU activation function when the number of channels in the input feature map is inconsistent with the desired number of channels in the module. The first processing path sequentially includes the conditional projection layer, lightweight convolutional block, normalization layer and two-dimensional selective scanning unit; the lightweight convolutional block sequentially includes a depth convolutional layer, a first point convolutional layer and a second point convolutional layer; and the lightweight convolutional block adds its input feature map to the output feature map of the second point convolutional layer through residual connection, as the output of the lightweight convolutional block; A neck network is used to perform feature fusion on the multi-scale feature maps and output an enhanced feature map. The neck network is a Slimneck structure built based on GSConv and VoVGSCSP modules; The detection head module is used to receive the enhanced feature map and output the category label and bounding box coordinates of the abnormal regions on the fish body surface in the input image based on the Transformer decoder structure. The data processing module, the backbone network, the neck network, and the detection head module are connected in sequence to work together to identify abnormalities on the fish's body surface.

2. The lightweight real-time identification system for fish surface abnormalities according to claim 1, characterized in that, The preprocessing performed by the data processing module includes: performing contrast-limited adaptive histogram equalization on the input image; and performing image sharpening on the image after contrast-limited adaptive histogram equalization using a sharpening convolution kernel.

3. The lightweight real-time identification system for fish surface abnormalities according to claim 1, characterized in that, The initial downsampling module uses the GELU activation function after the first downsampling and the SiLU activation function after the second downsampling.

4. The lightweight real-time identification system for fish surface abnormalities according to claim 1, characterized in that, The second processing path is a local convolutional unit; the local convolutional unit includes, in sequence, a 3×3 depth convolutional layer, a batch normalization layer, and a SiLU activation function; The residuals of the output features of the two-dimensional selective scanning unit and the output features of the conditional projection layer are added together, and the result is used as the input of the local convolution unit.

5. The lightweight real-time identification system for fish surface abnormalities according to claim 4, characterized in that, The local visual state space module uses learnable weighted parameters to weight and fuse the output features of the two-dimensional selective scanning unit and the output features of the local convolution unit to obtain the module's final output.

6. The lightweight real-time identification system for fish surface abnormalities according to claim 1, characterized in that, The neck network receives multi-scale feature maps output by the backbone network; the multi-scale feature maps include a first-scale feature map, a second-scale feature map, and a third-scale feature map; the third-scale feature map is upsampled and then concatenated with the second-scale feature map, and the concatenation result is input to the first VoVGSCSP module; the feature map output by the first VoVGSCSP module is upsampled and then concatenated with the first-scale feature map; the concatenation result is processed sequentially through the second VoVGSCSP module and the first GSConv unit; The feature map output by the second VoVGSCSP module is processed sequentially through the second GSConv unit and the third VoVGSCSP module; The outputs of the first VoVGSCSP module, the second VoVGSCSP module, and the third VoVGSCSP module are used together as the enhanced feature map.

7. The lightweight real-time identification system for fish surface abnormalities according to claim 6, characterized in that, The GSConv unit divides the input feature map into two parts in the channel dimension, performs a depthwise convolution on one part and concatenates it with the other part of the feature map, and performs channel mixing on the concatenated feature map to obtain an output feature map with the same number of channels as the input. The VoVGSCSP module compresses and divides the input feature map into two branches along the channel dimension. After feature processing is performed on one branch using two GSConv units, it is concatenated with the other branch. The number of channels in the concatenated result is adjusted to the target number of channels through linear mapping, and the final feature map is output. Only the first GSConv unit uses the SiLU activation function, while the second GSConv unit does not use any activation function.

8. The lightweight real-time identification system for fish surface abnormalities according to claim 1, characterized in that, The detection head module is RTDETRDecoder; the loss function used by the detection head module is a weighted summation classification loss, bounding box L1 regression loss, and distribution focus loss.

Images