Method and device for identifying surface symptoms of diseased fish and network training and data augmentation method thereof

Abstract

The application discloses a method and device for identifying symptoms on the surface of diseased fish, and a network training and data augmentation method thereof, and belongs to the field of image recognition. To solve the problem of identifying fine-grained information such as disease symptoms of fish body surface images, the method comprises the following steps: a first C3K2-PMSB component is used to extract a global difference feature map and a local detail feature map according to a third feature map; a fourth feature map is extracted according to the global difference feature map and the local detail feature map; a first DBiFusion component is used to obtain a first spliced feature map and a second spliced feature map by splicing a feature map based on a fifth feature map with a second feature map and a third feature map respectively according to the fifth feature map; the first spliced feature map and the second spliced feature map are fused according to feature weights determined by an activation function; and a second DBiFusion component is used to obtain a third spliced feature map and a fourth spliced feature map by splicing a feature map based on a first fused feature map with a first feature map and a second feature map respectively according to the first fused feature map; and the third spliced feature map and the fourth spliced feature map are fused according to feature weights determined by an activation function.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image recognition, and relates to a method and device for recognizing symptoms on the surface of diseased fish, as well as a network training and data augmentation method thereof. BACKGROUND

[0002] Fish farming is an important part of aquaculture, and high-density farming mode has gradually become the main means to improve production efficiency. However, this high-density farming mode has brought many problems, especially frequent diseases, which seriously threaten fish health, increase the risk of farming and economic losses, and has become a key problem restricting the sustainable and healthy development of the industry. Disease diagnosis and treatment is an effective means to control diseases, and symptom recognition on the surface of diseased fish as a basic technology for disease diagnosis plays a crucial role. Through the method of image recognition, fine-grained recognition of external symptoms of diseased fish is realized, thereby providing a reliable basis for subsequent accurate disease identification, and then taking effective treatment measures, providing an important guarantee for fish farming, improving production efficiency and farming quality, and realizing the sustainable development of aquaculture.

[0003] However, to achieve this goal, there are two major challenges: insufficient samples and symptoms that are easily confused. In the field of aquaculture, due to the difficulty in obtaining samples of diseased fish, the image data available for training is relatively scarce, and this data shortage limits the learning ability of the model; at the same time, the visual representation of multiple diseases is highly similar, such as the symptoms of bleeding and ulceration often appearing at the same time, their appearance features are similar and the pathological processes are related to each other, which further increases the difficulty of recognition.

[0004] Data augmentation technology, as an effective means to solve the problem of insufficient data, expands the existing data set to increase the diversity of training samples, thereby improving the robustness and generalization ability of the model. Specifically, image data augmentation methods mainly include two aspects: image transformation-based methods and image synthesis-based techniques. Image transformation-based methods include flipping, cropping, random erasing, rotation, and color adjustment. The latter includes mixed images, mainly including mixup, feature space expansion, and generative adversarial networks (GANs), which can generate new synthetic samples with certain semantic meaning. Although the augmentation strategy based on image synthesis can significantly improve the diversity of the data set, this method usually needs to augment data based on a certain amount of data. On the contrary, the image transformation-based method only needs to perform simple transformation on the original image, but has certain limitations in increasing sample diversity.

[0005] Given the high similarity among fish surface symptoms, which can easily lead to identification confusion, a high-performance model is crucial. Computer vision, with its superior image processing capabilities, can accurately extract feature information from target objects, providing a technological foundation and possibility for the identification of diseased fish. With the combination of computer vision and deep learning technologies, more and more research attempts to achieve automatic identification of diseased fish by analyzing changes in their morphology. Huang et al. proposed a method combining CNN and online sequence extreme learning machine to automatically identify diseased fish in aquaculture. However, this method has high model complexity and consumes significant computational resources, making it difficult to deploy effectively in actual factory farming scenarios due to hardware limitations. To address this issue, Wang et al. improved YOLOv5 by replacing lightweight convolutions to facilitate deployment in factory farming. However, facing the problem of poor model training performance caused by occlusion from diseased fish in real-world scenarios, Cai et al. improved the YOLOv7 object detection algorithm using adaptive enhancement algorithms and attention mechanisms to better handle the occlusion problem in diseased fish detection. Furthermore, most of the aforementioned studies focus on coarse-grained fish disease identification tasks, that is, they can only determine whether fish are sick, while the identification of fine-grained information such as the severity of the disease and specific symptoms has not been explored in depth. To address this deficiency, Li et al. proposed the multi-task learning network YOLO-FD, which adds a semantic segmentation detection head to YOLOv8, enabling the network to determine the degree of infection of sick fish while identifying them, thus providing the possibility for more refined disease analysis. However, this method has not yet achieved the explicit identification of specific symptom types.

[0006] In summary, while computer vision technology has made some progress in identifying diseased fish, it still faces the following challenges:

[0007] (1) The limited amount of data on diseased fish affected the accuracy of identification;

[0008] (2) Most existing studies focus on coarse-grained diseased fish identification tasks, which can only determine whether a fish is diseased, while the identification of fine-grained information such as disease symptoms still needs further research. Summary of the Invention

[0009] To address the aforementioned problem of identifying fine-grained information such as disease symptoms in images of diseased fish, in a first aspect, according to some embodiments of this application, a diseased fish surface symptom identification device is based on a YOLO network, which includes a backbone network, a neck network, and a detection head;

[0010] The backbone of the YOLO network is used to input images of the fish's body surface, including...

[0011] The first C3K2 component extracts the first feature map;

[0012] The second C3K2 component extracts the second feature map;

[0013] The third C3K2 component extracts the third feature map;

[0014] The first C3K2-PMSB component extracts a global difference feature map and a local detail feature map based on the third feature map, and then extracts a fourth feature map based on the global difference feature map and the local detail feature map; and

[0015] The attention mechanism obtains the fifth feature map from the fourth feature map through the attention mechanism of the backbone network;

[0016] Among them, the neck network of the YOLO network includes

[0017] The first DBiFusion component, based on the fifth feature map, obtains a first concatenated feature map and a second concatenated feature map by concatenating the feature map based on the fifth feature map with the second and third feature maps respectively; it then fuses the first concatenated feature map and the second concatenated feature map according to the feature weights determined by the activation function to obtain a first fused feature map; and

[0018] The second DBiFusion component obtains a third and a fourth concatenated feature map by concatenating the feature map based on the first fused feature map with the first feature map and the second feature map, respectively; and fuses the third and fourth concatenated feature maps according to the feature weights determined by the activation function to obtain the second fused feature map.

[0019] The detection head predicts the type of symptoms on the fish's body surface based on the second fusion feature map.

[0020] According to some embodiments of the present application, a diseased fish surface symptom recognition device includes a backbone network, which further comprises...

[0021] The first Conv component is used to input the fish body surface image into the first Conv component.

[0022] The second Conv component, the output of the first Conv component is input to the second Conv component, and the output of the second Conv component is input to the first C3K2 component;

[0023] The third Conv component extracts the first feature map from the first C3K2 component and inputs it into the third Conv component. The output of the third Conv component is input into the second C3K2 component.

[0024] The fourth Conv component, the second C3K2 component extracts the second feature map and inputs it into the fourth Conv component, the output of the fourth Conv component is input into the third C3K2 component; and

[0025] The fifth Conv component extracts the third feature map from the third C3K2 component and inputs it into the fifth Conv component. The output of the fifth Conv component is input into the first C3K2-PMSB component.

[0026] The attention mechanism of the backbone network includes...

[0027] SPPF component, the fourth feature map extracted by the first C3K2-PMSB component is input to the SPPF component; and

[0028] C2PSA component, the input and output of SPPF component C2PSA component.

[0029] According to some embodiments of the present application, a diseased fish surface symptom recognition device includes a neck network, further comprising...

[0030] The sixth Conv component, the C2PSA component's output input to the sixth Conv component, and the sixth Conv component's output input to the first DBiFusion component;

[0031] The fourth C3K2 component is the input of the first fused feature map output from the first DBiFusion component.

[0032] The seventh Conv component, the output and input of the fourth C3K2 component, the seventh Conv component, and the output and input of the seventh Conv component are the second DBiFusion component.

[0033] According to some embodiments of the present application, a diseased fish surface symptom recognition device includes a neck network, further comprising...

[0034] The first component, including the fifth C3K2 component, the eighth Conv component, and the first Concat component, outputs the sixth feature map;

[0035] The second component includes the sixth C3K2 component, the ninth Conv component, and the second Concat component, and its output is the seventh feature map; and

[0036] The second C3K2-PMSB component extracts a global difference feature map and a local detail feature map based on the seventh feature map; it then extracts a tenth feature map based on the global difference feature map and the local detail feature map. The extraction of the tenth feature map includes the step of concatenating the seventh feature map, the global difference feature map, and the local detail feature map.

[0037] The detection head includes a first detection head, a second detection head, and a third detection head;

[0038] Among them, the fifth C3K2 component of the first component extracts the eighth feature map based on the second fusion feature map, and the first detection head predicts the type of fish surface symptoms based on the sixth feature map;

[0039] Among them, the sixth C3K2 component of the second component extracts the ninth feature map based on the sixth feature map, and the second detection head predicts the type of symptoms on the fish's body surface based on the ninth feature map;

[0040] Among them, the third detection head predicts the type of symptoms on the fish's body surface based on the tenth feature map.

[0041] According to some embodiments of the present application, a diseased fish surface symptom identification device includes a C3K2-PMSB component.

[0042] The tenth Conv component includes a first output and a second output, and the feature map is input to the tenth Conv component;

[0043] The first Split function component includes a first output and a second output, and the first output of the tenth Conv component is input to the first Split function component;

[0044] The eleventh Conv component includes the first output, the second output, and the third output, and the first output input of the first Split function component is the eleventh Conv component;

[0045] The twelfth Conv component includes the first output, the first output input of the eleventh Conv component, and the twelfth Conv component.

[0046] PMSB, including the first output, the first output input of the twelfth Conv component C3K2-PMSB component;

[0047] The first fusion unit includes a first input, a second input, and a first output, wherein the first output of the PMSB is input to the first input of the first fusion unit, and the second output of the eleventh Conv component is input to the second input of the first fusion unit.

[0048] The thirteenth Conv component includes the first output, the third output input of the eleventh Conv component, and the thirteenth Conv component.

[0049] The third Concat component includes a first input, a second input, and a first output; the first output input of the first fusion unit is the first input of the third Concat component; the first output input of the thirteenth Conv component is the second input of the third Concat component;

[0050] The fourteenth Conv component includes the first output, the first output input of the third Concat component, and the fourteenth Conv component.

[0051] The fourth Concat component includes a first input, a second input, a third input, and a first output; the first output input of the fourteenth Conv component is the first input of the fourth Concat component; the second output input of the tenth Conv component is the second input of the fourth Concat component; the second output input of the first Split function component is the third input of the fourth Concat component;

[0052] The fifteenth Conv component includes a first output, which is used as the output of the C3K2-PMSB component, and the first output input of the fourth Concat component is the fifteenth Conv component.

[0053] A diseased fish surface symptom recognition device according to some embodiments of this application, wherein the PMSB includes

[0054] The second Split function component includes a first output and a second output, with the feature map input to the second Split function component;

[0055] The Rearrange function component includes a first output and a second output, and the first output input of the second Split function component is the Rearrange function component;

[0056] The first convolution, the first output input of the Rearrange function component, extracts global differential features;

[0057] The second convolution, the second output input of the Rearrange function component, extracts local detail features;

[0058] The second fusion unit concatenates the second output of the second Split function component, global difference features, and local detail features for use as the output of the PMSB component.

[0059] According to some embodiments of the present application, a diseased fish surface symptom recognition device is used, wherein, based on a fifth feature map, a first spliced ​​feature map and a second spliced ​​feature map are obtained by splicing the feature map based on the fifth feature map with a second feature map and a third feature map, respectively; the first spliced ​​feature map and the second spliced ​​feature map are fused according to the feature weights determined by the activation function to obtain a first fused feature map, as shown in the following formula:

[0060]

[0061]

[0062] In the formula, This represents the first fused feature map. This represents the Sigmoid function. This means taking the mean of all elements in the feature map. This represents the first concatenated feature map. This represents the second spliced ​​feature map;

[0063] Based on the first fused feature map, a third and fourth concatenated feature map are obtained by concatenating the feature map based on the first fused feature map with the first feature map and the second feature map, respectively. The third and fourth concatenated feature maps are then fused according to the feature weights determined by the activation function to obtain the second fused feature map, as shown in the following formula:

[0064]

[0065]

[0066] In the formula, This represents the second fused feature map. This represents the Sigmoid function. This means taking the mean of all elements in the feature map. This represents the third spliced ​​feature map. This represents the fourth spliced ​​feature map.

[0067] In a second aspect, a data augmentation method for the YOLO network training set in a diseased fish surface symptom recognition device according to some embodiments of this application includes...

[0068] The fish in the fish body surface images in the original training set are used as the segmentation objects to segment the background image of the fish body surface images to obtain the segmentation objects;

[0069] Perform geometric transformations on the segmented objects;

[0070] The segmented objects are randomly combined and placed on the background image to obtain a composite image;

[0071] Adjust the composite image so that the combined area of ​​the segmented objects occupies 1 / 3 to 2 / 3 of the area of ​​the background image;

[0072] If the overlapping area of ​​randomly combined segmented objects exceeds a set value, the segmented objects will be adjusted according to the following formula:

[0073]

[0074]

[0075] In the formula:

[0076] The center point of the bounding box of the original segmented object is located at Coordinates along the axis, This indicates that the center point of the bounding box of the segmented object after adjustment is at... Coordinates along the axis;

[0077] The center point of the bounding box of the original segmented object is located at Coordinates along the axial direction, This indicates that the center point of the bounding box of the segmented object after adjustment is at... Coordinates along the axis;

[0078] This represents the distance between the center point of the bounding box of the original segmented object and the width of the bounding box. This indicates the distance between the center point of the bounding box of the segmented object and the width of the bounding box after adjustment.

[0079] This represents the distance between the center point of the bounding box of the original segmented object and the height of the bounding box. This represents the center point of the bounding box of the segmented object and the height of the bounding box.

[0080] Indicates rotation Width after angle ;

[0081] Indicates rotation The height after the angle, ;

[0082] Indicates the scaling factor. express Parameters for fine-tuning axial displacement. express Parameters for fine-tuning axial displacement.

[0083] In three aspects, according to the training method of the YOLO network in the diseased fish surface symptom recognition device according to some embodiments of this application, the training loss function is as follows:

[0084]

[0085] in, Represents the loss function. This represents the intersection-over-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Indicates the weight threshold;

[0086]

[0087] in, This represents the average IoU value of all current samples; This represents the average IoU value from the previous update; This represents the initial exponential moving weight decay rate; This represents the exponential moving average decay factor. Indicates the number of times the loss calculation has been updated; The time constant that controls the decay rate of time.

[0088] In a fourth aspect, the method for identifying the surface symptoms of diseased fish according to some embodiments of this application is based on a YOLO network, wherein the YOLO network includes a backbone network, a neck network, and a detection head; wherein the identification method includes...

[0089] The backbone network takes a fish body surface image as input and extracts global difference feature maps and local detail feature maps based on the third feature map.

[0090] The backbone network extracts a fourth feature map based on the global difference feature map and the local detail feature map; the extraction of the fourth feature map includes the step of concatenating the third feature map, the global difference feature map and the local detail feature map;

[0091] The fourth feature map is used to obtain the fifth feature map through the attention mechanism of the backbone network;

[0092] The neck network obtains a first concatenated feature map and a second concatenated feature map by concatenating the feature map based on the fifth feature map with the second feature map and the third feature map, respectively.

[0093] The first concatenated feature map and the second concatenated feature map are fused according to the feature weights determined by the activation function to obtain the first fused feature map;

[0094] The neck network obtains a third and a fourth spliced ​​feature map by concatenating the feature map based on the first fusion feature map with the first feature map and the second feature map, respectively.

[0095] The third and fourth spliced ​​feature maps are fused according to the feature weights determined by the activation function to obtain the second fused feature map;

[0096] The detection head predicts the type of symptoms on the fish's body surface based on the second fusion feature map.

[0097] Beneficial Effects: Effective disease prevention and control is crucial for ensuring the profitability of fish farming, and accurate identification of surface symptoms is the foundation for precise disease prevention and control in fish. Existing methods for identifying surface symptoms in diseased fish suffer from insufficient sample data and high similarity of symptom features, leading to poor identification results. This invention, a method for identifying surface symptoms in diseased fish, is based on the YOLOv11-SDiseasedFishNet network model. Firstly, to address the problem of insufficient symptom samples, a weighted random combination self-embedded augmentation strategy (RAFA) is designed. This method segments diseased fish images and randomly combines them, simulating possible occlusion in a realistic background environment to generate synthetic images with semantic plausibility and visual diversity.

[0098] To address the identification confusion caused by similar symptom features, this invention optimizes the model from two aspects: improving symptom information extraction capabilities and enhancing the dynamic utilization of low-dimensional features. Firstly, a multi-scale convolutional feature enhancement module, C3K2-PMSB, is designed. By introducing a multiple receptive field structure, it achieves simultaneous extraction of global features and local details. Secondly, a multi-dimensional adaptive fusion method, DBifusion, is proposed to enhance the model's dynamic utilization of low-dimensional features. Through the synergistic effect of the C3K2-PMSB component and DBifusion, the identification confusion caused by similar symptom features is effectively solved, achieving accurate identification of diseased fish symptoms.

[0099] To verify the effectiveness of the method, ablation and comparative experiments were conducted on a self-built dataset. Regarding data augmentation, compared with traditional augmentation methods, the mean accuracy, recall, and mean precision were improved by 4.1%, 4.2%, and 5.7%, respectively. In terms of network structure, YOLOv11-SDFishNet achieved mean accuracy, recall, and mean precision of 94.8%, 93.9%, and 97.1%, respectively. Compared with current detection models (such as YOLOv12, RT-DETR, and YOLOv10UD), the mean precision was improved by 5%, 16%, and 5.3%, respectively. The results indicate that YOLOv11-SDFishNet can accurately identify the symptoms of diseased fish, providing strong support for the prevention and control of fish diseases.

[0100] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0101] Figure 1 Images of the symptoms on the body surface of diseased fish.

[0102] Figure 2 Augmented flowchart.

[0103] Figure 3 Images generated from the symptoms on the surface of diseased fish.

[0104] Figure 4 Comparison of data volume before and after embedding augmentation.

[0105] Figure 5 : Tag effect image.

[0106] Figure 6 YOLOv11-SDFishNet network structure diagram.

[0107] Figure 7 :C3K2-PMSB component.

[0108] Figure 8 PMSBConv component structure diagram.

[0109] Figure 9 DBiFusion architecture diagram.

[0110] Figure 10 Visualization of feature maps before and after DBiFusion: (a) shows the effect of the features extracted by the backbone network mainly retaining high-dimensional semantic information; (b) shows the effect after fusing the low-dimensional features provided by the DBiFusion module.

[0111] Figure 11 Comparison of detection performance between YOLOv11 and YOLOv11-SDiseasedFishNet.

[0112] Figure 12 Performance comparison of YOLOv11 and YOLOv11-SDiseasedFishNetmAP@0.5 / %.

[0113] Figure 13 Comparison of heatmaps for each model. Detailed Implementation

[0114] The embodiments of this application are described in detail below with reference to the accompanying drawings. The method for identifying the surface symptoms of diseased fish according to the present invention includes...

[0115] S10. Dataset Construction;

[0116] S20. Network model construction;

[0117] S30. Network model training and evaluation; and

[0118] S40. Use network models to identify symptoms on the surface of diseased fish.

[0119] In some embodiments of the present invention, step S10, dataset construction, includes...

[0120] S11. Acquisition of the Basic Dataset for Diseased Fish Symptoms: The diseased fish surface symptom dataset of this invention covers a variety of common farmed fish, including grass carp, common carp, and tilapia, fully reflecting the diversity and representativeness of the data, which helps to evaluate the stability and generalization ability of the algorithm on different fish species. This dataset classifies fish surface diseases into four categories: hemorrhage, ulceration, white spot, and lag. This classification method is not only scientifically sound but also closely reflects the characteristics of diseases in actual aquaculture scenarios. The dataset design is based on real-world scenarios and includes two types of samples: healthy fish and diseased fish. Among them, the diseased fish samples are further subdivided into the above four typical surface symptoms, each of which is accurately labeled to ensure high-quality data and usability.

[0121] The original data for this invention primarily comes from images of the surface symptoms of diseased fish in real-world scenarios. Data was collected through various means, including collaboration with the Aquatic Animal Hospital of Dalian Ocean University, consulting relevant websites and books, and actual filming at the Daheishi Modern Fishery Base of Dalian Tianzheng Co., Ltd. Ultimately, 695 unique fish samples, including both diseased and healthy fish images from both above and below water, were collected for constructing a dataset of surface symptoms of diseased fish. Part of the dataset is shown below. Figure 1 As shown.

[0122] S12. Embedding Augmentation Methods: The success of a model largely depends on a high-quality dataset. In practical applications, due to the limited number of diseased fish image samples, direct training can easily lead to model overfitting. Therefore, data augmentation techniques are typically used to transform the original samples in various ways, generating new samples with differences, thereby effectively expanding the data scale, improving the model's generalization ability, and reducing the risk of overfitting.

[0123] Image augmentation methods based on image synthesis and image transformation do have limitations. The former can generate semantically rich synthetic samples but relies on a sufficient amount of data, while the latter involves simple transformations but has limited diversity. To overcome this limitation and to simulate realistic occlusion situations, this invention proposes a self-embedding augmentation method that integrates a weighted random combination strategy for generating a dataset of diseased fish symptoms. This invention first uses the original dataset and employs SAM to accurately segment diseased fish objects, assigning weights to each symptom category, with smaller sample categories receiving higher weights. In the selection phase of the segmented objects, a two-step random selection mechanism is used: first, the first symptom category and its samples are selected according to weights; then, samples from the second symptom category are selected from the remaining categories for combination. This method effectively solves the data imbalance problem by dynamically adjusting the sampling probabilities of different categories, ensuring that smaller sample categories have a higher chance of participating in the generation of new samples, thereby improving the diversity of the dataset and the effectiveness of model training.

[0124] Furthermore, considering the significant differences in appearance and characteristics between fish on land and fish in water, it is essential to perform independent augmentation processing on fish images in these two scenarios to maintain data reliability.

[0125] A flowchart illustrating how fish on land can increase in number is shown below. Figure 2 As shown, it includes:

[0126] (1) Selection of background image and segmentation objects: When generating the composite image, a background image is first randomly selected, and the segmentation objects are randomly grouped using a weighted random combination strategy. At the same time, the size of the two is controlled to be relatively balanced according to the maximum length and width of the objects.

[0127] (2) Transformation processing of segmented objects: In order not to interfere with the model's learning of the symptoms of diseased fish, it was decided not to modify the attributes of diseased fish such as color. Only geometric transformations such as horizontal flipping, scaling, and rotation were performed on the selected segmented objects.

[0128] (3) Placement and overlap detection of segmented objects: The transformed segmented objects are randomly placed on the background image. If the overlap between segmented objects is too large and key information is obscured, the system will try to rearrange the objects by means of position fine-tuning, scaling adjustment or rotation optimization, so as to reduce mutual occlusion and optimize the layout, as shown in formula (1).

[0129]

[0130]

[0131] in, The center point of the bounding box of the original segmented object is located at Coordinates along the axial direction, This indicates that the center point of the bounding box of the segmented object after adjustment is at... Coordinates along the axis; The center point of the bounding box of the original segmented object is located at Coordinates along the axis, This indicates that the center point of the bounding box of the segmented object after adjustment is at... Coordinates along the axis; This represents the distance between the center point of the bounding box of the original segmented object and the width of the bounding box. This indicates the distance between the center point of the bounding box of the segmented object and the width of the bounding box after adjustment. This represents the distance between the center point of the bounding box of the original segmented object and the height of the bounding box. This represents the center point of the bounding box of the segmented object and the height of the bounding box. Indicates rotation Width after angle ; Indicates rotation The height after the angle, ; Indicates the scaling factor. express Parameters for fine-tuning axial displacement. express Parameters for fine-tuning axial displacement.

[0132] (4) Optimization and adjustment of the composite image: Finally, the composite image is optimized by adjusting the blank area (the area where the non-segmented object is located) to improve the proportion of the image, ensuring that each embedded image is unique and as close to the real situation as possible, so that the area of ​​the combination of segmented objects occupies 1 / 3 to 2 / 3 of the area of ​​the background image.

[0133] like Figure 3 The diagram shows the augmentation effect for each symptom type. (a) illustrates the data augmentation effect in an underwater scene, while (b) presents an example of data augmentation in a shore-based scene. The comparison of the number of augmentations is shown in the graph. Figure 4 .

[0134] Compared to traditional methods that only augment the entire image, the method proposed in this invention first accurately segments the diseased fish and then uses weighted random combinations to effectively alleviate the data imbalance problem. Image augmentation is performed separately for different scenarios, ensuring the authenticity and reliability of the data. By performing geometric transformations on the segmented objects rather than modifying color or texture, interference with disease features is avoided, while simulating realistic occlusion enhances the model's robustness, thereby generating new samples with semantic rationality and visual diversity. This approach not only overcomes the limitations of the original data distribution and enhances the model's ability to learn the features of the target itself, but also effectively alleviates the overfitting problem caused by insufficient data, significantly improving the model's generalization performance.

[0135] S13. Data Preprocessing and Labeling: First, the 1016 images of diseased fish surface symptoms were labeled using the rectangular bounding box tool of Labelme labeling software. Multiple labels were applied for healthy fish, diseased fish, bleeding, ulceration, white spots, and missing parts, with the categories named: "Healthy fish," "Sick fish," "Hemorrhage," "Ulceration," "WhiteSpot," and "Lack," respectively. Each diseased fish was labeled with its corresponding symptoms. The labeled result is shown below. Figure 5 As shown, after annotation, a txt file containing location coordinates and category names will be generated. Next, the annotated data will be divided into training, validation, and test sets in a 7:1:2 ratio. To ensure the reliability of the evaluation results, the entire process is automated by computer.

[0136] In some embodiments of the present invention, step S20, network model construction, includes...

[0137] S21. Overall Design of the YOLOv11-SDFishNet Network Model: The similarity in characteristics of different symptoms on the body surface of diseased fish, and the different characteristics of the same symptoms in different diseased fish, greatly complicate the identification of diseased fish symptoms. Furthermore, to achieve factory farming and facilitate rapid adjustments to farming strategies, a lightweight model is essential. Therefore, the selected model must possess both powerful feature extraction capabilities and lightweight design. YOLOv11 is one version of the YOLO series of algorithms, surpassing previous YOLO versions and other mainstream models of the same period in terms of the balance between accuracy and detection speed. Considering the model's efficiency, effectiveness, and practical application environment, the YOLOv11 model series was ultimately selected. Among the YOLOv11 model series, YOLOv11n balances lightweight design and feature extraction capabilities. Therefore, YOLOv11n is used as the baseline model in this invention.

[0138] The YOLOv11 model consists of three parts: the input layer, the backbone network, and the head network. The input layer unifies the image to a resolution of 640×640. The backbone network comprises convolutional neural networks for extracting image features, utilizing a C3K2 component structure to obtain richer gradient flows, facilitating the extraction and learning of more abundant image features. The head network includes a neck and a detector layer, used for feature fusion and target prediction, respectively. The neck is an intermediate layer connecting the backbone and detector, significantly improving the predictive performance of the detector through feature fusion and resolution adjustment strategies. The detector integrates a classifier and a regressor, predicting the target's location and category based on the output features of the neck.

[0139] Although YOLOv11 is an advanced real-time object detection algorithm that performs well in most scenarios, the similarity of symptoms directly reduces its accuracy in identifying symptoms in diseased fish. Furthermore, the imbalanced sample problem exacerbates the neglect of a minority of sample types, increasing the difficulty of identifying diseased fish symptoms. To address these issues, this invention proposes a network for identifying the surface symptoms of diseased fish, enabling accurate differentiation of individual fish exhibiting diseased symptoms in images. Its network structure is as follows: Figure 6 As shown, firstly, based on the original feature extraction module C3K2, a multi-scale convolutional PMSB method is combined to enhance the extraction of diseased fish symptom information. Simultaneously, the DBifusion multi-dimensional feature adaptive fusion method is proposed to optimize the traditional stitching module and achieve efficient utilization of low-dimensional features of diseased fish symptoms. Finally, the EMASlideLoss function is introduced to address the insufficient model attention caused by limited sample data, thereby achieving a significant improvement in the overall performance of diseased fish symptom recognition.

[0140] In one embodiment of the present invention, such as Figure 6 As shown, a network for recognizing the surface symptoms of diseased fish is based on YOLO network recognition, namely the YOLOv11-SDFishNet network of this invention. The overall network structure includes a backbone network, a neck network, and a detection head.

[0141] The YOLO network backbone is used to input fish surface images, including a first Conv component (the fish surface image is input into the first Conv component); a second Conv component (the output of the first Conv component is input into the second Conv component, and the output of the second Conv component is input into the first C3K2 component); the first C3K2 component extracts a first feature map; a third Conv component (the first C3K2 component extracts the first feature map and inputs it into the third Conv component, and the output of the third Conv component is input into the second C3K2 component); the second C3K2 component extracts a second feature map; and a fourth Conv component (the second C3K2 component extracts the second feature map and inputs it into the fourth Conv component). The output of the nv component is input to the third C3K2 component; the third C3K2 component extracts the third feature map; the fifth Conv component, the third feature map extracted by the third C3K2 component is input to the fifth Conv component, and the output of the fifth Conv component is input to the first C3K2-PMSB component; the first C3K2-PMSB component extracts a global difference feature map and a local detail feature map from the feature map output by the fifth Conv component, and extracts a fourth feature map based on the global difference feature map and the local detail feature map; the SPPF component, the fourth feature map extracted by the first C3K2-PMSB component is input to the SPPF component; the C2PSA component, the output of the SPPF component is input to the C2PSA component.

[0142] The YOLO network's neck network includes a sixth Conv component, whose output is input to the C2PSA component, and whose output is input to the first DBiFusion component. The first DBiFusion component acquires the feature map output from the sixth Conv component (based on the fifth feature map) and concatenates it with the second and third feature maps respectively to form a first concatenated feature map and a second concatenated feature map. It then fuses the first and second concatenated feature maps according to the feature weights determined by the activation function to obtain a first fused feature map. A fourth C3K2 component receives the first fused feature map output from the first DBiFusion component as input. A seventh Conv component receives the output from the fourth C3K2 component as input, and its output feature map (based on the first fused feature map) is input to the second DBiFusion component. The system comprises: a first component, including a fifth C3K2 component, an eighth Conv component, and a first Concat component, which outputs a sixth feature map; a second component, including a sixth C3K2 component, a ninth Conv component, and a second Concat component, which outputs a seventh feature map; a second C3K2-PMSB component, which extracts a global difference feature map and a local detail feature map from the seventh feature map; and a tenth feature map from the global difference feature map and the local detail feature map. The extraction of the tenth feature map includes the step of concatenating the seventh feature map, the global difference feature map, and the local detail feature map.

[0143] The detection head includes a first detection head, a second detection head, and a third detection head. The fifth C3K2 component of the first component extracts an eighth feature map based on the second fusion feature map, and the first detection head predicts the type of fish surface symptoms based on the sixth feature map. The sixth C3K2 component of the second component extracts a ninth feature map based on the sixth feature map, and the second detection head predicts the type of fish surface symptoms based on the ninth feature map. The third detection head predicts the type of fish surface symptoms based on the tenth feature map.

[0144] S22.C3K2-PMSB (Partial MultiScale Block) Feature Extraction Module: Because different symptoms on the body surface of diseased fish exhibit similar features, this significantly increases the difficulty for the network to identify these symptoms. To address this issue, existing research mainly employs concatenated convolutions to increase the receptive field and achieve better feature extraction, thus enabling better comparison and differentiation. However, while this method obtains a large receptive field through concatenated convolutions, it fails to extract subtle features between different symptoms and also significantly increases the number of model parameters.

[0145] The C3K2 component module is a specially designed feature extraction component in YOLOv11, which is an improvement on the traditional C3 module and provides stronger feature extraction capabilities. Therefore, this invention further optimizes the C3K2 component module by introducing PMSB. Considering factors such as computational complexity, grouping and channel rearrangement techniques are used to reduce computational load. This new module is named the PMSB module and is further combined with the C3K2 component feature extraction module to form the C3K2-PMSB component module, as shown below. Figure 7 As shown, this module utilizes multi-scale convolution to achieve efficient feature extraction through multiple receptive fields. It uses large convolution to control the overall features while using small convolution to extract subtle features, thus making up for the shortcomings of single convolution in feature extraction.

[0146] like Figure 7As shown, the C3K2-PMSB component includes a tenth Conv component, including a first output and a second output, with the feature map input to the tenth Conv component; a first Split function component, including a first output and a second output, with the first output of the tenth Conv component input to the first Split function component; an eleventh Conv component, including a first output, a second output, and a third output, with the first output of the first Split function component input to the eleventh Conv component; a twelfth Conv component, including a first output, with the first output of the eleventh Conv component input to the twelfth Conv component; a PMSB, including a first output, with the first output of the twelfth Conv component input to the C3K2-PMSB component; a first fusion unit, including a first input, a second input, and a first output, with the first output of the PMSB input to the first input of the first fusion unit, and the second output of the eleventh Conv component input to the second input of the first fusion unit; and a thirteenth Conv component, including a first output, with the first output of the eleventh Conv component input to the first Split function component. The thirteenth Conv component has a third output input; the third Concat component includes a first input, a second input, and a first output, the first output input of the first fusion component, the first input of the third Concat component, the first output input of the thirteenth Conv component, and the second input of the third Concat component; the fourteenth Conv component includes a first output, the first output input of the third Concat component, the fourteenth Conv component; the fourth Concat component includes a first input, a second input, a third input, and a first output, the first output input of the fourteenth Conv component, the first input of the fourth Concat component, the second output input of the tenth Conv component, the second input of the fourth Concat component, the second output input of the first Split function component, and the third input of the fourth Concat component; the fifteenth Conv component includes a first output, the output of the C3K2-PMSB component, and the first output input of the fourth Concat component.

[0147] like Figure 8 As shown, PMSB includes a second Split function component, which includes a first output and a second output, with the feature map input to the second Split function component; a Rearrange function component, which includes a first output and a second output, with the first output of the second Split function component input to the Rearrange function component; a first convolution, with the first output of the Rearrange function component input to the first convolution, extracts global differential features; a second convolution, with the second output of the Rearrange function component input to the second convolution, extracts local detail features; and a second fusion unit, which concatenates the second output of the second Split function component, the global differential features, and the local detail features for use as the output of the PMSB component.

[0148] PMSB module such as Figure 8 As shown, the input feature channels are first split into two parts using the Rearrange function component. One part is used for feature extraction through multiple receptive fields to extract rich features; the other part is directly retained for subsequent concatenation. After the Split function component operation, Stage 1 uses convolutional kernels of different sizes to construct a multiple receptive field structure, where... Convolution is responsible for capturing global differential features between diseased and healthy fish, while Convolution is used to extract local detail features to enhance the ability to distinguish between similar symptoms. Subsequently, these two feature sets are concatenated with Stage2 along the channel dimension. This not only achieves diversity and richness in feature extraction but also reduces the overall computational cost of the model, thereby improving computational efficiency and model performance. The Rearrange function component controls the grouping of feature vectors to prevent all feature vectors from entering the model. convolution, Convolution reduces computational cost.

[0149] S23. DBiFusion Fusion: The high similarity between diseased and healthy fish due to the indistinct surface symptoms of diseased fish directly leads to a decrease in the recall rate of diseased fish symptom identification. Therefore, fully utilizing the rich local semantic information in low-dimensional features is crucial for accurate identification of diseased fish symptoms. However, traditional methods often simply concatenate high- and low-dimensional features along channels, failing to dynamically adjust the importance of different channels in the low-dimensional feature map. Therefore, a DBiFusion fusion method was designed in the neck network, which adaptively fuses high-dimensional feature information and low-dimensional features from different channels to achieve efficient reuse of low-dimensional features of diseased fish, resulting in more accurate differentiation.

[0150] DBiFusion employs a multi-dimensional feature fusion strategy to process feature maps at different levels. For example... Figure 9 As shown, given feature maps at three different scales , and ,in This represents a high-dimensional feature map rich in semantic information, while and These are low-dimensional feature maps for different channels. In the first DBiFusion component, correspond Figure 6 The sixth Conv component outputs a feature map, corresponding to F2. Figure 6 The feature map output by the second C3K2 component, F3 corresponds to Figure 6 The feature map output by the third C3K2 component. And in the second DBiFusion component, correspond Figure 6 The seventh Conv component outputs a feature map, corresponding to F2. Figure 6 The feature map output by the first C3K2 component, F3 corresponds to Figure 6 The feature map output by the second C3K2 component.

[0151] To enable better feature map fusion, a high-dimensional feature information-guided approach is adopted, firstly... and application Convolution operations adjust the channels so that they are aligned with... The number of channels is consistent. At the same time, in order to align the spatial dimensions, After upsampling, and Then, downsampling is used to match the spatial resolution of other feature maps. Next, the upsampled... Each with the adjusted and The features are concatenated along the channel dimension to generate a fused feature map. and .

[0152] The feature extraction capability of a single C3K2 component in the backbone network of this invention may vary at different times. The DBiFusion method described above uses two low-dimensional features extracted from C3K2 components from different sources, which can overcome the stability problem of extraction by a single C3K2 component to some extent. However, this may lead to excessive low-dimensional local semantic information affecting high-dimensional information.

[0153] Therefore, the high-dimensional features of this invention respectively with and First, assemble; then assemble. and Instead of using, respectively with and The purpose of splicing in this way is to make the two features used for splicing the final product more compatible. and The ratio of high-dimensional information to low-dimensional information is approximately 1:1, which avoids excessive influence of low-dimensional local semantic information on high-dimensional information and optimizes the fusion effect.

[0154] Furthermore, the present invention DBiFusion introduces a path-weight-based mechanism, which calculates the value using the Sigmoid activation function. and The importance weight of each path is determined by the difference in mean values, and the weight output formula is (2). The final output feature map is obtained by weighted summation. The Sigmoid function can be used in the final feature concatenation to allocate weights according to importance and increase the proportion of important features.

[0155]

[0156]

[0157] in, This means taking the mean of all elements in the feature map. For the Sigmoid function, Mapped to the interval (0,1).

[0158] To further verify the effectiveness of the optimized fusion module in identifying symptoms of diseased fish, a visual analysis was performed on the feature maps processed by the backbone network and DBiFusion. For example... Figure 10 As shown, (a) primarily retains high-dimensional semantic information in the features extracted by the backbone network; (b) shows the effect after fusing the low-dimensional features provided by the DBiFusion module, demonstrating that low-dimensional local semantic information is effectively supplemented. This combination is crucial for accurately identifying the symptoms of diseased fish because it considers not only overall structural features but also integrates key detailed information, thereby improving the accuracy and reliability of disease identification.

[0159] In some embodiments of the present invention, step S30, network model training and evaluation, includes...

[0160] S31. EMASlideLoss: In real aquaculture, the number of diseased fish samples is relatively small, accounting for a low proportion of the total. Furthermore, when classifying different types of diseased fish symptoms, the number of images of each type of diseased fish symptom is even scarcer. This imbalance in quantity poses a severe challenge to network training, resulting in insufficient attention to scarce diseased fish symptom samples. Therefore, EMASlideLoss is introduced as a new loss function. The sliding weighted function and exponential moving average function used are shown in formulas (3) and (4), respectively, to enhance the network's learning ability for difficult samples and improve the network's overall fish recognition performance.

[0161]

[0162] in, The Intersection over Union (IoU) is the ratio between the predicted bounding box and the ground truth bounding box. The weight threshold is automatically updated by calling the exponential moving average function, and is used to distinguish between hard and easy samples.

[0163]

[0164] in, This represents the average IoU value across all current samples. This represents the average IoU value from the previous update; This is the initial exponential moving weight decay rate; This represents the exponential moving average decay factor. This refers to the number of times the loss calculation is updated. It is the time constant that controls the rate of time decay.

[0165] EMASlideLoss combines the high weighting of SlideLoss on difficult samples, making the model pay more attention to these samples during training, effectively solving the problem of imbalanced samples. It also combines the exponential moving average (EMA) function with SlideLoss, using the characteristics of EMA to smooth changes in the classification loss value. This avoids drastic fluctuations in the loss value caused by anomalies in a single batch of data, thus helping the model converge more stably.

[0166] S32. Network Model Training and Evaluation

[0167] Experimental Platform and Hyperparameter Settings: To ensure the accuracy and reliability of the experimental data, the training and testing of the underwater diseased fish identification network were conducted in the same environment. The experimental environment is shown in Table 1.

[0168] Experimental hyperparameter settings: batch size is 8, training epochs are 300, and initial learning rate is 0.0001.

[0169] Table 1 Experimental Environment

[0170]

[0171] Evaluation metrics: Accuracy of identification and high inference speed play a crucial role in the real-time monitoring of symptoms in diseased fish. To comprehensively evaluate the performance of the proposed method, a confusion matrix was used as the evaluation tool, including the following key metrics: True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN).

[0172] Regarding network recognition accuracy, precision (P), recall (R), and mean average precision (mAP@0.5) were selected as evaluation metrics. The calculation formulas are shown in (5)-(8).

[0173]

[0174]

[0175]

[0176]

[0177] Where P represents precision, R represents recall, and mAP represents the average precision across all classes. mAP is calculated at an IOU threshold of 0.5, and is also known as mAP@0.5. A higher mAP indicates better overall model performance in detection.

[0178] S33. Experimental Results and Analysis

[0179] S331. Data Augmentation Experiment

[0180] Experiments on Synthetic Image Augmentation Based on Object Quantity and Overlap Mechanism: In object detection tasks, especially fish disease detection, real-world data is often limited. Therefore, data augmentation techniques have become an important means to improve the generalization ability and detection accuracy of models. Due to random self-embedding augmentation, N segmented objects are randomly placed into the background. Considering that real-world scenes may produce occlusion, to verify whether the proposed synthetic image data augmentation method is optimal, two key factors need to be studied: the number of objects (N) and the overlap mechanism. The number of objects refers to the number of segmented objects contained in a synthetic image, and the overlap mechanism refers to whether severe overlap between target objects is allowed.

[0181] To easily distinguish different data augmentation configurations, the names consist of two parts: a prefix and the number of objects. The prefix "Sparse" indicates that the overlap mechanism is enabled, allowing slight occlusion between target objects; the prefix "Overlap" indicates that the overlap mechanism is not enabled, meaning occlusion is not considered. "1Obj", "2Obj", and "3Obj" represent images containing 1, 2, and 3 target objects, respectively. The model was trained and tested on the unified model YOLOv11-SDiseasedFishNet to evaluate the impact of various factors on model performance. Experimental results are shown in Table 2.

[0182] Experimental results show that introducing the overlap mechanism significantly improves detection accuracy (P), recall (R), and mean precision (mAP@0.5), especially in the setting with 2 objects and the overlap mechanism enabled (Env3), where these metrics reach 94.8%, 93.9%, and 97.1%, respectively. This is mainly because each synthesized image contains two segmented objects and allows for some occlusion between them, which not only expands the training data in quantity, allowing the model to better learn the features of diseased fish symptoms, but also enhances content diversity. In the data augmentation setting with 1 object, although the amount of data increases, the improvement in diversity is limited due to the simple object layout, and it still performs better than the baseline model. In the case of 2 objects without the overlap mechanism enabled, some data are severely occluded, making it difficult to effectively learn target features. This is more pronounced in the case of 3 objects without the overlap mechanism enabled. Although the amount of data increases further, the performance actually decreases. Due to the lack of reasonable spatial layout design, abnormally dense data can cause diseased fish symptoms to be occluded, even leading the model to learn incorrect information and misjudge, thus affecting the final performance. This demonstrates that simply increasing the number of objects without considering mutual occlusion between targets increases the learning difficulty of the model, thus affecting the final performance. Therefore, having 2 objects and enabling the overlap mechanism is the optimal solution for embedding augmentation.

[0183] Table 2. Effects of object number and overlap mechanism on composite images (experiment)

[0184]

[0185] Comparative Experiment with Augmented Data: To verify the effectiveness of the augmentation method proposed in this invention for detecting symptoms in diseased fish, a comparative experiment was designed with several existing data augmentation methods. Since data augmentation methods such as Mixup and Generative Adversarial Networks (GANs) are typically trained and augmented on relatively large and diverse initial datasets to improve the model's generalization ability, the amount of data in the basic dataset in this invention is relatively small, making it difficult to support the effective training and application of these complex augmentation methods. Therefore, this comparative experiment focuses only on the performance of different augmentation methods under limited data conditions, emphasizing their improvement effect on model performance in small sample situations. This more realistically reflects the practicality and advantages of the augmentation method proposed in this invention in resource-constrained scenarios. The experimental results are shown in Table 3.

[0186] Experimental results show that in YOLOv11-SDiseasedFishNet and YOLOv11 models, although YOLO's data augmentation methods YOLO-Augment and DAGIF can improve image diversity to some extent, they essentially transform the entire image, processing both the background and the diseased fish region simultaneously. With limited raw data, this makes it difficult for the model to focus on learning the key features of diseased fish symptoms. In contrast, RAFA achieves innovative image content reconstruction by segmenting diseased fish objects in existing images and recombining them into different backgrounds. This approach not only effectively increases the diversity of training samples but also allows the model to focus more on learning the features of the diseased fish themselves, thus significantly improving the model's stability and generalization ability in recognizing diseased fish symptoms. Compared with the baseline data, it achieved improvements of 4.6%, 5%, and 5.9% in accuracy (P), recall (R), and average precision (mAP@0.5), respectively.

[0187] Table 3 Comparison Experiment of Augmented Data

[0188]

[0189] S332. Model Experiment

[0190] Overall Model Ablation Experiment: To further verify the effectiveness of the C3K2-PMSB module, DBiFusion fusion strategy, and EMASlideLoss loss function in the diseased fish detection task, the three key modules, C3K2-PMSB (denoted as G), DBiFusion (denoted as D), and EMASlideLoss (denoted as E), were integrated into the basic network structure. After data augmentation, several model variants with different configurations were constructed, named YOLOv11-G, YOLOv11-D, YOLOv11-E, YOLOv11-GD, YOLOv11-GE, and YOLOv11-DE, respectively. Each suffix represents the combination of modules included in the model. The experimental results are shown in Table 4, clearly demonstrating the specific improvement effects of each module on accuracy (P), recall (R), and mean precision (mAP@0.5).

[0191] Because different symptoms on the body surface of diseased fish exhibit similar characteristics, the difficulty for the network to identify these symptoms is increased. The C3K2-PMSB module, by introducing a multi-scale convolutional structure, effectively enhances the detailed features of the diseased fish's body surface while preserving channel information, improving mAP@0.5 by 1.2%. The DBiFusion module, through multi-dimensional adaptive fusion, improves the model's ability to fuse local semantic information of diseased fish, thereby increasing the accuracy and recall of diseased fish symptoms from 91.5% and 90.2% to 92.7% and 91.2%, respectively. EMASlideLoss, through a sliding weighting mechanism, strengthens the model's learning ability for difficult samples, significantly improving robustness to class imbalance, increasing mAP@0.5 to 93.5%. When the two modules are combined, the model performance is further enhanced. After a series of improvements, the final model accuracy reaches 94.8%, and the diseased fish recall reaches 93.9%, representing significant improvements of 3.3% and 3.7% respectively compared to the baseline model. Figure 11 , Figure 12 The comparison of detection performance and mAP@0.5 / % between the baseline model and the proposed method is presented, further validating the effectiveness of the proposed method.

[0192] Table 4 Overall Comparison Experiment of Model Ablation

[0193]

[0194] Model Ablation Symptom Experiment: Given that this model focuses on distinguishing different types of diseased fish symptoms, special attention was paid to the improvement effect on each specific symptom. After introducing the C3K2-PMSB feature enhancement module, the DBiFusion fusion method, and the EMASlideLoss loss function, the model not only significantly improved the overall performance, but also substantially enhanced its ability to identify specific diseased fish symptoms such as hemorrhage, ulceration, white spot, and lack. The experimental results are shown in Table 5.

[0195] The C3K2-PMSB feature enhancement module constructs multiple receptive fields, enabling the model to improve global feature perception while accurately capturing local features, thus distinguishing subtle differences in symptoms among diseased fish. It is suitable for differentiating symptoms with similar manifestations, such as bleeding and ulceration, significantly improving the model's accuracy and recall for identifying these similar symptoms. The DBiFusion fusion method uses adaptive weighted fusion of high- and low-dimensional features to further enhance the effective utilization of low-dimensional local semantic information, allowing the model to better learn subtle symptoms such as bleeding, ulceration, and white spots. Addressing the problem of class imbalance in diseased fish samples, especially with limited data such as missing data or white spots, the EMASlideLoss loss function plays a crucial role, prompting the model to focus more on difficult-to-distinguish samples, thereby improving overall learning effectiveness and model performance. Results show that the model significantly improves overall performance while also enhancing the recognition of each symptom.

[0196] Table 5 Comparison of ablation symptoms in the model experiments

[0197]

[0198] Model Comparison Experiment: To further verify the effectiveness and advancement of the network model of this invention in underwater diseased fish identification, comparative experiments were conducted using models with high identification performance currently available in this field, including RT-DETR, YOLOv8, YOLOv10, YOLOv12, and the baseline model YOLOv11. The experimental results are shown in Table 6. Compared to other algorithms, the method proposed in this invention achieves the best overall performance.

[0199] Compared to RT-DETR, the method of this invention significantly improves accuracy and recall by 12.3% and 15.4%, respectively. Compared to the original YOLOv11, the improved model shows substantial improvements in precision, recall, and mean precision. Furthermore, applying the improved strategy to YOLOv8 and YOLOv10 also yields significant performance gains; for example, YOLOv8-SDFishNet and YOLOv10-SDFishNet achieve mAP@0.5% of 94.9% and 95.8%, respectively, demonstrating the scalability of the proposed method. However, compared to the YOLOv10-UDfishnet diseased fish model, which focuses on small regions in the image but lacks a large receptive field, resulting in insufficient control over overall symptoms and thus poor performance, the heatmap visually demonstrates the recognition performance of each model for diseased fish symptoms; the color intensity in the heatmap directly reflects the degree of attention the model pays to that region. The color comparison of the heatmap demonstrates YOLOv11-SDFishNet's highly focused and accurate identification capabilities for the symptoms of diseased fish.

[0200] Table 6 Model Comparison Experiment

[0201]

[0202] This invention proposes a novel network, YOLOv11-SDFishNet, for identifying the symptoms of diseased fish, aiming to accurately and efficiently recognize these symptoms. By introducing the Reinforced Embedding Algorithm (RAFA), the problem of poor model training performance due to insufficient training samples is effectively alleviated. The optimized feature splicing module DBiFusion and the C3K2-PMSB multi-scale feature enhancement module significantly enhance the model's ability to perceive symptom regions. Combined with the design of the EMASlideLoss loss function, the model's attention to symptoms in difficult samples is further improved. In terms of data augmentation, compared with traditional augmentation methods, the mean accuracy, recall, and average precision are improved by 4.1%, 4.2%, and 5.7%, respectively. In terms of network structure, YOLOv11-SDFishNet achieves mean accuracy, recall, and average precision of 94.8%, 93.9%, and 97.1%, respectively. Compared with current detection models (such as YOLOv12, RT-DETR and YOLOv10UD), the mean accuracy is improved by 5%, 16% and 5.3%, respectively.

[0203] Based on the above embodiments, this application also provides a computer program that, when run on a computer, causes the computer to execute the methods provided in the above embodiments.

[0204] Based on the above embodiments, this application also provides a computer storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the method provided in the above embodiments.

[0205] The storage medium can be any available medium that a computer can access. For example, but not limited to, a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0206] Based on the above embodiments, this application also provides a chip for reading a computer program stored in a memory to implement the method provided in the above embodiments.

[0207] Based on the above embodiments, this application provides a computer program product that implements the methods provided in the above embodiments when the computer program product is run on an electronic device.

[0208] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0210] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0211] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0212] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A device for identifying symptoms on the body surface of diseased fish, characterized in that, Identification is based on the YOLOv11n network, which includes a backbone network, a neck network, and a detection head. The backbone of the YOLOv11n network is used to input images of the fish's body surface, including... The first C3K2 component extracts the first feature map; The second C3K2 component extracts the second feature map; The third C3K2 component extracts the third feature map; The first C3K2-PMSB component extracts a global difference feature map and a local detail feature map based on the third feature map, and then extracts a fourth feature map based on the global difference feature map and the local detail feature map; and The attention mechanism obtains the fifth feature map from the fourth feature map through the attention mechanism of the backbone network; Among them, the neck network of the YOLOv11n network includes The first DBiFusion component, based on the fifth feature map, obtains a first concatenated feature map and a second concatenated feature map by concatenating the feature map based on the fifth feature map with the second and third feature maps respectively; it then fuses the first concatenated feature map and the second concatenated feature map according to the feature weights determined by the activation function to obtain a first fused feature map; and The second DBiFusion component obtains a third and a fourth concatenated feature map by concatenating the feature map based on the first fused feature map with the first feature map and the second feature map, respectively; and fuses the third and fourth concatenated feature maps according to the feature weights determined by the activation function to obtain the second fused feature map. The detection head predicts the type of symptoms on the fish's body surface based on the second fusion feature map.

2. The device for identifying the surface symptoms of diseased fish according to claim 1, characterized in that, in, The backbone network also includes The first Conv component is used to input the fish body surface image into the first Conv component. The second Conv component, the output of the first Conv component is input to the second Conv component, and the output of the second Conv component is input to the first C3K2 component; The third Conv component extracts the first feature map from the first C3K2 component and inputs it into the third Conv component. The output of the third Conv component is input into the second C3K2 component. The fourth Conv component extracts the second feature map from the second C3K2 component and inputs it into the fourth Conv component. The output of the fourth Conv component is input into the third C3K2 component. as well as The fifth Conv component extracts the third feature map from the third C3K2 component and inputs it into the fifth Conv component. The output of the fifth Conv component is input into the first C3K2-PMSB component. The attention mechanism of the backbone network includes... The SPPF component is the fourth feature map extracted by the first C3K2-PMSB component. as well as C2PSA component, the input and output of SPPF component C2PSA component.

3. The device for identifying the surface symptoms of diseased fish according to claim 1, characterized in that, in, Neck network, also includes The sixth Conv component, the C2PSA component's output input to the sixth Conv component, and the sixth Conv component's output input to the first DBiFusion component; The fourth C3K2 component is the input of the first fused feature map output from the first DBiFusion component. The seventh Conv component, the output and input of the fourth C3K2 component, the seventh Conv component, and the output and input of the seventh Conv component are the second DBiFusion component.

4. The device for identifying symptoms on the body surface of diseased fish according to claim 1, characterized in that, in, Neck network, also includes The first component, including the fifth C3K2 component, the eighth Conv component, and the first Concat component, outputs the sixth feature map; The second component, which includes the sixth C3K2 component, the ninth Conv component, and the second Concat component, outputs the seventh feature map. as well as The second C3K2-PMSB component extracts a global difference feature map and a local detail feature map based on the seventh feature map; it then extracts a tenth feature map based on the global difference feature map and the local detail feature map. The extraction of the tenth feature map includes the step of concatenating the seventh feature map, the global difference feature map, and the local detail feature map. The detection head includes a first detection head, a second detection head, and a third detection head; Among them, the fifth C3K2 component of the first component extracts the eighth feature map based on the second fusion feature map, and the first detection head predicts the type of fish surface symptoms based on the sixth feature map; Among them, the sixth C3K2 component of the second component extracts the ninth feature map based on the sixth feature map, and the second detection head predicts the type of symptoms on the fish's body surface based on the ninth feature map; Among them, the third detection head predicts the type of symptoms on the fish's body surface based on the tenth feature map.

5. The device for identifying symptoms on the body surface of diseased fish according to claim 1, characterized in that, in, C3K2-PMSB components, including The tenth Conv component includes a first output and a second output, and the feature map is input to the tenth Conv component; The first Split function component includes a first output and a second output, and the first output of the tenth Conv component is input to the first Split function component; The eleventh Conv component includes the first output, the second output, and the third output, and the first output input of the first Split function component is the eleventh Conv component; The twelfth Conv component includes the first output, the first output input of the eleventh Conv component, and the twelfth Conv component. The PMSB component includes the first output and the first output input of the twelfth Conv component, C3K2-PMSB component; The first fusion unit includes a first input, a second input, and a first output, wherein the first output of the PMSB is input to the first input of the first fusion unit, and the second output of the eleventh Conv component is input to the second input of the first fusion unit. The thirteenth Conv component includes the first output and the third output input of the eleventh Conv component. The third Concat component includes a first input, a second input, and a first output; the first output input of the first fusion unit is the first input of the third Concat component; the first output input of the thirteenth Conv component is the second input of the third Concat component; The fourteenth Conv component includes the first output, the first output input of the third Concat component, and the fourteenth Conv component. The fourth Concat component includes a first input, a second input, a third input, and a first output; the first output input of the fourteenth Conv component is the first input of the fourth Concat component; the second output input of the tenth Conv component is the second input of the fourth Concat component; the second output input of the first Split function component is the third input of the fourth Concat component; The fifteenth Conv component includes a first output, which is used as the output of the C3K2-PMSB component, and the first output input of the fourth Concat component is the fifteenth Conv component.

6. The device for identifying symptoms on the body surface of diseased fish according to claim 5, characterized in that, in, PMSB components, including The second Split function component includes a first output and a second output, with the feature map input to the second Split function component; The Rearrange function component includes a first output and a second output, and the first output input of the second Split function component is the Rearrange function component; The first convolution, the first output input of the Rearrange function component, extracts global differential features; The second convolution, the second output input of the Rearrange function component, extracts local detail features; The second fusion unit concatenates the second output of the second Split function component, global difference features, and local detail features for use as the output of the PMSB component.

7. The device for identifying symptoms on the body surface of diseased fish according to claim 1, characterized in that, in, Based on the fifth feature map, obtain a first concatenated feature map and a second concatenated feature map by concatenating the feature map based on the fifth feature map with the second feature map and the third feature map, respectively; fuse the first concatenated feature map and the second concatenated feature map according to the feature weights determined by the activation function to obtain the first fused feature map, as shown in the following formula: In the formula, This represents the first fused feature map. This represents the Sigmoid function. This means taking the mean of all elements in the feature map. This represents the first concatenated feature map. This represents the second spliced ​​feature map; Based on the first fused feature map, a third and fourth concatenated feature map are obtained by concatenating the feature map based on the first fused feature map with the first feature map and the second feature map, respectively. The third and fourth concatenated feature maps are then fused according to the feature weights determined by the activation function to obtain the second fused feature map, as shown in the following formula: In the formula, This represents the second fused feature map. This represents the Sigmoid function. This means taking the mean of all elements in the feature map. This represents the third spliced ​​feature map. This represents the fourth spliced ​​feature map.

8. A data augmentation method for the YOLOv11n network training set applied to the diseased fish surface symptom recognition device according to any one of claims 1-7, characterized in that, include The fish in the fish body surface images in the original training set are used as the segmentation objects to segment the background image of the fish body surface images to obtain the segmentation objects; Perform geometric transformations on the segmented objects; The segmented objects are randomly combined onto the background image to obtain a composite image; Adjust the composite image so that the combined area of ​​the segmented objects occupies 1 / 3 to 2 / 3 of the area of ​​the background image; If the overlapping area of ​​randomly combined segmented objects exceeds a set value, the segmented objects will be adjusted according to the following formula: In the formula: The center point of the bounding box of the original segmented object is located at Coordinates along the axis, This indicates that the center point of the bounding box of the segmented object after adjustment is at... Coordinates along the axis; The center point of the bounding box of the original segmented object is located at Coordinates along the axis, This indicates that the center point of the bounding box of the segmented object after adjustment is at... Coordinates along the axis; This represents the distance between the center point of the bounding box of the original segmented object and the width of the bounding box. This indicates the distance between the center point of the bounding box of the segmented object and the width of the bounding box after adjustment. This represents the distance between the center point of the bounding box of the original segmented object and the height of the bounding box. This represents the center point of the bounding box of the segmented object and the height of the bounding box. Indicates rotation Width after angle ; Indicates rotation The height after the angle, ; Indicates the scaling factor. express Parameters for fine-tuning axial displacement. express Parameters for fine-tuning axial displacement.

9. A training method for the YOLOv11n network applied to the diseased fish surface symptom recognition device according to any one of claims 1-7, characterized in that, The training loss function is shown in the following equation: in, Represents the loss function. This represents the intersection-over-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. This represents the average IoU value of all current samples; in, This represents the average IoU value of all current samples; This represents the average IoU value from the previous update; This represents the initial exponential moving weight decay rate; This represents the exponential moving average decay factor. Indicates the number of times the loss calculation has been updated; The time constant that controls the decay rate of time.

10. A method for identifying symptoms on the body surface of diseased fish, characterized in that, YOLOv11n network-based identification, where the YOLOv11n network comprises a backbone network, a neck network, and a detection head; the identification method includes... The backbone network takes a fish body surface image as input and extracts global difference feature maps and local detail feature maps based on the third feature map. The backbone network extracts a fourth feature map based on the global difference feature map and the local detail feature map; the extraction of the fourth feature map includes the step of concatenating the third feature map, the global difference feature map and the local detail feature map; The fourth feature map is used to obtain the fifth feature map through the attention mechanism of the backbone network; The neck network obtains a first concatenated feature map and a second concatenated feature map by concatenating the feature map based on the fifth feature map with the second feature map and the third feature map, respectively. The first concatenated feature map and the second concatenated feature map are fused according to the feature weights determined by the activation function to obtain the first fused feature map; The neck network obtains a third and a fourth spliced ​​feature map by concatenating the feature map based on the first fusion feature map with the first feature map and the second feature map, respectively. The third and fourth spliced ​​feature maps are fused according to the feature weights determined by the activation function to obtain the second fused feature map; The detection head predicts the type of symptoms on the fish's body surface based on the second fusion feature map.

Images