Marine organism detection method, device and equipment and storage medium

Abstract

The invention discloses a marine organism detection method, device and equipment and a storage medium, and relates to the technical field of target detection, and the marine organism detection method comprises the steps: obtaining a marine organism data set, and dividing the marine organism data set into a training set and a test set; introducing a Swin Transform network and an efficient multi-scale attention mechanism to improve the initial marine organism detection model, and obtaining a reference marine organism detection model; training and testing the reference marine organism detection model by using the training set and the test set to obtain a target marine organism detection model; and detecting the marine organisms in the marine organism image by using the target marine organism detection model. The target marine organism detection model obtained by the invention can more accurately detect the marine organisms and automatically identify and position the marine organisms, so that the detection efficiency is improved, and reliable data support is provided for scientific research and environmental protection.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to marine life detection methods, devices, equipment and storage media. Background Art

[0002] Underwater target detection is a key technology in underwater robots, underwater surveys, and marine research. In the marine environment, accurate detection and identification of marine biological targets is of great significance for ecological protection, resource development, and scientific research. However, the complexity of the underwater environment makes marine biological target detection a very challenging task in the field of computer vision. Underwater images usually show severe color deviation, low contrast, and blurred edges due to light refraction, turbidity, and variable underwater light conditions. These environmental factors significantly increase the difficulty of marine biological target detection.

[0003] At present, the methods for marine biological detection mostly rely on traditional convolutional neural networks (CNN) and deep learning algorithms based on target detection, such as the YOLO series, Faster R-CNN, etc. These methods have achieved good results in target detection in terrestrial scenes and have been introduced into the field of underwater biological detection in the hope of achieving the same success.

[0004] However, the performance of existing detection methods in underwater environments is not satisfactory. First, marine biological targets are usually small in scale. As the convolutional network deepens, the target features are easily lost, resulting in a decrease in detection accuracy. Secondly, marine organisms easily blend with the seabed environment, making it difficult to distinguish between the target and the background, which in turn increases the probability of false detection and missed detection. In addition, when faced with complex underwater environments, traditional target detection methods often cannot make full use of limited feature information, resulting in poor detection of small targets or low-contrast targets. Therefore, how to detect marine organisms more accurately has become an urgent problem to be solved.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The purpose of this application is to provide a marine organism detection method, device, equipment and storage medium, aiming to solve the technical problem of how to detect marine organisms more accurately.

[0007] To achieve the above objectives, the present application proposes a method for detecting marine organisms, the method comprising:

[0008] Acquire a marine biological data set, and divide the marine biological data set into a training set and a test set;

[0009] The Swin Transformer network and efficient multi-scale attention mechanism were introduced to improve the initial marine life detection model and obtain the reference marine life detection model.

[0010] Using the training set and the test set to train and test the reference marine organism detection model to obtain a target marine organism detection model;

[0011] The target marine life detection model is used to detect marine life in a marine life image.

[0012] In one embodiment, the target marine life detection model includes a backbone network, a neck network, and a head network, the backbone network includes a split module and a Transformer layer, and the step of using the target marine life detection model to detect marine life in a marine life image includes:

[0013] Splitting the marine life image into non-overlapping image blocks by the splitting module, and flattening the non-overlapping image blocks into sequence vectors;

[0014] Processing the sequence vector through the Transformer layer to obtain a multi-scale feature map;

[0015] Processing the multi-scale feature map through the neck network to obtain a spatial attention feature map;

[0016] The spatial attention feature map is decoupled through the head network to obtain a marine life detection result.

[0017] In one embodiment, the Transformer layer includes a linear embedding layer, a merging layer, a linear layer and a SwinTransformer module, and the step of processing the sequence vector through the Transformer layer to obtain a multi-scale feature map includes:

[0018] Projecting the sequence vector into a linear tensor through the linear embedding layer;

[0019] The linear tensor is dimensionally reduced through the merging layer and the linear layer, and the linear tensor is feature transformed through the SwinTransformer module to obtain a multi-scale feature map.

[0020] In one embodiment, the neck network includes a convolution module, a cross-dimensional interaction module, an aggregation module, an upsampling module, and an efficient multi-scale attention mechanism module. The step of processing the multi-scale feature map through the neck network to obtain a spatial attention feature map includes:

[0021] Performing one-dimensional global pooling, feature extraction, and feature merging on the multi-scale feature map through the convolution module, the cross-dimensional interaction module, the aggregation module, and the upsampling module to obtain an intermediate feature map;

[0022] The intermediate feature map is subjected to two-dimensional global pooling and normalization through the efficient multi-scale attention mechanism module to obtain a spatial attention feature map.

[0023] In one embodiment, the step of introducing the Swin Transformer network and the efficient multi-scale attention mechanism to improve the initial marine life detection model to obtain a reference marine life detection model includes:

[0024] Use the Swin Transformer network to replace the backbone network of the initial marine life detection model;

[0025] An efficient multi-scale attention mechanism is added to the neck network of the initial marine life detection model to obtain a reference marine life detection model.

[0026] In one embodiment, the step of using the training set and the test set to train and test the reference marine life detection model to obtain a target marine life detection model includes:

[0027] Using the training set to train the reference detection model to obtain a second reference detection model;

[0028] Using the test set to test the second reference detection model, to obtain a test result;

[0029] When the test result does not meet the preset requirements, return to the step of using the training set to train the reference detection model until the test result meets the preset requirements to obtain a target marine life detection model.

[0030] In one embodiment, the step of using the training set to train the reference detection model to obtain a second reference detection model includes:

[0031] The total loss is obtained based on the positioning loss, confidence loss, classification loss and multi-task loss;

[0032] The reference detection model is trained using the training set until the total loss is lower than a preset loss value, thereby obtaining a second reference detection model.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a marine life detection device, the device comprising:

[0034] A data acquisition module, used to acquire a marine biological data set and divide the marine biological data set into a training set and a test set;

[0035] The model improvement module is used to introduce the Swin Transformer network and the efficient multi-scale attention mechanism to improve the initial marine life detection model and obtain the reference marine life detection model;

[0036] A model training module, used to train and test the reference marine organism detection model using the training set and the test set to obtain a target marine organism detection model;

[0037] The target detection module is used to detect marine organisms in the marine organism image using the target marine organism detection model.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a marine life detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the marine life detection method described above.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the marine biological detection method described above are implemented.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] A marine biological data set is obtained and divided into a training set and a test set; the Swin Transformer network and an efficient multi-scale attention mechanism are introduced to improve the initial marine biological detection model to obtain a reference marine biological detection model; the reference marine biological detection model is trained and tested using the training set and the test set to obtain a target marine biological detection model; the target marine biological detection model is used to detect marine organisms in marine biological images. The marine biological data set is obtained and divided into a training set and a test set, which provides structured data support for the model, ensures efficient learning during training and accuracy during evaluation, and not only ensures data diversity, but also improves the performance of the model in different environments. The Swin Transformer network and an efficient multi-scale attention mechanism are introduced to improve the initial detection model, and the detection accuracy of the model is significantly improved by enhancing the global feature capture capability and multi-scale information processing capability of the model. The improved model is trained using the training set, and the performance is evaluated using the test set, ensuring the accuracy and robustness of the model in practical applications. Ultimately, the target marine life detection model obtained through these steps can detect marine life more accurately, automatically identify and locate marine life, thereby improving detection efficiency and providing reliable data support for scientific research and environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A schematic diagram of a process flow provided for Embodiment 1 of the marine organism detection method of the present application;

[0045] Figure 2 A schematic diagram of the Swin Transformer network architecture provided for the first embodiment of the marine organism detection method of the present application;

[0046] Figure 3 A schematic diagram of the network structure of the EMA attention mechanism provided in Example 1 of the marine organism detection method of this application;

[0047] Figure 4A schematic diagram of the network structure of a reference marine organism detection model provided in Example 1 of the marine organism detection method of the present application;

[0048] Figure 5 A schematic diagram of the process provided for the second embodiment of the marine organism detection method of the present application;

[0049] Figure 6 This is a schematic diagram of the module structure of the marine life detection device according to an embodiment of the present application;

[0050] Figure 7 Schematic diagram of the equipment structure of the hardware operating environment involved in the marine life detection method in the embodiment of the present application.

[0051] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] Underwater target detection is crucial for underwater robots, surveys, and ocean research, but the detection task faces challenges due to poor image quality caused by underwater environments such as light refraction, turbidity, and light changes. Although traditional CNN and deep learning algorithms such as YOLO and Faster R-CNN perform well in terrestrial scenes, in underwater environments, due to the small size of the target, its blending with the background, and insufficient use of feature information, the detection accuracy and discrimination are low, and false detections and missed detections occur frequently.

[0055] The main solution of the embodiment of the present application is: by acquiring and dividing the marine life data set, introducing the SwinTransformer network and multi-scale attention mechanism to improve the detection model, the model efficiently learns on the training set and accurately evaluates on the test set, thereby improving the detection accuracy and robustness of marine life and realizing automated marine life recognition and positioning.

[0056] It should be noted that the execution subject of the embodiments of the present application may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an underwater robot, an underwater survey system, a target detection system, etc. The following takes the underwater survey system as an example to illustrate this embodiment and the following embodiments.

[0057] Based on this, the present application embodiment provides a method for detecting marine organisms, referring to Figure 1 , Figure 1 This is a schematic diagram of the flow chart of the first embodiment of the marine organism detection method of the present application.

[0058] In this embodiment, the marine organism detection method includes steps S10 to S40:

[0059] Step S10, obtaining a marine organism data set, and dividing the marine organism data set into a training set and a test set;

[0060] It should be noted that the marine life dataset refers to an image or video dataset used to train and evaluate marine life detection algorithms. It contains annotated data of various marine life. The dataset is usually composed of images or video frames of marine life, and these images or video frames have been annotated with the category and location of the target organism (for example, through bounding box annotation). These datasets may be composed of images taken in the field, simulated data, or manually annotated data. The quality and diversity of the dataset directly affect the effect of the detection algorithm. The training set is the part of the marine life dataset used to train the algorithm. The training set contains annotated image data. The algorithm learns how to extract features from the input image and perform target detection through this data. The training set usually has more data, covering various situations and changes of marine life, to help the algorithm build a powerful model that can perform well in the face of various marine life and environmental conditions. The test set is the part of the marine life dataset used to evaluate the performance of the algorithm. The test set also contains annotated image data, but this data is not used as training samples during the training process. The test set is used to test the generalization ability of the algorithm on unseen data to ensure that the algorithm can maintain good detection results in practical applications. The results of the test set are usually used to evaluate the performance indicators of the model, such as accuracy, recall rate, and precision. In this embodiment, the data set comes from the open source data set URPC published by the National Underwater Robot Competition. The images in the data set are all real marine environments. After removing the seaweed label, four types of marine biological targets are manually labeled, namely sea urchins (echinus), sea cucumbers (holothurian), scallops (scallop), and starfish (starfish), with a total of 5543 pictures.

[0061] It is understandable that first, a dataset containing images or videos of various marine organisms needs to be obtained. These images or videos should be annotated to indicate the category and location of each marine organism. The source of the dataset can be field shooting, simulation generation, or acquisition from an existing database. The acquired dataset usually needs to cover marine organisms of different species, sizes, and environmental conditions to ensure the diversity and representativeness of the data. Next, the entire dataset is divided into two parts: a training set and a test set. The training set is used in the training process of the model and contains a large amount of annotated data for the algorithm to learn how to detect and classify marine organisms. The test set is used to evaluate the performance of the model and contains a part of the data that is not used for training to test the generalization ability and accuracy of the algorithm in practical applications. Usually, the division ratio of the training set and the test set is commonly 80% (training set) and 20% (test set), but the specific ratio may be adjusted according to actual needs. This process ensures the scientific use of data and helps the algorithm to effectively predict and detect when facing new data.

[0062] Step S20, introducing the Swin Transformer network and the efficient multi-scale attention mechanism to improve the initial marine life detection model, and obtaining a reference marine life detection model;

[0063] The Swin Transformer network is a Transformers-based visual model specifically designed to process image data. Unlike traditional convolutional neural networks (CNNs), the Swin Transformer network divides images into small blocks and uses a self-attention mechanism to capture global information in the image, giving it a strong advantage in processing image details and contextual information. The features of the Swin Transformer network include its hierarchical design and sliding window mechanism, which enable the model to effectively process features of different scales and remain efficient when processing large-scale data.

[0064] Please refer to Figure 2 , Figure 2The schematic diagram of the Swin Transformer network architecture provided for the first embodiment of the marine biological detection method of this application shows a deep learning model architecture based on Swin Transformer. The Swin Transformer network is mainly composed of a block module (Patch Partition) and four stages (Stage). Among them, Stage1 consists of a linear embedding layer (Linear Embedding) and a Swin Transformer Block. Stage2-Stage4 adopt a similar structure, which is composed of Patch (patch refers to non-overlapping image blocks) merging (Patch Merging) and Swin Transformer Block stacking.

[0065] The Swin Transformer network is used to process image data. From input to output, the model gradually extracts and fuses features through multiple stages, ultimately achieving efficient image recognition. Specifically: (1) Input stage: The image is first sent to the "Patch Partition" module, where the image is divided into small patches (patches) to prepare for subsequent feature extraction. (2) Linear Embedding: Each image patch is embedded through a linear layer, which helps map the features of the image patch to a higher-dimensional space, laying the foundation for subsequent complex feature extraction. (3) Swin Transformer Blocks: The image patch then enters multiple Swin Transformer blocks. Each block consists of multiple sub-modules: Window Multi-Head Self-Attention Mechanism (W-MSA, Window-MSA): Focuses on image patches in local areas and enhances the representation of important features through the attention mechanism. Transformer Block: After W-MSA, each block contains a Transformer module for further processing features and capturing dependencies globally through the self-attention mechanism. Layer Normalization (LN): Layer normalization is used before and after each Swin Transformer block to stabilize the training process and accelerate convergence. (4) Stage 1 to Stage 4: The model is divided into four stages, each of which contains multiple Swin Transformer blocks. As the stage goes deeper, the size of the image patch gradually decreases, while the dimension of the feature increases, which helps the model capture more abstract image features. (5) Patch Merging: At the end of each stage, the "Patch Merging" operation is used to reduce the number of image patches while increasing the feature dimension of each patch. This helps the model further abstract and integrate features while maintaining spatial information. (6) Multi-layer Perceptron (MLP): After the last stage, a multi-layer perceptron is used to perform the final integration and classification of the features. MLP is able to learn complex nonlinear relationships to support the final decision. (7) Output: The model output is the result of classifying the object in the image. Each category corresponds to a probability score, indicating the model's confidence that the image belongs to that category. The entire architecture uses carefully designed feature extraction and fusion strategies to enable the model to effectively process image data, especially in complex or changing environments, such as application scenarios such as marine life detection.

[0066] The Swin Transformer network first splits the input H×W×3 (H refers to the height, i.e. the vertical coordinate value, W refers to the width, i.e. the horizontal coordinate value) RGB image into non-overlapping N×(P2×3) patches (patch refers to non-overlapping image blocks) through the Patch Partition module. Each P2×3 patch is regarded as a patch token (token in the field of computer vision refers to cutting an image into a non-overlapping patch sequence). More specifically, a patch of size P2=4×4 and number of channels C=3 is used (where C refers to the number of channels. The most common RGB color space has three channels, namely R (red), G (green), and B (blue)). Therefore, the feature dimension of each flattened patch is P×P×C=4×4×3=48. There are a total of N=H / 4×W / 4=HW / 16 image patches. Each patch is flattened into a 48-dimensional token vector. The overall is a flattened N×(P2×3)=(H / 4×W / 4)×48-dimensional 2D patch sequence. The linear embedding layer (i.e., the fully connected layer) projects the tensor of dimension (H / 4×W / 4)×48 at this time to an arbitrary dimension C, and obtains a linear embedding of dimension (H / 4×W / 4)×C. Subsequently, these patch tokens (which are already linear embeddings at this time) are fed into several Swin Transformer Blocks with improved self-attention. The first Swin Transformer Block keeps the number of input and output tokens constant at H / 4×W / 4, and is designated as Stage1 together with the linear embedding layer. To produce a hierarchical representation, the number of tokens is gradually reduced through the Patch Merging layer as the network deepens. The first Patch Merging layer concatenates each group of 2×2 adjacent patches, so the number of patch tokens becomes 1 / 4 of the original, that is, H / 8×W / 8, and the dimension of the patch token is expanded by 4 times, that is, 4C. Then a linear layer is used on the 4C-dimensional patch concatenation features to reduce the output dimension to 2C. Swin Transformer blocks are then used for feature transformation, and its resolution remains unchanged at H / 8×W / 8. The first Patch Merging layer and the feature transformation Swin Transformer block are designated as Stage2. The same process as Stage2 is repeated twice, designated as Stage3 and Stage4 respectively. The output resolution / patch token number are H / 16×W / 16 and H / 32×W / 32 respectively.Each stage changes the dimension of the tensor to form a hierarchical representation. Therefore, this architecture can easily replace the existing backbone network for various visual tasks.

[0067] Efficient Multi-Scale Attention (EMA) is an improved attention mechanism that aims to enhance the object detection model's ability to process multi-scale features. EMA introduces a multi-scale self-attention module, which enables the model to capture more key information at different scales, thereby better detecting objects of various sizes. This mechanism improves the detection accuracy and robustness of the model when facing objects of different scales, and optimizes computational efficiency to reduce the consumption of computing resources.

[0068] Please refer to Figure 3 , Figure 3The network structure diagram of the EMA attention mechanism provided in the first embodiment of the marine biological detection method of the present application shows the network structure of the EMA (Excitation and Manipulation Attention) attention mechanism, which is an attention module for enhancing the representation of image features. EMA finely adjusts the channel relationship of the feature map to strengthen important features and suppress unimportant features, thereby improving the recognition ability of the model. The specific structure is as follows: (1) Input: The feature map C×H×W enters the module, where C is the number of channels, H and W are the height and width respectively. (2) Grouping: The feature map is first divided into G groups, each group contains C / G channels, forming a C / G×H×W intra-group feature map. (3) Average pooling: In each group, the feature map is subjected to horizontal and vertical average pooling to obtain C / G×1×W and C / G×H×1 feature maps respectively. (4) Convolution operation: The feature map after average pooling is convolved through a 3×3 convolution kernel, and then the two convolution results are spliced ​​in the channel dimension and processed through a 1×1 convolution kernel to enhance the feature expression ability. (5) Sigmoid activation: The concatenated feature map is activated by the Sigmoid activation function to generate two weight maps of C / G×H×W, which are used to reweight the original feature map. (6) Reweighting: The original feature map is reweighted by multiplying it with the weight map generated by the Sigmoid activation function, thereby highlighting important features and suppressing unimportant features. (7) Group norm: The reweighted feature map is normalized by the group norm to stabilize the training process. (8) Average pooling and Softmax: Average pooling is applied on the channel dimension, and the attention weight of each channel is generated by the Softmax function. (9) Matrix multiplication (Matmul): The attention weights are further adjusted using matrix multiplication to achieve cross-channel feature fusion. (10) Output: After all these processes, the feature map is reorganized into a C×H×W output format for subsequent image processing tasks. The EMA attention mechanism effectively improves the model's understanding and processing capabilities of image features through this complex feature reweighting and fusion strategy, especially when processing images with complex backgrounds and diverse features, which can significantly improve recognition accuracy.

[0069] In the EMA module, the input is first grouped and then processed by different branches: one branch performs one-dimensional global pooling through 1*1 convolution, and the other performs feature extraction through 3*3 convolution. The output features of the two branches are then modulated by the sigmoid function and normalization operation, and finally merged through the cross-dimensional interaction module (C2f) to capture the pairwise relationship at the pixel level. EMA adopts an information aggregation method across spatial dimensions, encoding the global spatial information output by the convolution branch through 2D global average pooling, and converting the output channel features of the smallest branch to the corresponding dimensional shape. On the output of 2D global average pooling, EMA uses the SoftMax function (normalized exponential function) to fit the linear transformation. Finally, the results of parallel processing are multiplied through the matrix dot product operation to obtain the first spatial attention map. The EMA attention mechanism improves the model's ability to focus on target features, allowing the model to obtain more feature information and improve the model's small target detection accuracy.

[0070] The initial marine life detection model refers to YOLOv8 (You Only Look Once version 8), an advanced target detection model with efficient detection speed and accuracy. YOLOv8 can quickly locate and classify target objects in images through improved network structure and algorithm. It uses technologies such as convolutional neural network (CNN) and feature pyramid network (FPN) to achieve real-time detection of various targets, and is suitable for the initial model of marine life detection. The reference marine life detection model refers to the model obtained by improving the initial YOLOv8 model by introducing Swin Transformer network and efficient multi-scale attention mechanism (EMA). This improved model combines the global information capture ability of Swin Transformer and the multi-scale processing advantages of EMA, aiming to improve the accuracy and robustness of marine life detection. After these technical improvements, the reference marine life detection model can better adapt to the detection task of marine life and provide more accurate detection results.

[0071] It is understandable that the structure of the model is optimized by introducing the Swin Transformer network and the efficient multi-scale attention mechanism (EMA) into the initial YOLOv8 marine life detection model. The Swin Transformer network enhances the model's feature extraction effect in complex underwater environments through its hierarchical global feature capture capability, while the EMA mechanism further improves the model's detection accuracy for marine life of different sizes by focusing on multi-scale important features. Through these two improvements, a more accurate and robust reference marine life detection model is finally obtained, which can better cope with the challenges of biological detection in marine environments.

[0072] As an example, the steps of introducing a Swin Transformer network and an efficient multi-scale attention mechanism to improve the initial marine life detection model and obtain a reference marine life detection model include: using the Swin Transformer network to replace the backbone network of the initial marine life detection model; adding an efficient multi-scale attention mechanism to the neck network of the initial marine life detection model to obtain a reference marine life detection model.

[0073] The backbone network is the basic part of the target detection model, responsible for extracting features from the input image. In YOLOv8, the backbone network is usually composed of multiple convolutional layers, pooling layers and other modules. Its main function is to extract low-level and high-level features of the image layer by layer, such as edges, textures, shapes, etc. Through the processing of the backbone network, the original image is converted into a series of feature maps, which contain rich information about the image content. The Swin Transformer network is introduced to replace the backbone network of the initial model, aiming to enhance the model's ability to capture global information and express complex features. The neck network is located between the backbone network and the detection head. Its main function is to further process and fuse the multi-scale features extracted by the backbone network. It fuses feature maps of different scales through a multi-layer feature pyramid network (FPN) or other feature fusion modules to better detect targets of various scales. In this step, an efficient multi-scale attention mechanism (EMA) is added to the neck network to enhance the model's ability to detect marine life of different sizes and complex backgrounds by focusing on important features and suppressing background noise.

[0074] First, the backbone network of the initial marine life detection model (YOLOv8) (the part responsible for extracting basic features from the image) was replaced with the Swin Transformer network. This replacement enhanced the model's ability to capture global features in complex marine environments, allowing the model to more effectively extract key information from images. Next, an efficient multi-scale attention mechanism (EMA) was introduced into the neck network of the model (the part responsible for fusing and processing multi-scale features). This mechanism further improved the model's detection accuracy and robustness for marine life of different sizes by focusing on important feature areas. After these improvements, a more accurate and adaptable reference marine life detection model was finally obtained, which can better cope with complex marine life detection tasks.

[0075] Please refer to Figure 4 , Figure 4A schematic diagram of the network structure of the reference marine biological detection model provided for the first embodiment of the marine biological detection method of this application shows the network structure of a deep learning model specifically used for marine biological detection, which includes multiple advanced feature extraction and processing modules. The following is a detailed explanation of each concept in the figure: 1) C2f (Cross-dimensional Interaction Module): a cross-dimensional interaction module used to interact information between different dimensions and enhance the expressiveness of features. In the figure, the C2f module appears repeatedly, indicating that it is used to process feature maps at multiple stages of the model. 2) SPPF (Spatial Pyramid Pooling Layer): a spatial pyramid pooling layer that integrates the global information of the image through multi-scale pooling, which helps the model capture features of different scales, and is particularly important for small target detection. 3) Conv (Convolutional Layer): a convolutional layer used to extract local features of an image. Multiple convolutional layers are stacked to capture more complex features. 4) CBS (Convolutional Block with BatchNorm and Sigmoid): a convolutional block with batch normalization and Sigmoid activation function. This structure helps the model learn more effective feature representations and enhances the model's expressiveness through nonlinear activation. 5) Upsample: Upsampling operation, used to increase the size of the feature map, usually used to convert low-resolution feature maps to high-resolution to retain more detail information. 6) Concat (Concatenation): Concatenation operation, used to merge feature maps from different sources in the channel dimension, so that feature information at different levels can be combined to enhance the model's feature fusion capability. 7) Bottleneck: Bottleneck layer, usually used to reduce the number of parameters and calculations of the model while trying to maintain feature information. In deep models, bottleneck layers help reduce the risk of overfitting. 8) EMA (Enhanced Multi-scale Attention): Enhanced multi-scale attention mechanism, used to focus on important parts of the image at different scales and improve the model's sensitivity to small targets and details. 9) Detect: Detection layer, usually contains a series of convolution operations and subsequent non-maximum suppression (NMS) and other processing, used to generate the final detection results, including the location, category and confidence of the target. 10) MaxPool2d (Max Pooling Layer): The maximum pooling layer is used to reduce the spatial dimension of the feature map while retaining the most representative features. 11) MaxPool2d followed by Conv2d: The maximum pooling layer is followed by a two-dimensional convolutional layer. This structure is usually used to further extract features and reduce the amount of calculation.The entire network structure uses a carefully designed feature extraction and fusion strategy, which enables the model to effectively process underwater images, especially to accurately detect marine life in complex environments. The cross-dimensional interaction module C2f is used multiple times in the model, showing its importance in enhancing feature representation.

[0076] Step S30, using the training set and the test set to train and test the reference marine organism detection model to obtain a target marine organism detection model;

[0077] It should be noted that the target marine life detection model refers to the final model obtained after the training and testing process and used for actual marine life detection tasks. It has the ability to detect marine life and can accurately detect and classify new images in practical applications.

[0078] It can be understood that, first, the reference marine life detection model is trained using the training set. The training set consists of annotated marine life images that provide real data about the target organism category and location. The model continuously adjusts its internal parameters through this data to optimize its feature extraction and target detection capabilities. During the training process, the performance of the model is continuously improved by calculating the loss function and optimization algorithm to better adapt to the task of marine life detection. After the training is completed, the trained model is evaluated using the test set. The test set contains unseen marine life images and is designed to test the generalization ability of the model in practical applications. Through the detection results on the test set, the model's performance indicators such as precision and recall are calculated to verify its actual effect. Finally, the trained and tested model is the "target marine life detection model", which is a final model that has been optimized and verified to effectively detect marine life in a real environment.

[0079] As an example, the reference marine biological detection model is trained and tested using the training set and the test set to obtain a target marine biological detection model, including the steps of: training the reference detection model using the training set to obtain a second reference detection model; testing the second reference detection model using the test set to obtain a test result; when the test result does not meet the preset requirements, returning to the step of training the reference detection model using the training set until the test result meets the preset requirements to obtain the target marine biological detection model.

[0080] The second reference detection model refers to the model obtained after training the reference marine life detection model using the training set. During the training process, the parameters of the reference model will be adjusted through the optimization algorithm so that the model can better learn and adapt to the features in the training data. After the training is completed, this new model is the second reference detection model, which has been optimized to facilitate the processing of marine life detection tasks. The test results refer to the performance indicators obtained after evaluating the second reference detection model using the test set. These results usually include various performance indicators of the model on the test set, such as accuracy, recall, precision, F1 score, etc. The test results are used to measure the generalization ability and actual detection effect of the model on unseen data to determine whether the model can meet the expected performance standards. The preset requirements refer to the performance standards or indicators set during the training and evaluation of the detection model. These requirements usually include the specific values ​​or ranges of indicators such as the detection accuracy, recall, F1 score, etc. of the model on the test set, which are used to evaluate whether the model meets the application requirements. For example, the preset requirements may be that the accuracy of the model must reach more than 95%, or the F1 score must exceed a certain threshold. If the test results fail to meet these preset requirements, the model needs to be further adjusted and trained until it reaches the expected standards.

[0081] First, the reference marine organism detection model is trained using the training set to optimize the model's parameters and generate a new version, called the second reference detection model. During this process, the model improves its ability to detect marine organisms by learning from the labeled data in the training set. Next, the second reference detection model is evaluated using the test set to obtain test results, which include performance indicators of the model on unseen data, such as detection accuracy and recall. The test results are then compared with the preset requirements. If the test results fail to meet these preset requirements, the training step is returned to continue to adjust and train the reference model to improve its performance. This process is repeated until the model's test results meet the preset performance standards and are ultimately determined as the target marine organism detection model, which ensures that the resulting model can effectively detect and classify marine organisms in practical applications.

[0082] As an example, the reference detection model is trained using the training set to obtain a second reference detection model, including: obtaining a total loss based on positioning loss, confidence loss, classification loss, and multi-task loss; and the reference detection model is trained using the training set until the total loss is lower than a preset loss value to obtain a second reference detection model.

[0083] The positioning loss is used to measure the error of the model when predicting the target location. It calculates the difference between the bounding box predicted by the model and the true labeled bounding box, usually expressed as coordinate error. This loss measures the accuracy of the model in locating the target object. Common positioning losses include Smooth L1 Loss or IoU Loss. Confidence loss measures the accuracy of the model's confidence in the existence of the target in each predicted box. This loss calculates the difference between the model's predicted probability of the existence of the target and the actual labeled value, usually measured by binary cross entropy loss. It ensures that the model makes accurate predictions on the confidence of the existence of the target. Classification loss is used to measure the accuracy of the model's prediction of the target category. This loss calculates the difference between the category probability distribution predicted by the model and the true category label, usually expressed as cross-entropy loss, which helps the model improve its ability to recognize the target object category. Multi-task loss is a comprehensive loss that combines positioning loss, confidence loss and classification loss to optimize the overall performance of the model. In the target detection task, this multi-task loss function simultaneously considers the model's performance in target positioning, target existence confidence and target classification to ensure the comprehensive ability of the model. The total loss is the weighted sum of all individual losses (positioning loss, confidence loss, classification loss). It measures the overall performance of the model on all tasks by combining various types of losses into a single indicator. The total loss function is used to guide the optimization during the training process, and the goal is to minimize this loss value. The preset loss value is a loss threshold set to evaluate the training effect of the model. For example, assuming the preset loss value is 0.5, this means that when the total loss of the model drops below 0.5 during training, the training will stop and the performance of the model is considered to have reached the expected standard. The setting of the preset loss value usually depends on the specific application requirements and model performance goals.

[0084] First, the total loss of the model is obtained by calculating the positioning loss, confidence loss, classification loss, and multi-task loss. The positioning loss evaluates the error between the target position predicted by the model and the actual position; the confidence loss measures the accuracy of the model's confidence in the existence of the target; the classification loss calculates the accuracy of the model's prediction of the target category; and the multi-task loss combines these losses to reflect the overall performance of the model in the target detection task. Next, the reference detection model is iteratively trained using the training set, and the total loss value is reduced by continuously optimizing the model parameters until the total loss is lower than the preset loss value (for example, 0.5), indicating that the model has reached the predetermined performance standard. The model obtained at this time is the second reference detection model, which can be optimized to perform more accurately in the target marine life detection task.

[0085] Step S40: Use the target marine life detection model to detect marine life in the marine life image.

[0086] It should be noted that marine life images refer to image data that contains marine life. These images can come from various sources, such as: Field images: images taken in actual marine environments by underwater cameras, remote-controlled submersibles, or underwater drones, showing real marine life and their natural habitats. Video frames: single-frame images extracted from underwater videos that record the dynamic behavior of marine life in natural environments. Simulated images: images generated by computer-generated technology or simulated environments, used to expand data sets or simulate specific scenarios. Images in databases: images from marine life image databases or public data sets, which are annotated and used to train or test target detection models.

[0087] It is understandable that the trained and optimized target marine life detection model is applied to actual marine life images. First, these images are input into the target detection model, and the model will identify various marine life in the image through its internal feature extraction and analysis mechanism. Specifically, the model will process each area in the image and generate prediction results, including the category, location (usually represented by a bounding box) of each target, and the confidence of the target's existence. The model will output this information in order to identify and locate different marine life appearing in the image. Through this process, marine life can be automatically detected, classified, and located, thereby providing support for subsequent data analysis or scientific research.

[0088] This embodiment provides a method for detecting marine life, which obtains a marine life data set and divides the marine life data set into a training set and a test set; introduces a Swin Transformer network and an efficient multi-scale attention mechanism to improve the initial marine life detection model to obtain a reference marine life detection model; uses the training set and the test set to train and test the reference marine life detection model to obtain a target marine life detection model; and uses the target marine life detection model to detect marine life in a marine life image. The marine life data set is obtained and divided into a training set and a test set, which provides structured data support for the model, ensures efficient learning during the training process and accuracy during the evaluation process, not only ensures the diversity of data, but also improves the performance of the model in different environments. The Swin Transformer network and the efficient multi-scale attention mechanism are introduced to improve the initial detection model, and the detection accuracy of the model is significantly improved by enhancing the global feature capture capability and multi-scale information processing capability of the model. The improved model is trained using the training set, and the performance is evaluated using the test set, which ensures the accuracy and robustness of the model in practical applications. Ultimately, the target marine life detection model obtained through these steps can detect marine life more accurately, automatically identify and locate marine life, thereby improving detection efficiency and providing reliable data support for scientific research and environmental protection.

[0089] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 5 , Figure 5 This is a flow chart of the second embodiment of the marine life detection method of the present application. The target marine life detection model includes a backbone network, a neck network and a head network. The backbone network includes a split module and a Transformer layer. Step S40 of the marine life detection method includes steps S41 to S44:

[0090] Step S41, splitting the marine life image into non-overlapping image blocks by the splitting module, and flattening the non-overlapping image blocks into sequence vectors;

[0091] It should be noted that the backbone network is the core component of the model, responsible for extracting important features from the input image. It generates high-level feature representations by processing image data, providing a basis for subsequent processing steps. In this model, the backbone network includes split modules and Transformer layers, etc., to assist in converting marine life images into feature representations. The neck network is located between the backbone network and the head network, responsible for further processing and integration of the features generated by the backbone network to provide more useful feature information for the head network. The neck network usually includes feature fusion and multi-scale feature processing to optimize the detection performance of the model. The head network is the last stage of the model, mainly responsible for making final predictions from the features output by the neck network, which includes specific tasks of target detection, such as locating the bounding box of the target, classifying the category of the target, and estimating the confidence of the target's existence. The split module is a component in the backbone network, which is used to split the input marine life image into several non-overlapping small image blocks. This method helps to process large-size images, so that each small block can be processed independently, thereby improving computational efficiency and feature extraction accuracy. The Transformer layer is a neural network structure that is good at processing sequence data. In the backbone network, the Transformer layer is used to process the sequence of image patches generated by the split module, and to capture long-distance dependencies and global context information through the self-attention mechanism, thereby improving the effect of feature extraction. Non-overlapping image patches refer to the division of an image into several non-overlapping small areas, each of which is independent of other areas. This division method can help reduce computational complexity and improve the locality and accuracy of feature extraction. Sequence vectors refer to the vector sequences formed by flattening the non-overlapping image patches generated by the split module. These vector sequences contain the feature information of each image patch and are used to input into the Transformer layer for further feature processing and analysis.

[0092] It can be understood that the target marine life detection model consists of a backbone network, a neck network, and a head network, where the backbone network is responsible for extracting features from the image. The splitting module in the backbone network first divides the input marine life image into several non-overlapping small image blocks, each of which covers a part of the image and does not overlap with other blocks. Then, these non-overlapping image blocks are flattened, that is, the two-dimensional pixel data of each image block is converted into a one-dimensional sequence vector. This flattening process arranges the pixel information in each image block in the form of a linear sequence for subsequent processing. The flattened sequence vector will be sent to the Transformer layer, which uses the self-attention mechanism to capture the relationship between each sequence and integrate global context information. In this way, the backbone network can effectively convert the local features of the image block into a global feature representation, thereby providing accurate feature information for the neck network and the head network, and improving the overall performance of target detection.

[0093] Step S42, processing the sequence vector through the Transformer layer to obtain a multi-scale feature map;

[0094] It should be noted that multi-scale feature maps refer to feature maps with different resolutions or scales generated after multi-layer processing in a neural network (especially in target detection tasks). These feature maps capture various information of target objects in the input image at different scales. Specifically, in the target marine life detection model, when the flattened sequence vector is processed by the Transformer layer, the model captures the global and local features of the input data through the self-attention mechanism and hierarchical structure. These processing results are output in the form of feature maps, but feature maps at different levels have different resolutions and scales. For example, feature maps at lower levels usually have higher resolutions and can capture detailed information, while feature maps at higher levels have lower resolutions but can capture more global and abstract features. The advantage of multi-scale feature maps is that they can capture target objects of different sizes in the image at the same time, allowing the model to perform well when detecting both small and large targets. This multi-scale feature map provides rich information for the subsequent head network, helping the model to more accurately identify and locate marine biological targets.

[0095] It can be understood that first, the flattened sequence vectors are input into the Transformer layer, which uses the self-attention mechanism and hierarchical structure to process these vectors and capture the global and local information in the image. In this process, the Transformer layer uses a multi-head self-attention mechanism to simultaneously focus on different feature dimensions, so that it can understand the features of the image at different scales. The processing results generate a series of feature maps with different resolutions or scales, which are called multi-scale feature maps. These feature maps capture various information of the target objects in the image at different scales. The higher-resolution feature maps can capture details, while the lower-resolution feature maps capture more global abstract features. Ultimately, these multi-scale feature maps provide rich information for the subsequent detection of the model, enabling the model to more accurately identify and locate marine biological targets of different sizes.

[0096] As an example, the Transformer layer includes a linear embedding layer, a merging layer, a linear layer and a SwinTransformer module. The step of processing the sequence vector through the Transformer layer to obtain a multi-scale feature map includes: projecting the sequence vector into a linear tensor through the linear embedding layer; reducing the dimension of the linear tensor through the merging layer and the linear layer, and performing feature conversion on the linear tensor through the Swin Transformer module to obtain a multi-scale feature map.

[0097] The Linear Embedding Layer is a key component in the Transformer layer, responsible for projecting the input sequence vector into a high-dimensional vector space. Specifically, this process is to transform the flattened sequence vector into a linear tensor with a fixed dimension through a linear transformation (i.e., matrix multiplication). This step can be regarded as embedding the original feature vector into a new feature space for subsequent feature processing. The Merging Layer is used to reduce the dimension and process the feature map at a specific stage. It reduces the resolution of the feature map by merging adjacent image blocks or feature blocks. This process helps to compress information and extract more global features, preparing for subsequent feature processing and the generation of multi-scale feature maps. The Linear Layer further processes the linear tensor and reduces the dimension of the tensor to the required size through a linear transformation (usually a fully connected layer). The tensor after dimensionality reduction still retains important feature information, but reduces the computational complexity, which helps to improve the efficiency of the model. The Swin Transformer Module is the core component of the Transformer layer. It uses a sliding window to perform self-attention operations in feature maps. It can capture long-distance dependencies in feature maps and process feature maps through a multi-head self-attention mechanism. The Swin Transformer module performs feature conversion at different scales to generate feature maps of different resolutions, thereby forming multi-scale feature maps. A linear tensor is a tensor that is projected by a linear embedding layer. It is a high-dimensional vector representation that retains the main feature information of the original sequence vector and is converted into a format suitable for further processing for subsequent feature conversion and dimensionality reduction operations.

[0098] The Transformer layer consists of a linear embedding layer, a merging layer, a linear layer, and a Swin Transformer module. The steps for processing sequence vectors are as follows: First, the linear embedding layer projects the input sequence vector into a linear tensor. This process converts the sequence vector into a high-dimensional tensor representation through a linear transformation, so that it can better capture feature information. Next, the linear tensor is processed by the merging layer and the linear layer for dimensionality reduction. The merging layer merges adjacent feature blocks to reduce the resolution of the feature map, thereby compressing the information. The linear layer reduces the dimension of the tensor through further linear transformation to reduce computational complexity and maintain key features. Finally, the processed tensor is sent to the Swin Transformer module, which performs feature transformation on the dimensionality-reduced tensor through the self-attention mechanism of the sliding window to generate feature maps of different resolutions. These feature maps have multi-scale features and can capture features of different sizes and levels in the image, providing rich contextual information for the subsequent detection tasks of the model.

[0099] Step S43, processing the multi-scale feature map through the neck network to obtain a spatial attention feature map;

[0100] It should be noted that the spatial attention feature map refers to the feature map generated in the target detection model by emphasizing the features of key areas in the image while suppressing the features of unimportant or background areas. This feature map can highlight the parts of the image related to the detection task and help the model identify and locate the target object more accurately. Specifically, in the marine life detection model, after the multi-scale feature map is processed by the neck network, the model applies the spatial attention mechanism to analyze the spatial information in the feature map. This mechanism calculates the importance of each position and focuses on areas in the image where targets may exist. For example, if an area contains features of marine life, this area will be enhanced in the spatial attention feature map, while other irrelevant areas will be weakened. In this way, the model can focus on important features more accurately and improve the accuracy and robustness of detection.

[0101] It can be understood that, first, the input multi-scale feature map is processed by the neck network, which further integrates and optimizes these feature maps through a series of operations (such as feature fusion, feature pyramid, etc.). In this process, the neck network will apply the spatial attention mechanism to focus on analyzing the spatial information of each position in the feature map and identify which areas may contain target objects. Through this mechanism, the model can enhance the regional features related to the target detection task while suppressing the features of the background or irrelevant areas. Finally, after this process, the generated spatial attention feature map can highlight the important target areas in the image, allowing the model to more accurately identify and locate the target object, improving the accuracy and reliability of detection.

[0102] As an example, the neck network includes a convolution module, a cross-dimensional interaction module, an aggregation module, an upsampling module and an efficient multi-scale attention mechanism module. The step of processing the multi-scale feature map through the neck network to obtain a spatial attention feature map includes: performing one-dimensional global pooling, feature extraction and feature merging on the multi-scale feature map through the convolution module, the cross-dimensional interaction module, the aggregation module and the upsampling module to obtain an intermediate feature map; performing two-dimensional global pooling and normalization on the intermediate feature map through the efficient multi-scale attention mechanism module to obtain a spatial attention feature map.

[0103] The convolution module is a basic unit in the neural network, which is used to extract local features in the input feature map through convolution operations. These convolution operations can capture information such as edges and textures in different regions of the image, thereby extracting more meaningful features. The cross-dimensional interaction module is a module that can exchange and fuse information in different dimensions. It allows features to interact between different channels or spatial dimensions, thereby enriching feature expressions and enhancing the model's ability to understand complex scenes. The aggregation module is responsible for fusing features from different levels or different sources to generate a more comprehensive feature representation. Through the aggregation module, the model can combine multi-level feature information to better detect targets. The upsampling module is used to increase the resolution of the feature map so that low-resolution feature maps can be restored to higher resolutions, which is very important for recovering detail information and improving detection accuracy, especially when dealing with small targets. The efficient multi-scale attention mechanism module focuses on the most important feature areas in the image by calculating attention at different scales and suppressing irrelevant or minor background information. It can improve the model's feature extraction ability and detection accuracy when dealing with complex scenes. One-dimensional global pooling is a pooling operation that performs an average or maximum operation on the spatial dimension of the feature map to obtain a more compact global feature. This operation is usually used to reduce the size of the feature map while retaining global information. Feature extraction refers to extracting discriminative features from the input image or feature map through convolution operations and other means. These features can be information such as edges, textures, shapes, etc., which are the basis for subsequent classification and detection. Feature merging refers to integrating feature maps from different sources (such as feature maps from different scales or different levels) to generate a comprehensive feature representation. This process can improve the performance of the model, especially in multi-scale object detection. Intermediate feature maps are feature maps generated after passing through convolution modules, cross-dimensional interaction modules, aggregation modules, and upsampling modules. These feature maps are the basis for the final generation of spatial attention feature maps and contain rich information extracted and integrated through the above processing. Two-dimensional global pooling is to perform global average or maximum pooling on each channel of the feature map in its spatial dimension to generate a value representing global information. This operation is performed in two-dimensional space, which helps extract global features and reduce the size of the feature map. Normalization is the process of standardizing feature maps, adjusting feature values ​​to a specific range (such as 0 to 1) or distribution (such as a mean of 0 and a variance of 1). Normalization helps the model to be more stable when processing features and helps accelerate the convergence of the model.

[0104] In this step, the neck network processes the multi-scale feature map through the multiple modules it contains. Specifically, the convolution module first extracts local features from the feature map, and the cross-dimensional interaction module allows information exchange and fusion between different dimensions to enhance feature expression. Then, the aggregation module fuses features from different levels or sources to generate a more comprehensive feature representation, and the upsampling module increases the resolution of the feature map to restore details and improve the detection of small objects. After these steps are completed together, the feature map is compressed through a one-dimensional global pooling operation to extract global features and generate an intermediate feature map. Subsequently, the intermediate feature map is processed by an efficient multi-scale attention mechanism module, which includes two-dimensional global pooling and normalization operations. Two-dimensional global pooling extracts global information, and normalization standardizes the feature values ​​to ensure the stability of the model processing. Finally, this series of processing steps generates a spatial attention feature map, which can highlight important target areas in the image and improve the accuracy and robustness of target detection.

[0105] Step S44, decoupling the spatial attention feature map through the head network to obtain a marine life detection result.

[0106] It should be noted that decoupling refers to the process of processing different detection tasks (such as classification and position regression of targets) separately in target detection tasks. In traditional target detection models, classification and position regression are usually performed simultaneously through the same feature map, while the decoupling method processes these tasks separately in order to optimize the feature extraction and processing process of each task. Through decoupling, the model can handle complex target detection tasks more effectively and improve classification accuracy and positioning accuracy. The marine life detection result refers to the final detection information output by the model after processing the image, which usually includes the category, location (usually represented by a bounding box), and confidence (indicating the model's degree of trust in the detection result) of each detected marine organism. These detection results are used to identify and locate marine biological targets in the image, providing data support for subsequent analysis, research or application.

[0107] It can be understood that decoupling refers to processing the two subtasks of classification and positioning in the target detection task separately, extracting the most suitable features for each task respectively. Through this process, the head network can more accurately determine the categories of marine life in the image (such as fish, starfish, etc.) and their locations (usually represented by bounding boxes), and calculate the confidence of each detection result (indicating the degree of trust the model has in the detection result). Ultimately, the marine life detection results output by the model include the category, location and confidence information of each target. These results are used to identify and locate marine biological targets in the image, providing data support for subsequent scientific research, environmental monitoring or applications.

[0108] In this embodiment, the marine life image is split into non-overlapping image blocks through the splitting module, and the non-overlapping image blocks are flattened into sequence vectors; the sequence vector is processed by the Transformer layer to obtain a multi-scale feature map; the multi-scale feature map is processed by the neck network to obtain a spatial attention feature map; the spatial attention feature map is decoupled by the head network to obtain a marine life detection result. First, the image is split into non-overlapping small blocks through the splitting module, and these small blocks are flattened into sequence vectors. This process enables subsequent processing to be performed on smaller feature units, improves computational efficiency, and enables the model to process fine-grained features. Then, the Transformer layer processes these sequence vectors and generates a multi-scale feature map through the self-attention mechanism. The feature map can capture target information of different scales and enhance the detection ability of marine life of various sizes. Subsequently, the neck network further processes the multi-scale feature map and generates a spatial attention feature map using the spatial attention mechanism. This processing step can highlight the key areas in the image and suppress irrelevant background noise, thereby improving the attention to the target area. Finally, the head network decouples the spatial attention feature map and processes the classification and positioning tasks separately, thereby optimizing the target category recognition and position regression respectively. This decoupled processing method ultimately outputs the marine life detection results, including the target category, location and confidence, providing high-quality data support for the accurate identification and positioning of marine life.

[0109] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the marine organism detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0110] This application also provides a marine life detection device, please refer to Figure 6 , the marine life detection device comprises:

[0111] A data acquisition module 10 is used to acquire a marine biological data set and divide the marine biological data set into a training set and a test set;

[0112] A model improvement module 20 is used to introduce a Swin Transformer network and an efficient multi-scale attention mechanism to improve the initial marine life detection model and obtain a reference marine life detection model;

[0113] A model training module 30, for training and testing the reference marine organism detection model using the training set and the test set to obtain a target marine organism detection model;

[0114] The target detection module 40 is used to detect marine life in the marine life image using the target marine life detection model.

[0115] The marine life detection device provided by the present application adopts the marine life detection method in the above embodiment, which can solve the technical problem of how to detect marine life more accurately. Compared with the prior art, the beneficial effects of the marine life detection device provided by the present application are the same as the beneficial effects of the marine life detection method provided by the above embodiment, and the other technical features of the marine life detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0116] The present application provides a marine life detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the marine life detection method in the above-mentioned embodiment one.

[0117] Reference below Figure 7 , which shows a schematic diagram of the structure of a marine life detection device suitable for implementing the embodiment of the present application. The marine life detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The marine life detection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0118] like Figure 7As shown, the marine life detection device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. In RAM 1004, various programs and data required for the operation of the marine life detection device are also stored. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the marine life detection device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a marine life detection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0119] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0120] The marine life detection device provided by the present application adopts the marine life detection method in the above embodiment, which can solve the technical problem of how to detect marine life more accurately. Compared with the prior art, the beneficial effects of the marine life detection device provided by the present application are the same as the beneficial effects of the marine life detection method provided by the above embodiment, and the other technical features of the marine life detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0121] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0122] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0123] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the marine life detection method in the above-mentioned embodiment.

[0124] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above. The computer-readable storage medium may be included in the marine life detection device; or may exist independently without being assembled into the marine life detection device.

[0125] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the marine life detection device, the marine life detection device: obtains a marine life data set, and divides the marine life data set into a training set and a test set; introduces a Swin Transformer network and an efficient multi-scale attention mechanism to improve the initial marine life detection model to obtain a reference marine life detection model; uses the training set and the test set to train and test the reference marine life detection model to obtain a target marine life detection model; and uses the target marine life detection model to detect marine life in a marine life image.

[0126] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0129] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned marine organism detection method, and can solve the technical problem of how to detect marine organisms more accurately. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the marine organism detection method provided in the above-mentioned embodiment, and will not be repeated here.

[0130] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned marine organism detection method when executed by a processor. The computer program product provided by the present application can solve the technical problem of how to detect marine organisms more accurately. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the marine organism detection method provided by the above-mentioned embodiment, which will not be repeated here.

[0131] The above descriptions are only some embodiments of the present application and do not limit the patent scope of the present application. All equivalent structural changes made by using the contents of the present application specification and drawings under the technical concept of the present application, or directly / indirectly applied in other related technical fields, are within the patent protection scope of the present application.

Claims

1. A method for detecting marine organisms, characterized in that: The method comprises: Acquire a marine biological data set, and divide the marine biological data set into a training set and a test set; The Swin Transformer network and efficient multi-scale attention mechanism were introduced to improve the initial marine life detection model and obtain the reference marine life detection model. Using the training set and the test set to train and test the reference marine organism detection model to obtain a target marine organism detection model; The target marine life detection model is used to detect marine life in a marine life image.

2. The method according to claim 1, characterized in that The target marine life detection model includes a backbone network, a neck network and a head network, the backbone network includes a split module and a Transformer layer, and the step of using the target marine life detection model to detect marine life in a marine life image includes: Splitting the marine life image into non-overlapping image blocks by the splitting module, and flattening the non-overlapping image blocks into sequence vectors; Processing the sequence vector through the Transformer layer to obtain a multi-scale feature map; Processing the multi-scale feature map through the neck network to obtain a spatial attention feature map; The spatial attention feature map is decoupled through the head network to obtain a marine life detection result.

3. The method according to claim 2, characterized in that The Transformer layer includes a linear embedding layer, a merging layer, a linear layer and a Swin Transformer module. The step of processing the sequence vector through the Transformer layer to obtain a multi-scale feature map includes: Projecting the sequence vector into a linear tensor through the linear embedding layer; The linear tensor is dimensionally reduced through the merging layer and the linear layer, and the linear tensor is feature transformed through the SwinTransformer module to obtain a multi-scale feature map.

4. The method according to claim 2, characterized in that The neck network includes a convolution module, a cross-dimensional interaction module, an aggregation module, an upsampling module, and an efficient multi-scale attention mechanism module. The step of processing the multi-scale feature map through the neck network to obtain a spatial attention feature map includes: Performing one-dimensional global pooling, feature extraction, and feature merging on the multi-scale feature map through the convolution module, the cross-dimensional interaction module, the aggregation module, and the upsampling module to obtain an intermediate feature map; The intermediate feature map is subjected to two-dimensional global pooling and normalization through the efficient multi-scale attention mechanism module to obtain a spatial attention feature map.

5. The method according to any one of claims 1 to 4, characterized in that The steps of introducing the Swin Transformer network and the efficient multi-scale attention mechanism to improve the initial marine life detection model and obtain the reference marine life detection model include: Use the Swin Transformer network to replace the backbone network of the initial marine life detection model; An efficient multi-scale attention mechanism is added to the neck network of the initial marine life detection model to obtain a reference marine life detection model.

6. The method according to claim 1, characterized in that The step of using the training set and the test set to train and test the reference marine organism detection model to obtain a target marine organism detection model comprises: Using the training set to train the reference detection model to obtain a second reference detection model; Using the test set to test the second reference detection model, to obtain a test result; When the test result does not meet the preset requirements, return to the step of using the training set to train the reference detection model until the test result meets the preset requirements to obtain a target marine life detection model.

7. The method according to any one of claims 6, characterized in that The step of using the training set to train the reference detection model to obtain a second reference detection model comprises: The total loss is obtained based on the positioning loss, confidence loss, classification loss and multi-task loss; The reference detection model is trained using the training set until the total loss is lower than a preset loss value, thereby obtaining a second reference detection model.

8. A marine life detection device, characterized in that: The device comprises: A data acquisition module, used to acquire a marine biological data set and divide the marine biological data set into a training set and a test set; The model improvement module is used to introduce the Swin Transformer network and the efficient multi-scale attention mechanism to improve the initial marine life detection model and obtain the reference marine life detection model; A model training module, used to train and test the reference marine organism detection model using the training set and the test set to obtain a target marine organism detection model; The target detection module is used to detect marine life in the marine life image using the target marine life detection model.

9. A marine life detection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the marine organism detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the marine organism detection method according to any one of claims 1 to 7 are implemented.

Images