Fish school detection method and system, electronic device and storage medium

Abstract

The application provides a fish school detection method and system, an electronic device and a storage medium, and the method comprises the following steps: inputting a to-be-detected fish school image into a fish school detection model; wherein the fish school detection model comprises a feature extraction layer, a feature fusion layer and a feature recognition layer; based on the feature extraction layer, feature information of the to-be-detected fish school image is extracted, and based on an attention mechanism, a fish school feature map and an attention feature map are determined; based on the feature fusion layer, the fish school feature map and the attention feature map are fused to determine a target fusion feature map; and based on the feature recognition layer, a target fish school detection result is determined according to the target fusion feature map. The method can exclude the interference of environmental factors on the detection result and effectively improve the accuracy of fish school detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method and system for detecting schools of fish, an electronic device, and a storage medium. Background Technology

[0002] Fish school detection is of great value in detecting the activity patterns of fish schools in lakes and oceans, and in analyzing the size and species of fish schools. Furthermore, in aquaculture production, detecting fish density is a crucial step in effective production management.

[0003] Currently, fish school detection technologies mainly rely on sensor detection, digital image processing, and deep learning object detection. Sensor detection primarily uses acoustic and optical sensors, but the results are easily affected by noise, water quality, and light. Digital image processing relies on traditional visual algorithms to extract features and combines them with the experience of fish keepers to determine the monitoring results, but its detection accuracy is low when fish are highly clustered or their trajectories are changing. Limited by the typically small size of fish datasets and the complexity of fish characteristics, existing deep learning object detection methods often suffer from low recognition accuracy and slow detection speed.

[0004] Therefore, it is necessary to provide a fish detection method, system, electronic equipment, and storage medium that can eliminate the interference of environmental factors on the detection results and effectively improve the accuracy of fish detection. Summary of the Invention

[0005] To address the shortcomings of existing technologies, embodiments of the present invention provide a fish school detection method and system, electronic device, and storage medium.

[0006] This invention provides a method for detecting fish schools, comprising:

[0007] The image of the fish school to be tested is input into the fish school detection model; the fish school detection model includes: a feature extraction layer, a feature fusion layer, and a feature recognition layer;

[0008] Based on the feature extraction layer, feature information of the fish group image to be tested is extracted, and the fish group feature map and attention feature map are determined according to the attention mechanism;

[0009] Based on the feature fusion layer, the fish swarm feature map and the attention feature map are fused to determine the target fused feature map;

[0010] Based on the feature recognition layer, the target fish group detection result is determined according to the target fusion feature map.

[0011] The fish swarm detection method provided by the present invention includes a feature extraction layer comprising an initial feature extraction layer and an attention feature extraction layer;

[0012] Correspondingly, based on the feature extraction layer, feature information of the fish swarm image is extracted. According to the attention mechanism, the fish swarm feature map and attention feature map are determined, specifically including:

[0013] Based on the initial feature extraction layer, feature information of the fish school image to be tested is extracted to determine the fish school feature map;

[0014] Based on the attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism, channel attention mechanism and spatial attention mechanism to determine the attention feature map.

[0015] The fish swarm detection method provided by the present invention includes an attention feature extraction layer comprising a coordinate attention feature extraction layer and a convolutional block attention feature extraction layer; and an attention feature map comprising a coordinate attention feature map and a spatiotemporal attention feature map.

[0016] Correspondingly, based on the attention feature extraction layer, the fish swarm feature map is transformed according to coordinate attention mechanism, channel attention mechanism, and spatial attention mechanism to determine the attention feature map, specifically including:

[0017] Based on the coordinate attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism to determine the coordinate attention feature map;

[0018] Based on the convolutional block attention feature extraction layer, the fish swarm feature map is transformed according to the channel attention mechanism to determine the channel attention feature map. Based on the spatial attention mechanism, the channel attention feature map is transformed to determine the spatiotemporal attention feature map.

[0019] According to the fish detection method provided by the present invention, a target fusion feature map is determined by fusing a fish swarm feature map and an attention feature map based on a feature fusion layer, specifically including:

[0020] Based on the feature fusion layer, and according to the feature pyramid network, the fish swarm feature map, coordinate attention feature map, and spatiotemporal attention feature map are fused to determine the target fusion feature map at three different scales.

[0021] According to the fish school detection method provided by the present invention, based on the feature recognition layer, the target fish school detection result is determined according to the target fusion feature map, specifically including:

[0022] Based on the feature recognition layer, the species and quantity of target fish are determined in the image of the fish school to be tested according to the target fusion feature map;

[0023] Based on the nonmaximum suppression algorithm, duplicate detection values ​​are eliminated to determine the target fish population detection results.

[0024] According to the fish detection method provided by the present invention, before the step of inputting the image of the fish to be detected into the fish detection model, the method further includes: determining the network structure of the fish detection model.

[0025] Determine the network structure of the fish swarm detection model, specifically including:

[0026] Based on the YOLOv5s algorithm network structure, coordinate attention modules and convolutional block attention modules are sequentially embedded in the backbone feature extraction network.

[0027] According to the fish detection method provided by the present invention, after the step of determining the network structure of the fish detection model, the method further includes: training the fish detection model.

[0028] Training the fish swarm detection model specifically includes:

[0029] Acquire several sample fish images, create labels, and determine the sample fish image set;

[0030] A fish detection model was trained based on a sample set of fish images.

[0031] Based on the objective loss function, the network parameters of the fish swarm detection model are updated using cosine annealing. The fish swarm detection model is then iteratively trained based on the updated network parameters until it converges.

[0032] The present invention also provides a fish school detection system, comprising: an image input unit, a feature extraction unit, a feature fusion unit, and a feature recognition unit;

[0033] The image input unit is used to input the image of the fish swarm to be tested into the fish swarm detection model; wherein, the fish swarm detection model includes: a feature extraction layer, a feature fusion layer and a feature recognition layer;

[0034] The feature extraction unit is used to extract feature information of the fish group image to be tested based on the feature extraction layer, and to determine the fish group feature map and the attention feature map according to the attention mechanism.

[0035] The feature fusion unit is used to fuse the fish swarm feature map and the attention feature map based on the feature fusion layer to determine the target fused feature map;

[0036] The feature recognition unit is used to determine the target fish group detection result based on the target fusion feature map, according to the feature recognition layer.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the fish detection methods described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the fish swarm detection methods described above.

[0039] The fish detection method, system, electronic device, and storage medium provided by this invention achieve fish image detection by constructing a fish detection model by combining an attention mechanism, enabling rapid screening of meaningful features, effectively improving the training and detection speed of the fish detection model, saving computing resources, and improving the accuracy of fish detection for overlapping and partially occluded targets while reducing the number of convolution kernels. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 The flowchart of the fish school detection method provided by the present invention;

[0042] Figure 2 This is a schematic diagram of the fish school detection model structure provided by the present invention;

[0043] Figure 3 A schematic diagram of the improved YOLOv5s algorithm network structure provided by this invention;

[0044] Figure 4 This is a schematic diagram of the fish detection system provided by the present invention;

[0045] Figure 5 A schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] Fish detection mainly relies on sensor detection methods, digital image processing methods, and deep learning object detection methods.

[0048] Sensor-based detection methods primarily utilize acoustic and optical sensors to detect schools of fish. Acoustic sensors measure the location of underwater objects by the time it takes for emitted sound waves to travel back after hitting them. Sonar sensors are unaffected by water turbidity, but the presence of noise in the aquatic environment can easily interfere with fish school signals. Optical sensors infer the size of underwater objects based on the amount of light transmitted, but they are highly dependent on water quality and lighting conditions and are easily affected by background interference.

[0049] Digital image processing methods, traditional visual algorithms, combine color feature extraction, texture feature extraction, geometric feature extraction and other methods with the experience of artificial breeders to use classifiers to process fish targets. When the fish population density is small and the artificially designed extractor is suitable for fish populations with specific breeding conditions, the detection accuracy is high. However, as the fish population density increases, the number of fish species increases, and the fish population trajectory changes due to the influence of a certain external factor, the detection accuracy will decrease and the robustness of the model will be low.

[0050] Deep learning-based object detection methods, particularly underwater fish detection algorithms, are currently a hot research topic. The network model can automatically extract effective features from images of fish farming and continuously adjust weights to achieve object recognition. Different datasets, network models, and optimization methods allow the model to adapt to various underwater environments.

[0051] While deep learning-based object detection methods offer the advantages mentioned above, existing methods often suffer from low accuracy and slow detection speed due to limitations such as the typically small size of fish datasets and the complexity of fish features. Therefore, this invention provides a fish school detection method to address these issues.

[0052] Figure 1 The flowchart of the fish school detection method provided by the present invention is as follows: Figure 1 As shown, this embodiment of the invention provides a method for detecting fish schools, including:

[0053] Step S1: Input the image of the fish school to be tested into the fish school detection model; wherein, the fish school detection model includes: a feature extraction layer, a feature fusion layer and a feature recognition layer;

[0054] Step S2: Based on the feature extraction layer, extract the feature information of the fish group image to be tested, and determine the fish group feature map and attention feature map according to the attention mechanism;

[0055] Step S3: Based on the feature fusion layer, fuse the fish swarm feature map and the attention feature map to determine the target fused feature map;

[0056] Step S4: Based on the feature recognition layer, determine the target fish group detection result according to the target fusion feature map.

[0057] Specifically, before conducting fish school detection, it is necessary to acquire images of the target fish school. It is understood that this invention can be used to detect images of fish schools in natural water bodies (e.g., lakes, seas, rivers, etc.) or in aquaculture environments (glass tanks or fishponds, etc.), without limiting the source of the images of the target fish school, and without limiting the species and quantity of the target fish school.

[0058] In addition, the fish school image can be obtained by taking pictures with an underwater camera or by acquiring the video in real time, and the video frame is used as the fish school image to be tested. This invention does not limit this method.

[0059] After obtaining the target fish group image, in step S1, the target fish group image is input into the trained fish group detection model.

[0060] Figure 2 This is a schematic diagram of the fish school detection model structure provided by the present invention, as shown below. Figure 2 As shown, the fish swarm detection model includes a feature extraction layer, a feature fusion layer, and a feature recognition layer. It is understood that the specific structure of the fish swarm detection model, the recognition algorithm used, and the training method can all be adjusted according to actual needs, and this invention does not limit these aspects.

[0061] In step S2, based on the feature extraction layer of the fish swarm detection model, the feature information of the fish swarm image to be tested is extracted to obtain the fish swarm feature map, and on this basis, the attention mechanism is combined to obtain the attention feature map of the fish swarm.

[0062] It should be noted that in the actual application of this invention, the type of attention mechanism can be selected according to actual needs, such as channel attention mechanism, spatial attention mechanism, coordinate attention mechanism, self-attention mechanism and hybrid attention mechanism, etc., and this invention does not limit it.

[0063] After determining the fish swarm feature map and the attention feature map, they are input into the feature fusion layer of the model. In step S3, the fish swarm feature map and the attention feature map are fused based on the feature fusion layer to determine the target fused feature map.

[0064] It should be noted that in the actual application of this invention, the specific method of feature fusion can be selected according to actual needs, such as Path Aggregation Network (PANet) and weighted bi-directional feature pyramid network (BiFPN), etc., and this invention does not limit it.

[0065] The fused target feature map is input into the feature recognition layer. In step S4, the target fish group detection result is determined based on the target fusion feature map and the feature recognition layer.

[0066] It is understood that, in the practical application of this invention, the detection result of the target fish group can be to determine the number of a certain type of fish in the image of the fish group to be tested (such as determining the specific number of saury in a mixed fish group), or to determine the species of fish in the image of the fish group to be tested and the number of each species (such as detecting that it contains three species in an ornamental fish group: white-breasted wrasse, light-cheeked seabream, and neon characin, and determining the number of each species), or to determine whether the fish group contains the target species (such as determining whether there is an invasive alien species, alligator gar, in a fish group image of a lake). The specific detection result can be adjusted according to actual needs, and this invention does not limit it in this regard.

[0067] The fish detection method provided by this invention achieves fish image detection by constructing a fish detection model using an attention mechanism, enabling rapid screening of meaningful features and focusing on regions of interest. Compared to existing target detection schemes, this invention effectively improves the training and detection speed of the fish detection model by adding an attention mechanism, saving computational resources, and improving the accuracy of fish detection for overlapping and partially occluded targets while reducing the number of convolution kernels.

[0068] Optionally, in the fish swarm detection method provided by the present invention, the feature extraction layer includes: an initial feature extraction layer and an attention feature extraction layer;

[0069] Correspondingly, based on the feature extraction layer, feature information of the fish swarm image is extracted. According to the attention mechanism, the fish swarm feature map and attention feature map are determined, specifically including:

[0070] Based on the initial feature extraction layer, feature information of the fish school image to be tested is extracted to determine the fish school feature map;

[0071] Based on the attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism, channel attention mechanism and spatial attention mechanism to determine the attention feature map.

[0072] Specifically, the feature extraction layer includes: an initial feature extraction layer and an attention feature extraction layer;

[0073] Correspondingly, based on the feature extraction layer, feature information of the fish swarm image is extracted. According to the attention mechanism, the fish swarm feature map and attention feature map are determined, specifically including:

[0074] Based on the initial feature extraction layer, feature information of the fish school image to be tested is extracted to determine the fish school feature map;

[0075] Based on the attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism, channel attention mechanism and spatial attention mechanism to determine the attention feature map.

[0076] The fish detection method provided by this invention realizes fish image detection by constructing a fish detection model by combining an attention mechanism, thereby achieving rapid screening of meaningful features, effectively improving the training speed and detection speed of the fish detection model, saving computing resources, and improving the accuracy of fish detection for overlapping and partially occluded targets while reducing the number of convolution kernels.

[0077] Optionally, according to the fish swarm detection method provided by the present invention, the attention feature extraction layer includes: a coordinate attention feature extraction layer and a convolutional block attention feature extraction layer; the attention feature map includes: a coordinate attention feature map and a spatiotemporal attention feature map;

[0078] Correspondingly, based on the attention feature extraction layer, the fish swarm feature map is transformed according to coordinate attention mechanism, channel attention mechanism, and spatial attention mechanism to determine the attention feature map, specifically including:

[0079] Based on the coordinate attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism to determine the coordinate attention feature map;

[0080] Based on the convolutional block attention feature extraction layer, the fish swarm feature map is transformed according to the channel attention mechanism to determine the channel attention feature map. Based on the spatial attention mechanism, the channel attention feature map is transformed to determine the spatiotemporal attention feature map.

[0081] Specifically, this invention employs three attention mechanisms to combine features in three directions: space, channel, and coordinate. This allows for the focus on regions of interest while preserving feature information as much as possible, thereby improving the model's ability to select features and increasing its detection efficiency.

[0082] like Figure 2 As shown, the attention feature extraction layer includes a coordinate attention feature extraction layer and a convolutional block attention feature extraction layer. Correspondingly, the attention feature maps include a coordinate attention feature map and a spatiotemporal attention feature map.

[0083] When determining the attention feature map, the fish swarm feature map is first input into the coordinate attention feature extraction layer. Based on the coordinate attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism to determine the coordinate attention feature map.

[0084] The coordinate attention feature map is then input into the convolutional attention feature extraction module. Based on the convolutional block attention feature extraction layer, the fish swarm feature map is transformed according to the channel attention mechanism to determine the channel attention feature map. Furthermore, the channel attention feature map is transformed based on the spatial attention mechanism to determine the spatiotemporal attention feature map.

[0085] Fish school characteristics are divided into individual characteristics and overall characteristics. Individual characteristics include features such as shape, size, form, color, and texture, while overall characteristics include fish school aggregation degree, movement direction, and position information.

[0086] Coordinate attention primarily extracts the target's location information, focusing on global features and feature dependencies to aid in subsequent extraction of key fish school features. Spatiotemporal attention, divided into channel and spatial attention, extracts local features, enabling the extraction of overall fish school features over a given period.

[0087] If we use coordinate attention to extract global features separately and spatiotemporal attention to extract local features separately, when performing feature fusion, the global part missing by spatiotemporal attention and the local details missing by coordinate attention are slower than extracting local features from the global part, and effective information cannot be directly selected.

[0088] Building upon global coordinate attention, spatiotemporal attention feature extraction further enhances the extraction of effective features. Compared to approaches that first employ spatiotemporal attention followed by coordinate attention (local first, then global), this approach effectively avoids the loss of some information.

[0089] This invention employs a feature extraction-coordinate attention mechanism-spatiotemporal attention mechanism approach, acquiring different feature maps at each stage to obtain fish swarm feature maps, coordinate attention feature maps, and spatiotemporal attention feature maps. This approach combines the advantages of different attention mechanisms, ensuring that more information is retained and making the extracted features more accurate and effective.

[0090] The fish detection method provided by this invention realizes fish image detection by constructing a fish detection model by combining an attention mechanism, thereby achieving rapid screening of meaningful features, effectively improving the training speed and detection speed of the fish detection model, saving computing resources, and improving the accuracy of fish detection for overlapping and partially occluded targets while reducing the number of convolution kernels.

[0091] Optionally, according to the fish swarm detection method provided by the present invention, based on the feature fusion layer, the fish swarm feature map and the attention feature map are fused to determine the target fused feature map, specifically including:

[0092] Based on the feature fusion layer, and according to the feature pyramid network, the fish swarm feature map, coordinate attention feature map, and spatiotemporal attention feature map are fused to determine the target fusion feature map at three different scales.

[0093] Specifically, the input to the feature fusion layer is a fused fish swarm feature map, a coordinate attention feature map, and a spatiotemporal attention feature map. After feature map input feature fusion, the fish swarm feature map, coordinate attention feature map, and spatiotemporal attention feature map are fused according to the feature pyramid network. The three feature maps are upsampled and fused. Combining feature information from three different scales, three target fused feature maps of different scales are determined.

[0094] The fish detection method provided by this invention achieves fish image detection by constructing a fish detection model using an attention mechanism. This enables rapid screening of meaningful features, effectively improving the training and detection speed of the fish detection model, saving computational resources, and increasing the accuracy of fish detection for overlapping and partially occluded targets. During feature fusion, a feature pyramid feature fusion method is used to determine target fusion feature maps at different scales, ensuring that targets of different sizes can have appropriate feature representations at their respective scales. This allows for the detection of targets of different sizes at different scales, effectively improving model performance.

[0095] Optionally, according to the fish school detection method provided by the present invention, based on the feature recognition layer, the target fish school detection result is determined according to the target fusion feature map, specifically including:

[0096] Based on the feature recognition layer, the species and quantity of target fish are determined in the image of the fish school to be tested according to the target fusion feature map;

[0097] Based on the nonmaximum suppression algorithm, duplicate detection values ​​are eliminated to determine the target fish population detection results.

[0098] Specifically, after determining the target fusion feature map, it is input into the feature recognition layer to perform target recognition, determining the species and quantity of target fish in the fish school image. Understandably, fish in the fish school image may overlap or partially occlude. To further avoid missed detections and false detections...

[0099] The non-maximum suppression algorithm is used to remove duplicate detection values ​​and determine the target fish population detection results.

[0100] It is understandable that the specific form of the target fish group detection result can be to label different types of fish with different colored boxes in the original image of the fish group to be tested, and display the total number of different types of fish. Alternatively, it can be to count only the number of a certain type of fish in the fish group. The specific form can be determined according to the needs of the actual application of the present invention, and the present invention does not limit it.

[0101] The fish detection method provided by this invention realizes fish image detection by constructing a fish detection model by combining an attention mechanism, achieving rapid screening of meaningful features, effectively improving the training speed and detection speed of the fish detection model, saving computing resources, and improving the accuracy of fish detection for overlapping and partially occluded targets while reducing the number of convolution kernels. Furthermore, it improves the detection accuracy by removing redundant values ​​in the detection results through a non-maximum suppression algorithm.

[0102] Optionally, according to the fish detection method provided by the present invention, before the step of inputting the image of the fish to be detected into the fish detection model, the method further includes: determining the network structure of the fish detection model;

[0103] Determine the network structure of the fish swarm detection model, specifically including:

[0104] Based on the YOLOv5s algorithm network structure, coordinate attention modules and convolutional block attention modules are sequentially embedded in the backbone feature extraction network.

[0105] Specifically, before inputting the image of the fish swarm to be tested into the fish swarm detection model, it is necessary to determine the network structure of the fish swarm detection model. The fish swarm detection model provided by this invention is actually a fish swarm detection model based on triple attention mechanism and YOLOv5s (TAM-YOLOv5s).

[0106] Figure 3 A schematic diagram of the improved YOLOv5s algorithm network structure provided by this invention is shown below. Figure 3 As shown, this model improves upon the YOLOv5s algorithm network structure by sequentially embedding coordinate attention modules and convolutional block attention modules into the backbone feature extraction network.

[0107] The invention will be described in conjunction with the specific structure of the fish swarm detection model in practical applications.

[0108] The fish swarm detection model (TAM-YOLOv5s) network structure consists of three parts: Backbone, Neck, and Head.

[0109] The backbone feature extraction network structure consists of a Focus layer, four CSP layers (five convolutional layers and two residual structures), one SPP layer (three different pooling kernels: 5, 9, and 13), five CBS layers (consisting of one convolutional layer, one batch normalization (BN) layer, and SiLU activation function), and after the Csp_2 and Csp_4 layers, a coordinate attention mechanism (CA) and a spatiotemporal attention mechanism (convolutional block attention module, CBAM) are added respectively.

[0110] The backbone feature extraction network extracts features and outputs three effective feature maps (fish swarm feature map, coordinate attention feature map, and spatiotemporal attention feature map).

[0111] For example, a CA module is added after the Csp_2 layer of the backbone structure. To preserve spatial structure information, the input fish swarm feature map F (80x80x128) is subjected to global average pooling based on width or height. One-to-one one-dimensional feature encoding is then performed to obtain two feature maps (80x1x128) and (1x80x128). The features from the two directions are then concatenated to obtain a feature map that contains both channel and spatial information, and a series of transformations are performed to obtain f. (c。r)x1x(H+W) Where r is 0.3, This is the Sigmoid activation function.

[0112]

[0113]

[0114]

[0115] f (c。r)x1x(H+W ) Perform a segmentation based on width or height to obtain f (c。r)x1xH f (c。r)x1xW The features in both directions are subjected to dimensionality increase operations, and the output F is of the same size as the original input F (80x80x128). h F w After passing through the activation function, the attention weights g of the feature map in height and width are obtained. h g w Finally, the two-dimensional features are multiplied by the original feature map to obtain the coordinate attention feature map y. c .

[0116]

[0117]

[0118]

[0119] After the Csp_4 layer in the backbone structure, a CBAM module is added. Different pooling operations, Global Max Pooling and Global Avg Pooling, are applied to the 512 channels of the input Csp_4_F (20x20x512) to obtain two richer 1x1x512 high-level features. These features are then input into an MLP (the first layer has 1 / 16 neurons, and the second layer has 512 neurons) to obtain two weights. and By superimposing two weights, we obtain dual channel and spatial weights, and then pass them through an activation function to obtain the channel attention feature map Csp_4_F1 (i.e., Mc(F)).

[0120]

[0121] First, the result of the bitwise multiplication of Csp_4_F1 and Csp_4_F (20x20x512) is input into Global Max Pooling and Global Avg Pooling based on 512 channels to obtain two features (20x20x1). Second, the two features are concatenated (20x20x2). Finally, a series of operations, including convolution, activation function, and bitwise multiplication with F, are performed to obtain the spatiotemporal attention feature map Csp_4_F2 (i.e., Ms(F)).

[0122]

[0123] In summary, three effective feature maps are output: Feature1 (80x80x128) (fish swarm feature map), Feature2 (40x40x256) (coordinate attention feature map), and Feature3 (coordinate attention feature map). c Feature3 (20x20x512) (Spatiotemporal attention feature map Csp_4_F2).

[0124] It is understood that the above feature map transformation is only used as a specific example to illustrate the present invention. In the actual application of the present invention, the size of the input image and the size of the output feature map can be adjusted according to actual needs, and the present invention does not limit this.

[0125] The neck structure performs feature fusion to enhance feature extraction, forming a feature pyramid PANet (Path Aggregation Network). It fuses feature maps output from coordinate attention and spatiotemporal attention mechanisms, with all convolutional layers using CBS (Convolutional Baseline Layout). Upsampling and downsampling are performed on the Feature1, Feature2, and Feature3 feature layers in the backbone, combining feature information from three different scales.

[0126] The Head consists of three 1*1 convolutional layers; except for the attention mechanism module which uses h-swish as the activation function, the other layers use SiLU as the activation function. The convolutional layers judge the feature points to determine whether there is an object corresponding to the feature points.

[0127] The fish detection method provided by this invention incorporates spatiotemporal attention and coordinate attention mechanisms into the YOLOv5s backbone network and enhanced feature extraction network, which not only improves the training speed of the model, but also improves the accuracy of overlapping and partially occluded targets while reducing the number of convolution kernels.

[0128] Optionally, according to the fish detection method provided by the present invention, after the step of determining the network structure of the fish detection model, the method further includes: training the fish detection model.

[0129] Training the fish swarm detection model specifically includes:

[0130] Acquire several sample fish images, create labels, and determine the sample fish image set;

[0131] A fish detection model was trained based on a sample set of fish images.

[0132] Based on the objective loss function, the network parameters of the fish swarm detection model are updated using cosine annealing. The fish swarm detection model is then iteratively trained based on the updated network parameters until it converges.

[0133] Specifically, after determining the network structure of the fish swarm detection model, the fish swarm detection model needs to be trained.

[0134] Taking the fish school detection model for identifying zebrafish fry as an example, firstly, several sample fish school images are acquired, labels are created, and the sample fish school image set is determined.

[0135] Zebra fry, ranging in length from 0.5 to 1.5 cm, were raised in glass aquariums. Images of the fish in a real-world aquaculture setting were captured every 5 seconds by a mobile phone camera pointed directly at the glass tank. Valid data were selected and labeled using annotation software. Basic image processing methods such as rotation, flipping, and cropping resulted in 692 experimental images, representing a total of 15,081 fry. The training, validation, and test sets were randomly divided in a 5:1:1 ratio.

[0136] It is understood that the method for preprocessing sample images in this invention can be selected according to actual needs, and this invention does not limit it.

[0137] After determining the sample fish swarm image set, the basic parameters of the model were configured, and the fish swarm detection model was trained using the sample fish swarm image set. During training, the input images were normalized to 640x640x3, the positive sample matching process was integrated into the data encapsulation process, the pre-trained weights of the COCO dataset were transferred, and the weights of the Exponential Moving Average (EMA) model were increased. The backbone network was first frozen and trained for 50 epochs with a batch size of 16, and then the backbone network was unfrozen and trained for 8 epochs with a batch size of 8. Mosaic data augmentation was used during training, but it was turned off at the 70th epoch after unfreezing the training. The activation factor r of CBAM was 16, and the activation factor r of CA was 0.3.

[0138] The maximum learning rate of the model is 1e-2, and the minimum learning rate is 0.01 times the maximum learning rate. The learning rate decay strategy is cosine annealing (cos). Based on the objective loss function, the network parameters of the fish swarm detection model are updated, and the fish swarm detection model is iteratively trained based on the updated network parameters until the fish swarm detection model converges. The optimal result is selected as the fish swarm detection model.

[0139] This invention employs a freeze-training and unfreeze-training method, which can accelerate training efficiency and prevent weights from being corrupted, thereby improving the training efficiency and effectiveness of the network.

[0140] It is understood that the training method for the fish detection model based on the target loss function (e.g., gradient descent and backpropagation) and the conditions for stopping iterative training (e.g., the number of training iterations reaches a preset threshold and the target function meets preset conditions) can be selected according to the actual situation. This invention does not limit these methods.

[0141] After training, the model parameters for the fish swarm detection model that identifies zebrafish fry are as follows:

[0142]

[0143]

[0144] It should be noted that the above model training method is only used as a specific example to illustrate the present invention. In the actual application of the present invention, the types and quantities of fish used as model samples, as well as the model parameters, can be adjusted according to actual needs. The present invention does not limit these adjustments.

[0145] The fish detection method provided by this invention realizes fish image detection by constructing a fish detection model by combining an attention mechanism, thereby achieving rapid screening of meaningful features, effectively improving the training speed and detection speed of the fish detection model, saving computing resources, and improving the accuracy of fish detection for overlapping and partially occluded targets while reducing the number of convolution kernels.

[0146] Figure 4 This is a schematic diagram of the fish detection system provided by the present invention, as shown below. Figure 4 As shown, the present invention also provides a fish school detection system, including: an image input unit 401, a feature extraction unit 402, a feature fusion unit 403, and a feature recognition unit 404;

[0147] The image input unit 401 is used to input the image of the fish swarm to be tested into the fish swarm detection model; wherein, the fish swarm detection model includes: a feature extraction layer, a feature fusion layer and a feature recognition layer;

[0148] The feature extraction unit 402 is used to extract feature information of the fish group image to be tested based on the feature extraction layer, and to determine the fish group feature map and the attention feature map according to the attention mechanism.

[0149] The feature fusion unit 403 is used to fuse the fish swarm feature map and the attention feature map based on the feature fusion layer to determine the target fused feature map;

[0150] The feature recognition unit 404 is used to determine the target fish group detection result based on the target fusion feature map according to the feature recognition layer.

[0151] Specifically, before conducting fish school detection, it is necessary to acquire images of the target fish school. It is understood that this invention can be used to detect images of fish schools in natural water bodies (e.g., lakes, seas, rivers, etc.) or in aquaculture environments (glass tanks or fishponds, etc.), without limiting the source of the images of the target fish school, and without limiting the species and quantity of the target fish school.

[0152] In addition, the fish school image can be obtained by taking pictures with an underwater camera or by acquiring the video in real time, and the video frame is used as the fish school image to be tested. This invention does not limit this method.

[0153] After acquiring the target fish group image, the image input unit 401 is used to input the target fish group image into the trained fish group detection model.

[0154] Figure 2This is a schematic diagram of the fish school detection model structure provided by the present invention, as shown below. Figure 2 As shown, the fish swarm detection model includes a feature extraction layer, a feature fusion layer, and a feature recognition layer. It is understood that the specific structure of the fish swarm detection model, the recognition algorithm used, and the training method can all be adjusted according to actual needs, and this invention does not limit these aspects.

[0155] The feature extraction unit 402 is used as the feature extraction layer based on the fish swarm detection model to extract the feature information of the fish swarm image to be tested, obtain the fish swarm feature map, and on this basis, combine the attention mechanism to obtain the attention feature map of the fish swarm.

[0156] It should be noted that in the actual application of this invention, the type of attention mechanism can be selected according to actual needs, such as channel attention mechanism, spatial attention mechanism, coordinate attention mechanism, self-attention mechanism and hybrid attention mechanism, etc., and this invention does not limit it.

[0157] After determining the fish swarm feature map and the attention feature map, they are input into the feature fusion layer of the model. The feature fusion unit 403 is used to fuse the fish swarm feature map and the attention feature map based on the feature fusion layer to determine the target fused feature map.

[0158] It should be noted that in the actual application of this invention, the specific method of feature fusion can be selected according to actual needs, such as Path Aggregation Network (PANet) and weighted bi-directional feature pyramid network (BiFPN), etc., and this invention does not limit it.

[0159] The fused target feature map is input into the feature recognition layer. The feature recognition unit 404 is used to determine the target fish group detection result based on the target fusion feature map.

[0160] It is understood that, in the practical application of this invention, the detection result of the target fish group can be to determine the number of a certain type of fish in the image of the fish group to be tested (such as determining the specific number of saury in a mixed fish group), or to determine the species of fish in the image of the fish group to be tested and the number of each species (such as detecting that it contains three species in an ornamental fish group: white-breasted wrasse, light-cheeked seabream, and neon characin, and determining the number of each species), or to determine whether the fish group contains the target species (such as determining whether there is an invasive alien species, alligator gar, in a fish group image of a lake). The specific detection result can be adjusted according to actual needs, and this invention does not limit it in this regard.

[0161] The fish detection system provided by this invention achieves fish detection in images by constructing a fish detection model using an attention mechanism, enabling rapid screening of meaningful features and focusing on regions of interest. Compared to existing target detection schemes, this invention effectively improves the training and detection speed of the fish detection model by adding an attention mechanism, saving computational resources, and improving the accuracy of fish detection for overlapping and partially occluded targets while reducing the number of convolution kernels.

[0162] It should be noted that the fish detection system provided by the present invention is used to execute the above-mentioned fish detection method, and its specific implementation method is the same as that of the method implementation method, and will not be repeated here.

[0163] Figure 5 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503, and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504. The processor 501 can call logical instructions in the memory 503 to execute a fish school detection method, which includes: inputting a fish school image to be tested into a fish school detection model; wherein the fish school detection model includes: a feature extraction layer, a feature fusion layer, and a feature recognition layer; based on the feature extraction layer, extracting feature information from the fish school image to be tested, and determining a fish school feature map and an attention feature map according to an attention mechanism; based on the feature fusion layer, fusing the fish school feature map and the attention feature map to determine a target fused feature map; and based on the feature recognition layer, determining the target fish school detection result according to the target fused feature map.

[0164] Furthermore, the logical instructions in the aforementioned memory 503 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the fish detection method provided by the above methods. The method includes: inputting a fish school image to be tested into a fish detection model; wherein the fish detection model includes: a feature extraction layer, a feature fusion layer, and a feature recognition layer; based on the feature extraction layer, extracting feature information of the fish school image to be tested, and determining a fish school feature map and an attention feature map according to an attention mechanism; based on the feature fusion layer, fusing the fish school feature map and the attention feature map to determine a target fused feature map; and based on the feature recognition layer, determining a target fish school detection result according to the target fused feature map.

[0166] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the fish school detection methods provided above. The method includes: inputting a fish school image to be detected into a fish school detection model; wherein the fish school detection model includes: a feature extraction layer, a feature fusion layer, and a feature recognition layer; based on the feature extraction layer, extracting feature information from the fish school image to be detected, and determining a fish school feature map and an attention feature map according to an attention mechanism; based on the feature fusion layer, fusing the fish school feature map and the attention feature map to determine a target fused feature map; and based on the feature recognition layer, determining a target fish school detection result according to the target fused feature map.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting fish schools, characterized in that, include: The image of the fish school to be tested is input into the fish school detection model; wherein, the fish school detection model includes: a feature extraction layer, a feature fusion layer and a feature recognition layer; Based on the feature extraction layer, the feature information of the fish group image to be tested is extracted. According to the attention mechanism, the fish group feature map and the attention feature map are determined. The attention feature map includes a coordinate attention feature map and a spatiotemporal attention feature map. Based on the feature fusion layer, the fish swarm feature map, coordinate attention feature map, and spatiotemporal attention feature map are fused to determine the target fusion feature map; Based on the feature recognition layer, the target fish group detection result is determined according to the target fusion feature map; Before the step of inputting the image of the fish school to be tested into the fish school detection model, the method further includes: determining the network structure of the fish school detection model; Determining the network structure of the fish swarm detection model specifically includes: Based on the YOLOv5s algorithm network structure, coordinate attention modules and convolutional block attention modules are sequentially embedded in the backbone feature extraction network; The fish swarm detection model is a detection model based on a combination of coordinate attention, spatiotemporal attention mechanisms, and YOLOv5s. The network structure of the fish swarm detection model includes three parts: Backbone, Neck, and Head. The Backbone feature extraction network structure includes one Focus layer, four CSP layers, one SPP layer, and five CBS layers. Coordinate attention is added after the second CSP layer. A spatiotemporal attention mechanism is added after the fourth CSP layer. The spatiotemporal attention mechanism adopts a convolutional block attention module. The Backbone feature extraction network extracts features and outputs three effective feature maps: a fish swarm feature map, a coordinate attention feature map, and a spatiotemporal attention feature map. The Neck structure is a PANet feature pyramid network used for feature fusion and enhanced feature extraction. It performs upsampling and downsampling of the fish swarm feature map, coordinate attention feature map, and spatiotemporal attention feature map output by the Backbone network, combining feature information from three different scales. The Head consists of three 1*1 convolutional layers. Except for the attention mechanism module, which uses h-swish as the activation function, the other layers use SiLU as the activation function. The convolutional layers judge whether there is an object corresponding to the feature point.

2. The fish school detection method according to claim 1, characterized in that, The feature extraction layer includes: an initial feature extraction layer and an attention feature extraction layer; Correspondingly, the step of extracting feature information of the fish swarm image based on the feature extraction layer, and determining the fish swarm feature map and attention feature map according to the attention mechanism, specifically includes: Based on the initial feature extraction layer, the feature information of the fish group image to be tested is extracted, and the fish group feature map is determined; Based on the attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism, channel attention mechanism and spatial attention mechanism to determine the attention feature map.

3. The fish school detection method according to claim 2, characterized in that, The attention feature extraction layer includes a coordinate attention feature extraction layer and a convolutional block attention feature extraction layer; the attention feature map includes a coordinate attention feature map and a spatiotemporal attention feature map. Correspondingly, the step of transforming the fish swarm feature map based on the attention feature extraction layer according to coordinate attention mechanism, channel attention mechanism and spatial attention mechanism to determine the attention feature map specifically includes: Based on the coordinate attention feature extraction layer, the fish swarm feature map is transformed according to the coordinate attention mechanism to determine the coordinate attention feature map; Based on the convolutional block attention feature extraction layer, the fish swarm feature map is transformed according to the channel attention mechanism to determine the channel attention feature map, and the channel attention feature map is transformed according to the spatial attention mechanism to determine the spatiotemporal attention feature map.

4. The fish school detection method according to any one of claims 1-3, characterized in that, The determination of the target fish group detection result based on the feature recognition layer and the target fusion feature map specifically includes: Based on the feature recognition layer, the species and quantity of the target fish are determined in the image of the fish swarm to be tested according to the target fusion feature map; Based on the nonmaximum suppression algorithm, duplicate detection values ​​are eliminated to determine the target fish population detection results.

5. The fish school detection method according to claim 1, characterized in that, After the step of determining the network structure of the fish detection model, the method further includes: training the fish detection model; Training the fish detection model specifically includes: Acquire several sample fish images, create labels, and determine the sample fish image set; The fish detection model is trained based on the sample fish image set; Based on the objective loss function, the network parameters of the fish detection model are updated using cosine annealing. The fish detection model is then iteratively trained based on the updated network parameters until it converges.

6. A fish school detection system, characterized in that, include: Image input unit, feature extraction unit, feature fusion unit, and feature recognition unit; The image input unit is used to input the image of the fish school to be tested into the fish school detection model; wherein, the fish school detection model includes: a feature extraction layer, a feature fusion layer and a feature recognition layer; The feature extraction unit is used to extract feature information of the fish group image to be tested based on the feature extraction layer, and to determine the fish group feature map and the attention feature map according to the attention mechanism. The attention feature map includes a coordinate attention feature map and a spatiotemporal attention feature map. The feature fusion unit is used to fuse the fish swarm feature map, coordinate attention feature map and spatiotemporal attention feature map based on the feature fusion layer to determine the target fused feature map; The feature recognition unit is used to determine the target fish group detection result based on the feature recognition layer and the target fusion feature map; Before the step of inputting the image of the fish school to be tested into the fish school detection model, the method further includes: determining the network structure of the fish school detection model; Determining the network structure of the fish swarm detection model specifically includes: Based on the YOLOv5s algorithm network structure, coordinate attention modules and convolutional block attention modules are sequentially embedded in the backbone feature extraction network; The fish swarm detection model is a detection model based on a combination of coordinate attention, spatiotemporal attention mechanisms, and YOLOv5s. The network structure of the fish swarm detection model includes three parts: Backbone, Neck, and Head. The Backbone feature extraction network structure includes one Focus layer, four CSP layers, one SPP layer, and five CBS layers. Coordinate attention is added after the second CSP layer. A spatiotemporal attention mechanism is added after the fourth CSP layer. The spatiotemporal attention mechanism adopts a convolutional block attention module. The Backbone feature extraction network extracts features and outputs three effective feature maps: a fish swarm feature map, a coordinate attention feature map, and a spatiotemporal attention feature map. The Neck structure is a PANet feature pyramid network used for feature fusion and enhanced feature extraction. It performs upsampling and downsampling of the fish swarm feature map, coordinate attention feature map, and spatiotemporal attention feature map output by the Backbone network, combining feature information from three different scales. The Head consists of three 1*1 convolutional layers. Except for the attention mechanism module, which uses h-swish as the activation function, the other layers use SiLU as the activation function. The convolutional layers judge whether there is an object corresponding to the feature point.

7. An electronic device, characterized in that, The device includes a memory and a processor, which communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, and the processor can execute the fish detection method as described in any one of claims 1 to 5 by calling the program instructions.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the fish detection method as described in any one of claims 1 to 5.

Images