Fry automatic counting method based on feature perception aggregation network
By using a feature-aware aggregation network, combined with a backbone network, attention module, and feature enhancement module, the problems of occlusion and overlap in high-density fish counting are solved, improving the accuracy and adaptability of fish fry counting.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HENAN UNIVERSITY
- Filing Date
- 2026-01-21
- Publication Date
- 2026-05-01
AI Technical Summary
Existing technologies suffer from overlapping and occlusion issues in high-density fish counting, leading to decreased counting accuracy, especially in fish fry scenarios where the counting effect is poor.
A feature-aware aggregation network is adopted, which combines a backbone network, an attention module, a feature enhancement module, and a decoding stage. Data augmentation and a dual cross-attention mechanism are used to improve the accuracy of fish fry counting.
It effectively solved the problems of shading and overlapping in high-density fish schools, and improved the accuracy and generalization ability of fish fry counting.
Smart Images
Figure CN121963254A_ABST
Abstract
Description
An Automatic Fish Fry Counting Method Based on Feature-Aware Aggregation Network Technical Field
[0001] This invention relates to the field of target counting technology, and specifically to an automatic fish fry counting method based on a feature-aware aggregation network. Background Technology
[0002] In recent years, aquaculture has played an increasingly important role in ensuring global food security and buffering the impact of environmental changes on fishery resources. In 2022, the global total output of fisheries and aquaculture reached 223 million tons, with aquaculture production exceeding that of wild-caught fisheries for the first time. Fish alone accounted for 63% of the total output, highlighting its growing value in alleviating hunger and malnutrition. As a crucial link in the aquaculture process, accurate estimation of fish density not only helps farmers rationally allocate feed, reduce resource waste and production costs, but also assists in judging the timing of harvesting to improve economic efficiency. Furthermore, it provides support for water quality control and disease early warning, directly impacting aquaculture efficiency and ecological sustainability.
[0003] However, traditional manual counting methods are no longer sufficient to meet the demands of modern aquaculture. Manual operation is not only time-consuming and labor-intensive, but also extremely inefficient. Furthermore, direct contact between personnel and fish, or environmental disturbances during the counting process, can negatively impact fish growth and even increase the risk of disease transmission, thus hindering the industry's large-scale and intelligent development. Therefore, developing non-invasive, precise counting technology has become a core issue that urgently needs to be addressed in this field.
[0004] Non-invasive automated counting methods mainly include acoustic-based and computer vision-based methods. Acoustic-based methods rely on sonar equipment and have certain advantages in low-light or murky environments, but they still have significant limitations: their signals are easily interfered with by factors such as water flow, water temperature, and impurities, leading to decreased accuracy. They are also more suitable for larger, more dispersed fish schools; in densely packed schools or small fry scenarios, their recognition performance declines. Furthermore, sonar equipment is expensive and requires specialized personnel to operate, making it difficult to promote in small and medium-sized fish farms. In contrast, computer vision-based methods, thanks to the development of deep learning and image processing technologies, have gradually become a research hotspot. These methods are superior to acoustic technologies in terms of cost and adaptability, but they still face technical bottlenecks.
[0005] Specifically, detection-based methods identify individual fish through object detection or instance segmentation, performing well in low-density, minimally occluded scenarios. However, they are prone to missed detections and duplicate counts in high-density fish populations, leading to a significant drop in accuracy. Density map-based methods, on the other hand, have advantages in high-density scenarios through density estimation and can balance accuracy and computational cost to some extent. However, current methods still fall short when dealing with severe occlusion, particularly in feature extraction from overlapping regions, resulting in discrepancies between the counting results and the actual situation, thus leading to poor counting performance. Summary of the Invention
[0006] To address the technical problem of poor counting results caused by overlapping and occlusion during fish fry counting in high-density situations, this invention proposes an automatic fish fry counting method based on a feature-aware aggregation network.
[0007] This invention provides an automatic fish fry counting method based on a feature-aware aggregation network. The method includes: collecting a fish fry dataset, partitioning the dataset and performing data augmentation to obtain an augmented training set; using a backbone network for basic feature extraction; designing an attention module; designing a feature augmentation module; using dilated convolution in the decoding stage; training a model consisting of a backbone network, attention module, feature augmentation module, and decoding stage based on the augmented training set; and inputting a test image into the trained model to obtain the number of fish.
[0008] Optionally, the process of collecting the fish fry dataset, dividing the dataset, and performing data augmentation to obtain an augmented training set includes: after collecting the fish fry dataset, dividing it into a training set, a validation set, and a test set, and performing data augmentation on the training set and the validation set. The data augmentation operations include: horizontal flipping, vertical flipping, Gaussian noise, salt and pepper noise, and reducing image contrast.
[0009] Optionally, the basic feature extraction using a backbone network includes: using the first 16 convolutional layers and the first four pooling layers of the VGG19 model as the backbone network to extract basic features, obtaining a basic feature map, where each convolutional layer consists of a 3×3 convolutional kernel and a ReLU activation function; and upsampling the basic feature map using bilinear interpolation, the formula for which is: in, The point to be interpolated. Represents the eigenvalues of the point to be interpolated. Points to be interpolated The integer index of the known point in the top left corner of the grid. , , , The positions of the four known feature points adjacent to the point to be interpolated. For the interpolation point at The direction is relative to the normalized distance of the known feature points. For the interpolation point at The direction is the normalized distance relative to the known feature points.
[0010] Optionally, the attention module design includes: the attention module consists of 3 attention layers and 2 convolutional layers; each attention layer is composed of a spatial attention mechanism and a channel attention mechanism connected in series, and the formula corresponding to the attention layer is expressed as: in, The Sigmoid activation function is used to map the output value to the interval [0, 1], thereby normalizing the attention weights. This represents a multilayer perceptron; It is average pooling. It is max pooling; For feature splicing operations; This indicates element-wise multiplication; It is the input feature map. It is the channel attention weight. It is the feature map after channel attention weighting. It is spatial attention weight. It is the final feature map after channel and spatial attention weighting.
[0011] Optionally, the design feature enhancement module includes: a parallel local structure awareness branch and a global attention branch; the local structure awareness branch focuses on enhancing fine-grained edge and texture information, and the input feature map... This is achieved through two parallel depthwise separable convolutions, using 3×3 and 5×5 kernels respectively, with the corresponding formulas as follows: The obtained feature map and Attention weights are generated by concatenating along the channel dimension and performing a series of convolutions and activation functions. : Using attention weights Element-wise multiplication of the original input feature map yields the local enhancement features. The corresponding formula is: The global attention branch aims to enhance the perception of the overall distribution of fish fry and capture long-range dependencies, focusing on the input feature map. The number of channels is adjusted by three 1×1 convolutions. ,get , and ,in, , and This represents the query, key, and value in that order; for and The attention map is reshaped and multiplied to generate attention weights, which are then applied to an activation function to generate attention weights. Finally, these attention weights are multiplied by... Multiplication, followed by a 1×1 convolutional layer, batch normalization, and residual connections to output global features. The formula corresponding to this process is: A dual cross-attention mechanism is introduced to mutually optimize the local structure perception branch and the global attention branch; a cross-attention mechanism outputs features from the global branch. As a query, the features of the demodulation local branches Injecting high-resolution structural priors into In contrast, another type of cross-attention focuses on the characteristics of local branches. As a query, to optimize the features of the global branch. This embeds broader contextual guidance information into the local representation.
[0012] Optionally, the step of training a model consisting of a backbone network, an attention module, a feature enhancement module, and a decoding stage based on the data-enhanced training set, and inputting the test image into the trained model to obtain the number of fish, includes: optimizing the network training using the Adam optimizer. The mathematical model of the Adam algorithm is expressed as: in, It is an exponential moving average of the gradient. It is the exponential moving average of the squared gradient. and Both are used to control the decay rate of the moving average. It is a time step The gradient at time is also the gradient of the parameters. It is after deviation correction , It is after deviation correction , It's about time. The parameters, It's the learning rate. It is a small constant used to ensure numerical stability; the Adam optimizer is used to train the model until it converges, the test image is input into the trained model to obtain the density map, and the density map is summed to obtain the number of fish.
[0013] This invention offers the following advantages: The technical solution provided by this invention comprehensively considers the severe occlusion and overlap problems faced by fish fry counting under high-density conditions. The designed network has strong generalization ability and can effectively estimate the number of fish fry. This invention proposes an improved fish fry estimation network based on feature-aware aggregation. This invention uses a combination of a basic feature extraction network and a dual cross-attention enhancement module to solve the counting inaccuracy problem caused by occlusion and deformation in fish fry counting. This fusion can enhance the perception of fine-grained structure and global spatial distribution, effectively improving the accuracy of fish fry counting under different densities and occlusion scenarios. Attached Figure Description
[0014] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 is a flowchart of an automatic fish fry counting method based on a feature-aware aggregation network according to the present invention; Figure 2 is a diagram showing the effect of data augmentation according to the present invention; Figure 3 is a detailed diagram of the architecture of the backbone network according to the present invention; Figure 4 is a detailed diagram of the architecture of the channel attention module in the attention module of the feature-aware aggregation network according to the present invention; Figure 5 is a detailed diagram of the architecture of the spatial attention module in the attention module of the feature-aware aggregation network according to the present invention; Figure 6 is a detailed diagram of the architecture of the dual cross-attention feature enhancement module in the feature-aware aggregation network according to the present invention. Detailed Implementation
[0016] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solution proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.
[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0018] This invention provides a fish fry counting network based on feature-aware aggregation. The fish fry counting network consists of a basic feature extractor, an attention module, a feature enhancement module, a dilated decoding network, and a density map generation head. The basic feature extractor, in deep learning, especially in computer vision tasks, is often referred to as the backbone network.
[0019] Referring to Figure 1, the flowchart of some embodiments of the automatic fish fry counting method based on a feature-aware aggregation network according to the present invention is shown. This automatic fish fry counting method based on a feature-aware aggregation network includes the following steps: Step S1, collecting a fish fry dataset, performing dataset partitioning and data augmentation to obtain a data-augmented training set.
[0020] In some embodiments, a fish fry dataset can be collected and divided, and then data augmentation can be performed using methods such as image flipping and adding noise.
[0021] As an example, after collecting the fish fry dataset, it was divided into training, validation, and test sets. Then, data augmentation was performed on the training and validation sets. Data augmentation operations included: horizontal flipping, vertical flipping, Gaussian noise, salt-and-pepper noise, and reducing image contrast. The effects of data augmentation are shown in Figure 2.
[0022] Step S2: Use the backbone network to extract basic features.
[0023] In some embodiments, a backbone network can be used for basic feature extraction.
[0024] The basic feature extractor consists of the first 16 convolutional layers and the first four pooling layers of the VGG19 model.
[0025] It should be noted that the backbone network, as shown in Figure 3, consists of the first 16 convolutional layers and the first four pooling layers of the VGG19 model to extract basic features. Each convolutional layer consists of a 3×3 convolutional kernel and a ReLU activation function. Then, bilinear interpolation is used to upsample the basic feature map.
[0026] As an example, the first 16 convolutional layers and the first four pooling layers of the VGG19 model are used as the backbone network to extract basic features, resulting in a basic feature map. Each convolutional layer consists of a 3×3 convolutional kernel and a ReLU activation function. Then, bilinear interpolation is used to upsample the basic feature map. Bilinear interpolation can be expressed by the following formula: in The point to be interpolated. Represents the eigenvalues of the point to be interpolated. Points to be interpolated The integer index of the known point in the top left corner of the grid. , , , The positions of the four known feature points adjacent to the point to be interpolated. For the interpolation point at The direction is relative to the normalized distance of the known feature points. For the interpolation point at The direction is the normalized distance relative to the known feature points.
[0027] Step S3: Design the attention module.
[0028] In some embodiments, an attention module can be designed to learn salient features and suppress noise interference.
[0029] As an example, the designed attention module consists of three attention layers and two convolutional layers. As shown in the upper region of Figure 3, each attention layer is composed of a spatial attention mechanism and a channel attention mechanism connected in series. The purpose is to enhance key features related to fish counting, locate the spatial region where the fish are located, suppress redundant information and background interference, and help improve the accuracy of fish counting. The channel attention mechanism is shown in Figure 4, and the spatial attention mechanism is shown in Figure 5. The attention layer is represented by the following formula: in, The Sigmoid activation function is used to map the output value to the interval [0, 1], thereby normalizing the attention weights. This represents a multilayer perceptron; It is average pooling. Both max pooling and max pooling extract statistical information about features from different perspectives. For feature splicing operations; This indicates element-wise multiplication, achieving the fusion of attention weights and feature maps; It is the input feature map. It is the channel attention weight. It is the feature map after channel attention weighting. It is spatial attention weight. It is the final feature map after channel and spatial attention weighting.
[0030] Step S4: Design the feature enhancement module.
[0031] In some embodiments, feature enhancement modules can be designed to improve counting robustness under occlusion conditions.
[0032] It should be noted that the feature enhancement module is shown in Figure 6. This module, also known as the dual cross-attention enhancement module, comprises parallel local structure perception branches and global attention branches. The local structure perception branch focuses on enhancing fine-grained edge and texture information, which is crucial for identifying individual fish fry in crowded or partially occluded areas. It guides the model to selectively highlight the edges and fine structural features of the fish fry, improving the network's recognition ability under conditions of fry occlusion and deformation. The global attention branch focuses on enhancing the perception of the overall distribution of fish fry and capturing long-distance dependencies. Furthermore, a dual cross-attention mechanism is proposed. This dual cross-attention interaction allows each branch to retain its own advantages and compensate for its shortcomings, ultimately resulting in a more balanced and information-rich fused feature, which helps improve the accuracy of fish fry counting in occlusion scenarios.
[0033] As an example, the local structure-aware branch focuses on enhancing fine-grained edge and texture information, which is crucial for identifying individual fish fry in crowded or partially occluded areas. Input feature map Next, two parallel depthwise separable convolutions are used, employing 3×3 and 5×5 kernels respectively, to capture complementary local features, such as the fine texture and edges of fish fry. The corresponding formula is: The resulting feature map and Attention weights are generated by concatenating along the channel dimension and performing a series of convolutions and activation functions. : Finally, attention weights are utilized. Element-wise multiplication of the original input feature map yields the local enhancement features. This mechanism guides the model to selectively highlight the edges and fine structural features of fish fry, improving the network's recognition ability under conditions of fry occlusion and deformation. The corresponding formula is: The global attention branch focuses on enhancing the perception of the overall distribution of fish fry and capturing long-range dependencies. (This refers to the input feature map.) The number of channels is adjusted by three 1×1 convolutions. , get query ( ),key( ) and value ( After that, regarding and The attention map is reshaped and multiplied to generate attention weights, which are then applied to an activation function to generate attention weights. Finally, these attention weights are multiplied by... The data are multiplied and then passed through a 1×1 convolutional layer, batch normalization, and residual connections to output global features. The formula corresponding to this process is: Furthermore, a dual cross-attention mechanism is proposed. Since local branch features contain fine-grained spatial information, while global branch features capture long-range contextual dependencies but coarser details, directly summing these heterogeneous features may lead to inconsistent representations or poor fusion results. To achieve more efficient and adaptive fusion, a dual cross-attention mechanism is introduced to mutually optimize the two branches. Specifically, one cross-attention mechanism uses the features output by the global branch... As a query, the features of the demodulation local branches This allows for the injection of high-resolution structural priors into... In contrast, another type of cross-attention focuses on the characteristics of local branches. As a query, to optimize the features of the global branch. This embeds broader contextual guidance information into the local representation.
[0034] This dual cross-attention interaction allows each branch to retain its own advantages and make up for its shortcomings, ultimately resulting in a more balanced and information-rich fusion feature, which helps improve the accuracy of fish fry counting in scenarios such as occlusion.
[0035] Step S5, the decoding stage uses dilated convolution.
[0036] In some embodiments, dilated convolution can be used in the decoding stage to maintain resolution while improving counting accuracy.
[0037] It should be noted that a series of dilated convolutions are used in the decoding stage. Dilated convolutions can increase the receptive field of the network by adjusting the dilation rate without increasing the parameters and computational load, which helps to capture a wide range of dependencies in the image. Furthermore, it can obtain a wider range of contextual information without reducing the resolution, reduce the loss of spatial information and retain more detailed features.
[0038] Step S6: Based on the augmented training set, train a model consisting of a backbone network, attention module, feature enhancement module, and decoding stage. Input the test image into the trained model to obtain the number of fish.
[0039] In some embodiments, a training set and designed modules can be used to train the model, and test images can be input into the model to obtain the number of fish.
[0040] It should be noted that a density map can be generated by changing the number of channels to 1 using a density map generator, and then the density maps can be summed to obtain the estimated number of fish fry.
[0041] As an example, using the Adam optimizer to optimize network training, the mathematical model of the Adam algorithm can be expressed as: in, It is an exponential moving average of the gradient. It is the exponential moving average of the squared gradient. and Both are used to control the decay rate of the moving average. It is a time step The gradient at time is also the gradient of the parameters. It is after deviation correction , It is after deviation correction , It's about time. The parameters, It's the learning rate. It is a small constant used to ensure numerical stability.
[0042] Train the model using the Adam optimizer until it converges, then input the image into the trained model to obtain the density map. Sum the density maps to get the number of fish.
[0043] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. An automatic fish fry counting method based on a feature-aware aggregation network, characterized in that, Includes the following steps: Collect fish fry datasets, perform dataset partitioning and data augmentation to obtain the augmented training set; Basic feature extraction is performed using a backbone network; Design an attention module; Design feature enhancement module; The decoding stage uses dilated convolutions; based on the augmented training set, a model consisting of a backbone network, attention module, feature enhancement module, and decoding stage is trained, and the test image is input into the trained model to obtain the number of fish.
2. The automatic fish fry counting method based on a feature-aware aggregation network according to claim 1, characterized in that, The process of collecting fish fry datasets, dividing the datasets, and performing data augmentation to obtain augmented training sets includes: after collecting the fish fry datasets, dividing them into training, validation, and test sets, and performing data augmentation on the training and validation sets. Data augmentation operations include: horizontal flipping, vertical flipping, Gaussian noise, salt and pepper noise, and reducing image contrast.
3. The automatic fish fry counting method based on a feature-aware aggregation network according to claim 1, characterized in that, The basic feature extraction using a backbone network includes: using the first 16 convolutional layers and the first four pooling layers of the VGG19 model as the backbone network to extract basic features, resulting in a basic feature map. Each convolutional layer consists of a 3×3 convolutional kernel and a ReLU activation function. Bilinear interpolation is then used to upsample the basic feature map. The formula for bilinear interpolation is as follows: in, Represents the eigenvalues of the point to be interpolated. For the interpolation point at The direction is relative to the normalized distance of the known feature points. For the interpolation point at The direction is the normalized distance relative to the known feature points.
4. The automatic fish fry counting method based on a feature-aware aggregation network according to claim 1, characterized in that, The attention module design comprises: 3 attention layers and 2 convolutional layers; each attention layer consists of a spatial attention mechanism and a channel attention mechanism connected in series, and the formula corresponding to each attention layer is expressed as: in, The Sigmoid activation function is used to map the output value to the interval [0, 1], thereby normalizing the attention weights. This represents a multilayer perceptron; It is average pooling. It is max pooling; For feature splicing operations; This indicates element-wise multiplication; It is the input feature map. It is the channel attention weight. It is the feature map after channel attention weighting. It is spatial attention weight. It is the final feature map after channel and spatial attention weighting.
5. The automatic fish fry counting method based on a feature-aware aggregation network according to claim 1, characterized in that, The design feature enhancement module includes: a parallel local structure awareness branch and a global attention branch; the local structure awareness branch focuses on enhancing fine-grained edge and texture information, and the input feature map... This is achieved through two parallel depthwise separable convolutions, using 3×3 and 5×5 kernels respectively, with the corresponding formulas as follows: The obtained feature map and Attention weights are generated by concatenating along the channel dimension and performing a series of convolutions and activation functions. ： Using attention weights Element-wise multiplication of the original input feature map yields the local enhancement features. The corresponding formula is: The global attention branch aims to enhance the perception of the overall distribution of fish fry and capture long-range dependencies, focusing on the input feature map. The number of channels is adjusted by three 1×1 convolutions. ,get 、 and ,in, 、 and This represents the query, key, and value in that order; for and The attention map is reshaped and multiplied to generate attention map, then an activation function is used to generate attention weights. These attention weights are then multiplied, and the resulting data is passed through a 1×1 convolutional layer, batch normalization, and residual connections to output global features. The formula corresponding to this process is: A dual cross-attention mechanism is introduced to mutually optimize the local structure perception branch and the global attention branch; a cross-attention mechanism outputs features from the global branch. As a query, the features of the demodulation local branches Injecting high-resolution structural priors into In contrast, another type of cross-attention focuses on the characteristics of local branches. As a query, optimize the features of the global branch. This embeds broader contextual guidance information into the local representation.
6. The automatic fish fry counting method based on a feature-aware aggregation network according to claim 1, characterized in that, The process involves training a model consisting of a backbone network, an attention module, a feature enhancement module, and a decoding stage based on the augmented training set, and then inputting the test image into the trained model to obtain the number of fish. This includes optimizing the network training using the Adam optimizer. The mathematical model of the Adam algorithm is expressed as: in, It is an exponential moving average of the gradient. It is the exponential moving average of the squared gradient. and Both are used to control the decay rate of the moving average. It is a time step The gradient at time is also the gradient of the parameters. It is after deviation correction ， It is after deviation correction ， It's about time. The parameters, It's the learning rate. It is a small constant used to ensure numerical stability; the Adam optimizer is used to train the model until it converges, the test image is input into the trained model to obtain the density map, and the density map is summed to obtain the number of fish.