A high occlusion scene fish count method, system and apparatus

Abstract

The application belongs to the field of aquaculture and computing technology, and particularly relates to a high-shielding scene fish population counting method, system and device, which comprises the following steps: collecting an aquaculture fish population image; extracting shallow features by non-normalized double convolution stacking for the image; compressing and projecting the shallow features and performing dynamic multi-scale convolution processing through parallel multi-branch, obtaining multi-level features through residual connection, lightweight spatial attention and pooling; projecting the multi-level features to the same channel space for dynamic weighted fusion, obtaining a single feature map; performing multi-level residual transformation on the single feature map, and iteratively extracting deep semantic features; performing multi-branch pooling and upsampling splicing on the deep features, obtaining an enhanced feature map, and then performing channel compression, expansion and step-by-step dimension reduction projection to generate a single-channel density map; and performing pixel integration summation on the density map to output the total number of fish population. The application realizes high-precision, low-parameter and strong-generalization fish population counting under the condition of severe shielding and limited edge computing power.

Description

Technical Field

[0001] This invention belongs to the field of aquaculture and computational technology, specifically relating to a method, system, and device for counting fish in highly obscured scenes. Background Technology

[0002] Factory-scale recirculating aquaculture systems are developing towards ultra-intensive, high-density stocking and 24 / 7 automated management. Against this backdrop, accurate, real-time, and low-stress monitoring of fish populations has become a core technological bottleneck restricting the optimization of stocking density, adjustment of feeding strategies, and prediction of fish yield.

[0003] Traditional counting methods, such as manual visual fishing and counting, photoelectric interception counting, and physical sampling and weighing conversion, generally have three inherent defects: First, they are labor-intensive and inefficient, and cannot achieve continuous dynamic monitoring; second, contact operations are prone to inducing stress reactions in fish, leading to decreased immunity or even mechanical damage; and third, the counting accuracy decreases sharply with the increase of stocking density, making it difficult to meet the real-time management and control requirements of modern factory farming.

[0004] In existing technologies, there is often a conflict between density map reconstruction quality and counting accuracy. On the one hand, some networks (such as MCNN) can generate visually smooth density maps through extremely lightweight designs, but their overly simplified feature extraction capabilities prevent them from accurately perceiving the multi-scale distribution of fish populations, resulting in a significant increase in counting bias during cross-pool testing. On the other hand, some networks with large parameter counts (such as STEERER) can generate structurally complete density maps, but their density response exhibits a systematic scaling bias. That is, while the spatial distribution of the density map is correct, the density amplitude in each region is generally too high or too low, leading to a systematic error in the counting result after integration and summation. This manifests as a coexistence of high structural similarity index and low counting accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and apparatus for counting fish in highly occluded scenes.

[0006] A method for counting fish in a highly occluded scene includes the following steps: S1. Collect original fish swarm video streams from different breeding ponds, different feeding times, different water depths and different lighting conditions in a recirculating aquaculture environment. After preprocessing and labeling, obtain images of farmed fish swarms and true density maps. S2. Perform unnormalized double convolution stacking processing on the image of farmed fish to obtain shallow features, and perform parallel multi-branch processing on the shallow features. Each branch process is as follows: perform compressed projection processing and multi-scale convolution processing on shallow features to obtain preliminary features, perform residual connection with shallow features, and then perform lightweight spatial attention processing and max pooling processing to obtain multi-scale features. S3. Project the multi-scale features obtained from all branches onto the same channel, perform weighted summation to obtain fused features, and then perform lightweight spatial attention processing to generate a single-channel spatial map. The single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map. S4. Perform multi-branch parallel branch mapping on a single feature map, sum the results and concatenate them with the single feature map to obtain residual stacked features. Parallel multi-branch pooling is performed on the residual stacked features, and then upsampling is performed separately to obtain the corresponding pooled features. All pooled features are then concatenated to obtain the enhanced feature map. S5. While maintaining the spatial resolution, perform compression and dilation transformations on the channel dimension of the enhanced feature map, and then restore the channel dimension through stepwise convolution to obtain the key feature map; perform stepwise channel dimension reduction projection on the key feature map to obtain the density map; construct a joint loss function based on the difference between the density map and the real density map; optimize the model parameters through backpropagation until the model converges to obtain the trained fish counting model. S6. Input the images of the farmed fish populations to be counted into the fish population counting model to obtain their corresponding density maps. Perform pixel integral summation to obtain the total number of fish populations.

[0007] The compression projection processing in S2 is as follows: , in, For compressed projection functions, and For 1×1 convolution, This represents a 3×3 grouped convolution. It is the ReLU activation function. This is a shallow feature.

[0008] The lightweight spatial attention processing in S3 is as follows: , in, This is a single-channel spatial diagram. For the Sigmod function, As a feature of fusion, For activation function, For 1×1 convolution, This is a grouped convolution.

[0009] In S3, the single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map, specifically: , Where Z is a single feature map. For residual characteristics, This is a single-channel spatial diagram. This indicates element-wise multiplication.

[0010] In S5, the channel dimensions are restored through stepwise convolution to obtain the key feature maps, specifically: , in, Here is the key feature map, and A represents the attention enhancement processing. This is the enhanced feature map after compression and dilation transformation. These represent the weights for compressed and dilated convolutions, respectively, and GA stands for global pooling. For the Sigmod function, This is the activation function.

[0011] In S5, a stepwise channel-wise dimension reduction projection is performed on the key feature map to obtain the density map, specifically: By using four consecutive 1×1 convolutions, the number of channel outputs in each convolution decreases to 1 at each level. Then, a progressive channel dimensionality reduction projection is performed to obtain the density map.

[0012] S5 performs a stepwise channel-wise dimensionality reduction projection on the key feature map to obtain the density map, and also includes: For each labeled point in the true density map, its n nearest neighbors are retrieved using a KD tree. The average of the distances to the (n-2), (n-1), and n nearest neighbors is taken to determine the Gaussian kernel standard deviation of the labeled point. A two-dimensional Gaussian kernel is generated based on the Gaussian kernel standard deviation and convolved with the labeled point to obtain its density contribution. The density contributions of all labeled points are accumulated pixel by pixel to obtain the true value of the density map.

[0013] The multi-branch parallel branch mapping in S4 also includes: squeezing each branch at the position where the number of channels changes, stimulating channel attention processing, and using random deactivation for regularization.

[0014] A fish counting system for highly occluded scenes, used to implement a method for counting fish in highly occluded scenes, includes: The image acquisition module collects original fish school video streams from different breeding ponds, different feeding times, different water depths and different lighting conditions in a recirculating aquaculture environment. After preprocessing and annotation, it obtains images of farmed fish schools and true density maps. The shallow feature extraction module performs unnormalized double convolution stacking processing on the image of farmed fish to obtain shallow features, and performs parallel multi-branch processing on the shallow features. Each branch process is as follows: perform compressed projection processing and multi-scale convolution processing on shallow features to obtain preliminary features, perform residual connection with shallow features, and then perform lightweight spatial attention processing and max pooling processing to obtain multi-scale features. The fusion feature extraction module projects the multi-scale features obtained from all branches onto the same channel, performs weighted summation to obtain fusion features, and then performs lightweight spatial attention processing to generate a single-channel spatial map. The single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map. The feature enhancement module performs multi-branch parallel branch mapping on a single feature map, sums the results, and concatenates them with the single feature map to obtain residual stacked features. The residual stacked features are then subjected to parallel multi-branch pooling, and upsampled separately to obtain corresponding pooled features. All pooled features are then concatenated to obtain the enhanced feature map. The training module performs compression and dilation transformations on the channel dimensions of the enhanced feature map while maintaining the spatial resolution. Then, it restores the channel dimensions through successive convolutions to obtain the key feature map. The key feature map is then subjected to successive channel dimensionality reduction projection to obtain the density map. The density map is then summed by pixel integrals to obtain the total number of fish. A joint loss function is constructed based on the difference between the density map and the true density map. The model parameters are optimized through backpropagation until the model converges, resulting in a trained fish counting model. The counting module inputs the images of the farmed fish to be counted into the fish counting model to obtain the corresponding density map, performs pixel integral summation, and obtains the total number of fish.

[0015] A fish counting device for a highly occluded scene includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a fish counting method for a highly occluded scene.

[0016] The advantages of this invention compared to the prior art are: This invention resolves the contradiction between density map reconstruction quality and counting accuracy through a collaborative design of feature extraction and density map generation. By employing an efficient multi-scale branching and deep feature enhancement module, it captures full-scale semantic information of fish schools from individual edges to cluster distribution with extremely low parameter counts, ensuring accurate spatial distribution of the density map. Furthermore, through channel information filtering and pure channel dimensionality reduction via density projection, it ensures that the amplitude response of the density map accurately reflects the true density. The combination of these two approaches results in a density map that is not only visually clear and faithful, but whose integral summation result also highly matches the actual number of fish. This achieves optimal performance in both density map reconstruction quality and counting accuracy, representing a technological breakthrough that balances both aspects without sacrificing one for the other.

[0017] This invention achieves an effective balance between accuracy and lightweight design. The two-stage cascaded architecture for feature extraction and the back-end forward 1×1 convolutional active reduction design greatly compress the number of model parameters, ensuring improved accuracy while reducing computational load.

[0018] This invention employs a non-normalized double convolution stacked structure in the shallow feature extraction stage to eliminate the dependence on batch size and maintain statistical independence in both the training and inference stages. Attached Figure Description

[0019] Figure 1 is a schematic diagram of the dataset annotation of this invention. Detailed Implementation

[0020] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments.

[0021] This invention relates to a method for counting fish in a highly occluded scene, comprising the following steps: S1. Collect original fish swarm video streams from different breeding ponds, different feeding times, different water depths and different lighting conditions in a recirculating aquaculture environment. After preprocessing and labeling, obtain images of the cultured fish swarms and true density maps.

[0022] The model's counting accuracy and cross-scene generalization ability were verified by using a monocular camera to capture multi-temporal and multi-view videos of the spherical recirculating aquaculture pond (circular pond diameter approximately 3m, water depth range 0.5~0.8m, water temperature 20~24℃). These videos were acquired from different ponds, feeding times, water depths, and lighting conditions (original resolution 2704×1320, 30fps). After frame extraction and cropping preprocessing, the videos were divided into three sets according to different acquisition times and water depths: a training set (PartI_Train), a validation set (PartI_Val), and a test set (PartII_Test) within the same pond at different times, as well as a test set (PartIII_Test) across different ponds at different times.

[0023] During preprocessing, one frame is sampled every 15 frames and cropped to 1009×760 pixels as the model input. The dataset consists of 580 images in Part I (divided into training and validation sets in a 3:1 ratio), 98 test images from the same pool at different times in Part II, and 108 test images from another pool in Part III.

[0024] In addition, during the annotation process for creating the dataset, the following rules were followed: For unobstructed fish bodies, the annotation points were placed at the geometric center of the fish body (see “×” in the figure) to ensure that the Gaussian kernel in the density map accurately responds to the visible parts; for fish bodies with no more than two-thirds obstruction, the center of the visible part was annotated as much as possible; for severely obstructed fish bodies, such as those where only the head or tail is visible, or where only the edges of the fish body are visible, detailed annotation examples can be found in [reference needed].Figure 1 The circle in the picture "This is for cases with high occlusion; after preprocessing and annotation, we obtain images of farmed fish schools."

[0025] Furthermore, S2-S5 are performed on the training set (PartI_Train) and validation set (PartI_Val) in the same pool at different times to output the density map, calculate the loss function, which is the difference between the predicted value and the true value, and then perform backpropagation to update the model weights; the optimal model version is selected.

[0026] S2. Perform unnormalized double convolution stacking processing on the image of farmed fish to obtain shallow features, and perform parallel multi-branch processing on the shallow features. Each branch process involves: performing compressed projection processing and multi-scale convolution processing on shallow features to obtain preliminary features, performing residual connections with shallow features, and then performing lightweight spatial attention processing and max pooling processing to obtain multi-scale features.

[0027] Images of farmed fish are subjected to unnormalized double convolution stacking processing, specifically 3×3 convolution followed by ReLU activation, with channel transformation from 3→16→16. This aims to decouple discriminative low-level visual primitives such as the fish head, gill cover edge, and caudal peduncle contour from the images, obtaining shallow features. The specific operation is as follows: Suppose the input image of the farmed fish population is ,in , For spatial dimensions, shallow feature extraction employs a non-normalized double convolution stacked structure, formally defined as: , in, These are shallow features. , The kernel weight matrix; For bias terms; It is the ReLU activation function; This indicates a convolution operation with stride and padding both set to 1, maintaining the same resolution.

[0028] This invention follows three main principles: First, a batch normalization strategy is used. Given the non-stationary distribution of local contrast caused by color shift in aquaculture water, light attenuation, and fish scale reflection, batch normalization (BN) and group normalization (GN) are deliberately disabled to prevent the normalization operation from stretching or annihilating the weak edge gradients of the fish body, thus preserving the statistical authenticity of the original response. Second, an equivalent receptive field expansion is used. Two cascaded 3×3 convolutions are equivalent to a 5×5 receptive field. Without introducing the overhead of large kernel convolution parameters, the contextual information of local deformation of the fish body (such as the continuity prior of bent torso) is captured at the shallow stage. Third, channel dimensionality enhancement and cross-branch sharing are used. The 3-channel RGB input is mapped to a 16-channel mid-level feature space, providing a unified high-dimensional input basis for the subsequent three parallel and efficient multi-scale branches. This avoids repeated extraction of low-level features by each branch, and the number of parameters is reduced compared to independent multi-column structures.

[0029] The essence of shallow feature extraction is to construct a fish body edge-preserving encoder that is robust to water noise. While outputting thousands of layers of features, it also has local deformation sensitivity and global context awareness, laying a high-fidelity foundation for subsequent multi-scale semantic abstraction.

[0030] Parallel multi-branch processing is performed on shallow features. The different convolution scale differences between different branches facilitate the screening of fish of different sizes. The multi-scale convolution kernels of different branches in this invention are 3×3, 5×5 and 7×7 respectively. Furthermore, each branch process involves: performing compressed projection processing and multi-scale convolution processing on the shallow features to obtain preliminary features, performing residual connections with the shallow features, and then performing lightweight spatial attention processing and max pooling processing to obtain multi-scale features. For example, parallel multi-branch processing can consist of three parallel EMBs.

[0031] Branching by scale factor The convolution stacking depth is dynamically adjusted (corresponding to effective receptive field kernel sizes of 3×3, 5×5, and 7×7), respectively, to represent the projection scales of small, medium, and large fish bodies. The computational process can be formally represented as follows: , in, This is a compression projection function for the Ghost class. These are shallow features. It is a 2×2 max pooling with a step size of 2. For extremely simplified channel attention processing, global average pooling and 1×1 convolution are used, followed by scalar attention weights obtained through Sigmoid, achieving global context adaptive recalibration with a very low number of parameters; These are learnable weights, initialized to 0.5, used to implement residual connections between multi-scale features and the original Ghost output. This design allows the network to adaptively adjust the injection intensity of multi-scale information while mitigating gradient vanishing caused by deep stacking. This represents dynamic multi-scale convolution. This part dynamically adjusts the size of the receptive field according to the scale factor by stacking 3×3 convolutions and ReLU, avoiding the problem of a surge in parameters caused by directly using large convolution kernels.

[0032] The aim is to map the input channels to a high-dimensional feature space with extremely low parameter count. The specific processing procedure is a three-stage process: "1×1 convolution processing → group convolution processing → 3×3 convolution processing", which can be represented as follows: , in, For compressed projection functions, and For 1×1 convolution, This represents a 3×3 grouped convolution, with the number of groups being... , The number of channels output by each branch; the first two convolution operations are both standardized using group normalization, with the number of groups being [number missing]. , It is the ReLU activation function. This is a shallow feature.

[0033] This design avoids the risks of parameter redundancy and overfitting caused by directly using large kernel convolution from an information theory perspective.

[0034] Furthermore, the non-rigid bending of the fish body manifests as low-rank local deformation in space. Grouped convolution induces the network to learn the low-dimensional linear manifold embedding of the fish body contour by forcing decoupling between channels, rather than the overparameterized fitting of high-dimensional nonlinear deformation. This achieves the suppression of the fish scale highlight channel and the enhancement of the fish body edge channel with extremely low parameter count.

[0035] Downsampling the feature map to half the input size serves three purposes: reducing the computational burden on subsequent modules, increasing the effective receptive field to capture fish cluster-level context, and enhancing robustness to fish swimming jitter through local translation invariance introduced by spatial downsampling.

[0036] The outputs of the three branches maintain consistency in spatial resolution and channel dimension, providing aligned multi-scale features for subsequent adaptive fusion.

[0037] S3. Project the multi-scale features obtained from all branches onto the same channel, perform weighted summation to obtain fused features, and then perform lightweight spatial attention processing to generate a single-channel spatial map. The single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map.

[0038] This step dynamically fuses features at different scales into a single feature map. , This represents the number of channels in a single feature map, while preserving discriminative information about the boundaries and internal density gradients of fish clusters.

[0039] First, each branch is projected onto the same channel space through independent 1×1 convolution, achieving cross-scale channel semantic alignment. Then, learnable Softmax branch weights are introduced to balance the unified multi-scale features. By performing dynamic weighted summation, the fused characteristics are obtained. This avoids information redundancy and parameter inflation caused by traditional splicing or fixed weighting; the fused features Then, a lightweight spatial attention model is generated to produce a single-channel spatial map. Specifically: , Among them, among them, This is a single-channel spatial diagram. For the Sigmod function, As a feature of fusion, For activation function, For 1×1 convolution, The number of groups in the grouped convolution is 4; F 1×1 (·) represents a 1×1 convolution.

[0040] Then obtain the single-channel space map With fusion features Element-wise multiplication enables the network to focus on the visible area of ​​the fish school, suppress occluded areas, and reduce the impact of high-frequency local noise caused by suspended particles in the water.

[0041] Finally, regarding the fusion features Perform residual processing to obtain the reconstructed residual channel. The features are then multiplied element-wise between the single-channel spatial map and the fused features, followed by residual processing to obtain a single feature map. This achieves a complementary fusion of structural fidelity and semantic richness, avoiding gradient vanishing or feature homogenization problems found in single paths. Specifically: , Where Z is a single feature map. For residual characteristics, This is an element-wise multiplication operation.

[0042] S4. Perform multi-branch parallel multi-level residual mapping on a single feature map, sum the results and concatenate them with the single feature map to obtain stacked residual features. Parallel multi-branch pooling is performed on the residual stacked features, and then upsampling is performed separately to obtain the corresponding pooled features. All pooled features are then concatenated to obtain the enhanced feature map.

[0043] This step can be achieved by stacking multi-level simplified Inception Residual Blocks (IRBs) and Spatial Pyramid Pooling (SPP).

[0044] Specifically, let the input be a single feature map Z, which undergoes multiple levels of simplified Inception residual mapping and parallel multi-branch pooling processing to obtain the enhanced feature map. : ; ; in, For the first l Level IRB transformation, S For spatial pyramid pooling processing, For the first l Level residual stacking output characteristics .

[0045] It should be noted that the multi-level residual mapping (Inception residual mapping) of each branch contains different processing steps. The multi-level residual mapping of each branch has different levels and pooling sizes. Its squeezing and excitation channel attention processing deployment follows the principle of "activation at key transformation points" and is only enabled when the number of channels changes. The specific composition is shown in Table 1.

[0046] Table 1. Composition of Inception Residual Mapping Structures at Each Level Where MP represents max pooling, 2×2, with a step size of 2; here These are the input and output channels, respectively, and the number of channels is shown in the table; True indicates that the SE channel attention mechanism is used. Regularization is applied to random deactivation, indicating that network neurons have a probability of p being randomly discarded.

[0047] The Inception residual mapping design of this invention integrates four technologies: multi-branch, residual learning, channel attention, and lightweight regularization. The core improvement is that all branches output the same number of channels, thereby avoiding the tedious manual tuning of branch channels in traditional methods, and adding residual connections after the branch ends.

[0048] The improved multi-branch parallel multi-level residual mapping of this invention is specifically implemented through the following four branches, and the processing steps are shown in Table 2 below, where AP represents average pooling: Table 2 Improved Inception Branch Structure After multi-level branch mapping, the feature map has rich semantic information, but the density gradient inside the fish cluster may still be blurred due to local occlusion.

[0049] Multi-branch pooling uses three branches with different pooling sizes to aggregate multi-scale context without changing the spatial resolution, and then uses a final pooling branch to enhance the perception of the overall cluster structure and boundary sharpness. Let the input feature map... Multi-branch pooling processing output Defined as: ; Each branch pooling process performs parallel multi-branch pooling on the residual stacked features, and then performs upsampling separately to obtain the corresponding pooled features. k=1, 2, 3 represent different branches: ; Among them, pooling output size Upsampling Upsample Restore the feature map to its original size. All pooled features are concatenated to obtain an enhanced feature map. Further, the three branches are concatenated along the channel dimension and then subjected to a 1×1 convolution. The fusion yields an enhanced feature map that preserves spatial accuracy while incorporating contextual information from multi-scale receptive fields. This design is particularly suitable for scenarios involving clustered fish: smaller pooling branches provide constraints on the overall cluster count, while larger pooling branches preserve individual boundaries. The combination of these two approaches effectively mitigates the "holes" or "diffusion" phenomena in the density map.

[0050] This step aims to further extract discriminative information about the nonlinear density distribution within fish clusters and the adhesion boundaries between individuals from the aligned single feature map.

[0051] S5. While maintaining the spatial resolution, perform compression and dilation transformations on the channel dimensions of the enhanced feature map, and then restore the channel dimensions through stepwise convolution to obtain the key feature map; perform stepwise channel dimensionality reduction projection on the key feature map to obtain the density map, construct a joint loss function based on the difference between the density map and the real density map, optimize the model parameters through backpropagation until the model converges, and obtain the trained fish counting model.

[0052] Specifically, it is implemented by connecting the Channel Bottleneck Density Head (CBDH) and the Density Projection Head (DPH) in series, consisting entirely of 1×1 convolutions, completely eliminating deconvolution, upsampling, and fully connected layers.

[0053] Let the high-dimensional semantic feature map output by the front-end feature extractor be... Generator output density map The overall mapping can be decomposed into: , in, These are density projection heads and channel bottleneck density heads, respectively. For SE channel attention.

[0054] While maintaining spatial resolution, compression and dilation transformations (CBDH) are performed on the channel dimensions of the enhanced feature maps. The design follows the Information Bottleneck principle: by forcing features through a low-dimensional bottleneck, redundant information irrelevant to density estimation (such as water background texture and uneven lighting noise) is eliminated, retaining only discriminative features strongly correlated with density regression. The channel dimensions are reduced progressively: , , in, The number of variation groups for channel i. The compression weights for channel i are given, and the channel changes are as follows: Expansion path The number of channels is restored symmetrically with the compression path. An SE channel attention module is embedded at the end of the expansion path. By restoring the channel dimensions through successive convolutional processes, key feature maps are obtained, specifically: , in, Here is the key feature map, and A represents the attention enhancement processing. This is the enhanced feature map after compression and dilation transformation. These are the compression and dilation convolution weights, respectively. GA For global pooling processing, For the Sigmod function, This is the activation function.

[0055] Key Feature Map The input density map projection head DPH is processed through four consecutive 1×1 convolutions, with the number of channel outputs decreasing to 1 at each convolution level. This process performs a step-by-step channel-wise dimensionality reduction projection to obtain a single-channel density map. , , , in, This is the final output density map; Represents the convolution operation functions at each level; The weights for each convolution operation, This design forces density regression to rely entirely on a linear combination of channel dimensions, avoiding the neighborhood smoothing that spatial convolution might introduce. This is crucial for fish counting, as adjacent pixels may belong to different fish, and excessive smoothing would lead to a blurred density map.

[0056] Furthermore, for each labeled point in the true density map, its n nearest neighbors are retrieved using a KD tree. The average of the distances to the (n-2), (n-1), and n nearest neighbors is taken to determine the Gaussian kernel standard deviation of the labeled point. A two-dimensional Gaussian kernel is generated based on the Gaussian kernel standard deviation and convolved with the labeled point to obtain its density contribution. The density contributions of all labeled points are accumulated pixel by pixel to obtain the true value of the density map.

[0057] Specifically, in order to generate a density map true value that is adapted to the distribution of fish populations, this invention adopts a geometrically adaptive Gaussian kernel density map construction method.

[0058] Given an input image and its set of labeled points P = { p 1 , p 2 , ..., p N (N is the total number of labeled points), first generate a density map of the same size as the image. For each labeled point... p i The Gaussian kernel standard deviation of a point is adaptively determined by taking the average of the distances from the 2nd, 3rd, and 4th nearest neighbors, using a KD-tree to retrieve its four nearest neighbors. , in, , , These are the distances to the 2nd, 3rd, and 4th nearest neighbors of the i-th labeled point, respectively.

[0059] Subsequently, with A two-dimensional Gaussian kernel is generated for the standard deviation, and convolved with the unit impulse map at that point to obtain the first... i Standard Note Density contribution of points , x , y The x and y coordinates of a pixel are given, and the density contributions of all labeled points are accumulated pixel by pixel to obtain the final density map. ,satisfy .

[0060] The calculation formula for this method is as follows: , , , in, Represents pixels ( x , y The contribution of all labeled points; The position function of the fish school. Indicates the location of the fish. Represents the Gaussian kernel; Indicates adaptive parameters; express The average Euclidean distance to the nearest neighbor sample; Indicates the number of samples.

[0061] The total number of fish is obtained by summing the pixel integrals of the density map.

[0062] Furthermore, to improve the pixel accuracy and structural fidelity of density map estimation, a multi-loss joint optimization strategy is adopted during network training.

[0063] Construct a joint loss function based on the difference between the density map and the true density map. : , in, To account for the mean squared error loss, the squared difference between the predicted density map and the ground density map is calculated pixel by pixel to guide the network to learn the accurate density distribution. For structural similarity loss, a Gaussian window with a window size of 7 is used to calculate brightness, contrast, and structural similarity, defined as follows: This is used to enhance the local structure preservation capability of density maps; To smooth the L1 loss and balance gradient stability with robustness to sparse outliers, it is defined as follows: ,, To predict the number of fish, the weights of the three loss terms were determined through experimental tuning, so that MSE dominates global pixel regression, while SSIM and SmoothL1 contribute to structural consistency and gradient smoothness, respectively.

[0064] Furthermore, the model parameters are optimized through backpropagation until the model converges, resulting in a trained fish counting model.

[0065] S6. Input the images of the farmed fish populations to be counted into the fish population counting model to obtain their corresponding density maps. Perform pixel integral summation to obtain the total number of fish populations.

[0066] After the model is completely fixed, the same pool time-sharing test set (PartⅡ_Test) and different pool time-sharing test set (PartⅢ_Test) are used at once, and S2-S6 are executed to output the counting results. This verifies the model's adaptability to "unfamiliar pools" and performs cross-scenario generalization.

[0067] A fish counting system for highly occluded scenes, used to implement a method for counting fish in highly occluded scenes, includes: The image acquisition module collects original fish school video streams from different breeding ponds, different feeding times, different water depths and different lighting conditions in a recirculating aquaculture environment. After preprocessing and annotation, it obtains images of farmed fish schools and true density maps. The shallow feature extraction module performs unnormalized double convolution stacking processing on the image of farmed fish to obtain shallow features, and performs parallel multi-branch processing on the shallow features. Each branch process is as follows: perform compressed projection processing and multi-scale convolution processing on shallow features to obtain preliminary features, perform residual connection with shallow features, and then perform lightweight spatial attention processing and max pooling processing to obtain multi-scale features. The fusion feature extraction module projects the multi-scale features obtained from all branches onto the same channel, performs weighted summation to obtain fusion features, and then performs lightweight spatial attention processing to generate a single-channel spatial map. The single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map. The feature enhancement module performs multi-branch parallel branch mapping on a single feature map, sums the results, and concatenates them with the single feature map to obtain residual stacked features. The residual stacked features are then subjected to parallel multi-branch pooling, and upsampled separately to obtain corresponding pooled features. All pooled features are then concatenated to obtain the enhanced feature map. The training module performs compression and dilation transformations on the channel dimensions of the enhanced feature map while maintaining the spatial resolution. Then, it restores the channel dimensions through successive convolutions to obtain the key feature map. The key feature map is then subjected to successive channel dimensionality reduction projection to obtain the density map. The density map is then summed by pixel integrals to obtain the total number of fish. A joint loss function is constructed based on the difference between the density map and the true density map. The model parameters are optimized through backpropagation until the model converges, resulting in a trained fish counting model. The counting module inputs the images of the farmed fish to be counted into the fish counting model to obtain the corresponding density map, performs pixel integral summation, and obtains the total number of fish.

[0068] A fish counting device for a highly occluded scene includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a fish counting method for a highly occluded scene.

Claims

1. A method for counting fish in a highly occluded scene, characterized in that, Includes the following steps: S1. Collect original fish swarm video streams from different breeding ponds, different feeding times, different water depths and different lighting conditions in a recirculating aquaculture environment. After preprocessing and labeling, obtain images of farmed fish swarms and true density maps. S2. Perform unnormalized double convolution stacking processing on the image of farmed fish to obtain shallow features, and perform parallel multi-branch processing on the shallow features. Each branch process is as follows: perform compressed projection processing and multi-scale convolution processing on shallow features to obtain preliminary features, perform residual connection with shallow features, and then perform lightweight spatial attention processing and max pooling processing to obtain multi-scale features. S3. Project the multi-scale features obtained from all branches onto the same channel, perform weighted summation to obtain fused features, and then perform lightweight spatial attention processing to generate a single-channel spatial map. The single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map. S4. Perform multi-branch parallel branch mapping on a single feature map, sum the results and concatenate them with the single feature map to obtain residual stacked features. Parallel multi-branch pooling is performed on the residual stacked features, and then upsampling is performed separately to obtain the corresponding pooled features. All pooled features are then concatenated to obtain the enhanced feature map. S5. While maintaining the spatial resolution, perform compression and dilation transformations on the channel dimension of the enhanced feature map, and then restore the channel dimension through stepwise convolution to obtain the key feature map; perform stepwise channel dimension reduction projection on the key feature map to obtain the density map; construct a joint loss function based on the difference between the density map and the real density map; optimize the model parameters through backpropagation until the model converges to obtain the trained fish counting model. S6. Input the images of the farmed fish populations to be counted into the fish population counting model to obtain their corresponding density maps. Perform pixel integral summation to obtain the total number of fish populations.

2. The method for counting fish in a highly occluded scene according to claim 1, characterized in that, The compression projection processing in S2 is as follows: ， in, For compressed projection functions, and For 1×1 convolution, This represents a 3×3 grouped convolution. It is the ReLU activation function. This is a shallow feature.

3. The method for counting fish in a highly occluded scene according to claim 1, characterized in that, The lightweight spatial attention processing in S3 is as follows: ， in, This is a single-channel spatial diagram. For the Sigmod function, As a feature of fusion, For activation function, For 1×1 convolution, This is a grouped convolution.

4. The method for counting fish in a highly occluded scene according to claim 1, characterized in that, In S3, the single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map, specifically: ， Where Z is a single feature map. For residual characteristics, This is a single-channel spatial diagram. This indicates element-wise multiplication.

5. The method for counting fish in a highly occluded scene according to claim 1, characterized in that, In S5, the channel dimensions are restored through stepwise convolution to obtain the key feature maps, specifically: ， in, Here is the key feature map, and A represents the attention enhancement processing. This is the enhanced feature map after compression and dilation transformation. These represent the weights for compressed and dilated convolutions, respectively, and GA stands for global pooling. For the Sigmod function, This is the activation function.

6. The method for counting fish in a highly occluded scene according to claim 1, characterized in that, In S5, a stepwise channel-wise dimension reduction projection is performed on the key feature map to obtain the density map, specifically: By using four consecutive 1×1 convolutions, the number of channel outputs in each convolution decreases to 1 at each level. Then, a progressive channel dimensionality reduction projection is performed to obtain the density map.

7. The method for counting fish in a highly occluded scene according to claim 1, characterized in that, S5 performs a stepwise channel-wise dimensionality reduction projection on the key feature map to obtain the density map, and also includes: For each labeled point in the true density map, its n nearest neighbors are retrieved using a KD tree. The average of the distances to the (n-2), (n-1), and n nearest neighbors is taken to determine the Gaussian kernel standard deviation of the labeled point. A two-dimensional Gaussian kernel is generated based on the Gaussian kernel standard deviation and convolved with the labeled point to obtain its density contribution. The density contributions of all labeled points are accumulated pixel by pixel to obtain the true value of the density map.

8. The method for counting fish in a highly occluded scene according to claim 1, characterized in that, The multi-branch parallel branch mapping in S4 also includes: squeezing each branch at the position where the number of channels changes, stimulating channel attention processing, and using random deactivation for regularization.

9. A fish counting system for a highly occluded scene, used to implement the fish counting method for a highly occluded scene as described in any one of claims 1-8, characterized in that, include: The image acquisition module collects original fish school video streams from different breeding ponds, different feeding times, different water depths and different lighting conditions in a recirculating aquaculture environment. After preprocessing and annotation, it obtains images of farmed fish schools and true density maps. The shallow feature extraction module performs unnormalized double convolution stacking processing on the image of farmed fish to obtain shallow features, and performs parallel multi-branch processing on the shallow features. Each branch process is as follows: perform compressed projection processing and multi-scale convolution processing on shallow features to obtain preliminary features, perform residual connection with shallow features, and then perform lightweight spatial attention processing and max pooling processing to obtain multi-scale features. The fusion feature extraction module projects the multi-scale features obtained from all branches onto the same channel, performs weighted summation to obtain fusion features, and then performs lightweight spatial attention processing to generate a single-channel spatial map. The single-channel spatial map is multiplied element-wise with the fused features, and then residual processing is performed to obtain a single feature map. The feature enhancement module performs multi-branch parallel branch mapping on a single feature map, sums the results, and concatenates them with the single feature map to obtain residual stacked features. Parallel multi-branch pooling is performed on the residual stacked features, and then upsampling is performed separately to obtain the corresponding pooled features. All pooled features are then concatenated to obtain the enhanced feature map. The training module performs compression and dilation transformations on the channel dimensions of the enhanced feature map while maintaining the spatial resolution. Then, it restores the channel dimensions through successive convolutions to obtain the key feature map. The key feature map is then subjected to successive channel dimensionality reduction projection to obtain the density map. The density map is then summed by pixel integrals to obtain the total number of fish. A joint loss function is constructed based on the difference between the density map and the true density map. The model parameters are optimized through backpropagation until the model converges, resulting in a trained fish counting model. The counting module inputs the images of the farmed fish to be counted into the fish counting model to obtain the corresponding density map, performs pixel integral summation, and obtains the total number of fish.

10. A fish counting device for highly obscured scenes, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a fish counting method in a highly occluded scene as described in any one of claims 1-8.

Images