Fish feeding intensity real-time evaluation method and system based on multi-feature fusion

Abstract

The invention discloses a fish ingestion intensity real-time evaluation method and system based on multi-feature fusion. A hydrophone is used for collecting the ingestion audio frequency of the fish, and then the ingestion audio frequency of the fish is divided according to the fixed length; extracting an Mel frequency spectrum matrix from each audio clip, and extracting an MFCC frequency spectrum and an FBank frequency spectrum from the Mel frequency spectrum matrix; and the MFCC spectrum and the FBank spectrum are inputted to a MobileNetV4S-ADEFF-SENet model for classification and identification, and a fish feeding intensity evaluation result is obtained. According to the method, key feature responses in the ingestion sound signals are dynamically enhanced through multiple features, noise interference is weakened, fusion features are constructed, classification is carried out through the MobileNetV4S model with the optimized UIB structure, the feature extraction capacity and the category distinguishing precision in the complex noise environment are remarkably improved, and the classification accuracy is improved. And a more stable and reliable technical support is provided for accurate evaluation of the fish ingestion intensity.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition technology, and in particular relates to a method and system for real-time assessment of fish feeding intensity based on multi-feature fusion. Background Technology

[0002] In modern aquaculture, precision feeding offers significant advantages over traditional methods. It promotes healthier and faster fish growth while reducing the risk of water pollution from overfeeding, thus helping to maintain optimal water quality. However, many existing feeding systems still rely on manual operation or automated metering systems. These methods are often based on the operator's subjective judgment and cannot adapt to the dynamic influence of environmental factors such as water quality and temperature on fish feeding behavior, easily leading to underfeeding or overfeeding. Given these limitations, there is an urgent need for an intelligent method that can accurately assess fish feeding intensity in real time to achieve more scientific and efficient management practices. In recent years, deep learning has made significant progress in detecting fish feeding intensity. Current research in this field is generally divided into two categories: computer vision-based methods and acoustic signal-based methods.

[0003] Vision-based methods use cameras to capture images of the water surface and analyze features such as fish density, movement patterns, and splash patterns to determine feeding intensity. However, this method heavily relies on lighting conditions and water transparency. When the aquaculture water is turbid, lighting is insufficient, or the fish have underwater feeding habits, visual information is prone to loss or distortion. Therefore, vision-based detection methods have limitations in adapting to complex aquaculture environments and in not covering multiple feeding patterns.

[0004] Acoustic-based methods collect sound signals generated by fish during feeding using hydrophones and then classify and identify these acoustic features using machine learning or deep learning algorithms. This method is not limited by lighting conditions, water quality, or feeding methods, exhibiting greater environmental adaptability. Currently, feeding intensity identification based on acoustic signals mainly employs a single acoustic feature combined with a traditional classifier or deep neural network for training. However, this method faces severe noise interference problems in actual aquaculture environments. Background noise generated by water pumps, aeration equipment, etc., overlaps with the spectrum of feeding sound signals, leading to insufficient feature extraction and decreased model recognition accuracy.

[0005] With the development of deep learning technology and mobile computing platforms, food intensity recognition technology is gradually evolving towards lightweight and multi-feature fusion. Therefore, deeply fusing multiple complementary acoustic features and enhancing the model's responsiveness to key features through attention mechanisms, while maintaining a lightweight model to adapt to edge deployment requirements, has become an important direction for improving the performance of food intensity recognition.

[0006] Existing acoustic-based methods for assessing feeding intensity mostly construct input features by using a single audio spectrum or by simply splicing different spectra, and then classify them using a deep learning model.

[0007] Current acoustic-based methods for assessing feeding intensity suffer from shortcomings in feature extraction and noise resistance. The main problems are: existing methods either use a single spectral feature (such as Mel spectrum or linear spectrum) as model input, failing to integrate the advantages of different feature extraction methods, leading to performance degradation in noisy environments; or, although they employ multiple spectral features, the simple concatenation without considering the correlation and complementarity between features still results in the loss of useful information. Furthermore, the feature channel selection mechanisms of existing models are not refined enough, limiting their ability to enhance key features and suppress noise channels, further impacting the accuracy of assessment in complex noisy environments. Therefore, this invention provides a noise-resistant feeding intensity assessment method based on multi-feature fusion. By designing an adaptive feature fusion mechanism and an optimized channel attention mechanism, it achieves more accurate and stable feeding intensity assessment in complex noisy environments.

[0008] Terminology Explanation:

[0009] MFCC spectrum: Mel frequency cepstral coefficients spectrum.

[0010] FBank spectrum: Filter Bank Energy spectrum.

[0011] Mel spectrum: Mel-scale power spectrum.

[0012] ADEFF: Adaptive Detail-Enhanced Feature Fusion.

[0013] UIB: Universal Inverted Bottleneck Block.

[0014] FusedIB: Fused Inverted Bottleneck. Summary of the Invention

[0015] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for real-time evaluation of fish feeding intensity based on multi-feature fusion.

[0016] This invention is implemented as follows: Firstly, it provides a method for real-time assessment of fish feeding intensity based on multi-feature fusion, the method comprising:

[0017] The feeding audio of fish is collected using a hydrophone, and then the feeding audio is divided into segments of fixed length.

[0018] Extract the Mel spectrum matrix for each audio segment, and then extract the MFCC spectrum and FBank spectrum from the Mel spectrum matrix;

[0019] The MFCC spectrum and FBank spectrum were input into the MobileNetV4S-ADEFF-SENet model for classification and identification to obtain the fish feeding intensity assessment results.

[0020] Preferably, the MobileNetV4S-ADEFF-SENet model replaces the Stage 1 (i.e., input) 3×3 convolutions of the MobileNetV4-Small model with ADEFF modules, replaces the inverted bottleneck module UIB with the inverted bottleneck module UIB-SE integrating the SENet attention mechanism, and replaces the fused inverted residual bottleneck module FusedIB with the fused inverted residual bottleneck module Fused-SE integrating the SENet attention mechanism.

[0021] Preferably, the ADEFF module is used to perform feature fusion on the MFCC spectrum and the FBank spectrum. This is achieved by performing convolution processing on the MFCC spectrum and the FBank spectrum with different kernel sizes to obtain two local features; multiplying the two local features element-wise to obtain initial fused features; calculating channel attention and spatial attention in parallel on the initial fused features to obtain a coarse-grained attention map; using channel rearrangement and group convolution on the coarse-grained attention map to obtain fine-grained channel weights; performing dynamic weighted summation on the two local features combined with the fine-grained channel weights; and finally outputting the fused features through 1×1 convolutional linear projection.

[0022] Preferably, the ADEFF module is implemented as follows: First, a 7×7 convolution is applied to the FBank spectrum to capture the long-term dependence of the fine-grained spectrum and global acoustic events; then, a 3×3 convolution is applied to the MFCC spectrum to preserve the local discriminativeness of its compact representation, thereby generating the first local feature representation. Second local feature representation ;

[0023] Subsequently, the first local feature representations are fused by element-wise multiplication. Second local feature representation The initial fused feature map is obtained. ;

[0024] For the initial fused feature map Channel attention weights are obtained through the channel attention module and the spatial attention module, respectively. Spatial attention weights ;

[0025] Subsequently, channel attention weights Spatial attention weights Perform element-wise multiplication to generate a coarse-grained attention map. ;

[0026] Channel rearrangement operation in coarse-grained attention maps and fusion features The channels are alternately rearranged to achieve interactive learning of multiple features, resulting in fine-grained channel weights. ;

[0027] Fine-grained channel weights Representation of the first local feature Fusion results, fine-grained channel weights With the second local feature representation The fusion results are fused through dynamic weighted summation, and finally... The convolutional layer is linearly projected to obtain the fused features. .

[0028] Preferably, the inverted bottleneck module UIB-SE integrating the SENet attention mechanism and the fused inverted residual bottleneck module FusedIB-SE integrating the SENet attention mechanism are respectively connected in series with an SENet attention mechanism module at the output of the inverted bottleneck module UIB and the fused inverted residual bottleneck module FusedIB.

[0029] Preferably, the variants of the inverted bottleneck module UIB include three types: IB, ExtraDW, and ConvNext.

[0030] Preferably, the SENet attention mechanism module adaptively adjusts the feature response intensity of each channel by dynamically learning the dependencies between channels, thereby enhancing the model's sensitivity to key feature channels. The implementation process is as follows:

[0031] The first step is compression. Global average pooling is performed on the input feature map. and The width and height of the feature map are respectively compressed in the spatial dimension to generate a file of size [size missing]. The output feature map, which encodes Global information for each channel;

[0032] The next step is the activation operation, which inputs the global feature descriptor of the output feature map into two fully connected layers, through... The function generates channel-level attention weights;

[0033] Finally, the learned channel-level attention weights are applied to the original input feature map to achieve channel-level feature recalibration.

[0034] Preferably, in the Stage 5 of the MobileNetV4S-ADEFF-SENet model, all depthwise convolutional layers in the second ExtraDW-SE module and the first two IB-SE modules employ 3×3 depthwise convolutions.

[0035] Preferably, the MFCC spectrum and FBank spectrum are Z-score normalized before being input into the MobileNetV4S-ADEFF-SENet model.

[0036] Secondly, it provides a real-time assessment system for fish feeding intensity, including:

[0037] The data acquisition module is responsible for using a hydrophone to collect the feeding audio of the fish, and then dividing the feeding audio into segments of fixed length;

[0038] The feature extraction module is responsible for extracting the Mel spectrum matrix for each audio segment, and then extracting the MFCC spectrum and FBank spectrum from the Mel spectrum matrix;

[0039] The classification and recognition module is responsible for inputting the MFCC spectrum and FBank spectrum into the MobileNetV4S-ADEFF-SENet model for classification and recognition, and obtaining the fish feeding intensity assessment results.

[0040] The beneficial effects of this invention are at least as follows:

[0041] This invention first converts the audio signal acquired by the hydrophone into a noise-resistant MFCC spectrum and a highly correlated FBank spectrum. These two spectra are then fed into the ADEFF feature fusion module. By constructing a collaborative mechanism of "coarse-grained feature integration—detail enhancement strategy," the key feature responses in the feeding sound signal are dynamically enhanced, effectively reducing noise interference on feature extraction. This invention optimizes the UIB and FusedIB structures of MobileNetV4 by embedding the SENet channel attention mechanism at the end of each UIB module and FusedIB structure to selectively enhance the response intensity of key feature channels and suppress redundant noise channels. Through these improvements, the feature extraction capability and class discrimination accuracy in complex noise environments are significantly improved, providing more stable and reliable technical support for the accurate assessment of fish feeding intensity. Attached Figure Description

[0042] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is an overall flowchart of the real-time assessment method for fish feeding intensity provided in this embodiment of the invention.

[0044] Figure 2 This is a diagram of the MobileNetV4S-ADEFF-SENet model architecture provided in an embodiment of the present invention.

[0045] Figure 3 This is an ADEFF module architecture diagram provided in an embodiment of the present invention.

[0046] Figure 4 This is a diagram of the SENet attention mechanism module architecture provided in an embodiment of the present invention.

[0047] Figure 5 This is an architecture diagram of the inverted bottleneck module UIB-SE and the fused inverted residual bottleneck module FusedIB-SE, which integrate the SENet attention mechanism, provided in this embodiment of the invention.

[0048] Figure 6 These are confusion matrices under different models. Detailed Implementation

[0049] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0050] like Figure 1 As shown in the figure, this embodiment provides a method for real-time assessment of fish feeding intensity, the method comprising:

[0051] Step S1: Use a hydrophone to collect the feeding audio of the fish, then divide the feeding audio of the fish into fixed lengths (e.g., 3s in this embodiment), and label the feeding intensity as four categories: strong, medium, weak, and none, to build a dataset.

[0052] For example, in step S1, a voltage signal is acquired using an LST-SH50 hydrophone, then amplified 1000 times using a VK301, and then the voltage signal is acquired again using a VK-700 at a sampling frequency of 51.2 kHz. Finally, the voltage signal is converted into an audio signal and downsampled to 22.5 kHz. The above data was acquired through legal and compliant means.

[0053] Step S2: Extract the Mel spectrum matrix for each audio segment, and then extract the MFCC spectrum and FBank spectrum from the Mel spectrum matrix.

[0054] For example, in step S2, the audio signal processed in step S1 is first pre-emphasized using formula (1) as follows:

[0055]

[0056] Then, the mel spectrum is extracted from the pre-emphasized audio. The mel scale is a logarithmically compressed scale designed to simulate human auditory perception. Equation (2) describes the logarithmic relationship between sound frequencies and mel frequencies, where... Indicates the Mel frequency. Indicates the actual frequency (unit: Hz).

[0057]

[0058] The Mel filter bank is used to simulate the human auditory system's perception of sound signals, and its mathematical description is shown in equation (3). In this equation, the triangular frequency filter bank consists of 64 bandpass filters, denoted as... .

[0059]

[0060] Where k represents the frequency index in the spectrum, Indicates the index of the Mel filter. Indicates the total number of filters. This represents the index of the center frequency of the m-th Mel filter in the FFT frequency domain.

[0061] When designing Mel filter banks, windowed signals The frequency domain is converted from the time domain by the fast Fourier transform, as shown in equation (4).

[0062]

[0063] In this formula, Represents position in the frequency domain The spectral line index at the position, i represents the frame index of the speech signal, and n represents the index of the time-domain sampling point in the windowed speech signal of the i-th frame.

[0064] Subsequently, through application A Mel filter bank converts the obtained power spectrum into the power spectrum of each filter. The signal energy within is shown in equation (5).

[0065]

[0066] By applying the above methods and procedures, information containing signal energy can be obtained. 3D Mel spectrum matrix.

[0067] The Mel spectrum matrix is ​​then logarithmically transformed to compress the dynamic range of the signal energy, making the values ​​more suitable for subsequent analysis and modeling. This transformation also improves the sensitivity to low-frequency energy variations, reflecting the enhanced responsiveness of the human auditory system to lower frequencies. As shown in Equation (6), the FBank spectrum is obtained by applying the signal energy generated by each Mel filter. It is obtained by taking the logarithm.

[0068]

[0069] The MFCC spectrum is also a refined representation derived from the Mel spectrum. Cepstral coefficients are extracted by applying a DCT transform to the Mel spectrum. Specifically, let... Representing a frame The index is The energy of the Mel filter, let Indicates that the index is in the same frame The MFCC coefficients are then used. The Mel energy is then converted to cepstral coefficients using a DCT transform, as shown in equation (7):

[0070]

[0071] in, This indicates the number of linear Mel filters. Indicates the MFCC coefficient index. Here, is the normalization coefficient, and P is the dimension of MFCC, which is taken as 64 here. When hour, ;when hour After applying the DCT transform described above, the MFCC spectrum can be obtained.

[0072] Finally, Z-score normalization was performed on the obtained MFCC spectrum and FBank spectrum using equation (8).

[0073]

[0074] in This represents the value of the original data point. This represents the average of all data points. This represents the standard deviation of all data points.

[0075] Step S3: Construct the MobileNetV4S-ADEFF-SENet model and train it using the dataset.

[0076] The MobileNetV4S-ADEFF-SENet model is mainly used for 224×224 image recognition tasks. It replaces the Stage1 3×3 convolutions of the MobileNetV4-Small model with ADEFF modules, and replaces the Universal Inverted Bottleneck (UIB) and Fused Inverted Bottleneck (FusedIB) modules with inverted bottleneck modules UIB-SE and FusedIB-SE that integrate the SENet attention mechanism to enhance the expressive power of key feature channels. Its classification head achieves the final classification output through global average pooling and convolutional layers.

[0077] surface MobileNetV4S-ADEFF-SENet architecture

[0078] Wherein, "DWK1" and "DWK2" represent the kernel sizes of the two deep convolutional layers, DeptWise, respectively; "ExpandedDim" represents the channel expansion dimension; "OutputDim" represents the number of output channels; and "Stride" represents the convolution stride.

[0079] In one implementation, the ADEFF module is used to perform feature fusion on the MFCC spectrum and the FBank spectrum to adapt to dual-spectrum input. This is achieved by performing convolution processing on the MFCC spectrum and the FBank spectrum with different kernel sizes to obtain two local features. The two local features are then multiplied element-wise to obtain initial fused features. Channel attention and spatial attention are calculated in parallel on the initial fused features to obtain a coarse-grained attention map. The coarse-grained attention map is then processed using channel rearrangement and group convolution to obtain fine-grained channel weights. The two local features are combined with the fine-grained channel weights to perform dynamic weighted summation, and finally, the fused features are output through 1×1 convolutional linear projection.

[0080] Specifically, see Appendix Figure 3 The specific implementation process of the ADEFF module is as follows:

[0081] First, a 7×7 convolution is applied to the FBank spectrum to capture the long-term dependence of the fine-grained spectrum and global acoustic events. Then, a 3×3 convolution is applied to the MFCC spectrum to preserve the local discriminativeness of its compact representation, generating the first local feature representation respectively. Second local feature representation ;

[0082]

[0083]

[0084] in Indicates the kernel size as The convolutional layer.

[0085] Subsequently, the first local feature representations are fused by element-wise multiplication. Second local feature representation The initial fused feature map is obtained. ;

[0086]

[0087] in, This represents element-wise multiplication. Compared to simple addition, element-wise multiplication can emphasize the common activation region of the two features, highlighting collaborative information.

[0088] To guide subsequent attention-based adaptive fusion, the initial fused feature map is... Channel attention weights are obtained through the channel attention module and the spatial attention module, respectively. Spatial attention weights ;

[0089]

[0090]

[0091] in, This represents a ReLU activation function, while This indicates a cross-channel splicing operation. and These represent global average pooling and global max pooling operations, respectively. To reduce the number of parameters and limit model complexity, the first... Convolution reduces the channel dimension from Reduce to ( (Refers to the reduction ratio), the second one Convolution expands it back We chose to... Set as This reduces the channel dimension to a fixed value of 16.

[0092] Subsequently, channel attention weights Spatial attention weights Perform element-wise multiplication to generate a coarse-grained attention map. This will generate channel-specific coarse-grained weights through a broadcast mechanism. .

[0093]

[0094] To obtain the final fine-grained channel-weighted... coarse-grained attention map Each channel undergoes adaptive optimization based on its corresponding input features. Specifically, the input features are used as guiding information to generate the final channel-specific weights. Specifically, this is achieved through channel rearrangement operations in the coarse-grained attention map. and fusion features The channels are alternately rearranged to achieve interactive learning of multiple features, resulting in fine-grained channel weights. .

[0095]

[0096] Where δ represents the Sigmoid activation function, This indicates a channel rearrangement operation. This represents a grouped convolutional layer with a kernel size of . .

[0097] Fine-grained channel weights Representation of the first local feature Fusion results, fine-grained channel weights With the second local feature representation The fusion results are fused through dynamic weighted summation, and finally... The convolutional layer is linearly projected to obtain the fused features. .

[0098]

[0099] In one implementation, the inverted bottleneck module UIB-SE and FusedIB-SE, which integrate the SENet attention mechanism, are achieved by cascading an SENet attention mechanism module at the output of the inverted bottleneck module UIB and the FusedIB module, thus realizing channel-level feature recalibration. UIB variants include three types: IB (Inverted Bottleneck), ExtraDW, and ConvNext. See appendix. Figure 5 .

[0100] Specifically, the SENet attention mechanism module dynamically learns the dependencies between channels and adaptively adjusts the feature response intensity of each channel, thereby enhancing the model's sensitivity to key feature channels. SENet consists of three operations: compression, activation, and recalibration, and its structure is as follows: As shown. Its implementation process is as follows:

[0101] The first step is compression. Global average pooling is performed on the input feature map. and The width and height of the feature map are respectively compressed in the spatial dimension to generate a file of size [size missing]. The output feature map, which encodes Global information for each channel;

[0102]

[0103] in, This indicates a compression operation. For the first input feature map One channel, This is the output global feature descriptor. and These represent the width and height of the feature map, respectively. These are the spatial coordinates on the feature map.

[0104] The next step is the activation operation, which inputs the global feature descriptor of the output feature map into two fully connected layers, through... The function generates channel-level attention weights, with an output dimension of .

[0105]

[0106] in, Indicates an incentive operation. Here is the channel attention weight vector. and They are two fully connected layers. Represents the ReLU activation function. yes Activation function.

[0107] Finally, the learned channel-level attention weights are applied to the original input feature map to achieve channel-level feature recalibration.

[0108]

[0109] in, This indicates a feature recalibration operation. This is the weighted output feature map. For the original number Feature map of each channel This represents the attention weight corresponding to this channel.

[0110] In one implementation, this invention replaces the original 5×5 depthwise convolutions "DWK1" and "DWK2" in the second ExtraDW module of Stage 5 of the MobileNetV4-Small model, as well as "DWK2" in the first two IB modules, with 3×3 depthwise convolutions to avoid receptive field overflow and computational redundancy. Specifically, the input in this embodiment is a 64×128 audio spectral feature. After downsampling in the first four stages, the feature map size of Stage 5 is reduced to 2×4. At this size, the 5×5 convolution kernels used in the original model cause the receptive field to exceed the feature map boundary, resulting in computational redundancy and decreased feature extraction efficiency. Therefore, this embodiment adjusts all 5×5 convolution kernels in Stage 5 (ExtraDW and subsequent IB modules in rows 11-16 of Table 1) to 3×3 convolution kernels, so that the convolution kernel size matches the feature map size, ensuring the effectiveness of feature extraction.

[0111] During the model training phase, this invention uses the cross-entropy loss function to measure the difference between the model output and the true labels. Let the model's predicted probability distribution for the input samples be... The one-hot encoded vector corresponding to the real label is Where C represents the number of categories, the cross-entropy loss function is defined as:

[0112]

[0113] By minimizing the aforementioned cross-entropy loss function, the model is guided to continuously optimize network parameters during training, thereby improving its ability to discriminate audio spectral features and ultimately enhancing overall classification performance.

[0114] Step S5: During application, a hydrophone is used to collect the feeding audio of the fish. The feeding audio is then divided into segments of fixed length. Two features, MFCC spectrum and FBank spectrum, are extracted from each audio segment. These two features are then input into the trained MobileNetV4S-ADEFF-SENet model for classification and recognition to obtain the fish feeding intensity assessment results.

[0115] To verify the performance of the fish feeding intensity assessment method proposed in this invention, extensive experimental verification was conducted. This embodiment uses the Real-FFIA dataset collected from an actual operational recirculating aquaculture pond in a certain region. The pond has a diameter of 8 meters and a water depth of approximately 1.5 meters, housing approximately 3000 California bass with an average weight of approximately 150 grams per fish. The actual sampling scenario involved complex environmental noises such as pump operation and bubble bursting, which more realistically reflect the acoustic challenges of industrial aquaculture. The software program was written in the PyTorch framework and trained on a computer configured with an NVIDIA RTX 6000. Specific training configurations are as follows... As shown.

[0116] surface Main hardware environment and parameter configuration

[0117] Configuration Model / version CPU Intel(R) Xeon(R) Gold 6248R CPU @3.00GHz GPU NVIDIA RTX 6000 24G Accelerated environment CUDA 11.8 Audio processing library Torchaudio 2.4.1 Optimiser Adamw Learning rate decay strategy CosineAnnealingLR Batch size 128 Peak learning rate 0.008 Training epoch 500 Warmup epochs 5

[0118] Comparison of MobileNetV4S-ADEFF-SENet model with the original MobileNetV4S (i.e., MobileNetV4-Small) model paired with traditional feature fusion methods:

[0119] See appendix Figure 6 On the Real-FFIA dataset, the proposed method maintains this advantage, achieving classification accuracies of 95.76%, 85.54%, 87.67%, and 93.21% for the four categories, respectively, significantly reducing inter-class confusion compared to other methods. These results demonstrate that the combination of the ADEFF module and the SENet mechanism can effectively extract and fuse complementary features, exhibiting stronger noise robustness and superior recognition capabilities in complex acoustic environments and real underwater scenarios.

[0120] To verify the effectiveness of the proposed model structure in assessing fish feeding intensity based on underwater acoustic signals, an ablation experiment of the model structure was designed. The experimental results are as follows: As shown. The experiment uses the lightweight network MobileNetV4S as the base model, and uses the stacked input MFCC spectrum and FBank spectrum as the baseline input form (Stack). Then, the SENet channel attention module and the ADEFF (Adaptive Detail-enhanced Feature Fusion) module are introduced in turn to analyze and compare the model performance step by step.

[0121] First, the baseline model (MobileNetV4S+Stack) using MFCC and FBank spectrum stacked inputs achieves performance of 88.89% accuracy, 88.86% recall, 88.77% precision, and 88.80% F1-Score.

[0122] Building upon this foundation, the introduction of the SENet channel attention mechanism (MobileNetV4S+Stack+SENet) further enhances model performance, achieving accuracy, recall, precision, and F1-score of 89.98%, 89.95%, 89.84%, and 89.88%, respectively. The SENet module, through adaptive weighting of feature channel importance, effectively enhances the response of key frequency bands related to feeding sounds, suppresses underwater noise and invalid channel signals, thereby improving the model's feature selection capability and anti-interference performance.

[0123] When the ADEFF module was introduced into the model (MobileNetV4S+ADEFF), the performance was significantly improved, with accuracy, recall, precision, and F1-score increasing to 90.45%, 90.45%, 90.40%, and 90.34%, respectively. The ADEFF module, through a collaborative mechanism of "coarse-grained feature integration—detail enhancement strategy," dynamically strengthens the key acoustic feature responses in feeding sound signals, optimizes the fusion effect of multi-scale features, makes the model more sensitive to changes in fish feeding intensity, and significantly reduces the impact of environmental noise on the recognition results.

[0124] By simultaneously introducing the ADEFF module and the SENet mechanism (MobileNetV4S+ADEFF+SENet), the model achieved optimal performance, with accuracy, recall, precision, and F1-score of 90.92%, 90.93%, 90.82%, and 90.85%, respectively. This result demonstrates the complementary roles of ADEFF and SENet in feature fusion and attention allocation: the ADEFF module enhances the representational depth of multi-source features, while the SENet mechanism optimizes the response intensity of inter-channel features. Their synergistic effect significantly improves the model's feature extraction capability and class discrimination accuracy in complex noisy environments.

[0125] Comprehensive analysis shows that the MobileNetV4S-ADEFF-SENet model proposed in this invention achieves efficient fusion and adaptive enhancement of acoustic features, significantly outperforming traditional structures and single-feature models. This method effectively improves the model's ability to suppress underwater noise interference and its sensitivity to feeding signal features, providing stable and reliable technical support for the accurate assessment of fish feeding intensity, and possesses good engineering deployability and real-time monitoring application value.

[0126] surface Ablation experimental results of SENet and ADEFF modules

[0127] Methods accuracy Recall rate Accuracy F1-Score MobileNetV4S+Stack 88.89 88.86 88.77 88.80 MobileNetV4S+Stack+SENet 89.98 89.95 89.84 89.88 MobileNetV4S+ADEFF 90.45 90.45 90.40 90.34 MobileNetV4S+ADEFF+SENet 90.92 90.93 90.82 90.85

[0128] Characteristic ablation experiments:

[0129] To further verify the impact of the acoustic feature fusion strategy of this invention on model performance, a feature ablation experiment was designed. The experimental results are as follows: As shown in the figure. The experiments examined the effects of noise-resistant MFCC features, highly correlated FBank features, and their combined use in the ADEFF (Adaptive Detail-enhanced Feature Fusion) module to evaluate the impact of different feature combinations on the model's recognition performance.

[0130] First, when using Mel-frequency cepstral coefficients (MFCC) features as model input (MobileNetV4S+MFCC), the accuracy, precision, recall, and F1-score are 86.85%, 86.80%, 86.70%, and 86.73%, respectively. MFCC features can effectively extract the envelope information and temporal variation characteristics of acoustic signals and exhibit strong anti-interference ability in noisy environments. However, due to the limited expressive power of MFCC in terms of high-frequency details and energy distribution, the model's sensitivity to differences in feeding sound energy remains insufficient.

[0131] When the filter bank energy features (FBank) are used as input (MobileNetV4S+FBank), the model performance is improved to a certain extent, with accuracy, precision, recall, and F1-score reaching 87.79%, 87.73%, 87.50%, and 87.53%, respectively. FBank features can more directly reflect the energy spectrum distribution of underwater acoustic signals, retain rich spectral details, and have a higher correlation with acoustic changes caused by feeding behavior, thereby improving the model's feature discrimination ability.

[0132] When MFCC and FBank features are simultaneously input into the model and fused using the ADEFF module (MobileNetV4S+ADEFF+MFCC+FBank), model performance is significantly improved. At this point, accuracy, precision, recall, and F1-score reach 90.45%, 90.45%, 90.40%, and 90.34%, respectively, all showing substantial improvements compared to single feature input. The ADEFF module, through a collaborative mechanism of "coarse-grained feature integration—detail enhancement strategy," adaptively mines and fuses the noise-resistant advantages of MFCC with the highly correlated features of FBank, enabling the model to achieve efficient complementarity and enhanced representation of time-frequency information at the feature layer. Simultaneously, the ADEFF module can dynamically suppress the propagation of noise interference features, thereby further improving the model's robustness and accuracy in complex underwater acoustic environments.

[0133] In summary, MFCC and FBank features are significantly complementary in terms of noise resistance and correlation. The ADEFF module achieves deep integration and dynamic enhancement of these two features through an adaptive fusion mechanism, effectively improving the model's feature representation ability and noise resistance. Experimental results fully verify the effectiveness of the multi-feature fusion strategy proposed in this invention, providing reliable technical support and performance assurance for underwater acoustic signal analysis and fish feeding intensity assessment.

[0134] surface Ablation experimental results of MFCC and FBank features

[0135] Methods Accuracy (%) Accuracy (%) Recall rate (%) F1-Score (%) MobileNetV4S+MFCC 86.85 86.80 86.70 86.73 MobileNetV4S+FBank 87.79 87.73 87.50 87.53 MobileNetV4S+ADEFF+MFCC+FBank 90.45 90.45 90.40 90.34

[0136] To further verify the comprehensive advantages of the proposed method in terms of model accuracy and lightweight performance, the performance of the proposed MobileNetV4S-ADEFF-SENet model was compared with that of mainstream convolutional neural network structures ResNet18, ResNet34, and EfficientNet-b1. The experimental results are as follows: As shown. The comparison metrics include accuracy, precision, recall, F1-Score, number of model parameters, and computational cost.

[0137] The experimental results show that the traditional residual network ResNet18 achieves accuracy, recall, precision, and F1-score of 88.11%, 88.08%, 88.04%, and 88.02%, respectively, with 11.18 M parameters and a computational cost of 290.77 MFLOPs. While possessing some recognition capability, its complex model structure and high computational cost make it challenging. ResNet34, with its increased network depth, slightly improves accuracy (reaching 89.36%), but the number of parameters and computational cost increase to 21.29 M and 593.58 MFLOPs, respectively, making it insufficient for real-time underwater acoustic monitoring deployment requirements. EfficientNet-b1, representing a lightweight approach, achieves an accuracy of 89.05%, with 12.06 M parameters and a computational cost of 105.38 MFLOPs. While it balances some efficiency and performance, its structural complexity remains high.

[0138] In comparison, the MobileNetV4S baseline model significantly reduces the number of parameters and computational cost, to only 2.51M and 30.90 MFLOPs, respectively, demonstrating extremely high lightweight characteristics. Furthermore, this invention introduces the ADEFF (Adaptive Enhanced Detail Feature Fusion) module and the SENet channel attention mechanism (MobileNetV4S-ADEFF-SENet), resulting in a significant performance improvement. Accuracy, precision, recall, and F1-score reach 90.92%, 90.93%, 90.82%, and 90.85%, respectively, representing an accuracy improvement of approximately 1.56 percentage points compared to ResNet34, while the number of parameters is only about 1 / 8 and the computational cost is reduced by more than 13 times. These results demonstrate that the method of this invention achieves superior recognition accuracy and stability compared to mainstream models while maintaining extremely low computational complexity and storage cost.

[0139] The model in this invention fully embodies efficiency and practicality in its structural design: the ADEFF module enhances the expressive power of key time-frequency features in acoustic signals through adaptive multi-scale feature fusion; the SENet mechanism improves feature channel selectivity through channel attention weighting, effectively suppressing noise interference. The synergistic effect of these two mechanisms enables the model to maintain high feature discrimination and robustness even in complex underwater acoustic environments.

[0140] A comprehensive comparison shows that MobileNetV4S-ADEFF-SENet achieves an ideal balance between accuracy, parameter count, and computational cost, possessing both high precision and high efficiency. This model is not only suitable for assessing fish feeding intensity in experimental environments but also feasible for deployment in edge computing devices or real-time monitoring systems, providing reliable technical support and application prospects for behavior recognition and feeding strategy optimization in intelligent aquaculture.

[0141] surface Comparison of feeding intensity assessment results using commonly used audio classification models

[0142] Methods Accuracy (%) Accuracy (%) Recall rate (%) F1-Score (%) Number of parameters (M) Computational cost (MFLOPs) ResNet18 88.11 88.04 88.08 88.02 11.18 290.77 ResNet34 89.36 89.31 89.29 89.29 21.29 593.58 Efficient-b1 89.05 88.92 88.92 88.91 12.06 105.38 MobileNetV4S+Stack 88.89 88.86 88.77 88.80 2.51 30.90 MobileNetV4S+ADEFF+SENet 90.92 90.93 90.82 90.85 2.54 43.90

[0143] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for real-time assessment of fish feeding intensity based on multi-feature fusion, characterized in that, The method includes: The feeding audio of fish is collected using a hydrophone, and then the feeding audio is divided into segments of fixed length. Extract the Mel spectrum matrix for each audio segment, and then extract the MFCC spectrum and FBank spectrum from the Mel spectrum matrix; The MFCC spectrum and FBank spectrum were input into the MobileNetV4S-ADEFF-SENet model for classification and identification to obtain the fish feeding intensity assessment results.

2. The method according to claim 1, characterized in that, The MobileNetV4S-ADEFF-SENet model replaces the 3×3 convolutions in Stage 1 of the MobileNetV4-Small model with ADEFF modules, replaces the inverted bottleneck module UIB with the inverted bottleneck module UIB-SE which integrates the SENet attention mechanism, and replaces the fused inverted residual bottleneck module FusedIB with the fused inverted residual bottleneck module Fused-SE which integrates the SENet attention mechanism.

3. The method according to claim 2, characterized in that, The ADEFF module is used to perform feature fusion on the MFCC spectrum and the FBank spectrum. It obtains two local features by performing convolution processing on the MFCC spectrum and the FBank spectrum with different kernel sizes. The two local features are then multiplied element-wise to obtain the initial fused features. Channel attention and spatial attention are calculated in parallel on the initial fused features to obtain a coarse-grained attention map. The coarse-grained attention map is then processed by channel rearrangement and group convolution to obtain fine-grained channel weights. The two local features are combined with the fine-grained channel weights to perform dynamic weighted summation. Finally, the fused features are output by 1×1 convolutional linear projection.

4. The method according to claim 3, characterized in that, The specific implementation process of the ADEFF module is as follows: First, a 7×7 convolution is applied to the FBank spectrum to capture the long-term dependence of fine-grained spectrum and global acoustic events, and a 3×3 convolution is applied to the MFCC spectrum to preserve the local discriminativeness of its compact representation, thereby generating the first local feature representation and the second local feature representation respectively. Subsequently, the first local feature representation and the second local feature representation are fused by element-wise multiplication to obtain the initial fused feature map; The initial fused feature map is processed by a channel attention module and a spatial attention module to obtain channel attention weights and spatial attention weights, respectively. Subsequently, the channel attention weights and spatial attention weights are multiplied element-wise to generate a coarse-grained attention map; By rearranging the channels alternately between the coarse-grained attention map and the fused features through channel rearrangement, interactive learning of multiple features is achieved, resulting in fine-grained channel weights. The fusion results of fine-grained channel weights and the first local feature representation, and the fusion results of fine-grained channel weights and the second local feature representation, are fused through dynamic weighted summation. Finally, after... The convolutional layer is linearly projected to obtain the fused features.

5. The method according to claim 2, characterized in that, The inverted bottleneck module UIB-SE, which integrates the SENet attention mechanism, and the fused inverted residual bottleneck module FusedIB-SE, which integrates the SENet attention mechanism, each have an SENet attention mechanism module connected in series at the output of the inverted bottleneck module UIB and the fused inverted residual bottleneck module FusedIB, respectively.

6. The method according to claim 5, characterized in that, Variants of the inverted bottleneck module UIB include IB, ExtraDW, and ConvNext.

7. The method according to claim 5, characterized in that, The SENet attention mechanism module dynamically learns the dependencies between channels and adaptively adjusts the feature response intensity of each channel, thereby enhancing the model's sensitivity to key feature channels. The implementation process is as follows: The first step is compression. Global average pooling is performed on the input feature map. and The width and height of the feature map are respectively compressed in the spatial dimension to generate a file of size [size missing]. The output feature map, which encodes Global information for each channel; The next step is the activation operation, which inputs the global feature descriptor of the output feature map into two fully connected layers, through... The function generates channel-level attention weights; Finally, the learned channel-level attention weights are applied to the original input feature map to achieve channel-level feature recalibration.

8. The method according to claim 2, characterized in that, In the Stage 5 of the MobileNetV4S-ADEFF-SENet model, all depthwise convolutional layers in the second ExtraDW-SE module and the first two IB-SE modules use 3×3 depthwise convolutions.

9. The method according to claim 1, characterized in that, Before being input into the MobileNetV4S-ADEFF-SENet model, the MFCC spectrum and FBank spectrum are Z-score normalized.

10. A real-time fish feeding intensity assessment system implementing the method of any one of claims 1-9, characterized in that, include: The data acquisition module is responsible for using a hydrophone to collect the feeding audio of the fish, and then dividing the feeding audio into segments of fixed length; The feature extraction module is responsible for extracting the Mel spectrum matrix for each audio segment, and then extracting the MFCC spectrum and FBank spectrum from the Mel spectrum matrix; The classification and recognition module is responsible for inputting the MFCC spectrum and FBank spectrum into the MobileNetV4S-ADEFF-SENet model for classification and recognition, and obtaining the fish feeding intensity assessment results.

Images