A method and system for monitoring feeding behavior of large yellow croaker based on target detection

Abstract

The application discloses a kind of based on target detection's large yellow croaker feeding behavior monitoring method and system, the method includes: the feature extraction of real-time acquisition large yellow croaker feeding video, obtain visual behavior characteristics, and the feature extraction of real-time acquisition large yellow croaker acoustic signal, obtain acoustic behavior characteristics;By preset cross-modal phase synchronization network, visual behavior characteristics and acoustic behavior characteristics are time aligned, obtain first feature pair, and by preset attention mechanism, first feature pair is semantically aligned, obtain second feature pair;Second feature pair is fused, and obtains multimodal feature, and by preset first feeding behavior identification model, according to multimodal feature, output feeding behavior identification result;When feeding behavior result belongs to abnormal feeding behavior category, according to preset grading early warning mechanism, feeding behavior abnormal early warning is carried out.The application can improve the accuracy of large yellow croaker feeding behavior monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent monitoring, and in particular to a large yellow croaker feeding behavior monitoring method and system based on target detection. BACKGROUND

[0002] In the field of modern aquaculture, as an important economic fish, the feeding state of large yellow croaker is directly related to the breeding benefit, feed cost and fish health. By monitoring the feeding behavior in real time, the breeder can scientifically optimize the feeding strategy to avoid slow growth due to insufficient feeding or feed waste and water pollution caused by overfeeding. More importantly, abnormal feeding behavior is often an early signal of fish stress, disease or water quality deterioration. Timely detection and intervention of these abnormal behaviors have great significance for preventing disease outbreaks, reducing mortality, improving breeding survival rate and fish quality, and are a key technical link to achieve precise, efficient and green breeding.

[0003] Current large yellow croaker feeding behavior monitoring technology is often based on single visual monitoring or acoustic monitoring. When abnormal feeding behavior is detected, a single global warning technology is also used. However, the current large yellow croaker feeding behavior monitoring technology still has obvious defects. First, behavior recognition based on a single signal source will result in missing information due to the limitations of single technology, thereby affecting the accuracy of the final feeding behavior recognition result. For example, when monitoring based on a single visual signal, the image information is difficult to extract due to the blocking of bait or water debris, low water visibility, etc. When monitoring based on a single audio feature, the small part of the abnormal feeding fish population is ignored due to the higher volume of the majority of healthy feeding fish population. Second, due to the current large scale of large yellow croaker breeding, feeding behavior recognition based on a single cloud model will seriously affect the calculation efficiency. Finally, the single global warning technology has a rigid warning mechanism, which is not conducive to the intelligent management of the breeding farm. SUMMARY

[0004] The present application provides a large yellow croaker feeding behavior monitoring method and system based on target detection, which can improve the accuracy of large yellow croaker feeding behavior monitoring.

[0005] In a first aspect, the present application provides a large yellow croaker feeding behavior monitoring method based on target detection, comprising:

[0006] extracting features from the real-time collected large yellow croaker feeding video to obtain visual behavior features, and extracting features from the real-time collected large yellow croaker acoustic signals to obtain acoustic behavior features;

[0007] time align the visual behavior feature and the acoustic behavior feature through a preset cross-modal phase synchronization network to obtain a first feature pair, and perform semantic alignment on the first feature pair through a preset attention mechanism to obtain a second feature pair;

[0008] perform feature fusion on the second feature pair to obtain a multi-modal feature, and output a feeding behavior recognition result according to the multi-modal feature through a preset first feeding behavior recognition model; wherein the first feeding behavior recognition model is a lightweight model, and is updated by migrating the knowledge of a pre-trained second feeding behavior recognition model to the first feeding behavior recognition model through a knowledge distillation technology; the second feeding behavior recognition model is generated according to a preset large database training;

[0009] when the feeding behavior result belongs to an abnormal feeding behavior category, feeding behavior abnormality warning is performed according to a preset hierarchical warning mechanism.

[0010] The embodiment of the application extracts fish behavior key information from video and sound signals respectively, constructs a multi-modal data base, aligns the visual behavior feature and the acoustic behavior feature in time through a cross-modal phase synchronization network, solves the problem of audio-visual signal time asynchronization, ensures feature time alignment, performs semantic alignment through a preset attention mechanism, eliminates modal semantic difference, and improves feature correlation; a lightweight feeding behavior recognition model is obtained through a knowledge distillation technology to perform feeding behavior recognition, so as to deploy an edge node to consider the efficiency and accuracy of feeding behavior recognition; through a hierarchical warning mechanism, feedback of abnormal feeding behavior is performed hierarchically, layered response is performed according to abnormal severity, and excessive alarm is avoided. Compared with the prior art, the application can improve the accuracy of large yellow croaker feeding behavior monitoring.

[0011] Further, the visual behavior feature is obtained by performing feature extraction on the real-time collected large yellow croaker feeding video, specifically:

[0012] frame-level feeding image sequences are obtained by performing frame processing on the real-time collected large yellow croaker feeding video; wherein the frame-level feeding image sequences include a plurality of time-continuous frame-level feeding image sequences;

[0013] the frame-level feeding image is decomposed into a plurality of spatial frequency bands through a Laplacian pyramid, and a band-pass filter processing is performed on the high frequency band in the plurality of spatial frequency bands to obtain a motion magnification frame sequence; wherein the high frequency band is a plurality of spatial frequency bands containing image details and motion information;

[0014] According to the motion amplification frame sequence, spatial-temporal key points are extracted, and the spatial-temporal key points are grouped according to a semantic mask of the spatial-temporal key points to obtain a structured key point sequence; wherein the spatial-temporal key points include a mouth, a pectoral fin and a caudal peduncle;

[0015] The structured key point sequence is compared with a preset feeding-non-feeding prototype library to extract feeding behavior features;

[0016] The feeding behavior features are physically semantically mapped to obtain visual behavior features; wherein the visual behavior features include a fish mouth opening and closing frequency, a body swing amplitude and a swim speed quantitative index.

[0017] The embodiment of the application amplifies high-frequency motion details and suppresses background noise through Laplacian pyramid and band-pass filtering; the structured key point sequence is obtained by grouping the spatial-temporal key points, focusing on anatomical sites strongly related to feeding, and structuring the behavior; the pre-defined behavior template-based rapid classification is realized by comparing the structured key point sequence with the preset feeding-non-feeding prototype library, reducing the amount of calculation; the image features are converted into quantifiable biomechanical indexes through the preset physical semantic mapping, improving the feature interpretability.

[0018] Further, the real-time collected large yellow croaker acoustic signals are subjected to feature extraction to obtain acoustic behavior features, specifically:

[0019] The real-time collected large yellow croaker acoustic signals are continuously input into a preset adaptive gate to extract signal energy and kurtosis values in the large yellow croaker acoustic signals, and the initial feeding signal segment is screened according to the signal energy and kurtosis values;

[0020] The initial feeding signal segment is subjected to two-classification through a preset lightweight time sequence convolution network to obtain a final feeding signal segment;

[0021] The final feeding signal segment is subjected to tensor decomposition to obtain a time tensor, a frequency tensor and a space tensor, and a frequency spectrum type and a space response are extracted according to the time tensor, the frequency tensor and the space tensor;

[0022] According to the frequency spectrum type and the space response, an individual fingerprint vector is constructed, and through a preset incremental spectrum clustering algorithm, the individual feeding acoustic signal segment of the large yellow croaker is separated from the fish school according to the individual fingerprint vector to obtain the individual feeding acoustic signal segment of each large yellow croaker;

[0023] The individual feeding acoustic signal segment is subjected to feature extraction to obtain the acoustic behavior features of each large yellow croaker.

[0024] The embodiment of the application captures the feeding sound pulse characteristics through signal energy and kurtosis value, quickly screens effective signal segments, performs two classifications through a preset lightweight time sequence convolution network, distinguishes feeding and non-feeding sounds, and reduces the complexity of subsequent processing, separates the multi-dimensional characteristics of acoustic signals through tensor decomposition, avoids feature coupling, provides data preparation for subsequent individual-level feature extraction, constructs individual fingerprint vectors, and obtains individual feeding acoustic signal segments of each large yellow croaker through a preset incremental spectrum clustering algorithm, thereby separating individuals from fish group mixed sound, solving the multi-fish interference problem, and realizing accurate feature extraction.

[0025] Further, the visual behavior feature and the acoustic behavior feature are time-aligned through a preset cross-modal phase synchronization network to obtain a first feature pair, specifically:

[0026] The visual behavior feature and the acoustic behavior feature are respectively converted into visual phase latent variables and acoustic phase latent variables of the same dimension through the preset cross-modal phase synchronization network.

[0027] The visual phase latent variables and the acoustic phase latent variables are time-aligned through a preset timestamp to construct the first feature pair.

[0028] The visual behavior feature and the acoustic behavior feature are respectively converted into visual phase latent variables and acoustic phase latent variables of the same dimension through the preset cross-modal phase synchronization network, the heterogeneous features are mapped to a unified mathematical space, and time alignment is facilitated; time alignment is realized through a preset timestamp to realize consistency of different modal features in the time dimension.

[0029] Further, the first feature pair is semantically aligned through a preset attention mechanism to obtain a second feature pair, specifically:

[0030] The visual phase latent variables and the acoustic phase latent variables in the first feature pair are respectively mapped to a preset hyperbolic space according to a preset feeding semantic prototype dictionary to calculate the semantic distance between the first feature pair and each prototype in the feeding semantic prototype dictionary; wherein the feeding semantic prototype dictionary is constructed in a hyperbolic Poincare ball space; each prototype corresponds to a posture-voiceprint joint semantic of a large yellow croaker.

[0031] The visual phase latent variables and the acoustic phase latent variables with the smallest semantic distance are matched one by one according to the semantic distance through a preset contrastive attention mechanism to obtain the second feature pair.

[0032] The embodiment of the application uses hyperbolic geometric expression hierarchical semantics to enhance the semantic consistency between modalities through hyperbolic space semantic distance calculation.

[0033] Further, the second feature pair is fused to obtain a multi-modal feature, specifically:

[0034] The visual phase latent variable and the acoustic phase latent variable in the second feature pair are respectively compressed to reconstruct the visual phase latent variable and the acoustic phase latent variable into a two-dimensional visual feature map and a two-dimensional acoustic map, respectively.

[0035] The two-dimensional visual feature map and the two-dimensional acoustic map are spliced in the channel dimension to obtain a multi-modal feature map.

[0036] The multi-modal feature map is adaptively fused by a preset convolution block to obtain a multi-modal feature.

[0037] The embodiment of the application preserves spatial structure information by feature map reconstruction and channel splicing, avoiding feature flattening loss; and dynamically adjusts the contributions of visual and acoustic features by adaptive weight fusion.

[0038] Further, the knowledge of the pre-trained second feeding behavior recognition model is migrated to the first feeding behavior recognition model through a knowledge distillation technique, specifically:

[0039] Within a preset time period, the feeding behavior class probability distribution output by the second feeding behavior recognition model and the feeding behavior class probability distribution output by the first feeding behavior recognition model are obtained, and the KL divergence between the outputs of the first feeding behavior recognition model and the second feeding behavior recognition model is calculated according to the feeding behavior class probability distributions of the two.

[0040] The cross-entropy loss of the first feeding behavior recognition model is calculated according to the feeding behavior probability distribution of the first feeding behavior recognition model and the pre-stored true probability distribution value.

[0041] The overall loss function is constructed according to the KL divergence and the cross-entropy, and the first feeding behavior recognition model is updated according to the overall loss function through the knowledge distillation technique.

[0042] The embodiment of the application updates the model through the joint loss of the KL divergence and the cross-entropy, constrains the lightweight model to learn the output distribution of the teacher model and the real label at the same time, and prevents overfitting during distillation.

[0043] Further, the second feeding behavior recognition model is trained according to a preset large database, specifically:

[0044] The second feeding behavior recognition model is trained through self-supervised contrast learning according to a preset large database; wherein the large database is stored by storing pre-collected large yellow croaker acoustic data, large yellow croaker image data, environmental variables and historical feeding behavior recognition results through cold and hot hierarchical storage technology.

[0045] The embodiment of the application trains the model through self-supervised contrast learning to enhance the generalization of the model, and stores the data hierarchically to optimize the reading efficiency of high-frequency access data through the cold and hot storage technology of the large database.

[0046] Further, when the feeding behavior result belongs to the abnormal feeding behavior category, the feeding behavior abnormality early warning is performed according to the preset hierarchical early warning mechanism, specifically:

[0047] In a preset time period, the latest feeding behavior result is obtained, if the feeding behavior result belongs to the abnormal feeding behavior category, the corresponding bait feeding speed control instruction and oxygen pump control instruction are issued according to the abnormal feeding behavior category in the feeding behavior result and the preset bait-behavior mapping table; wherein the abnormal feeding category includes food stealing, weak feeding, food stopping, abnormal chewing and pathological feeding;

[0048] If the latest feeding behavior result still belongs to the abnormal feeding behavior category in the next time period, the corresponding feeding information and the breeding location information are sent to the staff.

[0049] The embodiment of the application maps the behavior category to the control instruction through the bait-behavior mapping table to realize closed-loop control; through two-level early warning, the automatic response is preferred, and the manual intervention is required only when the abnormality is continuous, thereby reducing the operation and maintenance cost.

[0050] In a second aspect, the embodiment of the application provides a large yellow croaker feeding behavior monitoring system based on target detection, which comprises a behavior feature acquisition module, a feature pair acquisition module, a feeding behavior recognition module and a behavior abnormality early warning module, wherein,

[0051] The behavior feature acquisition module is used for extracting features from the real-time collected large yellow croaker feeding video to obtain visual behavior features, and extracting features from the real-time collected large yellow croaker acoustic signal to obtain acoustic behavior features;

[0052] The feature pair acquisition module is used for time aligning the visual behavior features and the acoustic behavior features through a preset cross-modal phase synchronization network to obtain a first feature pair, and performing semantic alignment on the first feature pair through a preset attention mechanism to obtain a second feature pair;

[0053] The feeding behavior recognition module is used to perform feature fusion on the second feature pair to obtain multimodal features, and output feeding behavior recognition results based on the multimodal features using a preset first feeding behavior recognition model; wherein, the first feeding behavior recognition model is a lightweight model, and is updated by transferring knowledge from a pre-trained second feeding behavior recognition model to the first feeding behavior recognition model through knowledge distillation technology; the second feeding behavior recognition model is generated by training on a preset large database;

[0054] The abnormal behavior warning module is used to issue an abnormal eating behavior warning according to a preset graded warning mechanism when the eating behavior result belongs to the abnormal eating behavior category.

[0055] This invention employs a behavior feature acquisition module to extract key information about fish behavior from video and audio signals, constructing a multimodal data foundation. A feature pair acquisition module, using a cross-modal phase synchronization network, performs time alignment between the visual and acoustic behavior features, resolving the time asynchrony issue between audiovisual signals and ensuring feature time alignment. A preset attention mechanism is used for semantic alignment, eliminating semantic differences between modalities and improving feature relevance. A feeding behavior recognition module utilizes knowledge distillation technology to obtain a lightweight feeding behavior recognition model for feeding behavior identification, thus balancing efficiency and accuracy through edge node deployment. Finally, a behavior anomaly warning module, based on a tiered warning mechanism, provides feedback on abnormal feeding behavior in stages, responding in layers according to the severity of the anomaly to avoid over-alarming.

[0056] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of a method for monitoring the feeding behavior of large yellow croaker based on target detection, provided in an embodiment of the present invention.

[0058] Figure 2 This is a structural diagram of a large yellow croaker feeding behavior monitoring device based on target detection, provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Example 1:

[0061] like Figure 1 As shown in the figure, a method for monitoring the feeding behavior of large yellow croaker based on target detection is provided by an embodiment of the present invention, including the following steps:

[0062] S11, extract features from the real-time acquired large yellow croaker feeding video to obtain visual behavior features, and extract features from the real-time acquired large yellow croaker acoustic signal to obtain acoustic behavior features;

[0063] Preferably, a synchronously triggered dual-mode camera array is deployed above the aquaculture pond. First, a multispectral polarization camera is used to acquire 8-band polarization images of 450 nm–700 nm to enhance the fish's outline in turbid water. Second, an event camera is used to record pixel-level brightness change events at a time resolution of 1 kHz, directly capturing micro-movements such as high-speed tail wagging and instantaneous mouth opening.

[0064] In this embodiment, feature extraction is performed on the real-time acquired large yellow croaker feeding video to obtain visual behavior features. Specifically, the real-time acquired large yellow croaker feeding video is processed into frames to obtain a frame-level feeding image sequence; wherein, the frame-level feeding image sequence includes several temporally consecutive frame-level feeding image sequences; the frame-level feeding images are decomposed into multi-level spatial frequency bands using a Laplacian pyramid, and the high-frequency bands in the multi-level spatial frequency bands are bandpass filtered to obtain a motion-magnified frame sequence; wherein, the high-frequency bands are multi-level spatial frequency bands containing image details and motion... The system extracts several spatial frequency bands of motion information; based on the motion magnification frame sequence, it extracts spatiotemporal key points and groups them according to the semantic mask of the spatiotemporal key points to obtain a structured key point sequence; wherein the spatiotemporal key points include the mouth, pectoral fins, and caudal peduncle; the structured key point sequence is compared with a preset feeding-non-feeding prototype library to extract feeding behavior features; the feeding behavior features are subjected to physical semantic mapping to obtain visual behavior features; wherein the visual behavior features include fish mouth opening and closing frequency, body swaying amplitude, and swimming speed quantification index.

[0065] In one specific embodiment, feature extraction is performed on the real-time acquired feeding video of large yellow croaker to obtain visual behavioral features, specifically:

[0066] First, using the original 60fps image sequence as input, each frame is decomposed into 4 spatial frequency bands through the Laplacian pyramid; the high frequency band is bandpass filtered by 0.4–2.5Hz to amplify the small motion amplitude by 8–12 times, resulting in the "motion magnification frame sequence"; the low frequency band retains its original amplitude value to preserve the background and fish outline.

[0067] Furthermore, a lightweight YOLO-Nano was run on the motion magnification frame sequence to detect three semantic masks: mouth, pectoral fin, and caudal peduncle. 32-dimensional spatiotemporal key points (ST-Key) were extracted in each mask: x, y, t three-dimensional position + motion direction angle + pixel-level confidence. The number of key points was dynamically adjusted by adaptive nonmaximum suppression, and 15-25 key points were sufficient to cover the key poses of a single fish.

[0068] Furthermore, a temporal prototype library is constructed using 2 million fine-grained segments of "feeding / non-feeding" data; the current 0.5s ST-Key sequence is compared with the prototype library using cosine similarity, and a 128-dimensional "prototype residual feature" is output; to reduce computational cost, "locality-sensitive hashing + product quantization" is used to compress the prototype library to 8MB, with a single comparison time of <1ms.

[0069] Furthermore, the prototype residual features are reduced to 32 dimensions through 1×1 convolution, which is the final visual behavior feature.

[0070] In this embodiment, feature extraction is performed on the real-time acquired acoustic signals of large yellow croaker to obtain acoustic behavior features. Specifically, the real-time acquired acoustic signals of large yellow croaker are continuously input into a preset adaptive gating system to extract the signal energy and kurtosis value from the acoustic signals. Based on the signal energy and kurtosis value, an initial feeding signal segment is selected. The initial feeding signal segment is binary classified using a preset lightweight temporal convolutional network to obtain a final feeding signal segment. The final feeding signal segment is decomposed into a tensor to obtain a time tensor, a frequency tensor, and a spatial tensor. Based on the time tensor, frequency tensor, and spatial tensor, a frequency spectrum and a spatial response are extracted. Based on the frequency spectrum and spatial response, an individual fingerprint vector is constructed. Using a preset incremental spectral clustering algorithm, the individual feeding acoustic signal segments of large yellow croakers are separated from the fish population based on the individual fingerprint vectors to obtain the individual feeding acoustic signal segment of each large yellow croaker. Feature extraction is performed on the individual feeding acoustic signal segments to obtain the acoustic behavior features of each large yellow croaker.

[0071] In one specific embodiment, acoustic behavior features are extracted from the real-time acquired acoustic signals of large yellow croaker. Specifically, a microphone array is constructed, and the sound source localization technology (TDOA) is used to locate the sound source. Then, the individual position information from visual tracking is combined to perform cross-modal individual-sound association.

[0072] Furthermore, a 1 / 3 octave band filter is applied to the single-channel signal to generate a 24-dimensional logarithmic energy envelope; an "energy-kurtosis dual-threshold sliding window" is introduced: when the window energy is greater than the background and 6 dB and the kurtosis is greater than 5, it is marked as a candidate feeding window; a 1D-TCN (temporal convolutional network) is used for binary classification to confirm whether it is a real feeding pulse train, and the start and end frame indices are output;

[0073] Furthermore, a third-order tensor is constructed using the "time × frequency × space" framework, and an online tensor decomposition algorithm with sparse nonnegativity constraints is employed to obtain temporal activation, frequency spectrum, and mixed channel spatial response.

[0074] Furthermore, individual fingerprint vectors are constructed based on frequency spectrum and spatial response. Incremental spectral clustering is used to accurately separate the acoustic signals of individual large yellow croakers from the fish population, and the feeding acoustic behavior characteristics of each fish are output for subsequent abnormal behavior identification.

[0075] S12, through a preset cross-modal phase synchronization network, the visual behavioral features and the acoustic behavioral features are time-aligned to obtain a first feature pair, and through a preset attention mechanism, the first feature pair is semantically aligned to obtain a second feature pair;

[0076] In this embodiment, the visual behavior features and the acoustic behavior features are time-aligned using a preset cross-modal phase synchronization network to obtain a first feature pair. Specifically, the visual behavior features and the acoustic behavior features are converted into visual phase latent variables and acoustic phase latent variables of the same dimension using the preset cross-modal phase synchronization network; the visual phase latent variables and acoustic phase latent variables are time-aligned using a preset timestamp to construct the first feature pair.

[0077] In one specific embodiment, the visual behavior features are convolved and encoded into 64-dimensional visual phase latent variables through the visual branch of the cross-modal Transformer model; the acoustic behavior features are convolved and encoded into 64-dimensional acoustic phase latent variables through a learnable bandpass filter bank through the acoustic branch of the cross-modal Transformer model.

[0078] Furthermore, the visual phase latent variable and acoustic phase latent variable are input into a complex multiplication layer to output the instantaneous phase difference; the number of sampling points that need to be shifted forward / backward is directly predicted through a complex convolutional gating network.

[0079] Furthermore, a self-supervised contrastive loss is used to maximize the cosine similarity after alignment and minimize the similarity of unaligned segments.

[0080] In this embodiment, the step of semantically aligning the first feature pair to obtain the second feature pair through a preset attention mechanism specifically involves: mapping the visual phase latent variables and acoustic phase latent variables in the first feature pair to a preset hyperbolic space according to a preset feeding semantic prototype dictionary, so as to calculate the semantic distance between the first feature pair and each prototype in the feeding semantic prototype dictionary; wherein, the feeding semantic prototype dictionary is constructed in hyperbolic Poincaré sphere space; each prototype corresponds to a posture-voiceprint joint semantic of a large yellow croaker; and through a preset contrastive attention mechanism, matching the visual phase latent variables and acoustic phase latent variables with the smallest semantic distance one by one according to the semantic distance to obtain the second feature pair.

[0081] In one specific embodiment, using 500,000 multimodal fragments with "known identities" as the base library, interpretable prototypes are learned in the hyperbolic Poincaré sphere model. Each prototype corresponds to a joint semantic of "feeding intensity-posture-voiceprint" to construct a hyperbolic space prototype dictionary.

[0082] Furthermore, the input aligned first feature pairs are mapped to hyperbolic space by hyperbolic mapping of the visual phase latent variable and acoustic phase latent variable of each fish, respectively, and the hyperbolic distance with all prototypes is calculated.

[0083] Furthermore, based on an attention mechanism, a shared semantic vector is generated with the goal of visual-acoustic-prototype triadic consistency, serving as a second feature pair.

[0084] Preferably, the prototype center is updated periodically using an exponential moving average, and prototype fission is triggered when the prototype nearest neighbor entropy exceeds the limit, in order to maintain the robustness and interpretability of online semantic alignment.

[0085] S13, perform feature fusion on the second feature pair to obtain multimodal features, and output the feeding behavior recognition result based on the multimodal features using a preset first feeding behavior recognition model; wherein, the first feeding behavior recognition model is a lightweight model, and is updated by transferring knowledge from a pre-trained second feeding behavior recognition model to the first feeding behavior recognition model through knowledge distillation technology; the second feeding behavior recognition model is generated by training on a preset large database;

[0086] In this embodiment, the feature fusion of the second feature pair to obtain multimodal features specifically involves: compressing the visual phase latent variable and the acoustic phase latent variable in the second feature pair to reconstruct them into a two-dimensional visual feature map and a two-dimensional acoustic map, respectively; concatenating the two-dimensional visual feature map and the two-dimensional acoustic map along the channel dimension to obtain a multimodal feature map; and performing adaptive weight fusion on the multimodal feature map using a preset convolutional block to obtain the multimodal features.

[0087] In one specific embodiment, the feature fusion of the second feature pair to obtain multimodal features is performed as follows: First, a 1×1 convolutional layer is used to compress the visual phase latent variable, reducing the number of input channels from the original C_v to 64 channels; the convolutional kernel size is fixed at 1×1, the stride is 1, and there is no padding operation to ensure that the spatial dimension H×W remains unchanged; the compressed output dimension is [T, 64, H, W], where T is the time step, and H and W are the feature map height and width, respectively;

[0088] Furthermore, the acoustic phase latent variables are upsampled to the target size of 128×128 using a bilinear interpolation algorithm; a 1×1 convolutional layer is used to unify the number of channels, compressing the number of input channels from C_a to 64 channels; the output is reconstructed into a two-dimensional acoustic feature map with dimensions [T, 64, 128, 128].

[0089] Furthermore, the spatial dimensions of the visual feature map (H×W) and the acoustic feature map (128×128) are uniformly adjusted to 128×128 and spliced ​​along the channel axis to generate a multimodal feature map with dimensions [T, 128, 128, 128].

[0090] Furthermore, spatial local features of the multimodal feature map are extracted through the first convolutional layer, and the importance weights of each channel are calculated based on the channel attention module;

[0091] Furthermore, the output of the second convolutional layer is multiplied with the channel weights channel by channel to finally output a multimodal fusion feature with dimensions [T, 128, 128, 128].

[0092] It should be noted that the structure of the first convolutional layer is as follows: it uses a 3×3 convolutional kernel, and the number of input and output channels is 128; pixel padding (padding=1) is used to keep the feature map space size unchanged; and the activation function is ReLU non-linear transformation.

[0093] The second convolutional layer has the following structure: it uses a 1×1 convolutional kernel, and the number of input and output channels remains unchanged at 128; it generates channel-level statistics through global average pooling based on the channel attention module (SE mechanism); it uses two fully connected layers to form a bottleneck structure; and it uses the Sigmoid function to generate channel weights in the 0-1 range.

[0094] In this embodiment, knowledge distillation is used to transfer knowledge from a pre-trained second feeding behavior recognition model to the first feeding behavior recognition model to update the first feeding behavior recognition model. Specifically, within a preset time period, the latest output feeding behavior category probability distribution of the second feeding behavior recognition model and the output feeding behavior category probability distribution of the first feeding behavior recognition model are obtained. Based on the feeding behavior category probability distributions of the two models, the KL divergence between the outputs of the first and second feeding behavior recognition models is calculated. Based on the feeding behavior probability distribution of the first feeding behavior recognition model and the pre-stored true probability distribution values, the cross-entropy loss of the first feeding behavior recognition model is calculated. Based on the KL divergence and the cross-entropy, an overall loss function is constructed, and the first feeding behavior recognition model is updated based on the overall loss function using knowledge distillation.

[0095] In one specific embodiment, distillation is initiated periodically, and the same batch of data is input in parallel into the teacher model (i.e., the second feeding behavior recognition model) and the student model (i.e., the first feeding behavior recognition model). The original output (logits vector) of the teacher network is divided by the temperature coefficient τ=3, and then the teacher probability distribution is generated by the softmax function. The original output of the student network is subjected to the same softmax transformation with the same temperature coefficient τ=3 to generate the student probability distribution.

[0096] Furthermore, the relative entropy of the probability distribution of teachers and the probability distribution of students is calculated for each sample; the arithmetic mean of the KL divergence of all samples in the batch is calculated; and the square of the temperature coefficient is multiplied as a compensation factor to finally obtain the KL divergence loss.

[0097] Furthermore, the true labels are converted into one-hot encodings (6-dimensional binary vectors); for each sample, the negative of the sum of the logarithmic products of the true label and the student model output probability is calculated; the arithmetic mean of the cross-entropy of all samples in the batch is calculated to obtain the cross-entropy loss.

[0098] Furthermore, the KL divergence loss and cross-entropy loss are dynamically weighted to obtain an overall loss function value, and the student model is updated based on the overall loss function value. The dynamic weighting rule is as follows: when the student model accuracy is below 85%, the KL divergence loss weight λ = 0.7, and the cross-entropy loss weight 0.3; when the student model accuracy is 85% or above, the KL divergence loss weight λ = 0.3, and the cross-entropy loss weight 0.7.

[0099] It should be noted that the teacher model used in this invention is also updated periodically based on the latest data, and each knowledge distillation process is implemented using the latest updated parameters of the teacher model.

[0100] In this embodiment, the second feeding behavior recognition model is trained according to a preset large database. Specifically, the second feeding behavior recognition model is trained through self-supervised comparative learning based on the preset large database. The large database stores pre-collected acoustic data of large yellow croaker, large yellow croaker image data, environmental variables, and historical feeding behavior recognition results using a hot and cold hierarchical storage technology.

[0101] In one specific embodiment, the large database is constructed using the hot and cold tiered storage technology as follows: the feeding video / acoustic signal collected in real time within 7 days is stored in the hot storage layer, with NVMe solid-state drives as the storage medium, the daily access frequency is greater than 100 times, and a rolling over mechanism is used as the retention strategy, which can retain the data for a maximum of 7 days.

[0102] The system stores historical abnormal behavior fragments within 6 months in a warm storage layer, using a SATA solid-state drive as the storage medium. The daily access frequency is 5-10 times, and the fixed retention period is 180 days.

[0103] Long-term environmental monitoring data is stored in the cold storage layer, using a Blu-ray disc library as the storage medium; finally, manually labeled real datasets are stored in the archive layer.

[0104] In one specific embodiment, the second feeding behavior recognition model is trained using self-supervised contrastive learning based on a pre-set large database, specifically as follows:

[0105] First, multimodal data preprocessing is performed to perform spatiotemporal alignment and enhancement operations on the raw acquired data: the 30 frames / second video stream is downsampled to 5 frames / second to extract key action frames, the acoustic signal is simultaneously segmented into 2-second segments and precisely aligned with the video frames, and the environmental variables acquired at the minute level are boosted to a 5Hz sampling rate through linear interpolation.

[0106] Furthermore, cross-modal data augmentation was implemented: random HSV perturbations (and Gaussian blur) were applied to the visual data to simulate underwater optical properties; 15±3dB white noise was injected into the acoustic data and the time axis was distorted by ±20%; and sensor-level jitter was added to the environmental data.

[0107] Furthermore, a contrastive learning framework is constructed: positive sample pairs are constructed using trimodal data combinations (video clips + acoustic clips + environmental data) with the same timestamp and homogeneous data after different enhancement processes; data from the same time period of different individuals in the same fish group are used as hard negative samples, and randomly selected data from different time periods are used as easy negative samples, thereby forming negative sample pairs through the hard negative samples and easy negative samples; wherein, the ratio of positive to negative samples is fixed at 1:5.

[0108] Furthermore, the visual branch uses ResNet-50 to output 512-dimensional feature vectors, the acoustic branch outputs 512-dimensional feature vectors through 1D-CNN and BiLSTM networks, and the environmental branch generates 128-dimensional feature vectors using a three-layer MLP. The loss function is calculated based on the cosine similarity of the three-modal feature vectors. Specifically, it is calculated by dividing the dot product of two feature vectors by the product of their respective L2 norms, and then scaling the similarity value using a temperature coefficient τ=0.07. The final loss function is expressed as the sum of the three-modal positive sample similarity index in the numerator and the negative logarithm of the ratio to the denominator.

[0109] Furthermore, the LAMB optimizer is used in conjunction with a learning rate scheduling mechanism—the learning rate is linearly increased to its peak value in the first 5000 steps, and then a cosine annealing strategy is used to reduce the learning rate to the minimum of 1e-6, with a total training cycle of 100 rounds. During training, the similarity matrix of the validation set is monitored every 1000 steps: when the diagonal similarity is >0.85 and the off-diagonal mean is <0.15, the positive and negative samples are considered to be effectively separated; if the validation loss decreases by <0.1% for three consecutive rounds, an early stopping mechanism is triggered.

[0110] Furthermore, the feature space distribution is visualized and verified by t-SNE dimensionality reduction, requiring data points of the same row to cluster together. When adapting to downstream tasks, the weights of the feature encoder are frozen, and only the classification head composed of two fully connected layers is fine-tuned. This process only requires 10% of labeled data to achieve the effect of fully supervised training.

[0111] S14, when the result of the feeding behavior belongs to the category of abnormal feeding behavior, an abnormal feeding behavior warning will be issued according to the preset graded warning mechanism.

[0112] In this embodiment, when the feeding behavior result belongs to the abnormal feeding behavior category, an abnormal feeding behavior warning will be issued according to a preset graded warning mechanism. Specifically, within a preset time period, the latest feeding behavior result is obtained. If the feeding behavior result belongs to the abnormal feeding behavior category, the corresponding feed dispensing speed control command and oxygen pump control command are issued according to the abnormal feeding behavior category in the feeding behavior result and the preset feed-behavior mapping table. The abnormal feeding categories include competitive feeding, weak feeding, cessation of feeding, abnormal chewing, and pathological feeding. If the latest feeding behavior result still belongs to the abnormal feeding behavior category in the next time period, the corresponding feeding information and breeding location information are sent to the staff.

[0113] In one specific embodiment, when the feeding behavior result belongs to the abnormal feeding behavior category, an abnormal feeding behavior warning will be issued according to a preset graded warning mechanism. Specifically, the following steps are taken: the feed-behavior mapping table is queried to obtain control parameters, and the command value is dynamically calculated. The feed dispenser, oxygen pump, and bottom aerator are then controlled according to the command value. Next, the feed flow meter is used to monitor the descent rate in real time and the dissolved oxygen sensor detects changes in DO value through feedback from IoT sensors. When the number of consecutive abnormal cycles reaches the mapping table threshold (e.g., weak feeding lasts for 3 cycles) or a specific category (stopping feeding / pathological feeding), the warning will be upgraded, and staff will be notified to take manual intervention measures.

[0114] This invention constructs a multimodal data foundation by extracting key information about fish behavior from video and audio signals respectively; it uses a cross-modal phase synchronization network to time-align the visual and acoustic behavioral features, solving the problem of time asynchrony between audio-visual signals and ensuring feature time alignment; it uses a preset attention mechanism for semantic alignment, eliminating semantic differences between modalities and improving feature relevance; it uses knowledge distillation technology to obtain a lightweight feeding behavior recognition model for feeding behavior recognition, thereby balancing efficiency and accuracy of feeding behavior recognition through edge node deployment; and it uses a hierarchical early warning mechanism to provide feedback on abnormal feeding behavior in stages, responding in layers according to the severity of the anomaly to avoid over-alarms. Compared with existing technologies, this invention can improve the accuracy of monitoring the feeding behavior of large yellow croaker.

[0115] Example 2:

[0116] like Figure 2 As shown, this embodiment provides a large yellow croaker feeding behavior monitoring system based on target detection, including a behavior feature acquisition module 001, a feature pair acquisition module 002, a feeding behavior recognition module 003, and a behavior anomaly early warning module 004, wherein...

[0117] The behavioral feature acquisition module 001 is used to extract features from the real-time collected large yellow croaker feeding video to obtain visual behavioral features, and to extract features from the real-time collected large yellow croaker acoustic signal to obtain acoustic behavioral features.

[0118] In this embodiment, the behavior feature acquisition module 001 extracts features from the real-time acquired large yellow croaker feeding video to obtain visual behavior features. Specifically, the behavior feature acquisition module 001 performs frame-by-frame processing on the real-time acquired large yellow croaker feeding video to obtain a frame-level feeding image sequence; wherein, the frame-level feeding image sequence includes several temporally consecutive frame-level feeding image sequences; the frame-level feeding images are decomposed into multi-level spatial frequency bands using a Laplacian pyramid, and bandpass filtering is performed on the high-frequency bands in the multi-level spatial frequency bands to obtain a motion-magnified frame sequence; wherein, the high-frequency bands are multi-level spatial... The frequency band contains several spatial frequency bands containing image details and motion information; based on the motion magnification frame sequence, spatiotemporal key points are extracted, and based on the semantic mask of the spatiotemporal key points, the spatiotemporal key points are grouped to obtain a structured key point sequence; wherein, the spatiotemporal key points include the mouth, pectoral fins, and caudal peduncle; the structured key point sequence is compared with a preset feeding-non-feeding prototype library to extract feeding behavior features; the feeding behavior features are subjected to physical semantic mapping to obtain visual behavior features; wherein, the visual behavior features include fish mouth opening and closing frequency, body swaying amplitude, and swimming speed quantification indicators.

[0119] In this embodiment, the behavior feature acquisition module 001 extracts features from the real-time acquired acoustic signals of the large yellow croaker to obtain acoustic behavior features. Specifically, the behavior feature acquisition module 001 continuously inputs the real-time acquired acoustic signals of the large yellow croaker into a preset adaptive gating system to extract the signal energy and kurtosis value from the acoustic signals, and filters the initial feeding signal segment based on the signal energy and kurtosis value; the initial feeding signal segment is then binary-classified using a preset lightweight temporal convolutional network to obtain the final feeding signal segment; and the final feeding signal segment is then... Tensor decomposition is performed on the final feeding signal segment to obtain time tensor, frequency tensor, and spatial tensor. Based on the time tensor, frequency tensor, and spatial tensor, frequency spectrum and spatial response are extracted. Based on the frequency spectrum and spatial response, an individual fingerprint vector is constructed. Using a preset incremental spectral clustering algorithm, the individual feeding acoustic signal segment of the large yellow croaker is separated from the fish group based on the individual fingerprint vector, obtaining the individual feeding acoustic signal segment of each large yellow croaker. Feature extraction is performed on the individual feeding acoustic signal segment to obtain the acoustic behavior characteristics of each large yellow croaker.

[0120] The feature pair acquisition module 002 is used to: time-align the visual behavior features and the acoustic behavior features through a preset cross-modal phase synchronization network to obtain a first feature pair; and semantically align the first feature pair through a preset attention mechanism to obtain a second feature pair.

[0121] In one specific embodiment, the feature pair acquisition module 002 uses a preset cross-modal phase synchronization network to time-align the visual behavior features and the acoustic behavior features to obtain a first feature pair. Specifically, the feature pair acquisition module 002 uses the preset cross-modal phase synchronization network to convert the visual behavior features and the acoustic behavior features into visual phase latent variables and acoustic phase latent variables of the same dimension, respectively; and uses a preset timestamp to time-align the visual phase latent variables and acoustic phase latent variables to construct the first feature pair.

[0122] In one specific embodiment, the feature pair acquisition module 002 performs semantic alignment on the first feature pair through a preset attention mechanism to obtain a second feature pair. Specifically, according to a preset feeding semantic prototype dictionary, the visual phase latent variable and acoustic phase latent variable in the first feature pair are mapped to a preset hyperbolic space to calculate the semantic distance between the first feature pair and each prototype in the feeding semantic prototype dictionary. The feeding semantic prototype dictionary is constructed in a hyperbolic Poincaré sphere space. Each prototype corresponds to a posture-voiceprint joint semantic of a large yellow croaker. Through a preset contrastive attention mechanism, the visual phase latent variable and acoustic phase latent variable with the smallest semantic distance are matched one by one according to the semantic distance to obtain the second feature pair.

[0123] The feeding behavior recognition module 003 is used to perform feature fusion on the second feature pair to obtain multimodal features, and output feeding behavior recognition results based on the multimodal features using a preset first feeding behavior recognition model; wherein, the first feeding behavior recognition model is a lightweight model, and is updated by transferring knowledge from a pre-trained second feeding behavior recognition model to the first feeding behavior recognition model through knowledge distillation technology; the second feeding behavior recognition model is generated by training on a preset large database;

[0124] In this embodiment, the feeding behavior recognition module 003 performs feature fusion on the second feature pair to obtain multimodal features. Specifically, the feeding behavior recognition module 003 compresses the visual phase latent variable and acoustic phase latent variable in the second feature pair to reconstruct the visual phase latent variable and acoustic phase latent variable into a two-dimensional visual feature map and a two-dimensional acoustic map, respectively. The two-dimensional visual feature map and the two-dimensional acoustic map are then concatenated along the channel dimension to obtain a multimodal feature map. The multimodal feature map is then adaptively weighted and fused using a preset convolutional block to obtain the multimodal features.

[0125] In this embodiment, the feeding behavior recognition module 003 uses knowledge distillation technology to transfer knowledge from the pre-trained second feeding behavior recognition model to the first feeding behavior recognition model to update the first feeding behavior recognition model. Specifically, within a preset time period, the feeding behavior recognition module 003 acquires the latest output feeding behavior category probability distribution of the second feeding behavior recognition model and the output feeding behavior category probability distribution of the first feeding behavior recognition model, and calculates the KL divergence between the outputs of the first and second feeding behavior recognition models based on the feeding behavior category probability distributions of the two models; calculates the cross-entropy loss of the first feeding behavior recognition model based on the feeding behavior probability distribution of the first feeding behavior recognition model and the pre-stored true probability distribution value; constructs an overall loss function based on the KL divergence and the cross-entropy, and updates the first feeding behavior recognition model based on the overall loss function using knowledge distillation technology.

[0126] In this embodiment, the feeding behavior recognition module 003 trains the second feeding behavior recognition model based on a preset large database. Specifically, the feeding behavior recognition module 003 trains the second feeding behavior recognition model through self-supervised comparative learning based on the preset large database. The large database stores pre-collected acoustic data of large yellow croaker, large yellow croaker image data, environmental variables, and historical feeding behavior recognition results using a hot and cold hierarchical storage technology.

[0127] The abnormal behavior warning module 004 is used to issue an abnormal eating behavior warning according to a preset graded warning mechanism when the eating behavior result belongs to the abnormal eating behavior category.

[0128] In this embodiment, when the feeding behavior result belongs to the abnormal feeding behavior category, an abnormal feeding behavior warning will be issued according to a preset graded warning mechanism. Specifically, the abnormal behavior warning module 004 obtains the latest feeding behavior result within a preset time period. If the feeding behavior result belongs to the abnormal feeding behavior category, it issues corresponding feed dispensing speed control instructions and oxygen pump control instructions based on the abnormal feeding behavior category in the feeding behavior result and the preset feed-behavior mapping table. The abnormal feeding categories include competitive feeding, weak feeding, cessation of feeding, abnormal chewing, and pathological feeding. If the latest feeding behavior result still belongs to the abnormal feeding behavior category in the next time period, the corresponding feeding information and breeding location information will be sent to the staff.

[0129] For a more detailed explanation of the working principle and procedures of this embodiment, please refer to the relevant description in Embodiment 1.

[0130] This invention employs a behavior feature acquisition module 001 to extract key information about fish behavior from video and audio signals, constructing a multimodal data foundation. A feature pair acquisition module 002, using a cross-modal phase synchronization network, performs time alignment between the visual and acoustic behavior features to address the time asynchrony between audiovisual signals, ensuring feature time alignment. A preset attention mechanism is also used for semantic alignment, eliminating semantic differences between modalities and improving feature relevance. A feeding behavior recognition module 003 utilizes knowledge distillation technology to obtain a lightweight feeding behavior recognition model for feeding behavior identification, thus balancing efficiency and accuracy through edge node deployment. Finally, a behavior anomaly warning module 004, based on a tiered warning mechanism, provides tiered feedback on abnormal feeding behavior, responding according to the severity of the anomaly to avoid over-alarms.

[0131] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0132] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for monitoring the feeding behavior of large yellow croaker based on target detection, characterized in that, include: Feature extraction was performed on real-time acquired videos of large yellow croaker feeding to obtain visual behavioral features, and feature extraction was performed on real-time acquired acoustic signals of large yellow croaker to obtain acoustic behavioral features. The visual behavioral features and the acoustic behavioral features are temporally aligned using a pre-defined cross-modal phase synchronization network to obtain a first feature pair. The first feature pair is then semantically aligned using a pre-defined attention mechanism to obtain a second feature pair. The second feature pair is fused to obtain multimodal features, and a feeding behavior recognition result is output based on the multimodal features using a preset first feeding behavior recognition model. The first feeding behavior recognition model is a lightweight model, updated by transferring knowledge from a pre-trained second feeding behavior recognition model to the first feeding behavior recognition model using knowledge distillation technology. The second feeding behavior recognition model is generated by training on a preset large database. When the feeding behavior identification result belongs to the abnormal feeding behavior category, an abnormal feeding behavior warning will be issued according to the preset graded warning mechanism.

2. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 1, characterized in that, The process of extracting visual behavioral features from real-time acquired videos of large yellow croaker feeding is as follows: The real-time acquired video of large yellow croaker feeding is processed into frames to obtain a frame-level feeding image sequence; wherein, the frame-level feeding image sequence includes several temporally consecutive frame-level feeding image sequences; The frame-level feeding image is decomposed into multiple spatial frequency bands using the Laplacian pyramid, and the high-frequency bands in the multiple spatial frequency bands are bandpass filtered to obtain a motion magnified frame sequence; wherein, the high-frequency bands are several layers of spatial frequency bands containing image details and motion information in the multiple spatial frequency bands. Based on the motion magnification frame sequence, spatiotemporal key points are extracted, and based on the semantic mask of the spatiotemporal key points, the spatiotemporal key points are grouped to obtain a structured key point sequence; wherein, the spatiotemporal key points include the mouth, pectoral fins and caudal peduncle. The structured key point sequence is compared with a preset feeding-non-feeding prototype library to extract feeding behavior features; The feeding behavior characteristics are subjected to physical semantic mapping to obtain visual behavior characteristics; wherein, the visual behavior characteristics include fish mouth opening and closing frequency, body swaying amplitude and swimming speed quantitative indicators.

3. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 1, characterized in that, The acoustic behavior features of the real-time acquired large yellow croaker acoustic signals are extracted, specifically as follows: The acoustic signals of large yellow croaker collected in real time are continuously input into a preset adaptive gating system to extract the signal energy and kurtosis value from the acoustic signals of large yellow croaker, and to filter the initial feeding signal segment based on the signal energy and kurtosis value; The initial feeding signal segment is classified into two categories using a pre-defined lightweight temporal convolutional network to obtain the final feeding signal segment. Tensor decomposition is performed on the final feeding signal segment to obtain time tensor, frequency tensor and spatial tensor, and frequency spectrum and spatial response are extracted based on the time tensor, frequency tensor and spatial tensor; Based on the frequency spectrum and spatial response, an individual fingerprint vector is constructed, and the individual feeding acoustic signal segment of the large yellow croaker is separated from the fish group by a preset incremental spectral clustering algorithm, based on the individual fingerprint vector, to obtain the individual feeding acoustic signal segment of each large yellow croaker. Feature extraction was performed on the individual feeding acoustic signal segments to obtain the acoustic behavioral characteristics of each large yellow croaker.

4. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 1, characterized in that, The first feature pair is obtained by temporally aligning the visual behavioral features with the acoustic behavioral features through a preset cross-modal phase synchronization network, specifically: The visual behavioral features and acoustic behavioral features are transformed into visual phase latent variables and acoustic phase latent variables of the same dimension through a preset cross-modal phase synchronization network. By using a preset timestamp, the visual phase latent variable and the acoustic phase latent variable are time-aligned to construct a first feature pair.

5. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 4, characterized in that, The second feature pair is obtained by semantically aligning the first feature pair using a preset attention mechanism, specifically as follows: According to the preset feeding semantic prototype dictionary, the visual phase latent variable and acoustic phase latent variable in the first feature pair are respectively mapped to the preset hyperbolic space to calculate the semantic distance between the first feature pair and each prototype in the feeding semantic prototype dictionary; wherein, the feeding semantic prototype dictionary is constructed in the hyperbolic Poincaré sphere space; each prototype corresponds to a posture-voiceprint joint semantic of a large yellow croaker. By using a pre-defined contrastive attention mechanism, the visual phase latent variable and the acoustic phase latent variable with the smallest semantic distance are matched one by one according to the semantic distance to obtain the second feature pair.

6. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 5, characterized in that, The feature fusion of the second feature pair to obtain multimodal features is specifically as follows: The visual phase latent variable and acoustic phase latent variable in the second feature pair are respectively compressed to reconstruct the visual phase latent variable and acoustic phase latent variable into a two-dimensional visual feature map and a two-dimensional acoustic map, respectively. The two-dimensional visual feature map and the two-dimensional acoustic map are stitched together in the channel dimension to obtain a multimodal feature map; The multimodal feature map is adaptively weighted and fused using a preset convolutional block to obtain multimodal features.

7. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 1, characterized in that, The knowledge of the pre-trained second feeding behavior recognition model is transferred to the first feeding behavior recognition model through knowledge distillation, specifically as follows: Within a preset time period, the latest output probability distribution of the feeding behavior category of the second feeding behavior recognition model and the output probability distribution of the feeding behavior category of the first feeding behavior recognition model are obtained, and the KL divergence between the output of the first feeding behavior recognition model and the output of the second feeding behavior recognition model is calculated based on the two feeding behavior category probability distributions. Based on the feeding behavior probability distribution of the first feeding behavior recognition model and the pre-stored true probability distribution value, calculate the cross-entropy loss of the first feeding behavior recognition model; Based on the KL divergence and the cross-entropy, an overall loss function is constructed, and the first feeding behavior recognition model is updated based on the overall loss function using knowledge distillation technology.

8. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 7, characterized in that, The second feeding behavior recognition model is trained based on a pre-set large database, specifically as follows: The second feeding behavior recognition model is trained through self-supervised comparative learning based on a pre-set large database. The large database stores pre-collected acoustic data, image data, environmental variables, and historical feeding behavior recognition results of large yellow croaker using a hot-cold hierarchical storage technology.

9. The method for monitoring the feeding behavior of large yellow croaker based on target detection as described in claim 1, characterized in that, When the feeding behavior identification result belongs to the abnormal feeding behavior category, an abnormal feeding behavior warning will be issued according to a preset graded warning mechanism, specifically as follows: Within a preset time period, the latest feeding behavior identification result is obtained once. If the feeding behavior identification result belongs to the abnormal feeding behavior category, the corresponding bait delivery speed control command and oxygen pump control command are issued according to the abnormal feeding behavior category in the feeding behavior identification result and the preset bait-behavior mapping table. The abnormal feeding behavior categories include snatching food, weak feeding, stopping feeding, abnormal chewing, and pathological feeding. If the latest feeding behavior identification result still falls under the category of abnormal feeding behavior in the next time period, the corresponding feeding information and breeding location information will be sent to the staff.

10. A system for monitoring the feeding behavior of large yellow croaker based on target detection, characterized in that, It includes a behavioral feature acquisition module, a feature pair acquisition module, a feeding behavior recognition module, and a behavioral anomaly warning module, among which, The behavioral feature acquisition module is used to extract features from real-time acquired large yellow croaker feeding videos to obtain visual behavioral features, and to extract features from real-time acquired large yellow croaker acoustic signals to obtain acoustic behavioral features. The feature pair acquisition module is used to time-align the visual behavior features and the acoustic behavior features through a preset cross-modal phase synchronization network to obtain a first feature pair, and to semantically align the first feature pair through a preset attention mechanism to obtain a second feature pair. The feeding behavior recognition module is used to perform feature fusion on the second feature pair to obtain multimodal features, and output feeding behavior recognition results based on the multimodal features using a preset first feeding behavior recognition model; wherein, the first feeding behavior recognition model is a lightweight model, and is updated by transferring knowledge from a pre-trained second feeding behavior recognition model to the first feeding behavior recognition model through knowledge distillation technology; the second feeding behavior recognition model is generated by training on a preset large database; The abnormal behavior warning module is used to issue an abnormal eating behavior warning according to a preset hierarchical warning mechanism when the eating behavior identification result belongs to the abnormal eating behavior category.

Images