A method and system for early intelligent warning of diseases in high-density cultured fish

By transmitting acoustic stimulus signals to fish schools and collecting acoustic and behavioral data, an acoustic-behavioral dual-modal stress response representation system is constructed. Combined with a cross-modal comparison network for self-supervised training, the problem of delayed early warning of fish diseases in existing technologies is solved, and early and accurate early warning of fish diseases is achieved.

CN123100537APending Publication Date: 2026-11-17HUAXIA ZHIYU (SHANDONG) AGRICULTURAL DEVELOPMENT CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611253308.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-18
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing fish disease early warning technologies mainly rely on passive observation, which cannot achieve proactive acquisition of the real-time status of fish populations, resulting in delayed early warnings and insufficient early detection capabilities.

Method used

By transmitting acoustic stimuli to fish schools and collecting acoustic signals and sonar images, an acoustic-behavioral dual-modal stress response representation system is constructed. Acoustic and behavioral feature recognition is used, combined with self-supervised training of a cross-modal contrastive network, to generate a health baseline vector and set a threshold, thereby achieving early warning of diseases.

Benefits of technology

It enables precise, non-invasive early warning of fish diseases, breaking through the passive observation paradigm. It can trigger early warning at a very early stage before disease signs become apparent, improving the sensitivity and timeliness of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123100537A_ABST
    Figure CN123100537A_ABST
Patent Text Reader

Abstract

This invention relates to the field of fish disease early warning technology, and provides a method and system for early intelligent early warning of diseases in high-density farmed fish. The method includes triggering a fish school stress response through active acoustic stimulation, and simultaneously acquiring water acoustic signals and sonar images. Using voiceprint recognition, time-frequency decomposition is performed on the acoustic frames, and the baseline mean component is subtracted to extract the stress acoustic feature sequence. Using behavioral feature recognition, a fish school velocity field is generated through optical flow calculation to extract behavioral features. Using machine learning, a cross-modal contrastive network with bidirectional cross-modal attention is constructed. A healthy baseline representation is obtained by training with self-supervised InfoNCE loss. By calculating the cosine distance between the real-time fused representation and the baseline, an early warning is triggered when the distance exceeds a threshold multiple times consecutively. This method overcomes the limitations of passive observation and can achieve early and accurate early warning before disease signs become apparent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of fish disease early warning technology, and more specifically, to a method and system for early intelligent early warning of diseases in high-density farmed fish. Background Technology

[0002] The content in this section only provides background information related to this invention and may not constitute prior art.

[0003] With the large-scale promotion of high-density aquaculture, early warning of fish diseases has become a key factor restricting aquaculture efficiency and the quality and safety of aquatic products. Currently, fish health monitoring technology is evolving along three main technical paths.

[0004] The first approach involves computer vision-based fish anomaly detection technology. This method uses underwater cameras to capture image sequences of fish schools and analyzes parameters such as fish appearance, swimming posture, and population density to determine their health status. However, this technology can only detect diseased individuals that have already shown obvious external symptoms or behavioral abnormalities. It is difficult to effectively detect early or subclinical diseases that have not yet shown visible signs.

[0005] The second approach is fish vocalization analysis based on passive underwater acoustic monitoring. This technique uses underwater microphones to collect vocal signals from fish and analyzes changes in vocal frequency, intensity, and rhythm to infer their health status. However, this type of technology relies on the natural vocalizations of fish as a detection basis. In many disease processes, changes in the vocalization characteristics of fish are not significant, and environmental noise interference can easily lead to misdiagnosis and missed diagnosis.

[0006] The third approach involves environmental parameter monitoring technology based on water quality sensors. This technology indirectly assesses the health risks of fish populations by monitoring water quality indicators such as water temperature, dissolved oxygen, pH, and ammonia nitrogen. However, this type of technology can only reflect environmental stress factors and cannot directly characterize the physiological state of the fish population itself. Furthermore, there is a time lag between water quality abnormalities and fish disease outbreaks, resulting in insufficient timely warnings.

[0007] To improve detection accuracy, Chinese patent CN119478650A discloses a multimodal fish health early warning system. This system fuses and analyzes image data from an underwater camera with audio data from an underwater microphone, extracting visual spatiotemporal features through a slow-fast dual-channel network and combining these with audio channel features for joint discrimination. While this approach improves detection accuracy to some extent through multimodal fusion, it essentially remains a passive observation paradigm. The system passively waits for disease signals to manifest naturally before identifying the problem, failing to actively detect the real-time physiological state of the fish population.

[0008] In summary, existing fish disease early warning technologies are all based on passive observation, requiring the disease to develop to a stage where it can be detected by visual, acoustic, or environmental sensors before an early warning can be triggered. This results in problems such as delayed warnings and insufficient early detection capabilities. Therefore, how to achieve proactive acquisition of the real-time status of fish populations, thereby moving the disease early warning point forward, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] The purpose of this invention is to provide an intelligent early warning method and system for diseases in high-density farmed fish, in order to improve the aforementioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows: Firstly, this application provides an early intelligent warning method for diseases in high-density farmed fish, including: S101, an acoustic stimulation signal is emitted to the fish school through an acoustic emission transducer set in the target water body. At the same time, the water body acoustic signal and fish school sonar image are collected before and after the stimulation through a hydrophone array and sonar. The water body acoustic signal is sliced ​​into an acoustic frame sequence according to a fixed time window, and the fish school sonar image is arranged in frame order to obtain a sonar image sequence. S102, the acoustic frame sequence is divided into a baseline acoustic frame group and a response acoustic frame group according to the stimulus time point. After performing time-frequency decomposition on each frame in the response acoustic frame group, the mean component of the corresponding frequency band of the baseline acoustic frame group is subtracted to obtain the stress response feature components. These components are then arranged in frame order to generate a stress acoustic feature sequence. S103, the sonar image sequence is divided into baseline image group and response image group according to the stimulation time point, and optical flow calculation is performed on each group to obtain the fish swarm velocity field sequence; the velocity amplitude distribution feature, motion direction consistency feature and vortex intensity feature are extracted from the fish swarm velocity field sequence to generate the stress behavior feature sequence. S104: The stress acoustic feature sequence is unfolded into a two-dimensional acoustic map, and the stress behavioral feature sequence is unfolded into a two-dimensional behavioral map. The two-dimensional acoustic map and the two-dimensional behavioral map are respectively input into a preset cross-modal contrast network. The contrast network includes an acoustic branch convolutional network and a behavioral branch convolutional network, with a bidirectional cross-modal attention module between the two branches. The bidirectional cross-modal attention module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, and outputs acoustic representation vectors and behavioral representation vectors. The acoustic representation vectors and behavioral representation vectors are fused to obtain a fused representation vector. The contrast network adopts a self-supervised training method to construct cross-modal positive sample pairs and negative sample pairs. The distance between positive sample pairs in the representation space is constrained to be less than the distance between negative sample pairs in the representation space through InfoNCE loss to complete the training. A healthy baseline vector is generated based on the training samples, and a preset threshold is set based on the healthy baseline vector. S105, calculate the cosine distance between the fused representation vector and the healthy baseline vector, compare the cosine distance with a preset threshold, and generate an early warning signal for the disease when the cosine distance corresponding to multiple consecutive stimuli exceeds the preset threshold.

[0010] Furthermore, the acoustic signal of the water body is sliced ​​according to a fixed time window to obtain an acoustic frame sequence, specifically including: A Hanning window of a preset duration is selected as the analysis window function. The time-domain waveform of the water body acoustic signal is multiplied by the Hanning window segment by segment to achieve windowing. The window moves forward by a fixed number of sampling points each time, so that the waveform intervals covered by adjacent windows partially overlap. Each segment of the time-domain waveform obtained by windowing is recorded as an acoustic frame. The acoustic frames are arranged in the order of extraction to obtain the acoustic frame sequence.

[0011] Furthermore, after performing time-frequency decomposition on each frame in the response acoustic frame group, the mean component of the corresponding frequency band of the baseline acoustic frame group is subtracted to obtain the stress response characteristic components, specifically including: Perform a complex wavelet transform on each response sound frame in the response sound frame group to obtain the complex time spectrum, and take its modulus to obtain the time-frequency amplitude matrix of the sound frame; For the time-frequency amplitude matrix of all baseline sound frames in the baseline sound frame group, calculate the arithmetic mean along the time dimension for each frequency point to obtain the baseline mean spectrum in the corresponding frequency direction. For each response acoustic frame's time-frequency amplitude matrix, the amplitude at each frequency point in the matrix is ​​subtracted from the mean of the corresponding frequency point in the baseline mean spectrum. The amplitude with a negative difference is set to zero, and the positive value is retained as the stress response characteristic component of that response acoustic frame.

[0012] Furthermore, optical flow calculations were performed separately to obtain the fish swarm velocity field sequence, specifically including: For two temporally adjacent sonar images within the baseline image group and the response image group, a dense optical flow algorithm based on polynomial expansion is used to estimate the displacement of each pixel in the image plane. Remove the bit by the time interval between two frames to obtain the instantaneous velocity vector of the pixel; the instantaneous fish swarm velocity field of the interval is formed by the velocity vectors of all pixels within a frame interval. The instantaneous fish swarm velocity fields of all frame intervals within the baseline image group and the response image group are arranged in chronological order to obtain the fish swarm velocity field sequence.

[0013] Furthermore, velocity amplitude distribution features, motion direction consistency features, and vortex intensity features are extracted from the fish swarm velocity field sequence to generate a stress behavior feature sequence, specifically including: For each velocity field in the fish swarm velocity field sequence, calculate the arithmetic mean and standard deviation of the velocity vector magnitude of all pixels in the target area, which are used as the velocity amplitude distribution characteristics at the corresponding time. Normalize the velocity vector of each pixel in the target area into a unit direction vector, calculate the vector sum of all unit direction vectors, and obtain the motion direction consistency feature by dividing the magnitude of the vector sum by the total number of pixels involved in the summation. The curl of the velocity vector is calculated pixel by pixel in the velocity field to obtain the curl scalar field. Then, the spatial average value of the absolute value of the curl in the target area is calculated as the vortex intensity feature. The velocity amplitude distribution feature, motion direction consistency feature and vortex intensity feature at the same moment are combined into a behavior feature vector. The behavior feature vectors at each moment are arranged in chronological order to form a stress behavior feature sequence.

[0014] Furthermore, the bidirectional cross-modal attention module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, outputting acoustic representation vectors and behavioral representation vectors, specifically including: The acoustic feature map output by the acoustic branch convolutional network and the behavioral feature map output by the behavioral branch convolutional network are respectively mapped to the feature space of the same dimension. The mapping is achieved through their respective independent fully connected layers. The mapped acoustic features are used as queries, and the mapped behavioral features are used as keys and values. The cross-attention weights of the acoustic features and behavioral features are calculated, and then the behavioral features are weighted and summed according to the cross-attention weights to obtain the acoustic-side fusion features. The mapped behavioral features are used as queries, and the mapped acoustic features are used as keys and values. The cross-attention weights of the behavioral features and acoustic features are calculated, and then the acoustic features are weighted and summed according to the cross-attention weights to obtain the behavioral-side fusion features. The acoustic representation vector is obtained by adding the acoustic-side fused features to the original acoustic features, and the behavioral representation vector is obtained by adding the behavioral-side fused features to the original behavioral features.

[0015] Furthermore, a healthy baseline vector is generated based on the training samples, and a preset threshold is set based on the healthy baseline vector, specifically including: The fusion representation vectors corresponding to healthy samples during training are averaged to generate a healthy baseline vector. A preset threshold is set based on the maximum cosine distance between the fusion representation vectors and the healthy baseline vectors in the training samples.

[0016] Secondly, this application also provides an early intelligent warning system for diseases in high-density farmed fish, comprising: The data acquisition module is used to transmit acoustic stimulation signals to the fish school through an acoustic emission transducer set in the target water body. At the same time, it collects the water body acoustic signals and fish school sonar images before and after stimulation through a hydrophone array and sonar. The water body acoustic signals are sliced ​​into acoustic frame sequences according to a fixed time window, and the fish school sonar images are arranged in frame order to obtain a sonar image sequence. The stress acoustic module is used to divide the sound frame sequence into a baseline sound frame group and a response sound frame group according to the stimulus time point. After performing time-frequency decomposition on each frame in the response sound frame group, the mean component of the corresponding frequency band of the baseline sound frame group is subtracted to obtain the stress response feature components. These components are then arranged in frame order to generate a stress acoustic feature sequence. The stress behavior module is used to divide the sonar image sequence into a baseline image group and a response image group according to the stimulation time point, and perform optical flow calculations on each group to obtain the fish swarm velocity field sequence. From the fish swarm velocity field sequence, the velocity amplitude distribution characteristics, motion direction consistency characteristics, and vortex intensity characteristics are extracted to generate a stress behavior feature sequence. The cross-modal fusion module unfolds the stress acoustic feature sequence into a two-dimensional acoustic map and the stress behavioral feature sequence into a two-dimensional behavioral map. The two-dimensional acoustic and behavioral maps are then input into a pre-defined cross-modal contrast network. The contrast network includes an acoustic branch convolutional network and a behavioral branch convolutional network, with a bidirectional cross-modal attention module between the two branches. This module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, outputting acoustic and behavioral representation vectors. The acoustic and behavioral representation vectors are then fused to obtain a fused representation vector. The contrast network employs self-supervised training, constructing cross-modal positive and negative sample pairs. Training is completed by constraining the distance between positive sample pairs in the representation space to be less than the distance between negative sample pairs using InfoNCE loss. A healthy baseline vector is generated based on the training samples, and a pre-defined threshold is set based on this healthy baseline vector. The early warning module is used to calculate the cosine distance between the fused representation vector and the healthy baseline vector, compare the cosine distance with a preset threshold, and generate an early warning signal for the disease when the cosine distance corresponding to multiple consecutive stimuli exceeds the preset threshold.

[0017] Thirdly, this application also provides an electronic device, including: Memory, used to store computer programs; A processor is used to implement the method steps as described in the first aspect when executing a computer program.

[0018] Fourthly, this application also provides a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method steps of the first aspect.

[0019] The beneficial effects of this invention are as follows: This invention is based on the principle of active acoustic stimulus-stress response detection. By transmitting standardized acoustic stimulus signals into high-density aquaculture water bodies, and simultaneously collecting water acoustic signals and fish sonar images before and after the stimulus, an acoustic-behavioral dual-modal stress response characterization system is constructed.

[0020] In the acoustic dimension, using voiceprint recognition, the acoustic signal is sliced ​​into a sequence of acoustic frames according to a fixed time window. With the frequency mean of the baseline acoustic frame before stimulation as a reference, the response acoustic frame is decomposed in time and frequency and the baseline component is subtracted to extract the stress acoustic feature sequence that reflects the stress physiological state of the fish population, and to accurately capture the subtle drift of the fish's vocalization pattern in the early stage of disease.

[0021] In the behavioral dimension, by using behavioral feature recognition, optical flow calculations are performed on sonar image sequences to generate a fish swarm velocity field. Three types of features are extracted from this field: velocity amplitude distribution, motion direction consistency, and vortex intensity. A stress behavior feature sequence is constructed to quantify the degree of abnormal group movement of the fish swarm after being stimulated.

[0022] Building upon this foundation, a cross-modal contrastive network was designed using machine learning, incorporating convolutional networks with acoustic and behavioral branches. A bidirectional cross-modal attention module facilitates deep interaction and fusion of features from both branches. A self-supervised training method, employing InfoNCE loss to constrain the representation distance between positive and negative sample pairs, learns a fused representation baseline vector for healthy fish populations. During actual early warning, the cosine distance between the real-time fused representation vector and the healthy baseline vector is calculated. When the distance value exceeds a preset threshold for multiple consecutive stimuli, a pathological drift in the fish population's stress response pattern is identified, triggering an early disease warning signal. This approach overcomes the limitations of passive observation paradigms by integrating fish stress physiology with underwater acoustic engineering, enabling precise and non-invasive early warning even at the very early stages before disease symptoms become apparent. Attached Figure Description

[0023] Figure 1 A flowchart of an early intelligent warning method for diseases in high-density farmed fish provided by the present invention; Figure 2 This is a schematic diagram of an intelligent early warning system for diseases in high-density farmed fish provided by the present invention; Figure 3 This is a schematic diagram of an electronic device provided by the present invention.

[0024] In the diagram: 201, data acquisition module; 202, stress acoustic module; 203, stress behavior module; 204, cross-modal fusion module; 205, early warning module; 301, processor; 302, memory. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] like Figure 1 As shown in the embodiment of the present invention, an early intelligent warning method for diseases in high-density farmed fish includes: S101, an acoustic stimulation signal is emitted to the fish school through an acoustic emission transducer set in the target water body. At the same time, the water body acoustic signal before and after the stimulation and the fish school sonar image are collected by a hydrophone array and sonar. The water body acoustic signal is sliced ​​into a sound frame sequence according to a fixed time window, and the fish school sonar image is arranged in frame order to obtain a sonar image sequence.

[0027] This step aims to induce a stress response in fish populations through active acoustic stimulation and simultaneously collect raw data in both acoustic and image modalities, providing a high-quality input foundation for subsequent feature extraction and disease identification.

[0028] Specifically, acoustic stimulation signals are emitted to the fish population through acoustic emission transducers installed in the target water body. These transducers can be piezoelectric ceramic underwater acoustic transducers, positioned on the side walls and bottom of the aquaculture pond. The emission frequency band is selected from the low-frequency range sensitive to fish hearing, such as a single-frequency sinusoidal pulse signal in the range of 200Hz to 2kHz. The pulse duration can be set to 0.5 to 2 seconds, and the sound pressure level is controlled within a safe range to avoid causing damage to the fish. The principle of active acoustic stimulation is that healthy fish exhibit clear escape or avoidance behaviors in response to acoustic stimuli, while fish in the early stages of disease show significantly reduced intensity and timeliness of their stress response due to damage to their nervous system or decreased physical strength. By actively applying controllable stimulation, the physiological differences in the fish population can be transformed into observable acoustic and behavioral differences, thereby achieving early disease exposure. This method offers higher sensitivity and earlier warning compared to passive monitoring.

[0029] Simultaneously, a hydrophone array and sonar are used to collect water acoustic signals and fish sonar images before and after stimulation. The hydrophone array can consist of 4 to 8 hydrophone units arranged in an equally spaced linear or circular array in the aquaculture water (i.e., the hydrophone units are placed on pillars in the aquaculture pond, and signals from different locations and depths are obtained by setting different pillars). The sampling rate is set to 48kHz or higher to fully cover the frequency range of fish vocalizations and swimming sounds. The sonar uses a high-frequency imaging sonar with an operating frequency of, for example, 1MHz to 2MHz, and a frame rate of no less than 10 frames / second to ensure the temporal resolution of fish movement. During the acquisition process, the time of emission of the acoustic stimulus signal is used as the time reference point, and baseline data for a period of time before stimulation (e.g., 5 seconds) and response data for a period of time after stimulation (e.g., 10 seconds) are recorded synchronously. The principle of synchronous acquisition is that fish stress response is reflected in both vocalization and movement behaviors. Only when the two types of data are strictly aligned in time can the effectiveness of subsequent cross-modal analysis be guaranteed. Hydrophone arrays can obtain spatial orientation information of fish vocalization, while sonar can obtain spatial distribution information of fish movement. The two complement each other and can comprehensively characterize the stress state of fish.

[0030] The acoustic signal of the water body is sliced ​​according to a fixed time window to obtain an acoustic frame sequence. Specifically, this includes: selecting a Hanning window of a preset duration as the analysis window function; multiplying the time-domain waveform of the water body acoustic signal with the Hanning window segment by segment to achieve windowing; the window moves forward by a fixed number of sampling points each time, so that the waveform intervals covered by adjacent windows partially overlap; each segment of the time-domain waveform obtained by windowing is recorded as an acoustic frame; and the acoustic frames are arranged in the order of extraction to obtain the acoustic frame sequence. In an embodiment of the present invention, the Hanning window duration can be set to 1024 sampling points, corresponding to approximately 21.3 milliseconds (at a sampling rate of 48 kHz), and the window movement step is set to 512 sampling points, meaning that there is a 50% overlap between adjacent windows. The principle of using Hanning windows with overlapping is that the smooth transition to zero at both ends of the Hanning window can effectively suppress spectral leakage in subsequent time-frequency analysis and ensure frequency resolution; while the overlapping of adjacent frames can compensate for the information loss caused by the low weight at both ends of the window function, so that each segment of the time-domain waveform can be in the central high-weight region of the window function in a certain frame, thereby improving the temporal continuity and analysis accuracy of the acoustic frame sequence and avoiding the loss of boundary information caused by windowing.

[0031] The expression for the Hanning window as the analysis window function is as follows:

[0032] In the formula, For the discrete Hanning window function in the th The amplitude at each sampling point; The sampling point number within the window. =0, 1, ..., -1; This is the total length of the window function (number of sampling points).

[0033] The windowing process involves multiplying the time-domain waveform of the water acoustic signal by the Hanning window segment by segment, and its expression is as follows:

[0034] In the formula, For the first The first sound frame inside the window The sampled value of a point, i.e., the time-domain waveform after windowing; This is a discrete sampling sequence of acoustic signals from the water body, with the subscript representing the sampling point number; For the first The offset of the starting sampling point of each sound frame in the original signal; This represents the number of sampling points (steps) the window moves forward each time.

[0035] The sonar images of the fish school are arranged in frame order to obtain a sonar image sequence. The sonar image sequence is numbered and stored in chronological order of acquisition time, with each frame corresponding to a precise timestamp. This timestamp maintains a consistent reference with the time axis of the water acoustic signal, ensuring strict alignment of acoustic data and image data in the time dimension. This provides a time consistency guarantee for the synchronous analysis of acoustic and behavioral characteristics in subsequent steps.

[0036] S102, the acoustic frame sequence is divided into a baseline acoustic frame group and a response acoustic frame group according to the stimulus time point. After performing time-frequency decomposition on each frame in the response acoustic frame group, the mean component of the corresponding frequency band of the baseline acoustic frame group is subtracted to obtain the stress response feature components. These components are then arranged in frame order to generate a stress acoustic feature sequence. This step aims to separate the baseline components of environmental background noise and normal fish vocalizations from the original acoustic frame sequence, retaining only the acoustic feature increments corresponding to the stress response induced by acoustic stimuli. This transforms the differences in the physiological state of the fish population into quantifiable frequency domain feature differences, providing a high signal-to-noise ratio input for subsequent disease identification.

[0037] Specifically, using the emission time of the acoustic stimulus signal as the time reference point, the acoustic frames collected before the reference point constitute the baseline acoustic frame group, corresponding to the quiet state of the fish when they are not stimulated; the acoustic frames collected after the reference point constitute the response acoustic frame group, corresponding to the stress response stage of the fish after being stimulated. Taking parameters of 48kHz sampling rate, 1024 points of Hanning window, and 512 points of step as an example, the baseline data in the first 5 seconds of stimulation can generate approximately 468 baseline acoustic frames, and the response data in the last 10 seconds of stimulation can generate approximately 937 response acoustic frames. The principle of dividing the data into two groups according to the stimulation time point is that the baseline acoustic frame group reflects the background acoustic environment of the aquaculture water body under the condition of no external stimulation, including water flow noise, equipment operation noise, and normal sound emission of the fish swimming and feeding; the response acoustic frame group is superimposed on this background with the stress-induced sound increment of the fish. By comparing two sets of data, background interference unrelated to stress can be effectively eliminated, allowing the subsequently extracted features to purely reflect the fish's response to sound stimuli. Compared to directly extracting features from the response signal, this method can significantly improve feature discrimination and robustness.

[0038] Specifically, after performing time-frequency decomposition on each frame in the response acoustic frame group, the mean component of the corresponding frequency band of the baseline acoustic frame group is subtracted to obtain the stress response characteristic components, which specifically include: The first step involves performing a complex wavelet transform on each response frame in the response frame group to obtain the complex-valued time spectrum. The modulus value is then used to obtain the time-frequency amplitude matrix for that frame. Compared to the short-time Fourier transform, the complex wavelet transform offers higher frequency resolution in the low-frequency band and higher time resolution in the high-frequency band. This characteristic is highly compatible with the multi-scale frequency domain characteristics of fish vocalizations, where fish response sounds often simultaneously contain low-frequency body vibrations and mid-to-high-frequency muscle contraction sounds. The complex wavelet transform can simultaneously ensure analytical accuracy across different frequency bands within a single transform framework. In practice, the Morlet complex wavelet can be selected as the mother wavelet, with the scale parameter uniformly distributed at logarithmic intervals. The frequency coverage range is set from 100Hz to 10kHz to fully cover the main frequency bands of fish vocalizations and swimming sounds. The complex-valued time spectrum output by the complex wavelet transform contains both amplitude and phase information. The modulus value is then used to obtain the time-frequency amplitude matrix. The rows of this matrix correspond to the frequency scale, and the columns correspond to time points. The value of each element represents the signal energy at that time-frequency point.

[0039] The calculation formula for the complex wavelet transform is as follows:

[0040] The correspondence between scale and frequency is as follows:

[0041] In the formula, For the first Each sound frame at frequency ,time The complex wavelet transform coefficients (complex values) at a given point, i.e., the spectrum at the complex value; For the first Time-domain waveform of each sound frame (continuous representation). This is the mother wavelet function (in this example, it is the Morlet complex wavelet). Its conjugate; For the corresponding frequency Wavelet scaling parameters; The center frequency of the mother wavelet is determined by the selected mother wavelet type. Let be the dummy variable for integration, representing time; For the corresponding frequency.

[0042] Therefore, the time-frequency amplitude matrix is:

[0043] In the formula, For the first Each sound frame at frequency The time-frequency amplitude at a given point, that is, the energy value at the corresponding frequency point in the time-frequency amplitude matrix; For the first Each sound frame corresponds to a specific moment.

[0044] The second step involves calculating the arithmetic mean of the time-frequency amplitude matrix of all baseline sound frames within the baseline sound frame group, point by point along the time dimension, to obtain the baseline mean spectrum in the corresponding frequency direction. The value of this baseline mean spectrum at each frequency point represents the average energy level at that frequency under baseline conditions, reflecting the statistical distribution characteristics of the water background noise and the normal vocalizations of the fish in the frequency domain. The principle of averaging along the time dimension is that the baseline sound frame group contains a sufficient number of time samples, and time-domain averaging can effectively suppress random fluctuations, obtaining a stable and reliable frequency domain baseline, providing an accurate reference benchmark for subsequent difference calculations.

[0045] The baseline mean spectrum is calculated as follows:

[0046] In the formula, The baseline mean spectrum at frequency The value at that frequency represents the average energy level at that frequency under baseline conditions. This represents the total number of sound frames contained in the baseline sound frame group; For the first Each baseline acoustic frame at frequency The time-frequency amplitude at the location, superscript This indicates that it belongs to the baseline group; For audio frame index.

[0047] The third step involves subtracting the amplitude of each frequency point in the response sound frame from the mean of the corresponding frequency point in the baseline mean spectrum for the time-frequency amplitude matrix of each response sound frame. Amplitudes with negative differences are set to zero, while positive values ​​are retained as the stress response characteristic components of that response sound frame. The physical meaning of this subtraction operation is to remove the background component from the response signal, retaining only the incremental energy induced by the acoustic stimulus; setting negative values ​​to zero eliminates spurious negative responses caused by baseline fluctuations, ensuring that the stress response characteristic components only contain positive response components with energy higher than the baseline. This processing method effectively highlights the characteristic frequency bands of the fish school's response sound, suppresses background interference, and gives the characteristic components a clear physical meaning.

[0048] The stress response feature components corresponding to each response acoustic frame are arranged in frame order to generate a stress acoustic feature sequence. This sequence reflects the complete dynamic process of fish stress response from induction, enhancement to decay in the time dimension, and reflects the spectral distribution characteristics of stress response in the frequency dimension, providing a time-domain and frequency-domain acoustic feature foundation for subsequent cross-modal fusion and disease identification.

[0049] S103, the sonar image sequence is divided into a baseline image group and a response image group according to the stimulation time point, and optical flow calculation is performed on each group to obtain the fish swarm velocity field sequence; the velocity amplitude distribution characteristics, motion direction consistency characteristics and vortex intensity characteristics are extracted from the fish swarm velocity field sequence to generate a stress behavior characteristic sequence.

[0050] This step aims to extract quantitative features of fish movement behavior from sonar image sequences. By using optical flow calculations, pixel motion in the image domain is transformed into vector distribution in the velocity field, and multidimensional behavioral features reflecting the intensity and pattern of fish stress response are extracted. This provides a quantitative basis for behavioral dimensions for subsequent cross-modal fusion and disease identification.

[0051] Specifically, the sonar image sequence is divided into a baseline image group and a response image group based on the stimulus time point. Using the emission time of the acoustic stimulus signal as the time reference point, the sonar images acquired before the reference point constitute the baseline image group, corresponding to the normal swimming state of the fish under no external stimulus conditions; the sonar images acquired after the reference point constitute the response image group, corresponding to the dynamic stage of the fish exhibiting stress behavior after being stimulated by the acoustic stimulus. Taking a high-frequency imaging sonar frame rate of 15 frames / second, with 5 seconds of acquisition before stimulation and 10 seconds of acquisition after stimulation as an example, the baseline image group contains approximately 75 frames, and the response image group contains approximately 150 frames. The principle of dividing the two groups according to the stimulus time point is that the baseline image group reflects the baseline movement level of the fish in a quiet state, including the speed distribution generated by spontaneous behaviors such as normal cruising and feeding activities; the response image group superimposes the baseline movement with the incremental stress behaviors such as escape and avoidance induced by the acoustic stimulus. By processing two groups separately and then comparing and analyzing them, normal movement interference unrelated to stress can be effectively removed, so that the extracted behavioral characteristics can purely reflect the fish's response to sound stimuli, thereby improving the sensitivity and reliability of disease diagnosis.

[0052] Specifically, optical flow calculations are performed to obtain the fish swarm velocity field sequence, including: The first step involves using a dense optical flow algorithm based on polynomial expansion to estimate the displacement of each pixel in the image plane for two temporally adjacent sonar images within both the baseline and response image groups. The principle of this algorithm is to approximate the gray-level distribution of each pixel's neighborhood using a local polynomial. By comparing the changes in the polynomial coefficients of corresponding pixels in adjacent frames, the displacement vector of the pixel is calculated. Compared to gradient-based sparse optical flow algorithms, this method can output a velocity estimate for each pixel in the image, forming a dense velocity field covering the entire target area, which is crucial for characterizing the overall movement distribution of fish swarms. Furthermore, the polynomial expansion method is robust to imaging artifacts such as speckle noise and uneven gray-level distribution in sonar images, maintaining high optical flow estimation accuracy even under low signal-to-noise ratio conditions. In practical implementation, the order of the polynomial expansion can be set to second order, the smoothing kernel size to 5×5 pixels, and the number of iterations to three, achieving a balance between computational efficiency and estimation accuracy.

[0053] The second step involves removing bits by the time interval between two frames to obtain the instantaneous velocity vector of that pixel. The velocity vectors of all pixels within a frame interval constitute the instantaneous fish swarm velocity field for that interval. This instantaneous fish swarm velocity field describes the motion state of the fish swarm within the entire imaging plane at a given moment in the form of a two-dimensional vector field. The direction of each vector represents the direction of movement of the fish at that position, and the magnitude of the vector represents the magnitude of the motion velocity, thus transforming the apparent motion in the image sequence into a velocity distribution field with clear physical meaning.

[0054] The expression for the instantaneous velocity vector is:

[0055] In the formula, For the first Coordinates within the frame interval The instantaneous velocity vector of a pixel, a two-dimensional vector; The horizontal velocity vector ( Directional component; The perpendicular of the velocity vector ( Directional component, unit as above; , The first Frame to the +1 frame between pixels Estimated displacement values ​​in the horizontal and vertical directions; The time interval between two adjacent sonar frames is equal to the reciprocal of the frequency. These are the pixel coordinates within the image plane.

[0056] The third step involves arranging the instantaneous fish velocity fields of all frame intervals within the baseline and response image groups in chronological order to obtain a fish velocity field sequence. This sequence comprehensively records the entire process of the fish's movement evolution from its baseline state to its stress response after stimulation and then to its gradual recovery, providing a frame-by-frame velocity field data foundation for subsequent temporal analysis of behavioral characteristics.

[0057] Furthermore, velocity amplitude distribution features, motion direction consistency features, and vortex intensity features are extracted from the fish swarm velocity field sequence to generate a stress behavior feature sequence, specifically including: The first step is to calculate the arithmetic mean and standard deviation of the velocity vector magnitudes of all pixels within the target region for each velocity field in the fish swarm velocity field sequence, as the velocity amplitude distribution characteristics at the corresponding time point. The arithmetic mean of the velocity amplitude reflects the overall activity level of the fish swarm at that time; a higher mean indicates faster swimming and a stronger stress response. The standard deviation reflects the dispersion of the movement velocity within the fish swarm; a larger standard deviation indicates more significant speed differences between individuals within the swarm. Healthy fish swarms, after being stimulated by sound, often exhibit a rapid increase in the overall velocity mean and a synchronous increase in the standard deviation. However, in the early stages of disease, due to decreased physical strength or impaired motor coordination, the increase in the velocity mean is limited, and the standard deviation distribution is abnormal.

[0058] The arithmetic mean of the velocity vector magnitude is expressed as:

[0059] Wherein the velocity amplitude (magnitude):

[0060] In the formula, For the first The average velocity amplitude of the frame reflects the overall activity level of the fish school, and the unit is the same as that of velocity; For pixels The magnitude (amplitude) of the velocity vector at that point; The target monitoring area in the sonar image; For target area The area (number of pixels or actual area) is the total number of pixels included in the statistics.

[0061] Wherein, the standard deviation of the velocity vector magnitude is:

[0062] In the formula, denoted as the standard deviation of the velocity amplitude in the m-th frame, reflecting the degree of dispersion of the movement velocity within the fish swarm.

[0063] The second step involves normalizing the velocity vector of each pixel within the target area to a unit direction vector. The sum of all unit direction vectors is then calculated, and the motion direction consistency feature is obtained by dividing the magnitude of the sum by the total number of pixels involved in the summation. The physical meaning of this feature is to measure the degree of uniformity in the movement direction of the fish school: when all pixels move in the same direction, the magnitude of the unit vector sum equals the total number of pixels, and the feature value approaches 1; when the movement directions are randomly distributed, the vectors cancel each other out, and the feature value approaches 0. Healthy fish schools often exhibit highly consistent directional movement during escape behavior induced by sound stimulation, resulting in a significantly increased direction consistency feature value; however, in the early stages of disease, due to impaired neural regulation or decreased orientation ability, the escape behavior of fish is scattered, and the increase in the direction consistency feature value is significantly lower.

[0064] The expression for the consistency feature of motion direction is as follows:

[0065] In the formula, For the first The motion direction consistency feature value of the frame, with a value range of [0,1]; The closer it is to 1, the more consistent the directions are. The closer it is to 0, the more random and scattered the direction becomes.

[0066] The third step involves calculating the curl of the velocity vector pixel by pixel in the velocity field to obtain a curl scalar field. Then, the spatial average of the absolute curl values ​​within the target region is calculated as the vortex intensity characteristic. Curl describes the intensity and direction of local rotational motion in the velocity field; a larger absolute value indicates more intense rotational motion at that location. Healthy fish exhibit primarily directional translational movement during escape, resulting in low vortex intensity. However, in the early stages of disease, fish may exhibit abnormal movement patterns such as circling and imbalance, leading to an abnormally high vortex intensity characteristic.

[0067] The expression for the curl scalar field is:

[0068] In the formula, For the first Frame in pixel The curl of the velocity field at a given location describes the intensity and direction of the local rotational motion; Vertical velocity component Along the horizontal direction The partial derivatives; Horizontal velocity component along the vertical direction The partial derivatives of .

[0069] The expression for the vortex intensity characteristic is as follows:

[0070] In the formula, For the first The vortex intensity characteristics of the frame reflect the average intensity of the overall rotational component in the movement of the fish school; For the curl scalar field in pixels The absolute value at that position indicates the intensity of the rotational motion at that location.

[0071] The fourth step involves combining the velocity amplitude distribution characteristics, motion direction consistency characteristics, and vortex intensity characteristics at the same moment into a single behavioral feature vector. These behavioral feature vectors are then arranged chronologically to form a stress behavior feature sequence. This sequence comprehensively characterizes the fish school's stress behavior response from three dimensions: motion intensity, directional coordination, and motion pattern, providing rich behavioral feature inputs for subsequent cross-modal fusion and disease identification.

[0072] For example, in an embodiment of the present invention, 500 bass are raised in the target aquaculture pond. The sonar operates at a frequency of 1.5 MHz and a frame rate of 20 frames per second. The baseline image is acquired for 4 seconds before stimulation and the response image is acquired for 8 seconds after stimulation. The baseline image set contains 80 frames and the response image set contains 160 frames. After calculating the velocity field using the second-order polynomial expansion dense optical flow algorithm, the average velocity amplitude of the healthy fish population during the response phase rapidly increases from 2.3 cm / s at the baseline to 12.7 cm / s, the directional consistency increases from 0.28 to 0.86, and the vortex intensity remains at a low level. However, the average velocity amplitude of the fish population in the early stage of disease only increases to 6.1 cm / s, the directional consistency only increases to 0.42, but the vortex intensity increases by 2.1 times compared to the baseline. The abnormal combination of these three characteristics can effectively characterize the behavioral differences in the early stage of disease.

[0073] S104: The stress acoustic feature sequence is unfolded into a two-dimensional acoustic map, and the stress behavioral feature sequence is unfolded into a two-dimensional behavioral map. The two-dimensional acoustic map and the two-dimensional behavioral map are respectively input into a preset cross-modal contrast network. The contrast network includes an acoustic branch convolutional network and a behavioral branch convolutional network, with a bidirectional cross-modal attention module between the two branches. The bidirectional cross-modal attention module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, and outputs acoustic representation vectors and behavioral representation vectors. The acoustic representation vectors and behavioral representation vectors are fused to obtain a fused representation vector. The contrast network adopts a self-supervised training method to construct cross-modal positive sample pairs and negative sample pairs. The distance between positive sample pairs in the representation space is constrained to be less than the distance between negative sample pairs in the representation space by InfoNCE loss to complete the training. A healthy baseline vector is generated based on the training samples, and a preset threshold is set based on the healthy baseline vector.

[0074] This step aims to map stress acoustic features and stress behavioral features into a unified high-dimensional representation space through a cross-modal contrastive learning framework. It utilizes a bidirectional cross-modal attention mechanism to achieve deep interaction and complementary enhancement of the two types of modal information, thereby generating a fusion representation vector that can comprehensively characterize the stress state of fish populations. Furthermore, it establishes a health baseline and discrimination threshold through self-supervision, providing an objective and quantifiable basis for subsequent disease early warning.

[0075] Specifically, the stress acoustic feature sequence is unfolded into a two-dimensional acoustic map. The stress acoustic feature sequence is generated by S102, with each frame corresponding to a stress response feature component. This component is a one-dimensional vector distributed along the frequency dimension, reflecting the energy increment of the fish swarm's stress-induced sound at different frequency bands at that moment. The stress response feature components of each frame are arranged sequentially along the second dimension in chronological order, thus forming a two-dimensional acoustic map with frequency as the first dimension and time as the second dimension. This two-dimensional acoustic map is essentially a time-frequency image where energy amplitude is the pixel value. Its vertical dimension corresponds to the frequency scale, and its horizontal dimension corresponds to the time frame sequence. The value of each pixel represents the magnitude of the stress response energy at that time-frequency location. The principle of unfolding a one-dimensional feature sequence into a two-dimensional map is that convolutional networks are naturally good at extracting local structural features in two-dimensional space. The excitation sound of a school of fish often exhibits specific texture patterns and temporal evolution rules on the time-frequency plane. For example, the excitation sound of a healthy school of fish will form obvious energy bands in the mid-to-high frequency bands in a short time after stimulation and gradually decay over time. This time-frequency joint distribution pattern is difficult to fully capture through one-dimensional sequence analysis. However, inputting the two-dimensional map into the convolutional network can effectively utilize the network's spatial feature extraction capabilities and automatically learn high-order statistical features and local texture patterns on the time-frequency plane, thereby improving the representation and discrimination capabilities of acoustic features.

[0076] Similarly, the stress behavior feature sequence is unfolded into a two-dimensional behavioral atlas. The stress behavior feature sequence is generated by S103, with each moment corresponding to a behavioral feature vector. This vector contains three behavioral indicators: velocity amplitude distribution characteristics, motion direction consistency characteristics, and vortex intensity characteristics. The behavioral feature vectors at each moment are arranged sequentially along the second dimension, forming a two-dimensional behavioral atlas with behavioral feature dimension as the first dimension and time as the second dimension. This two-dimensional behavioral atlas uses behavioral feature type as the row and time step as the column, with each element's value representing the quantized value of the corresponding behavioral feature at that moment. The principle of unfolding the behavioral feature sequence into a two-dimensional atlas is similar to that of acoustic atlases; that is, through two-dimensional processing, the temporal evolution pattern of behavioral features can be effectively extracted by the convolutional network. Simultaneously, the spatial arrangement of the three types of behavioral features in the first dimension allows the convolutional kernel to simultaneously capture the correlation and cooperative change patterns between different behavioral features, such as the temporal correspondence between an increase in velocity amplitude and an increase in directional consistency. This cross-feature dimension correlation information is of great significance for accurately characterizing the overall pattern of fish school stress behavior.

[0077] Then, the two-dimensional acoustic map and the two-dimensional behavioral map are respectively input into a pre-defined cross-modal contrastive network. This cross-modal contrastive network adopts a dual-branch parallel structure to process the input data of the acoustic modality and the behavioral modality separately, and realizes the interactive fusion of features of the two modalities through a bidirectional cross-modal attention module in the middle. The principle of adopting a dual-branch structure is that there are significant differences in the data distribution, feature scale, and semantic level between the acoustic modality and the behavioral modality. Using their respective independent convolutional branches, dedicated feature extraction layers and receptive field configurations can be designed for the characteristics of different modalities, avoiding insufficient feature extraction due to modal differences in a single branch. By setting a cross-modal interaction module between the two branches, information complementarity can be achieved at the high-level feature level, enabling the network to make comprehensive judgments using evidence from both modalities simultaneously, thereby obtaining a more comprehensive and robust representation result than a single modality.

[0078] The contrasting networks include an acoustic branch convolutional network and a behavioral branch convolutional network. The acoustic branch convolutional network takes a two-dimensional acoustic map as input and consists of stacked convolutional, pooling, and activation layers. It extracts multi-level acoustic features, from low-level time-frequency textures to high-level semantic patterns, through layer-by-layer convolutional operations, ultimately outputting an acoustic feature map. The kernel size and stride configuration of this branch are optimized for the anisotropy of the time-frequency map, employing different receptive field configurations in the frequency and time dimensions to accommodate the relative stability of acoustic features in the frequency direction and their rapid changes in the time direction. The behavioral branch convolutional network takes a two-dimensional behavioral map as input. Its structure is similar to the acoustic branch but its parameters are independent. It extracts the temporal evolution patterns and cross-feature association patterns of behavioral features through convolutional operations, ultimately outputting a behavioral feature map. The independent parameterization of the two branches ensures that each modality obtains a feature extractor best suited to its own data distribution, avoiding inter-modal interference and feature degradation caused by parameter sharing.

[0079] Furthermore, a bidirectional cross-modal attention module is set up between the two branches. This module calculates cross-attention features using features from one branch as the query and features from the other branch as the key, outputting acoustic and behavioral representation vectors. The core function of this module is to achieve bidirectional information interaction between the acoustic and behavioral modalities, enabling each modality's feature representation to actively focus on and integrate the most relevant information from the other modality, thereby generating an enhanced representation that incorporates cross-modal context. The principle behind using bidirectional attention instead of unidirectional attention is that the correlation between acoustic and behavioral features is mutual: a specific frequency band response on the acoustic side may correspond to a specific motion pattern on the behavioral side, and vice versa; bidirectional interaction ensures that the correlation in both directions is fully explored, avoiding information omissions caused by unidirectional information flow. Its specific implementation is as follows: The first step involves mapping the acoustic feature maps output by the acoustic branch convolutional network and the behavioral feature maps output by the behavioral branch convolutional network to a feature space of the same dimension. This mapping is achieved through separate fully connected layers. Since the network structures of the acoustic and behavioral branches are different, their output feature maps may have differences in channel and spatial dimensions. Direct attention calculation would be impossible due to dimensionality mismatch. Mapping the two types of features to a common feature space of the same dimension through separate fully connected layers establishes a comparable dimensional basis for the two modalities, making subsequent similarity calculations and weighted summations mathematically feasible. Simultaneously, the independent mapping parameters ensure that each modality can be projected into the common space in a way that best suits its own characteristics, avoiding feature distortion or information loss caused by shared mapping parameters. The mapped acoustic and behavioral features have the same feature dimensions and retain the semantic information of the original modality, laying the foundation for subsequent cross-modal attention calculations.

[0080] The second step involves using the mapped acoustic features as the query and the mapped behavioral features as the key and value. The cross-attention weights of the acoustic features and behavioral features are calculated, and then the behavioral features are weighted and summed according to these cross-attention weights to obtain the acoustic-side fused features. The physical meaning of this process is that, using the feature representation of each position in the acoustic features as a reference, the most relevant content is found across all positions in the behavioral features, and the behavioral features are weighted and aggregated according to their relevance. This supplements each position of the acoustic features with relevant contextual information from the behavioral modality. The principle of calculating the cross-attention weights is to measure the semantic relevance between a certain position of the acoustic feature and all positions of the behavioral features through the dot product operation of the query vector and the key vector, and then normalize to obtain the weight distribution; a higher weight indicates that the information at that behavioral position is more important to the representation of the current acoustic position. The effect of this weighted aggregation method is that acoustic features are no longer isolated frequency domain response descriptions, but are actively associated with corresponding motion evidence in behavioral modalities. For example, when a strong response occurs in the mid-to-high frequency band, the network will automatically focus on the corresponding time period when the velocity amplitude in the behavioral features increases, so that the acoustic representation can be corroborated and enhanced at the behavioral level, thereby improving the discriminative power and reliability of the representation.

[0081] The third step involves using the mapped behavioral features as queries and the mapped acoustic features as keys and values. The cross-attention weights of the behavioral features on the acoustic features are calculated, and then the acoustic features are weighted and summed according to these cross-attention weights to obtain the behavioral-side fusion features. This third step is symmetrical to the second step; that is, using the behavioral features as a reference, relevant information is searched for and weighted within the acoustic features to supplement the behavioral features with contextual information from the acoustic modality. Its principle and beneficial effects are similar to acoustic-side fusion, only the information flow is reversed: the representation at each position in the behavioral features receives support from the relevant frequency band responses in the acoustic modality. For example, when the directional consistency in the behavioral features increases, the network automatically focuses on the corresponding frequency bands in the acoustic features that enhance the excitation sound, thus providing acoustic confirmation and supplementation to the behavioral representation. The bidirectional cross-attention design ensures that both modalities can obtain beneficial information from each other, achieving true bidirectional enhancement rather than unidirectional dependence.

[0082] The fourth step involves adding the acoustic-side fused features to the original acoustic features to obtain the acoustic representation vector, and adding the behavioral-side fused features to the original behavioral features to obtain the behavioral representation vector. The principle behind using residual addition instead of direct replacement is that while cross-modal fused features supplement information from another modality, the core information of the original features within the current modality is equally important. Direct replacement may lead to the loss or distortion of this modal information. By using residual addition, the network can adaptively superimpose cross-modal supplementary information while preserving the original features, ensuring that the fused representation possesses both the core discriminative power of the current modality and the enhanced information from the cross-modality, thus achieving a synergistic effect greater than the sum of its parts. This residual connection design also helps alleviate the vanishing gradient problem in deep network training, improving the network's training stability and convergence speed.

[0083] After outputting the acoustic and behavioral representation vectors, the acoustic and behavioral representation vectors are fused to obtain a fused representation vector. In practice, this can be achieved through either a concatenation fusion method, where the acoustic and behavioral representation vectors are concatenated end-to-end along their feature dimensions to form a fused representation vector whose dimension is the sum of the two; or a weighted summation method, where weights are assigned based on the confidence or importance of the two modalities and then summed to obtain the fused representation vector. The principle of concatenation fusion is to preserve the feature information of each modality to the greatest extent possible without additional information compression, making it suitable for subsequent discrimination layers to learn how to combine and utilize the two types of information. The principle of weighted summation is to dynamically adjust the contribution ratio of each modality based on modal reliability; when the quality of a modality deteriorates due to environmental interference, its weight can be reduced, improving the robustness of the system. As the final output of the cross-modal comparison network, the fused representation vector integrates the stress response information of both acoustic and behavioral modalities, comprehensively depicting the physiological state and stress response capabilities of the fish population from multiple dimensions, providing a high-quality feature foundation for subsequent disease identification.

[0084] Finally, the contrastive network employs a self-supervised training method, constructing cross-modal positive and negative sample pairs. Training is completed by constraining the distance between positive sample pairs in the representation space to be less than the distance between negative sample pairs using InfoNCE loss. The core idea of ​​self-supervised training is to utilize the inherent structure of the data itself to construct supervisory signals, eliminating the need for manual disease labeling. This is because early-stage disease samples are often difficult to obtain and labeling is costly, while self-supervised methods can utilize a large amount of unlabeled healthy fish data for pre-training. Specifically, cross-modal positive sample pairs are constructed as follows: in the same acoustic stimulus experiment, the fused representation vectors corresponding to the collected acoustic and behavioral data constitute positive sample pairs, because they describe the state of the same fish group in the same stress event and should be close to each other in the representation space. Conversely, cross-modal negative sample pairs are constructed as follows: the fused representation vectors corresponding to acoustic and behavioral data collected in different experiments, different fish groups, or at different times constitute negative sample pairs, because they describe different stress events or different fish group states and should be far apart in the representation space. The InfoNCE loss function maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs by comparing their similarity. Its principle is similar to contrastive loss in metric learning, enabling the network to learn a representation space with good intra-class aggregation and inter-class separation properties. Employing self-supervised contrastive learning allows the network to automatically discover semantic correspondences between acoustic and behavioral modalities. The learned representations exhibit stronger generalization and transferability, achieving high-quality feature representations even with scarce labeled samples, laying a solid foundation for subsequent disease identification.

[0085] The expression for the loss function is:

[0086] The cosine similarity is:

[0087] In the formula, The comparison loss function value for InfoNCE; This is the fused representation vector from the acoustic modes in the positive sample pair; This is the fused representation vector from the behavioral modality in the positive sample pair; For the first The behavioral fusion representation vectors of each negative sample, totaling... One negative sample; The total number of negative samples; Temperature coefficient; for The dot product of two vectors.

[0088] This includes generating a healthy baseline vector based on training samples and setting a preset threshold based on the healthy baseline vector, specifically including: The healthy baseline vector is generated by averaging the fusion representation vectors corresponding to healthy samples during training. Healthy samples refer to the fusion representation vectors generated by a known healthy fish population during an acoustic stimulus experiment. These samples are sufficient in number and have clearly defined states, representing the standard stress response pattern of a healthy fish population. The fusion representation vectors of all healthy samples are arithmetically averaged element-wise along each feature dimension, and the resulting average vector is the healthy baseline vector. This healthy baseline vector represents the "center" or "typical pattern" of the stress response of a healthy fish population in the representation space, with each feature value reflecting the average level of the healthy fish population in that feature dimension. The principle of averaging is that individual healthy samples may exhibit random variations due to individual differences, environmental fluctuations, etc., while the statistical average of a large number of samples can suppress these random fluctuations, obtaining a stable and reliable health status benchmark, providing a reference standard for subsequent anomaly detection.

[0089] The expression for the health baseline vector is:

[0090] In the formula, The healthy baseline vector represents the statistical center of the stress response pattern of healthy fish populations in space; The total number of healthy samples in the training set; For the first The fusion representation vector corresponding to each healthy sample, with superscript... Indicates the first One healthy sample.

[0091] A preset threshold is set based on the maximum cosine distance between the fused representation vector and the healthy baseline vector in the training samples. Cosine distance measures the degree of difference between two vectors in direction; a smaller value indicates greater similarity. For all healthy samples in the training set, the cosine distance between their fused representation vector and the healthy baseline vector is calculated. The maximum value among these distances is taken as the reference benchmark for the preset threshold, with a margin reserved to reduce the false alarm rate. The principle behind setting the threshold based on the maximum cosine distance of healthy samples is that the distance between the representation vector of all healthy samples and the baseline should be within this threshold. Therefore, when the distance of a new sample exceeds the threshold, it can be determined that its stress response pattern deviates from the normal range of healthy status, suggesting a possible early abnormality in disease. This threshold setting method based on statistical distribution has clear physical meaning and a controllable false alarm rate, providing an objective and quantifiable basis for subsequent early warning judgments.

[0092] S105, calculate the cosine distance between the fused representation vector and the healthy baseline vector, compare the cosine distance with a preset threshold, and generate an early warning signal for the disease when the cosine distance corresponding to multiple consecutive stimuli exceeds the preset threshold.

[0093] Specifically, cosine distance is obtained by first calculating the cosine similarity between two vectors and then subtracting that value from 1. The principle behind using cosine distance is that it measures the difference in vector direction rather than the difference in amplitude, which can effectively eliminate the interference of amplitude fluctuations caused by factors such as fish size and aquatic environment, so that the distance value purely reflects the degree of deviation of the response pattern.

[0094] The cosine distance is compared with a preset threshold. This preset threshold is set in S104 based on the maximum cosine distance between the fused representation vector of the training samples and the healthy baseline vector, with a certain margin reserved. When the cosine distance is less than or equal to the preset threshold, it indicates that the stress response pattern is still within the normal fluctuation range of a healthy state and is judged as normal; when the cosine distance exceeds the preset threshold, it indicates that it has deviated from the healthy baseline and there is a possibility of anomaly. The principle of directly comparing the cosine distance with the preset threshold is that the preset threshold is an objective benchmark determined based on the statistical distribution of healthy samples. A single comparison can quickly screen potential anomalies, providing an initial screening basis for subsequent continuous verification. At the same time, it converts continuous distance values ​​into binary judgment results, simplifying subsequent logical judgments.

[0095] An early warning signal for disease is generated when the cosine distance corresponding to multiple consecutive stimuli exceeds a preset threshold. In practice, the number of consecutive stimuli can be set to 3 to 5, with an interval of 30 minutes to 2 hours between each stimulation, to ensure the fish population fully recovers and avoid the cumulative effect of continuous stress affecting the accuracy of the judgment. The principle behind the continuous verification mechanism is that a single anomaly may be caused by sporadic factors, such as temporary abnormalities in individual fish, brief disturbances in the water, or instantaneous noise from equipment. Relying solely on a single anomaly for early warning would result in a high false alarm rate. Continuous verification effectively eliminates sporadic interference, ensuring the continuous abnormality of the physiological state of the fish population corresponding to the warning, thereby significantly reducing the false alarm rate and improving system reliability. The generated warning signal can be notified to management personnel through audible and visual alarms, terminal push notifications, etc.

[0096] like Figure 2 As shown, based on the same inventive concept, this embodiment provides an early intelligent warning system for diseases in high-density farmed fish, including: The data acquisition module 201 is used to transmit acoustic stimulation signals to the fish school through an acoustic emission transducer set in the target water body, and simultaneously collect water acoustic signals and fish sonar images before and after stimulation through a hydrophone array and sonar. The water acoustic signals are sliced ​​according to a fixed time window to obtain an acoustic frame sequence, and the fish sonar images are arranged in frame order to obtain a sonar image sequence. The stress acoustic module 202 is used to divide the sound frame sequence into a baseline sound frame group and a response sound frame group according to the stimulus time point. After performing time-frequency decomposition on each frame in the response sound frame group, the mean component of the corresponding frequency band of the baseline sound frame group is subtracted to obtain the stress response feature component, which is then arranged in frame order to generate a stress acoustic feature sequence. The stress behavior module 203 is used to divide the sonar image sequence into a baseline image group and a response image group according to the stimulation time point, and perform optical flow calculations on each group to obtain the fish swarm velocity field sequence; and extract the velocity amplitude distribution features, motion direction consistency features and vortex intensity features from the fish swarm velocity field sequence to generate a stress behavior feature sequence. The cross-modal fusion module 204 is used to expand the stress acoustic feature sequence into a two-dimensional acoustic map and the stress behavioral feature sequence into a two-dimensional behavioral map. The two-dimensional acoustic map and the two-dimensional behavioral map are respectively input into a preset cross-modal contrast network. The contrast network includes an acoustic branch convolutional network and a behavioral branch convolutional network, with a bidirectional cross-modal attention module between the two branches. The bidirectional cross-modal attention module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, outputting acoustic representation vectors and behavioral representation vectors. The acoustic representation vector and the behavioral representation vector are fused to obtain a fused representation vector. The contrast network uses a self-supervised training method to construct cross-modal positive and negative sample pairs. Training is completed by constraining the distance between positive sample pairs in the representation space to be less than the distance between negative sample pairs in the representation space using InfoNCE loss. A healthy baseline vector is generated based on the training samples, and a preset threshold is set based on the healthy baseline vector. The early warning module 205 is used to calculate the cosine distance between the fused representation vector and the healthy baseline vector, compare the cosine distance with a preset threshold, and generate an early warning signal for the disease when the cosine distance corresponding to multiple consecutive stimuli exceeds the preset threshold.

[0097] like Figure 3 As shown, based on the same inventive concept, this embodiment provides an electronic device, including: Memory 302 is used to store computer programs; Processor 301 is used to implement the method steps as described in the first aspect when executing a computer program.

[0098] Based on the same inventive concept, this embodiment provides a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method steps as described in the first aspect.

[0099] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0100] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for early intelligent warning of diseases in high-density farmed fish, characterized in that, include: Acoustic stimulation signals are emitted to the fish by an acoustic transducer installed in the target water body. At the same time, the water acoustic signals and fish sonar images are collected before and after the stimulation by a hydrophone array and sonar. The water acoustic signals are sliced ​​into a sound frame sequence according to a fixed time window, and the fish sonar images are arranged in frame order to obtain a sonar image sequence. The acoustic frame sequence is divided into a baseline acoustic frame group and a response acoustic frame group according to the stimulus time point. After performing time-frequency decomposition on each frame in the response acoustic frame group, the mean component of the corresponding frequency band of the baseline acoustic frame group is subtracted to obtain the stress response feature components. These components are then arranged in frame order to generate a stress acoustic feature sequence. The sonar image sequence is divided into a baseline image group and a response image group according to the stimulation time point, and optical flow calculation is performed on each group to obtain the fish swarm velocity field sequence. The velocity amplitude distribution feature, motion direction consistency feature and vortex intensity feature are extracted from the fish swarm velocity field sequence to generate a stress behavior feature sequence. The stress acoustic feature sequence is unfolded into a two-dimensional acoustic map, and the stress behavioral feature sequence is unfolded into a two-dimensional behavioral map. The two-dimensional acoustic map and the two-dimensional behavioral map are respectively input into a preset cross-modal contrast network. The contrast network includes an acoustic branch convolutional network and a behavioral branch convolutional network, and a bidirectional cross-modal attention module is set between the two branches. The bidirectional cross-modal attention module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, outputting an acoustic representation vector and a behavioral representation vector; the acoustic representation vector and the behavioral representation vector are fused to obtain a fused representation vector; the contrast network adopts a self-supervised training method to construct cross-modal positive sample pairs and negative sample pairs, and completes training by constraining the distance between the positive sample pairs in the representation space to be less than the distance between the negative sample pairs in the representation space through InfoNCE loss; A healthy baseline vector is generated based on the training samples, and a preset threshold is set based on the healthy baseline vector. Calculate the cosine distance between the fused representation vector and the healthy baseline vector, compare the cosine distance with a preset threshold, and generate an early warning signal for disease when the cosine distance corresponding to multiple consecutive stimuli exceeds the preset threshold.

2. The method for early intelligent warning of diseases in high-density farmed fish according to claim 1, characterized in that, The step of slicing the water body acoustic signal into an acoustic frame sequence according to a fixed time window specifically includes: A Hanning window of a preset duration is selected as the analysis window function. The time-domain waveform of the water body acoustic signal is multiplied by the Hanning window segment by segment to achieve windowing. The window moves forward by a fixed number of sampling points each time, so that the waveform intervals covered by adjacent windows partially overlap. Each segment of the time-domain waveform obtained by windowing is recorded as an acoustic frame. The acoustic frames are arranged in the order of extraction to obtain the acoustic frame sequence.

3. The method for early intelligent warning of diseases in high-density farmed fish according to claim 1, characterized in that, After performing time-frequency decomposition on each frame in the response acoustic frame group, the mean component of the corresponding frequency band of the baseline acoustic frame group is subtracted to obtain the stress response feature components, specifically including: Perform a complex wavelet transform on each response sound frame in the response sound frame group to obtain the complex time spectrum, and take its modulus to obtain the time-frequency amplitude matrix of the sound frame; For the time-frequency amplitude matrix of all baseline acoustic frames in the baseline acoustic frame group, the arithmetic mean is calculated point by point along the time dimension to obtain the baseline mean spectrum in the corresponding frequency direction. For each time-frequency amplitude matrix of the response acoustic frame, the amplitude of each frequency point in the matrix is ​​subtracted from the mean of the corresponding frequency point in the baseline mean spectrum, and the amplitude with a negative difference is set to zero, while the positive part is retained as the stress response characteristic component of the response acoustic frame.

4. The method for early intelligent warning of diseases in high-density farmed fish according to claim 1, characterized in that, The step of performing optical flow calculations to obtain the fish swarm velocity field sequence specifically includes: For two temporally adjacent sonar images within the baseline image group and the response image group, a dense optical flow algorithm based on polynomial expansion is used to estimate the displacement of each pixel in the image plane. The bit is removed by the time interval between two frames to obtain the instantaneous velocity vector of the pixel; the instantaneous fish swarm velocity field of the interval is formed by the velocity vectors of all pixels within a frame interval. The instantaneous fish swarm velocity fields of all frame intervals within the baseline image group and the response image group are arranged chronologically to obtain the fish swarm velocity field sequence.

5. The method for early intelligent warning of diseases in high-density farmed fish according to claim 4, characterized in that, The step of extracting velocity amplitude distribution features, motion direction consistency features, and vortex intensity features from the fish swarm velocity field sequence to generate a stress behavior feature sequence specifically includes: For each velocity field in the fish swarm velocity field sequence, the arithmetic mean and standard deviation of the velocity vector magnitudes of all pixels in the target area are calculated as the velocity amplitude distribution characteristics at the corresponding time. The velocity vector of each pixel in the target area is normalized to a unit direction vector. The sum of all unit direction vectors is calculated. The motion direction consistency feature is obtained by dividing the magnitude of the sum by the total number of pixels involved in the summation. The curl of the velocity vector is calculated pixel by pixel in the velocity field to obtain a curl scalar field. Then, the spatial average value of the absolute value of the curl in the target area is calculated as the vortex intensity feature. The velocity amplitude distribution feature, the motion direction consistency feature and the vortex intensity feature at the same moment are combined into a behavior feature vector. The behavior feature vectors at each moment are arranged in chronological order to form the stress behavior feature sequence.

6. The method for early intelligent warning of diseases in high-density farmed fish according to claim 1, characterized in that, The bidirectional cross-modal attention module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, outputting acoustic representation vectors and behavioral representation vectors, specifically including: The acoustic feature map output by the acoustic branch convolutional network and the behavioral feature map output by the behavioral branch convolutional network are respectively mapped to the feature space of the same dimension, and the mapping is achieved through their respective independent fully connected layers; The mapped acoustic features are used as queries, and the mapped behavioral features are used as keys and values. The cross-attention weights of the acoustic features and the behavioral features are calculated, and then the behavioral features are weighted and summed according to the cross-attention weights to obtain the acoustic-side fusion features. The mapped behavioral features are used as queries, and the mapped acoustic features are used as keys and values. The cross-attention weights of the behavioral features and acoustic features are calculated, and then the acoustic features are weighted and summed according to the cross-attention weights to obtain the behavioral-side fusion features. The acoustic-side fusion features are added to the original acoustic features to obtain the acoustic representation vector, and the behavioral-side fusion features are added to the original behavioral features to obtain the behavioral representation vector.

7. The method for early intelligent warning of diseases in high-density farmed fish according to claim 1, characterized in that, The step of generating a healthy baseline vector based on training samples and setting a preset threshold based on the healthy baseline vector specifically includes: The average value of the fusion representation vectors corresponding to healthy samples during training is used to generate a healthy baseline vector. A preset threshold is set based on the maximum cosine distance between the fusion representation vectors in the training samples and the healthy baseline vector.

8. An intelligent early warning system for diseases in high-density cultured fish, based on the intelligent early warning method for diseases in high-density cultured fish as described in claim 1, characterized in that, include: The data acquisition module is used to transmit acoustic stimulation signals to the fish school through an acoustic emission transducer set in the target water body, and simultaneously collect water acoustic signals and fish sonar images before and after stimulation through a hydrophone array and sonar. The water acoustic signals are sliced ​​into an acoustic frame sequence according to a fixed time window, and the fish sonar images are arranged in frame order to obtain a sonar image sequence. The stress acoustic module is used to divide the sound frame sequence into a baseline sound frame group and a response sound frame group according to the stimulation time point. After performing time-frequency decomposition on each frame in the response sound frame group, the mean component of the corresponding frequency band of the baseline sound frame group is subtracted to obtain the stress response feature component, and the components are arranged in frame order to generate a stress acoustic feature sequence. The stress behavior module is used to divide the sonar image sequence into a baseline image group and a response image group according to the stimulation time point, and perform optical flow calculations on each group to obtain the fish swarm velocity field sequence; from the fish swarm velocity field sequence, the velocity amplitude distribution features, the motion direction consistency features, and the vortex intensity features are extracted to generate a stress behavior feature sequence. A cross-modal fusion module is used to expand the stress acoustic feature sequence into a two-dimensional acoustic map and the stress behavioral feature sequence into a two-dimensional behavioral map. The two-dimensional acoustic map and the two-dimensional behavioral map are respectively input into a preset cross-modal comparison network. The comparison network includes an acoustic branch convolutional network and a behavioral branch convolutional network, and a bidirectional cross-modal attention module is set between the two branches. The bidirectional cross-modal attention module calculates cross-attention features using one branch feature as the query and the other branch feature as the key, outputting an acoustic representation vector and a behavioral representation vector; the acoustic representation vector and the behavioral representation vector are fused to obtain a fused representation vector; the contrast network adopts a self-supervised training method to construct cross-modal positive sample pairs and negative sample pairs, and completes training by constraining the distance between the positive sample pairs in the representation space to be less than the distance between the negative sample pairs in the representation space through InfoNCE loss; A healthy baseline vector is generated based on the training samples, and a preset threshold is set based on the healthy baseline vector. The early warning module is used to calculate the cosine distance between the fused representation vector and the health baseline vector, compare the cosine distance with a preset threshold, and generate an early warning signal for the disease when the cosine distance corresponding to multiple consecutive stimuli exceeds the preset threshold.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the method for early intelligent warning of diseases in high-density farmed fish as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements an early intelligent warning method for diseases in high-density farmed fish as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Fish health early warning system based on multiple modes

    CN119478650A