A method and system for identifying abnormal behavior in large yellow croaker based on radio frequency signals

Abstract

This invention relates to the field of fish anomaly recognition technology, and discloses a method and system for recognizing abnormal behavior in large yellow croaker based on radio frequency signals. The method includes: acquiring radio frequency signals of tagged fish; extracting features to obtain behavioral feature indicators; obtaining a first anomaly probability based on the behavioral feature indicators and a health baseline; acquiring the behavioral signals of a fish school; calculating the group behavioral entropy; obtaining a second anomaly probability based on the group behavioral entropy and an entropy threshold; obtaining an initial abnormal fish school based on the first and second anomaly probabilities; acquiring the underwater acoustic signals and image data of the initial abnormal fish school; extracting features from the underwater acoustic signals to obtain acoustic features; performing feature statistics on the image data to obtain image features; and inputting the acoustic and image features into an anomaly recognition model for secondary recognition of abnormal behavior. This invention achieves accurate recognition of abnormal behavior in large yellow croaker through the collaborative determination of group behavioral entropy and behavioral trajectory, and cross-validation through multimodal data fusion.

Description

Technical Field

[0001] This invention relates to the field of fish abnormal behavior recognition technology, and in particular to a method and system for recognizing abnormal behavior in large yellow croaker based on radio frequency signals. Background Technology

[0002] As an important economic fish species in my country, the large yellow croaker's health status is crucial for improving aquaculture efficiency and reducing economic losses due to the large-scale and intelligent development of aquaculture. Traditional aquaculture monitoring mainly relies on manual inspections or single-sensor data for fish status assessment. The limitations of traditional methods lie in the fact that signals from single sensors, such as image signals, are easily affected by environmental factors, leading to insufficient robustness of feature extraction and incomplete information, resulting in inaccurate assessment results. While some technologies attempt to assess fish status through the fusion of different modalities, these fusion methods are mostly simple feature splicing, failing to achieve deep semantic-level information complementarity.

[0003] Furthermore, current aquaculture of large yellow croaker includes not only nearshore aquaculture primarily using net cages, but also various methods such as deep-sea aquaculture platforms and deep-sea ecological enclosure aquaculture. Among these, marine ecological enclosure aquaculture is a sustainable aquaculture model that appropriately isolates the aquaculture area from the open sea through ecological enclosures, utilizing natural ocean currents to achieve water self-purification while maintaining interaction between the farmed organisms and the wild ecosystem. Compared to traditional net cage aquaculture, it has advantages such as large aquaculture capacity, high ecological compatibility, and product quality close to that of wild organisms, and has become the mainstream development direction of deep-sea aquaculture. However, marine ecological enclosures are deployed in open sea areas, characterized by strong environmental dynamics and complex disturbance factors. Most existing monitoring technologies are designed based on closed ponds or nearshore net cage scenarios, making it difficult to adapt to the semi-open ecosystem characteristics of ecological enclosures. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for identifying abnormal behavior in large yellow croaker based on radio frequency signals. This method solves the problems of insufficient accuracy in existing identification methods and difficulty in adapting to semi-open ecological aquaculture environments, thereby improving the accuracy of identifying abnormal behavior in large yellow croaker and providing more reliable decision support for precision aquaculture and disease prevention.

[0005] In a first aspect, embodiments of the present invention provide a method for identifying abnormal behavior in large yellow croaker based on radio frequency signals, including:

[0006] The radio frequency signals of tagged fish with radio frequency chips in the fish school are obtained. The radio frequency signals are processed and feature extracted to obtain behavioral feature indicators. Based on the behavioral feature indicators and a preset health baseline, a first abnormality probability is obtained.

[0007] Acquire the behavioral signals of the fish school, calculate the group behavior entropy of the fish school based on the behavioral signals, and obtain the second anomaly probability based on the group behavior entropy and a preset entropy threshold.

[0008] Based on the first abnormal probability and the second abnormal probability, the fish swarm is initially identified as having abnormal behavior, and an initial abnormal fish swarm is obtained.

[0009] Acquire the underwater acoustic signal and image data of the initial abnormal fish swarm, extract features from the underwater acoustic signal to obtain acoustic features, and perform target detection and feature extraction on the image data to obtain image features;

[0010] The acoustic features and image features are input into a preset anomaly recognition model to perform secondary recognition of the abnormal behavior of the initial abnormal fish group. Based on the recognition results, the abnormal fish group and the corresponding intervention plan are determined. The anomaly recognition model is constructed based on a neural network model.

[0011] Further, the step of performing data processing and feature extraction on the radio frequency signal to obtain behavioral feature indicators, and obtaining a first abnormality probability based on the behavioral feature indicators and a preset health baseline, includes:

[0012] The radio frequency signal is subjected to data processing, which includes noise removal and missing value completion;

[0013] Feature extraction and feature statistics are performed on the radio frequency signals after data processing to obtain behavioral characteristic indicators of the fish population. The behavioral characteristic indicators include spatial distribution indicators, movement pattern indicators, rhythm indicators, and environmental response indicators.

[0014] A health baseline is obtained based on the current environmental parameters and a preset baseline database. An indicator threshold is determined based on the health baseline. The behavioral characteristic indicator is compared with the indicator threshold, and a first abnormal probability is obtained based on the comparison result.

[0015] Furthermore, the step of acquiring the behavioral signals of the fish school and calculating the group behavioral entropy of the fish school based on the behavioral signals includes:

[0016] First image data of the fish school is acquired, and target detection and trajectory tracking are performed on the first image data to obtain the temporal position sequence of each fish;

[0017] Feature extraction is performed on the temporal position sequence to obtain behavioral feature parameters for each fish, including movement speed and orientation angle.

[0018] Based on the joint distribution of velocity direction, grid cells are divided. Histogram statistics are performed according to the behavioral feature parameters and the grid division results to obtain statistical results, which include the number of fish in each grid cell and the total number of fish in all grid cells.

[0019] Based on the statistical results, the probability distribution of each grid cell is calculated, and the entropy of the fish group behavior is calculated based on the probability distribution and the entropy formula.

[0020] Furthermore, the step of acquiring the behavioral signals of the fish school and calculating the group behavioral entropy of the fish school based on the behavioral signals includes:

[0021] Acquire the group acoustic signal of the fish school, and preprocess the group acoustic signal, including high-pass filtering, frame windowing and validity verification;

[0022] Sound event detection is performed on the preprocessed group acoustic signal to obtain the sound emission time. Based on the sound emission time, the sound emission interval is calculated, and based on the sound emission interval, the sound emission standard deviation is calculated.

[0023] The power spectrum of the preprocessed group acoustic signal is estimated and the dominant frequency is extracted to obtain the dominant frequency sequence. The dominant frequency variance is calculated based on the dominant frequency sequence.

[0024] The group behavior entropy is obtained by normalizing and weighting the standard deviation of vocalization and the variance of dominant frequency.

[0025] Furthermore, the step of acquiring the behavioral signals of the fish school and calculating the group behavioral entropy of the fish school based on the behavioral signals includes:

[0026] Acquire first image data of the fish school, perform target detection on the first image data, and determine the validity of the first image data based on the target detection results;

[0027] If the first image data is valid, then based on the target detection result, trajectory tracking and behavioral feature extraction are performed, and based on the extracted behavioral feature parameters, the group behavior entropy of the fish school is calculated.

[0028] If the first image data is invalid, the group acoustic signal of the fish school is obtained, and acoustic event detection and main frequency extraction are performed on the group acoustic signal. Based on the detected sound interval and the extracted main frequency sequence, the group behavior entropy of the fish school is calculated.

[0029] Further, the step of initially identifying abnormal behavior in the fish swarm based on the first abnormal probability and the second abnormal probability to obtain an initial abnormal fish swarm includes:

[0030] The marking rate is obtained based on the number of marked fish in the fish swarm, and the anomaly probability weight is determined based on the marking rate.

[0031] Based on the aforementioned anomaly probability weights, the first anomaly probability and the second anomaly probability are weighted and summed to obtain the comprehensive anomaly probability.

[0032] Based on the comprehensive anomaly probability, the initial abnormal fish population is obtained.

[0033] Furthermore, the step of performing target detection and feature extraction on the image data to obtain image features includes:

[0034] Fish body recognition is performed on the first underwater image data in the image data by a fish body detection model to obtain fish body recognition results. The fish body recognition results include fish body bounding boxes and key point coordinates. The fish body detection model is constructed based on a deep neural network model.

[0035] The fish body posture angle is calculated based on the coordinates of the key points. The rollover rate is calculated based on the fish body posture angle. The aggregation density is calculated based on the fish body bounding box.

[0036] Moving targets are extracted from the water image data in the image data using the background subtraction method, and the extracted moving target images are then filtered.

[0037] The filtered moving target image is subjected to target screening and trajectory tracking to obtain fish leap events. The number of fish leap events within a preset time period is counted to obtain the number of fish leaps.

[0038] The rollover rate, the clustering density, and the number of fish leaps are used as the first image features.

[0039] Furthermore, the step of performing target detection and feature extraction on the image data to obtain image features also includes:

[0040] Acquire second underwater image data of the fish school, perform target tracking and region extraction on the second underwater image data to obtain key regions, and use the spatial mean of the color channels as the region signal;

[0041] Environmental noise is removed from the regional signal of the key region to obtain the regional time signal, and the regional time signal is then dimensionality reduced to obtain the residual signal.

[0042] Independent component analysis was used to extract high correlation information of heart rate from the residual signal to obtain the relevant sub-component signals;

[0043] The relevant sub-component signals are decomposed into time-frequency components, and the individual heart rate value is calculated based on the decomposition results.

[0044] Data filtering and statistical analysis are performed on multiple individual heart rate values ​​to obtain a heart rate abnormality index and a group mean heart rate, and the heart rate abnormality index and the group mean heart rate are used as second image features.

[0045] Furthermore, the step of inputting the acoustic features and the image features into a preset anomaly recognition model to perform secondary recognition of abnormal behavior of the initial abnormal fish group includes:

[0046] The acoustic features and the first image features are fused at multiple scales using a feature pyramid network, and the weights are dynamically adjusted based on a squeeze excitation network to obtain the initial fused features.

[0047] The second image feature is normalized and then concatenated to the initial fusion feature to obtain the fusion feature;

[0048] The fused features are input into the anomaly detection model to obtain the detection result. The anomaly detection model is constructed based on a fully connected neural network model.

[0049] Secondly, embodiments of the present invention provide a system for identifying abnormal behavior in large yellow croaker based on radio frequency signals, comprising:

[0050] The behavior trajectory analysis module is used to acquire the radio frequency signals of tagged fish with radio frequency chips in the fish school, perform data processing and feature extraction on the radio frequency signals to obtain behavioral feature indicators, and obtain a first abnormal probability based on the behavioral feature indicators and a preset health baseline.

[0051] The group behavior analysis module is used to acquire the behavior signals of the fish school, calculate the group behavior entropy of the fish school based on the behavior signals, and obtain the second anomaly probability based on the group behavior entropy and the preset entropy threshold.

[0052] The initial anomaly identification module is used to perform initial anomaly behavior identification on the fish group based on the first anomaly probability and the second anomaly probability, so as to obtain the initial abnormal fish group.

[0053] The data processing module is used to acquire the underwater acoustic signal and image data of the initial abnormal fish swarm, extract features from the underwater acoustic signal to obtain acoustic features, and perform target detection and feature extraction on the image data to obtain image features.

[0054] The secondary anomaly identification module is used to input the acoustic features and the image features into a preset anomaly identification model to perform secondary identification of the abnormal behavior of the initial abnormal fish group, and to determine the abnormal fish group and the corresponding intervention plan based on the identification results. The anomaly identification model is constructed based on a neural network model.

[0055] This invention provides a method and system for identifying abnormal behavior in large yellow croaker based on radio frequency (RF) signals. The invention analyzes the behavioral trajectory of large yellow croaker using RF signals and quantifies the orderly state of the fish population's behavior using group behavior entropy. By synergistically combining group behavior entropy and behavioral trajectory, the invention makes an initial judgment on abnormal behavior in large yellow croaker, effectively correcting the risk of missed or false judgments caused by RF signal sample bias. Furthermore, through multimodal analysis and fusion of underwater acoustic and image signals, the initial judgment results are cross-validated using multimodal methods, effectively improving the accuracy of abnormal behavior identification. Through multimodal fusion and dual anomaly identification, this invention can accurately identify abnormal behavior in large yellow croaker, achieving accurate and efficient health status assessment, thus providing more reliable decision support for precision farming and disease control of large yellow croaker. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating the abnormal behavior identification method for large yellow croaker based on radio frequency signals according to an embodiment of the present invention.

[0057] Figure 2 This is a schematic diagram of the structure of the abnormal behavior recognition system for large yellow croaker based on radio frequency signals according to an embodiment of the present invention;

[0058] Figure label:

[0059] 10. Behavioral Trajectory Analysis Module; 20. Group Behavior Analysis Module; 30. Initial Anomaly Identification Module; 40. Data Processing Module; 50. Secondary Anomaly Identification Module. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Please see Figure 1 The first embodiment of the present invention proposes a method for identifying abnormal behavior of large yellow croaker based on radio frequency signals, including steps S10 to S50:

[0062] Step S10: Obtain the radio frequency signal of the tagged fish with radio frequency chip in the fish school, perform data processing and feature extraction on the radio frequency signal to obtain behavioral feature indicators, and obtain the first abnormal probability based on the behavioral feature indicators and the preset health baseline.

[0063] Step S20: Obtain the behavior signal of the fish school, calculate the group behavior entropy of the fish school based on the behavior signal, and obtain the second abnormal probability based on the group behavior entropy and a preset entropy threshold.

[0064] Step S30: Based on the first abnormal probability and the second abnormal probability, perform initial identification of abnormal behavior of the fish group to obtain an initial abnormal fish group;

[0065] Step S40: Acquire the underwater acoustic signal and image data of the initial abnormal fish swarm, extract features from the underwater acoustic signal to obtain acoustic features, and perform target detection and feature extraction on the image data to obtain image features;

[0066] Step S50: Input the acoustic features and the image features into a preset anomaly recognition model to perform secondary recognition of the abnormal behavior of the initial abnormal fish group, and determine the abnormal fish group and the corresponding intervention plan based on the recognition results. The anomaly recognition model is constructed based on a neural network model.

[0067] This invention targets the marine ecological fence-style large yellow croaker farming environment. It monitors the behavioral trajectory of the large yellow croaker using radio frequency (Passive Integrated Transponder, PIT) signals and combines image and underwater acoustic signals to accurately assess the health status of the large yellow croaker through multimodal fusion and dual abnormal behavior identification. Before describing the identification method provided by this invention, a brief description of the farming environment in which this invention is applied is provided. Marine ecological ranches are sustainable farming models that utilize natural ocean currents to achieve water self-purification while maintaining interaction between the farmed organisms and the wild ecosystem by deploying ecological fences within a specific sea area. The ecological fences can be implemented using flexible netting, semi-permeable barriers, or netless acoustic wave systems. To facilitate data collection, surface buoys and underwater supports are installed at multiple pre-set locations at the edge and center of the marine ecological ranch to mount relevant sensor equipment, including underwater acoustic sensors, cameras, radio frequency signal receivers, environmental sensors, GPS positioning devices, and UWB base stations.

[0068] In the marine ecological ranch environment, to better track the behavior of large yellow croaker and analyze its health status based on this behavior, this embodiment uses miniature radio frequency (RF) chips on the fish in the school and tracks the RF signals emitted by these chips to monitor the fish's behavior in real time. To balance cost and efficiency, this embodiment employs a sampling labeling method. For fish stocked in batches, a certain percentage (e.g., 10% to 25%) of each batch is sampled. The sampled fish are then RF-tagged, and those with RF chips are designated as tagged fish. The RF signals from the tagged fish are then acquired to analyze the fish's behavior. Specific steps include:

[0069] The radio frequency signal is subjected to data processing, which includes noise removal and missing value completion;

[0070] Feature extraction and feature statistics are performed on the radio frequency signals after data processing to obtain behavioral characteristic indicators of the fish population. The behavioral characteristic indicators include spatial distribution indicators, movement pattern indicators, rhythm indicators, and environmental response indicators.

[0071] A health baseline is obtained based on the current environmental parameters and a preset baseline database. An indicator threshold is determined based on the health baseline. The behavioral characteristic indicator is compared with the indicator threshold, and a first abnormal probability is obtained based on the comparison result.

[0072] In this embodiment, raw data is first acquired, including the raw trajectory data and coordinate data of the PIT chip, as well as environmental parameters (such as water temperature, dissolved oxygen, and salinity). The raw data is then processed, including outlier removal (e.g., filtering coordinate jumps caused by signal interference), trajectory completion (e.g., using linear interpolation to complete missing data), and coordinate transformation (e.g., converting latitude and longitude coordinates to Cartesian coordinates with the ranch center as the origin). Feature extraction is then performed on the processed data to obtain behavioral characteristic indicators, including spatial distribution indicators, movement pattern indicators, rhythm indicators, and environmental response indicators. Spatial distribution indicators include average daily activity range, core area proportion, and maximum diffusion distance; movement pattern indicators include average movement speed, movement frequency, and standard deviation of turning angle; rhythm indicators include diurnal movement ratio and peak activity at dawn and dusk; and environmental response indicators include temperature sensitivity coefficient. Specifically, the average daily activity range can be calculated using kernel density estimation to determine the area of ​​an individual's daily activity zone. This indicator reflects the strength of the fish's exploration ability; a value that is too small may indicate decreased vitality, while a value that is too large may indicate stress-induced escape behavior. The core area percentage can be calculated by the ratio of the time spent in the core habitat (e.g., 50% of the kernel density estimation area) to the total activity time. A value that is too large (e.g., greater than 70%) indicates a stable habitat preference, while a value that is too small (e.g., less than 30%) may indicate habitat drift caused by environmental discomfort or disease. The maximum diffusion distance is the farthest distance from the center point among the daily trajectory points. This value, along with the average daily activity range, reflects the boundary of the activity range. The average movement speed refers to the average distance between adjacent trajectory points divided by the time difference. This value reflects the fish's basic kinetic ability. The movement frequency is the number of trajectory segments with a speed >0.1 m / s per unit time. This value reflects the fish's activity level; a value that is too large may indicate stress-induced excitement, while a value that is too small may indicate illness. The resulting lack of vitality; the standard deviation of turning angle is the standard deviation of the turning angle of the trajectory point (the angle formed by three adjacent points), with an angle range of 0-180°. This value reflects the complexity of the movement path. If the value is too large (e.g., greater than 60%), it indicates a tortuous path, which may be feeding behavior. If the value is too small (e.g., <20°), it indicates linear movement, which may be migration or escape; the diurnal movement ratio refers to the total movement distance during the day (6:00-18:00) divided by the total movement distance at night (18:00-6:00). This value is the type of diurnal rhythm. Disease or hyperactivity can cause changes in diurnal rhythm; the peak activity at dawn and dusk is obtained by statistically analyzing the average movement speed from 6:00-8:00 and 16:00-18:00, and taking the time period corresponding to the maximum value. This value is related to the feeding rhythm; the temperature sensitivity coefficient refers to the ratio of the change in the average daily activity range between two adjacent days to the change in water temperature. This value is a behavioral elasticity indicator. A higher value indicates sensitivity to temperature changes, which may be a pathological condition.It should be noted that the above behavioral characteristic indicators are only preferred options and not specific limitations. The behavioral characteristic indicators can be flexibly selected according to the actual situation of the marine ecological ranch and the calculation needs.

[0073] For all tagged fish, behavioral characteristic indicators are statistically analyzed. Population statistics (such as mean ± standard deviation, median, 95% confidence interval, etc.) are used as behavioral characteristic indicators for the fish population. Simultaneously, a corresponding healthy baseline is selected from a pre-constructed baseline database. Since different environmental parameters have different effects on fish behavior—for example, the average daily activity range is higher in summer than in winter, and water temperature changes also affect the average daily activity range—multiple healthy baselines are stored in the baseline database based on different environmental parameters. In this embodiment, a corresponding healthy baseline is selected from the baseline database according to the current environmental parameters, and then the behavioral characteristic indicators are compared with the healthy baselines. In this embodiment, both the behavioral characteristic indicators and the healthy baselines are normalized feature vectors. During the comparison, an indicator threshold is determined based on the standard deviation σ of the healthy baseline, for example, using the healthy baseline ± 2σ as the threshold. The anomaly probability is determined based on the difference between the behavioral characteristic indicator and the indicator threshold. If the statistical parameters of the tagged fish exceed twice the standard deviation of the healthy baseline, it is inferred that the overall fish population may be abnormal.

[0074] Since the tagged fish are sampled, if the fish population exhibits non-uniform distribution anomalies, the statistical results may mask local anomalies, leading to a risk of missed detection. Furthermore, if untagged fish are infected with disease while the tagged fish are all healthy, the statistical characteristic values ​​will still be within the normal range, also resulting in missed anomaly detection. Therefore, relying solely on the behavioral characteristics of tagged fish to reflect the behavior of the fish population has certain limitations. To improve the accuracy of anomaly identification, this embodiment, based on radio frequency signals, acquires the behavioral signals of the fish population and analyzes the collective behavior of the fish population based on these signals to obtain the collective behavior entropy. The limitations of the statistical characteristics of tagged fish are then supplemented and corrected based on this collective behavior entropy. The specific steps include:

[0075] First image data of the fish school is acquired, and target detection and trajectory tracking are performed on the first image data to obtain the temporal position sequence of each fish;

[0076] Feature extraction is performed on the temporal position sequence to obtain behavioral feature parameters for each fish, including movement speed and orientation angle.

[0077] Based on the joint distribution of velocity direction, grid cells are divided. Histogram statistics are performed according to the behavioral feature parameters and the grid division results to obtain statistical results, which include the number of fish in each grid cell and the total number of fish in all grid cells.

[0078] Based on the statistical results, the probability distribution of each grid cell is calculated, and the entropy of the fish group behavior is calculated based on the probability distribution and the entropy formula.

[0079] In this embodiment, the fish swarming effect describes the phenomenon of fish forming coordinated group behaviors through simple biological mechanisms. Under normal circumstances, fish groups move collectively, but when some fish become infected with diseases, their abnormal behavior affects the synchronicity of group behavior. Based on this principle, this embodiment uses group behavior entropy to quantify the synchronicity and randomness of individual fish movements, thereby measuring the ordered-disordered state of group behavior patterns. The group behavior entropy ranges from [0,1]. If the group behavior entropy approaches zero, it indicates highly synchronized behavior (e.g., school migration, unified predator avoidance), while if it approaches 1, it indicates completely chaotic behavior (e.g., disease outbreak, panicked escape). Furthermore, this embodiment presets an entropy threshold for group behavior entropy. Preferably, the group behavior entropy under normal aquaculture conditions, i.e., the entropy threshold, is set between 0.3 and 0.7. This threshold represents moderate synchronicity and allows for individual differences. This embodiment improves the robustness of anomaly identification by synergistically analyzing group behavior entropy and the statistical parameters of tagged fish to correct the bias of solely relying on tagged fish samples.

[0080] Regarding the group behavior entropy, this embodiment acquires image data of the fish school using an image acquisition device, and calculates the group behavior entropy by performing group behavior analysis and quantification on the image data. Specifically, for the acquired underwater images of the fish school, fish bodies are first detected in the underwater images using a target detection algorithm, and the bounding box coordinates of each fish body are output. The target detection algorithm can be a conventional algorithm such as a deep convolutional neural network or a YOLO series algorithm. Preferably, this embodiment uses the YOLOv8 algorithm for fish body detection. It should be noted that the model used in this embodiment can be a conventional model with the same function, and the specific detection steps can refer to the detection steps of conventional models. There is no specific limitation on the selected model.

[0081] For the identified fish, a target tracking algorithm, such as DeepSORT, is used to track the trajectory of multiple consecutive frames of images. The movement trajectories of the same fish are associated through appearance feature matching to obtain the temporal position sequence of each fish. Based on the temporal position sequence, behavioral feature parameters of each fish are calculated, including movement speed and orientation angle. For movement speed, the single-frame displacement is obtained by measuring the displacement of the i-th fish in adjacent frames. A calibration board is used to determine the mapping relationship between image resolution and actual distance, converting the single-frame displacement into actual displacement. The frame interval, i.e., the time interval between two frames, is calculated based on the frame rate of the image frames. The movement speed is obtained by the quotient of the actual displacement and the time interval. To avoid interference from outliers, movement speeds exceeding a speed threshold are truncated; that is, the movement speed is forcibly set within the speed threshold range. For the orientation angle, the displacement vector of the i-th fish is calculated by the adjacent temporal positions, and the initial orientation angle is calculated by the arctangent function and quadrant correction is performed. Then the orientation is discretized, for example, divided into 8 equal intervals, and a mapping rule is set. The mapping rule refers to discretizing the angle range into directions, that is, mapping each angle interval to a direction, and using the discrete direction as the orientation angle feature. Finally, the behavioral feature parameters of each fish are obtained.

[0082] Based on the behavioral characteristic parameters of each fish, the "ordered-disordered" state of the group behavior is quantified through the probability distribution of the behavioral characteristic parameters. The original group behavior entropy is calculated and normalized to [0,1]. Specifically, firstly, grid cells are divided based on the joint distribution of speed and direction. For example, the range threshold of movement speed is divided into 5 levels to obtain speed range intervals. The direction angle is divided into 8 levels according to the discrete directions mentioned above. Based on the speed and direction levels, grid cells are divided, resulting in a total of 5*8=40 grid cells. Based on the divided grid cells, for N fish detected in the image, the grid cell to which the fish belongs is determined through its movement speed and direction angle in the behavioral characteristic parameters. Then, the number of fish in each grid cell is counted to obtain a histogram, and the total number of fish in all grid cells is obtained based on the number of fish in each grid cell.

[0083] Based on the number of fish in each grid cell and the total number of fish, the probability of each grid cell is calculated. The probability value is the quotient of the number of fish in each grid cell to the total number of fish. Then, the entropy value of the probability of each grid cell is calculated using the Shannon entropy formula to obtain the group behavior entropy, the expression of which is:

[0084]

[0085] In the formula, GBE represents the group behavior entropy, p(m) represents the probability of the m-th grid cell, and M represents the total number of grid cells.

[0086] Finally, the group behavior entropy is normalized to obtain the final group behavior entropy. This embodiment accurately quantifies the synchronicity and randomness of individual fish movements through group behavior entropy, thus providing accurate data supplementation for subsequent abnormal behavior identification.

[0087] In a preferred embodiment, the present invention also provides another method for calculating group behavior entropy, the specific steps of which include:

[0088] Acquire the group acoustic signal of the fish school, and preprocess the group acoustic signal, including high-pass filtering, frame windowing and validity verification;

[0089] Sound event detection is performed on the preprocessed group acoustic signal to obtain the sound emission time. Based on the sound emission time, the sound emission interval is calculated, and based on the sound emission interval, the sound emission standard deviation is calculated.

[0090] The power spectrum of the preprocessed group acoustic signal is estimated and the dominant frequency is extracted to obtain the dominant frequency sequence. The dominant frequency variance is calculated based on the dominant frequency sequence.

[0091] The group behavior entropy is obtained by normalizing and weighting the standard deviation of vocalization and the variance of dominant frequency.

[0092] In this embodiment, the analysis data of fish group behavior uses group acoustic signals. Specifically, group acoustic signals are collected in the vocal frequency band of large yellow croaker using a hydroacoustic signal sensor. The group acoustic signals are first subjected to noise filtering to remove low-frequency water flow noise and high-frequency electronic noise. Then, the time domain signal is segmented by framing and windowing to obtain the framed data structure. Preferably, the window function used is a Hamming window.

[0093] For each frame of data, sound event detection is performed. Common methods for sound event detection include signal processing-based methods and machine learning-based methods. Signal processing-based methods mainly utilize the time and frequency domain features of audio signals to detect sound events, such as short-time energy (STE), short-time zero-crossing rate (SZCR), and Mel-frequency cepstral coefficients. Machine learning-based methods can employ machine learning models, such as support vector machines, hidden Markov models, or deep learning models. Taking short-time energy as an example, the short-time energy (STE) of each frame of signal is first calculated. Then, an energy threshold is determined. Background noise without fish activity is collected for the first 30 seconds, and the mean and standard deviation of the noise energy are calculated. The sum of the mean noise energy and three times the standard deviation is used as the energy threshold. When the STE of three consecutive frames is greater than the energy threshold, it is marked as the sound start point; when the STE of three consecutive frames is less than the energy threshold, it is marked as the sound end point. Of course, other detection methods can also be used, and no specific detection method is limited here.

[0094] For each detected sound event segment, the timestamp corresponding to the first frame of the starting point is taken as the sound emission time. For multiple detected sound emission times, the intervals between adjacent sound emission times are calculated, and then the standard deviation of the sound interval sequence is calculated.

[0095] Simultaneously, the dominant frequency distribution features are extracted from the frame data. Specifically, for each frame signal after windowing, a Fast Fourier Transform is performed to obtain the power spectrum. The power spectrum is then transformed, and the frequency corresponding to the peak value is found to obtain the dominant frequency distribution. For all frame data within a preset duration, such as 5 minutes, the dominant frequency of each frame is extracted, and after outlier handling, the variance of the dominant frequency sequence is calculated to obtain the dominant frequency variance. Finally, the standard deviation of the sound interval and the dominant frequency variance are normalized and weighted to obtain the group behavior entropy based on the underwater acoustic signal. Preferably, the weight of the sound interval can be set to 0.6, and the weight of the dominant frequency distribution can be set to 0.4.

[0096] Underwater acoustic data is essentially the "acoustic fingerprint" of group behavior. Its variation patterns are highly coupled with the ecological needs of fish. The interval between vocalizations can reflect the synchronicity of group behavior, and the dominant frequency distribution can reflect the intensity of group behavior. Therefore, this embodiment quantifies the "orderly-disordered" state of group behavior by statistically analyzing the time interval characteristics and spectral distribution characteristics of group vocalizations, thereby achieving effective monitoring of the group behavior characteristics of fish schools.

[0097] In another preferred embodiment, the present invention calculates the entropy of group behavior by combining image data and underwater acoustic data, specifically including the following steps:

[0098] Acquire first image data of the fish school, perform target detection on the first image data, and determine the validity of the first image data based on the target detection results;

[0099] If the first image data is valid, then based on the target detection result, trajectory tracking and behavioral feature extraction are performed, and based on the extracted behavioral feature parameters, the group behavior entropy of the fish school is calculated.

[0100] If the first image data is invalid, the group acoustic signal of the fish school is obtained, and acoustic event detection and main frequency extraction are performed on the group acoustic signal. Based on the detected sound interval and the extracted main frequency sequence, the group behavior entropy of the fish school is calculated.

[0101] In this embodiment, underwater images of the fish school are first acquired, and then the validity of the images is determined by fish detection. If the fish detection box loss rate is high, such as more than 40% for three consecutive frames, the image data is considered invalid. In this embodiment, the fish detection box loss rate is used to characterize the quality of the image data. Poor shooting environment or turbid water will result in poor image data quality, and data quality is directly related to data validity.

[0102] This embodiment evaluates the effectiveness of an image by assessing the fish body detection box loss rate. However, effectiveness can also be determined by image sharpness. Sharpness can be assessed using image quality metrics or sharpness evaluation methods. Image quality metrics include peak signal-to-noise ratio, mean square error, and structural similarity. Commonly used sharpness evaluation methods include edge-based and gradient-based methods. The specific method for effectiveness detection is not limited here. For valid images, the group behavior entropy can be calculated using the valid image data. For invalid images, the corresponding time period's underwater acoustic data is used to calculate the group behavior entropy, ensuring its continuity. The specific steps for calculating group behavior entropy based on image and underwater acoustic data in this embodiment are the same as those in the previous embodiment and will not be repeated here.

[0103] The abnormal probability of the fish school is determined by comparing the calculated group behavior entropy with the corresponding entropy threshold. When the group behavior entropy is within the entropy threshold range, the abnormal probability is low; when it is outside the entropy threshold range, the abnormal probability is high. In this embodiment, the abnormal probability obtained based on the radio frequency signal is used as the first abnormal probability, and the abnormal probability obtained based on the group behavior entropy is used as the second abnormal probability. Then, the abnormal behavior of the fish school is initially judged based on the first and second abnormal probabilities. When making a comprehensive judgment, a decision matrix can be used. For example, the abnormal probability within the healthy baseline and the abnormal probability outside the healthy baseline can be uniformly set. Preferably, for the case exceeding ±2σ of the healthy baseline, the probability of abnormal statistical parameters of the marked fish is set to 1, and for the group behavior entropy exceeding the range of 0.3 to 0.7, the abnormal probability is set to 0.8. At this time, if either the first or second abnormal probability exceeds 60%, the fish school is considered abnormal. In addition, a comprehensive probability assessment can be performed. For example, the anomaly probability can be calculated based on the difference exceeding a threshold. The larger the difference, the higher the anomaly probability. Then, a weight is set for the first anomaly probability and the second anomaly probability. Preferably, the probability weight can be set based on the marking rate of the fish swarm. In this embodiment, radio frequency signals are used as the main signal, supplemented by behavioral signals, to determine anomalies. Therefore, the weight of the first anomaly probability is usually set to be greater than the weight of the second anomaly probability. Furthermore, the higher the marking rate of the fish with radio frequency chips, the greater the weight of the first anomaly probability. For example, when the marking rate is greater than or equal to 20%, the weight of the first anomaly probability is set to 0.7. If the marking rate is less than 20%, the weight of the first anomaly probability is set to 0.55 to increase the weight of the group behavior entropy. The above weight settings are only preferred methods, and the specific weights can be flexibly adjusted according to the actual situation.

[0104] The comprehensive anomaly probability is obtained by weighted summation of the first and second anomaly probabilities. The fish population is judged to be in an abnormal state based on whether the comprehensive anomaly probability exceeds the probability threshold. Furthermore, the fish population can be sorted according to the magnitude of the comprehensive anomaly probability to facilitate subsequent processing based on the initial degree of anomaly.

[0105] The above-mentioned radio frequency signals and behavioral signals can be used to make a preliminary judgment on abnormal fish behavior. In order to avoid misjudgment caused by sensor failure and environmental interference, this embodiment adopts a multimodal data cross-validation method to confirm the authenticity of the fish abnormality for fish with preliminary abnormality.

[0106] In this embodiment, the multimodal data used includes underwater acoustic signals and image data. The underwater acoustic signals and image data of the initial abnormal fish swarm are acquired, and feature extraction is performed on both signals to obtain multimodal features. Specifically, for the underwater acoustic signals, after high-pass filtering to remove noise and frame-by-frame windowing, acoustic features are extracted using Mel-frequency cepstral coefficients. These acoustic features include dominant frequency distribution, acoustic intensity, and fluctuation frequency. The dominant frequency distribution is obtained based on peak values ​​extracted from the Fast Fourier Transform, the acoustic intensity is calculated using the sound pressure level formula, and the fluctuation frequency is obtained by calculating the standard deviation of the acoustic intensity over one minute. The fluctuation frequency reflects signal stability.

[0107] For image data, this embodiment divides the image data into surface image data and underwater image data, and performs feature extraction on the surface image data and underwater image data respectively. The specific steps include:

[0108] Fish body recognition is performed on the first underwater image data in the image data by a fish body detection model to obtain fish body recognition results. The fish body recognition results include fish body bounding boxes and key point coordinates. The fish body detection model is constructed based on a deep neural network model.

[0109] The fish body posture angle is calculated based on the coordinates of the key points. The rollover rate is calculated based on the fish body posture angle. The aggregation density is calculated based on the fish body bounding box.

[0110] Moving targets are extracted from the water image data in the image data using the background subtraction method, and the extracted moving target images are then filtered.

[0111] The filtered moving target image is subjected to target screening and trajectory tracking to obtain fish leap events. The number of fish leap events within a preset time period is counted to obtain the number of fish leaps.

[0112] The rollover rate, the clustering density, and the number of fish leaps are used as the first image features.

[0113] In this embodiment, underwater images are primarily used for monitoring fish school status, while surface images are used for monitoring fish activity. For underwater images, a fish detection model based on a neural network, such as the YOLOv8 model, is first used to detect fish. Based on the detection results, features of the fish school status are extracted, including the rollover rate and clustering density. The rollover determination criterion is that when the angle θ between the fish's attitude angle and the vertical direction is greater than 60°, it is considered a rollover. The rollover rate is the quotient of the number of rollover fish detected in a single frame to the total number of fish detected in that frame. The clustering density is the quotient of the total number of fish detected in a single frame to the field of view of the camera. The camera's field of view can be calculated using the principle of similar triangles based on the camera's field of view angle and installation depth.

[0114] To facilitate feature extraction, this embodiment uses the YOLOv8-Pose pose recognition model to construct a fish detection model. This model, while detecting the target bounding box, can output the coordinates of predefined key points. These predefined key points include head and tail points. The head point is the tip of the fish's mouth, and the tail point is the base of the tail fin. The head and tail points form a key point pair. After YOLOv8-Pose inference, it outputs the bounding box, confidence score, and key point coordinates for each fish. For valid key point pairs, the angle between the line connecting the two points and the horizontal line is calculated and mapped to the range [0°, 180°] for easy rollover determination. If the angle is greater than 60°, the fish has rolled over. Preferably, if using the existing YOLOv8 detection model, a new regression branch can be added to the YOLOv8 detection head to specifically predict the pose angle. Its input is the feature maps extracted from the backbone and neck of the YOLOv8 model, and its output is the pose angle corresponding to each anchor box. Simultaneously, a pose angle regression loss is added to the model's loss function, using the MSE loss algorithm. To avoid affecting detection accuracy, the weight of the pose angle regression loss is set to 0.1. The pose angle regression loss is then weighted and summed with the original model's loss function to obtain the model's final loss function. After model training, the pose angle is output simultaneously for each detected fish body bounding box. Under this model, there is no need to additionally calculate keypoint coordinates.

[0115] For water images, moving targets are extracted using background subtraction. This involves acquiring the first 100 frames of calm water images without fish leaping, establishing an initial background model using a Gaussian mixture model, and subtracting the current frame from the background model to extract the foreground. Morphological filtering using opening operations removes interference from water ripples (small-area noise) and false targets caused by fish leaping. Then, the number of fish leaps is counted in the filtered target image. Specifically, connected components in the foreground image that match the morphological characteristics of a fish leaping out of the water are retained. For example, connected components in the foreground image with an area of ​​50-500 pixels and an aspect ratio of 0.5-2.0 for the bounding rectangle are retained. Alternatively, the YOLOv8 algorithm can be used to detect fish leaping targets and output the bounding box coordinates.

[0116] For targets within 5 consecutive frames, the position is predicted using trajectory association via Kalman filtering. Specifically, for the first frame or a newly appearing target (without a matching historical trajectory), the Kalman filter parameters are initialized. Based on the trajectories of the previous 4 frames, the target position in the current frame is predicted using Kalman filtering. Kalman filtering includes a state transition equation and a measurement equation. The prediction step uses the state transition equation to predict the next state, while the update step adjusts the predicted value based on the measurement value. Specific prediction steps can be found in the standard prediction steps of Kalman filtering. Then, the Interchange of Union (IOU) value between the predicted bounding box and the detection box in the current frame is calculated. If the IOU > 0.2, it is determined to be the same fish leap event, and the trajectory is updated; otherwise, it is marked as a new event. If no target is matched within 5 consecutive frames, the trajectory automatically terminates, completing one fish leap event count. Finally, the number of fish leap events within a preset time period, such as 5 minutes, is counted to obtain the number of fish leaps. The rollover rate, cluster density, and number of fish leaps are used as image features. This embodiment can accurately display the changes in the state and vitality of the fish school through image features, thereby characterizing the health status of the fish school.

[0117] Of course, other methods can also be used to count fish leap events. For example, deep learning models can be used to predict the target position and category, and fish leap events can be counted based on the predicted target position. When judging fish leap events, cosine similarity can also be used to measure the similarity of target feature vectors. For example, a threshold of <0.2 is considered as the same target. Here, only preferred fish leap event counting methods are given, without specific limitations.

[0118] After obtaining acoustic and image features through the above steps, the data from the two modalities are fused and input into the anomaly recognition model for secondary anomaly recognition. This yields the final recognition result of the abnormal behavior of the fish school. The anomaly recognition model can be constructed using a neural network model, such as a deep convolutional model or a long short-term memory neural network model.

[0119] In a preferred embodiment, to further improve the accuracy of anomaly detection, the present invention also provides another feature extraction method for image data, the specific steps of which include:

[0120] Acquire second underwater image data of the fish school, perform target tracking and region extraction on the second underwater image data to obtain key regions, and use the spatial mean of the color channels as the region signal;

[0121] Environmental noise is removed from the regional signal of the key region to obtain the regional time signal, and the regional time signal is then dimensionality reduced to obtain the residual signal.

[0122] Independent component analysis was used to extract high correlation information of heart rate from the residual signal to obtain the relevant sub-component signals;

[0123] The relevant sub-component signals are decomposed into time-frequency components, and the individual heart rate value is calculated based on the decomposition results.

[0124] Data filtering and statistical analysis are performed on multiple individual heart rate values ​​to obtain a heart rate abnormality index and a group mean heart rate, and the heart rate abnormality index and the group mean heart rate are used as second image features.

[0125] In this embodiment, remote photoplethysmography (rPPG) is employed to indirectly calculate heart rate by capturing the periodic fluctuations in light reflection caused by changes in blood flow in the microvessels on the surface of an organism using a camera. rPPG is based on Beer-Lambert's law, utilizing the specific absorption characteristics of hemoglobin for 520-580nm green light. Increased blood volume during cardiac systole enhances light absorption, while diastole weakens it, forming a periodic signal of light intensity changes synchronized with the heartbeat. Based on the specific absorption characteristics of hemoglobin, green light has a high absorption rate for oxyhemoglobin; therefore, green light can improve the accuracy of rPPG measurements in fish, and the signal exhibits good synchronization with electrocardiogram (ECG).

[0126] Based on the above principles, in this embodiment, underwater images of the fish school are captured using an RGB camera, and ambient light interference is suppressed using a green light filter. A target detection algorithm is employed to locate the fish bodies and track their trajectories in the underwater images, extracting regions of interest (ROIs), i.e., key regions. For each fish, the abdomen is cropped as the ROI. Furthermore, a relatively clean area in the background is selected as the background signal ROI, resulting in both the abdomen ROI and the background ROI. The size of each ROI is then standardized to reduce the impact of individual size differences.

[0127] For each ROI, the spatial mean of its three RGB color channels is calculated and sorted according to the frame number of the image to obtain the abdominal signal and background signal. The abdominal signal and background signal are decomposed into multiple source component vectors, and environmental noise is removed from the source component vectors to obtain the temporal signal sequence of the abdominal ROI. The reconstructed regional temporal signal of the abdominal ROI contains not only the specular reflection vector from the skin surface, but also the diffuse reflection component after the skin and subcutaneous tissue absorb and scatter the incident light, as well as the illumination component from the light source. Therefore, by performing signal dimensionality reduction on the regional temporal signal, a residual signal containing only the specular reflection component and the diffuse reflection component can be obtained, and its central pulse information is contained in the diffuse reflection component. Then, through independent component analysis, the specular reflection component and diffuse reflection component are separated. The sub-component with the strongest correlation to the green channel is selected as the basis for heart rate information. Finally, time-frequency decomposition is performed on the sub-component signal to obtain intrinsic mode functions (EMFs) at different time-frequency scales. Then, based on the heart rate distribution interval of the large yellow croaker, the frequency domain interval is determined, and the EMF with the highest energy within the frequency domain interval is selected as the basis for heart rate calculation. For the selected EMF, the frequency value corresponding to the maximum amplitude can be directly obtained from its spectrum. Multiplying the frequency value by 60 yields the number of heartbeats per minute for the individual fish. It should be noted that this embodiment uses a non-invasive fish heart rate measurement method based on remote photoplethysmography (rPPG). The specific steps are the same as the conventional steps of the rPPG algorithm based on fish heart rate measurement, and will not be repeated here.

[0128] After obtaining the individual heart rate value for each fish, outliers are first removed to obtain valid individual heart rate values. Then, through data filtering and statistical analysis of the valid individual heart rate values, a heart rate anomaly index and a group mean heart rate are obtained. Individual heart rate values ​​exceeding the heart rate threshold are considered abnormal heart rates. The heart rate anomaly index is calculated based on the ratio of the number of abnormal heart rates to the total number of individual heart rate values. The arithmetic mean ± standard deviation of the valid individual heart rate values ​​is then calculated as the group mean heart rate. Finally, the heart rate anomaly index and the group mean heart rate are used as heart rate indicators, i.e., the second image feature.

[0129] After obtaining the second image features, the first image features, the second image features, and the acoustic features are concatenated and fused, and then input into the anomaly recognition model to obtain the anomaly recognition result. The robustness of fish is related to their physiological and behavioral characteristics, and heart rate is an important physiological indicator for measuring fish status. In this embodiment, rPPG is used for long-distance heart rate monitoring, and the heart rate indicator is used as a key indicator for assessing the physiological status of the fish population. Combined with the aforementioned acoustic and image features, the accuracy of subsequent abnormal behavior recognition can be effectively improved.

[0130] In a preferred embodiment, the first image features and the second image features are used as image features and fused with acoustic features. This multimodal fusion achieves accurate identification of abnormal behavior. Specific steps include:

[0131] The acoustic features and the first image features are fused at multiple scales using a feature pyramid network, and the weights are dynamically adjusted based on a squeeze excitation network to obtain the initial fused features.

[0132] The second image feature is normalized and then concatenated to the initial fusion feature to obtain the fusion feature;

[0133] The fused features are input into the anomaly detection model to obtain the detection result. The anomaly detection model is constructed based on a fully connected neural network model.

[0134] In this embodiment, to improve the accuracy of anomaly detection, a multi-level fusion strategy is adopted to perform feature fusion and decision-making on image and acoustic features. First, a feature pyramid network is used to perform multi-scale feature fusion on acoustic features and the first image features. Then, a squeeze-excitation network (SENet) is used to dynamically adjust the channel weights. Specifically, the acoustic features are upsampled to match the dimension of the first image features, and the upsampled acoustic features are added element-wise to the first image features to obtain multi-scale fused features. Then, the multi-scale fused features are input into the SENet network, and global average pooling is performed on the multi-scale fused features through a squeeze operation to achieve global information compression. Channel weights are generated through a two-layer fully connected network with excitation operations to learn channel importance. Finally, the dynamic weights learned by SENet are multiplied channel-wise with the multi-scale fused features to obtain the initial fused features. It should be noted that if only acoustic features and the first image features are used as input data for the anomaly detection model, then after obtaining the initial fused features, they can be input into the anomaly detection model for abnormal behavior detection. If a second image feature is combined with the acoustic features and the first image features, then the initial fused features also need to be fused with the second image feature.

[0135] During fusion, the second image features are normalized and concatenated to the initial fusion features to obtain fusion features. These fusion features are then input into an anomaly recognition model for abnormal behavior identification. The anomaly recognition model can be constructed based on a neural network model. Preferably, this embodiment uses a fully connected classification network model to construct the anomaly recognition model. The network structure of the anomaly recognition model includes an input layer, two hidden layers, and an output layer, using weighted cross-entropy loss as the loss function. The input data of the input layer is the fusion features. The two hidden layers are successively dimensionalized, for example, to 521 dimensions and 256 dimensions respectively, and both use the ReLU function as the activation function. The output layer outputs the abnormal behavior classification judgment result through the Softmax activation function, including normal state and abnormal state. Abnormal states include stress state and disease state. Corresponding graded warnings are issued according to different abnormal states. For example, if it is an emergency state, a stress warning is triggered, and environmental parameters such as water quality or dissolved oxygen are checked according to the stress warning. If it is a disease state, a disease warning is triggered, and isolation and quarantine are carried out according to the disease warning. It should be noted that in the process of abnormal behavior identification, when processing multimodal data, it is also necessary to perform conventional data preprocessing steps such as data synchronization and normalization to unify dimensions. These data preprocessing steps are routine technical steps and will not be described in detail during the data processing.

[0136] In another preferred embodiment, the present invention also provides another data fusion method, the specific steps of which include:

[0137] The acoustic features and the first image features are concatenated to obtain the first concatenated features. Feature weights are generated through a modal gating mechanism, and a primary fusion feature is obtained based on the feature weights and the first concatenated features.

[0138] The second image feature is normalized and stitched to the initial fusion feature to obtain the second stitched feature. The weight of the rollover rate feature in the second stitched feature is adjusted according to the heart rate abnormality index to obtain the fusion feature.

[0139] In this embodiment, primary feature fusion is first performed using channel-level concatenation and modal attention. During primary fusion, acoustic features and the first image features are concatenated, thus preserving all feature channels. Then, a modal gating mechanism is used to assign weights to all concatenated features. During weight assignment, a weight matrix and a bias vector are set, and the Xavier initialization method is used to assign weights to the weight matrix. The weight matrix can be divided into four sub-matrices: the influence matrix of acoustic features on acoustic modal weights, the influence matrix of image features on acoustic modal weights, the influence matrix of acoustic features on image modal weights, and the influence matrix of image features on image modal weights. A linear transformation is performed on the concatenated features based on the weight matrix and bias vector to obtain inactive gating scores. These gating scores are then mapped to the (0,1) interval using the Sigmoid function to obtain the modal attention vector, which represents the contribution of each feature channel. Finally, the modal attention vector is multiplied element-wise with the concatenated features to achieve modal-level dynamic weighting, resulting in the primary fused features.

[0140] The second image features are normalized and concatenated to the initial fusion features to obtain the second concatenated features. The rollover rate feature channel is determined through prior knowledge or training localization. Then, the weights of the rollover rate feature channel are adjusted according to the heart rate abnormality index. Specifically, when the heart rate abnormality index is greater than a threshold, such as greater than 0.5, the weight of the rollover rate feature is increased to enhance disease relevance. When increasing the weights, an abnormality coefficient can be set, and the abnormality coefficient multiplied by the heart rate abnormality index is used as the weight increment. To prevent the rollover rate feature weight from being too high and masking subtle changes in other features, an upper limit on the weight increment is also set. This upper limit balances the contribution ratio of multimodal features and ensures the synergistic effect of multiple features. After obtaining the weight increment, the weights of other features remain unchanged, and the rollover rate feature is enhanced according to the weight increment to obtain the fusion features. Specifically, when enhancing the rollover rate feature according to the weight increment, a weight vector can be set where the weights of other channels in this weight vector are all 1, and the weight of the rollover rate feature channel is 1 + the weight increment. This achieves rollover rate feature enhancement, thereby improving the accuracy of abnormality identification.

[0141] In this embodiment, the reason for increasing the contribution of the rollover rate feature by the heart rate abnormality index is that an increase in the heart rate abnormality index does not directly indicate disease. This is because there are false anomalies in physiological indicators caused by environmental disturbances. For example, when the dissolved oxygen level in the aquaculture water suddenly drops (such as due to oxygen consumption from algal blooms), the fish will exhibit a stress-induced increase in heart rate, but at this time the fish are not diseased and only need oxygenation treatment. Furthermore, in the early stages of fish infection, the heart rate may not yet show obvious abnormalities, but rollover behavior has already begun to increase (such as unbalanced swimming posture and increased frequency of surface rollovers), that is, behavioral abnormalities precede physiological indicators. Rollovers in healthy fish are occasional and brief (such as when avoiding obstacles), while rollovers in diseased fish are persistent and clustered. Therefore, when the heart rate abnormality index is high, the weight of the rollover rate feature is increased to strengthen the synergistic judgment of heart rate abnormalities and behavioral abnormalities, so as to avoid misjudgment based on a single feature. When the heart rate abnormality index is low, a lower original weight is used to avoid triggering disease warnings due to occasional rollovers. This embodiment further improves the robustness and accuracy of the anomaly recognition model by complementing multimodal information from behavior and physiology.

[0142] This embodiment provides a method for identifying abnormal behavior in large yellow croaker based on radio frequency (RF) signals. This method analyzes the behavioral trajectory of the large yellow croaker using RF signals and quantifies the orderly state of the fish's group behavior using group behavior entropy. The initial judgment of abnormal behavior is made through the synergy of group behavior entropy and behavioral trajectory, effectively correcting the risk of missed or false judgments caused by RF signal sample bias. Furthermore, multimodal analysis and fusion of underwater acoustic and image signals are used to perform multimodal cross-validation of the initial judgment results, effectively improving the accuracy of abnormal behavior identification. This embodiment, through multimodal fusion and dual anomaly identification, can accurately identify abnormal behavior in large yellow croaker, achieving accurate and efficient health status assessment, thus providing more reliable decision support for precision farming and disease control of large yellow croaker.

[0143] Please see Figure 2 Based on the same inventive concept, the second embodiment of this invention proposes a system for identifying abnormal behavior of large yellow croaker based on radio frequency signals, comprising:

[0144] The behavior trajectory analysis module 10 is used to acquire the radio frequency signal of the tagged fish with radio frequency chips in the fish school, perform data processing and feature extraction on the radio frequency signal to obtain behavior feature indicators, and obtain a first abnormal probability based on the behavior feature indicators and a preset health baseline.

[0145] The group behavior analysis module 20 is used to acquire the behavior signals of the fish school, calculate the group behavior entropy of the fish school based on the behavior signals, and obtain the second anomaly probability based on the group behavior entropy and a preset entropy threshold.

[0146] The initial anomaly identification module 30 is used to perform initial anomaly behavior identification on the fish group based on the first anomaly probability and the second anomaly probability to obtain an initial abnormal fish group.

[0147] The data processing module 40 is used to acquire the underwater acoustic signal and image data of the initial abnormal fish swarm, extract features from the underwater acoustic signal to obtain acoustic features, and perform target detection and feature extraction on the image data to obtain image features.

[0148] The secondary anomaly identification module 50 is used to input the acoustic features and the image features into a preset anomaly identification model, perform secondary identification of the abnormal behavior of the initial abnormal fish group, and determine the abnormal fish group and the corresponding intervention plan based on the identification results. The anomaly identification model is constructed based on a neural network model.

[0149] The technical features and effects of the large yellow croaker abnormal behavior recognition system based on radio frequency signals proposed in this invention are the same as those of the method proposed in this invention, and will not be repeated here. Each module in the above-mentioned large yellow croaker abnormal behavior recognition system based on radio frequency signals can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0150] In summary, the present invention proposes a method and system for identifying abnormal behavior in large yellow croaker based on radio frequency signals. The method acquires radio frequency signals from tagged fish with radio frequency chips in a school of fish, processes and extracts features from the radio frequency signals to obtain behavioral feature indicators, and obtains a first abnormality probability based on the behavioral feature indicators and a preset health baseline. It then acquires behavioral signals from the school of fish, calculates the group behavioral entropy based on the behavioral signals, and obtains a second abnormality probability based on the group behavioral entropy and a preset entropy threshold. Based on the first and second abnormality probabilities, it performs initial identification of abnormal behavior in the school of fish to obtain an initial abnormal school of fish. It acquires underwater acoustic signals and image data of the initial abnormal school of fish, extracts features from the underwater acoustic signals to obtain acoustic features, and performs target detection and feature extraction on the image data to obtain image features. Finally, it inputs the acoustic features and image features into a preset abnormality identification model to perform secondary identification of abnormal behavior in the initial abnormal school of fish, and determines the abnormal school of fish and corresponding intervention schemes based on the identification results. The abnormality identification model is constructed based on a neural network model. This invention uses the synergy of group behavior entropy and behavior trajectory to initially determine abnormal behavior in large yellow croaker, effectively correcting the risk of missed or false judgments caused by radio frequency signal sample deviation. Through multimodal analysis and fusion of underwater acoustic and image signals, the initial determination results are cross-validated using multimodal methods, effectively improving the accuracy of abnormal behavior identification. This invention accurately identifies abnormal behavior in large yellow croaker through multimodal fusion and dual anomaly identification, achieving accurate and efficient health status assessment of large yellow croaker, thus providing more reliable decision support for precision farming and disease control of large yellow croaker.

[0151] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0152] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the technical principles of this invention, and these improvements and substitutions should also be considered within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the scope of the claims.

Claims

1. A method for identifying abnormal behavior of Pseudosciaena crocea based on radio frequency signals, characterized in that, The method comprises the following steps: acquiring a radio frequency signal of a tagged fish provided with a radio frequency chip in a fish school, performing data processing and feature extraction on the radio frequency signal to obtain a behavior characteristic index, and obtaining a first abnormal probability according to the behavior characteristic index and a preset health baseline; acquiring a behavior signal of the fish school, calculating a group behavior entropy of the fish school according to the behavior signal, and obtaining a second abnormal probability according to the group behavior entropy and a preset entropy threshold; performing initial identification of abnormal behavior on the fish school according to the first abnormal probability and the second abnormal probability to obtain an initial abnormal fish school; acquiring a water acoustic signal and image data of the initial abnormal fish school, performing feature extraction on the water acoustic signal to obtain acoustic features, and performing target detection and feature extraction on the image data to obtain image features; inputting the acoustic features and the image features into a preset abnormal identification model to perform secondary identification of abnormal behavior on the initial abnormal fish school, and determining an abnormal fish school and a corresponding intervention scheme according to the identification result, wherein the abnormal identification model is constructed based on a neural network model.

2. The method for identifying abnormal behavior of big yellow croaker based on radio frequency signals according to claim 1, characterized in that, The step of performing data processing and feature extraction on the radio frequency signal to obtain a behavior characteristic index, and obtaining a first abnormal probability according to the behavior characteristic index and a preset health baseline comprises: performing data processing on the radio frequency signal, wherein the data processing comprises noise removal and missing value completion; performing feature extraction and feature statistics on the radio frequency signal after data processing to obtain behavior characteristic indexes of the fish school, wherein the behavior characteristic indexes comprise spatial distribution indexes, motion mode indexes, rhythm indexes and environmental response indexes; obtaining a health baseline according to current environmental parameters and a preset baseline database, determining index threshold values according to the health baseline, comparing the behavior characteristic indexes with the index threshold values, and obtaining a first abnormal probability according to the comparison result.

3. The method according to claim 1, wherein, The step of acquiring a behavior signal of the fish school and calculating a group behavior entropy of the fish school according to the behavior signal comprises: acquiring first image data of the fish school, performing target detection and trajectory tracking on the first image data to obtain a time sequence position sequence of each fish; performing feature extraction on the time sequence position sequence to obtain behavior characteristic parameters of each fish, wherein the behavior characteristic parameters comprise moving speed and direction angle; performing grid cell division based on speed direction joint distribution, performing histogram statistics according to the behavior characteristic parameters and the grid division result to obtain a statistical result, wherein the statistical result comprises the number of fish in each grid cell and the total number of fish in all grid cells; calculating a probability distribution of each grid cell according to the statistical result, and calculating a group behavior entropy of the fish school according to the probability distribution and an entropy value formula.

4. The method according to claim 1, wherein, The step of acquiring a behavior signal of the fish school and calculating a group behavior entropy of the fish school according to the behavior signal comprises: acquiring a group acoustic signal of the fish school, performing preprocessing on the group acoustic signal, wherein the preprocessing comprises high-pass filtering, frame windowing and validity verification; performing sound event detection on the preprocessed group acoustic signal to obtain a sound emission time, calculating a sound emission interval according to the sound emission time, and calculating a sound emission standard deviation according to the sound emission interval; The power spectrum estimation and the main frequency extraction are performed on the preprocessed group acoustic signal to obtain a main frequency sequence, and a main frequency variance is calculated according to the main frequency sequence; The sound emission standard deviation and the main frequency variance are normalized and weighted summed to obtain the group behavior entropy.

5. The method according to claim 1, wherein, The step of obtaining the behavior signal of the fish group and calculating the group behavior entropy of the fish group according to the behavior signal comprises: obtaining first image data of the fish group, performing target detection on the first image data, and judging the validity of the first image data according to the target detection result; if the first image data is valid, performing trajectory tracking and behavior feature extraction according to the target detection result, and calculating the group behavior entropy of the fish group according to the extracted behavior feature parameters; if the first image data is invalid, obtaining group acoustic signals of the fish group, and performing sound event detection and main frequency extraction on the group acoustic signals, and calculating the group behavior entropy of the fish group according to the detected sound emission interval and the extracted main frequency sequence.

6. The method according to claim 1, wherein the method is characterized by, The step of obtaining the behavior signal of the fish group and calculating the group behavior entropy of the fish group according to the behavior signal comprises: obtaining first image data of the fish group, performing target detection on the first image data, and judging the validity of the first image data according to the target detection result; if the first image data is valid, performing trajectory tracking and behavior feature extraction according to the target detection result, and calculating the group behavior entropy of the fish group according to the extracted behavior feature parameters; if the first image data is invalid, obtaining group acoustic signals of the fish group, and performing sound event detection and main frequency extraction on the group acoustic signals, and calculating the group behavior entropy of the fish group according to the detected sound emission interval and the extracted main frequency sequence.

7. The method according to claim 1, wherein the method is characterized by, The step of obtaining the behavior signal of the fish group and calculating the group behavior entropy of the fish group according to the behavior signal comprises: obtaining first image data of the fish group, performing target detection on the first image data, and judging the validity of the first image data according to the target detection result; if the first image data is valid, performing trajectory tracking and behavior feature extraction according to the target detection result, and calculating the group behavior entropy of the fish group according to the extracted behavior feature parameters; if the first image data is invalid, obtaining group acoustic signals of the fish group, and performing sound event detection and main frequency extraction on the group acoustic signals, and calculating the group behavior entropy of the fish group according to the detected sound emission interval and the extracted main frequency sequence. The step of obtaining the behavior signal of the fish group and calculating the group behavior entropy of the fish group according to the behavior signal comprises: obtaining first image data of the fish group, performing target detection on the first image data, and judging the validity of the first image data according to the target detection result; 8. The method according to claim 7, wherein the method is characterized by, if the first image data is valid, performing trajectory tracking and behavior feature extraction according to the target detection result, and calculating the group behavior entropy of the fish group according to the extracted behavior feature parameters; if the first image data is invalid, obtaining group acoustic signals of the fish group, and performing sound event detection and main frequency extraction on the group acoustic signals, and calculating the group behavior entropy of the fish group according to the detected sound emission interval and the extracted main frequency sequence. The step of obtaining the behavior signal of the fish group and calculating the group behavior entropy of the fish group according to the behavior signal comprises: obtaining first image data of the fish group, performing target detection on the first image data, and judging the validity of the first image data according to the target detection result; if the first image data is valid, performing trajectory tracking and behavior feature extraction according to the target detection result, and calculating the group behavior entropy of the fish group according to the extracted behavior feature parameters; if the first image data is invalid, obtaining group acoustic signals of the fish group, and performing sound event detection and main frequency extraction on the group acoustic signals, and calculating the group behavior entropy of the fish group according to the detected sound emission interval and the extracted main frequency sequence. The step of obtaining the behavior signal of the fish group and calculating the group behavior entropy of the fish group according to the behavior signal comprises: obtaining first image data of the fish group, performing target detection on the first image data, and judging the validity of the first image data according to the target detection result; if the first image data is valid, performing trajectory tracking and behavior feature extraction according to the target detection result, and calculating the group behavior entropy of the fish group according to the extracted behavior feature parameters; if the first image data is invalid, obtaining group acoustic signals of the fish group, and performing sound event detection and main frequency extraction on the group acoustic signals, and calculating the group behavior entropy of the fish group according to the detected sound emission interval and the extracted main frequency sequence. Data screening and data statistics are performed on the plurality of individual heart rate values to obtain a heart rate abnormality index and a population heart rate mean, and the heart rate abnormality index and the population heart rate mean are taken as second image features.

9. The method according to claim 8, wherein the method is characterized by, The step of inputting the acoustic features and the image features into a preset abnormality recognition model to perform secondary recognition of abnormal behavior on the initial abnormal fish group comprises: Multi-scale feature fusion is performed on the acoustic features and the first image features through a feature pyramid network, and dynamic weight adjustment is performed based on a squeeze-and-excitation network to obtain initial fusion features; The second image features are normalized and spliced to the initial fusion features to obtain fusion features; The fusion features are input into an abnormality recognition model to obtain a recognition result, and the abnormality recognition model is constructed based on a fully connected neural network model. 10.A system for abnormal behavior recognition of Pseudosciaena crocea based on radio frequency signals, characterized in that, Comprise: The behavior trajectory analysis module is configured to acquire radio frequency signals of tagged fish provided with radio frequency chips in the fish school, perform data processing and feature extraction on the radio frequency signals, obtain behavior feature indicators, and obtain a first abnormality probability according to the behavior feature indicators and a preset health baseline. The group behavior analysis module is configured to acquire behavior signals of the fish school, calculate a group behavior entropy of the fish school according to the behavior signals, and obtain a second abnormality probability according to the group behavior entropy and a preset entropy threshold. The initial abnormality recognition module is configured to perform initial recognition of abnormal behavior on the fish school according to the first abnormality probability and the second abnormality probability, and obtain an initial abnormal fish group. The data processing module is configured to acquire underwater acoustic signals and image data of the initial abnormal fish group, perform feature extraction on the underwater acoustic signals to obtain acoustic features, and perform target detection and feature extraction on the image data to obtain image features. The secondary abnormality recognition module is configured to input the acoustic features and the image features into a preset abnormality recognition model to perform secondary recognition of abnormal behavior on the initial abnormal fish group, and determine an abnormal fish group and a corresponding intervention scheme according to a recognition result, and the abnormality recognition model is constructed based on a neural network model.

Images