A sturgeon health intelligent monitoring method and system based on multi-source data fusion

Abstract

The application provides a kind of sturgeon health intelligent monitoring method and system based on multi-source data fusion, first real-time synchronous acquisition underwater image data and infrared thermal image data of sturgeon breeding pond, and respectively pre-process;After fish positioning, behavior feature extraction and temperature feature extraction are carried out respectively, then the sturgeon behavior feature and fish body temperature feature are fused, finally the sturgeon fusion feature is input into the pre-trained anomaly detection model, and the anomaly detection result is obtained;The application integrates various sensing technologies and advanced data analysis algorithms, realizes comprehensive, real-time, intelligent monitoring of sturgeon breeding process, not only can timely find and handle abnormal situation, but also can provide scientific basis and optimization suggestion for breeding management through long-term data accumulation and analysis;At the same time, the implementation of the application also significantly improves the efficiency and product quality of sturgeon breeding, reduces the risk of disease and economic loss.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of aquaculture, more particularly, to a sturgeon health intelligent monitoring method and system based on multi-source data fusion. BACKGROUND

[0002] Sturgeon is a high economic value aquaculture species, and its caviar is highly sought after on the international market. However, sturgeon farming faces many challenges, the most prominent of which is the difficulty in timely detection and treatment of fish diseases. Traditional sturgeon farming monitoring methods mainly rely on manual patrols, which have the following shortcomings:

[0003] 1) Low monitoring frequency: manual patrols are usually performed a limited number of times per day, making it difficult to achieve around-the-clock monitoring;

[0004] 2) Limited observation range: the water depth of the breeding pond is usually around 1.5 meters, making it difficult for the human eye to fully observe the underwater situation;

[0005] 3) Strong subjectivity: differences in experience and judgment standards among different staff can lead to inconsistent monitoring results;

[0006] 4) Reaction lag: when abnormalities in fish are visible to the naked eye, the disease may have already progressed to a serious stage;

[0007] 5) Difficulty in data recording and analysis: manual monitoring makes it difficult to accumulate and analyze systematic data.

[0008] Some intelligent farming systems have already appeared, which have improved the monitoring capability of aquaculture in some aspects. For example, an existing patent document discloses a water farming monitoring system, which includes a pool water collector arranged in a water pool to be monitored, a water quality collecting device arranged in the pool water collector, a sensor module arranged in the water quality collecting device, the sensor module being used to monitor the temperature, oxygen content, PH value, and dissolved oxygen degree of water, the sensor being connected with a microprocessor through a control module, the microprocessor being arranged on a monitoring base station, the monitoring base station being connected with an output device display instrument, a command input device keyboard, a data storage device, and a power supply, the microprocessor receiving and transmitting the commands of the control module, the microprocessor being connected with a sewage discharge device, a water pump, an oxygen supply equipment, a water temperature adjusting device, a camera, an alarm module, a drainage valve, and a salinity adjusting device. However, this system collects farming environment data through water quality sensors, cameras, and other devices, but mainly focuses on water quality monitoring, and has limited ability to identify abnormal fish behavior.

[0009] In addition, existing intelligent farming systems still have the following shortcomings:

[0010] 1) Data source is single: most systems only rely on a single type of sensor data, which is difficult to fully reflect the health status of fish;

[0011] 2) Insufficient intelligent analysis capability: lack of fusion analysis of multi-source data, unable to fully utilize the complementarity of various data;

[0012] 3) Poor targeting: most are general aquaculture monitoring systems, not optimized for the characteristics of sturgeon, with low monitoring accuracy;

[0013] 4) Inconvenient human-computer interaction: it is difficult for managers to obtain monitoring information and perform remote operations at any time and anywhere. SUMMARY

[0014] To overcome the defects of low efficiency, real-time performance and strong subjectivity of manual monitoring in the prior art, the present application provides a sturgeon health intelligent monitoring method and system based on multi-source data fusion, which can comprehensively, intelligently and in real time monitor the health status of sturgeon, thereby improving the breeding efficiency and reducing economic losses.

[0015] To solve the above technical problems, the technical solution of the present application is as follows:

[0016] A sturgeon health intelligent monitoring method based on multi-source data fusion, comprising the following steps:

[0017] S1: Real-time synchronous acquisition of underwater image data and infrared thermal image data of a sturgeon breeding pond, and pre-processing respectively;

[0018] S2: inputting the pre-processed underwater image data into a pre-trained fish body detection model, locating all sturgeons in each image, obtaining sturgeon position information, and searching for sturgeon fish bodies in the pre-processed infrared thermal image data according to the sturgeon position information;

[0019] S3: behavior analysis of sturgeons according to the sturgeon position information, obtaining sturgeon behavior characteristics; the behavior analysis includes: swimming trajectory analysis, swimming speed analysis, posture analysis and group behavior analysis;

[0020] Extracting fish body temperature characteristics of each sturgeon in the pre-processed infrared thermal image data;

[0021] S4: feature fusion of the sturgeon behavior characteristics and fish body temperature characteristics, obtaining sturgeon fusion characteristics;

[0022] S5: inputting the sturgeon fusion characteristics into a pre-trained anomaly detection model, obtaining an anomaly detection result, and completing intelligent monitoring of sturgeon health.

[0023] Preferably, in the step S1, the underwater image data is processed to obtain preprocessed underwater image data as follows:

[0024] Image denoising: using Gaussian filtering or median filtering to remove noise caused by underwater environment;

[0025] Contrast enhancement: using a limited contrast adaptive histogram equalization algorithm to improve the clarity of the underwater image data;

[0026] Color correction: selecting a reference white point in the underwater image data, calculating the average value of the reference white point in the red, green and blue color channels, and comparing it with the pre-set standard value of the red, green and blue color channels respectively to obtain the gain coefficient of each color channel; using the gain coefficients of the red, green and blue color channels to perform white balance adjustment on the underwater image data;

[0027] The infrared thermal image data is processed as follows to obtain preprocessed infrared thermal image data:

[0028] Image segmentation: converting the infrared thermal image data into a grayscale image, using OTUS algorithm to calculate the segmentation threshold of the water surface and fish body area, and separating the water surface and fish body area according to the segmentation threshold. Figure Two

[0029] Preferably, in the step S2, the fish body detection model is specifically a YOLOv8 target detection model, and the sturgeon position information includes the fish body center point coordinates and the fish body contour points of the sturgeon.

[0030] Preferably, in the step S2, searching for the sturgeon fish body in the preprocessed infrared thermal image data according to the sturgeon position information includes the following steps:

[0031] Using affine transformation to establish the coordinate system mapping relationship between the underwater image data and the infrared thermal image data, which is represented as:

[0032]

[0033] Where (x i ,y i ) is the coordinate of the i-th pixel point in the underwater image data coordinate system, (x′ i ,y′ i ) is the corresponding pixel point coordinate in the infrared thermal image data coordinate system, (x i ,y i ) and (x′ i ,y′ i ) constitute a set of pixel point pairs. is a linear transformation matrix, is a translation matrix; ​

[0034] The following linear equation set is constructed using n groups of pixel points:

[0035]

[0036] wherein n is a positive integer;

[0037] The linear equation set constructed is solved using the least square method to obtain the calculation results of the linear transformation matrix and the translation matrix;

[0038] The sturgeon position information is mapped into the preprocessed infrared thermal image data according to the calculation results, forming a plurality of mapping centers representing the sturgeon positions in the infrared thermal image data, a search window is set with each mapping coordinate as the center, a connected region with a temperature higher than a preset value is searched in the search window, and a corresponding sturgeon fish body is obtained.

[0039] Preferably, in the step S3, the fish body temperature feature of each sturgeon includes:

[0040] In the preprocessed infrared thermal image data, for each located sturgeon fish body region, the temperature values of all pixel points in the region are extracted, the temperature mean of all pixel points in the region is calculated as the average temperature of the corresponding sturgeon fish body, and the average temperature and the standard deviation of all sturgeons in the school are further calculated;

[0041] According to the average temperature of the fish body at each time, the temperature change rate is calculated;

[0042] The average temperature of the fish body and the temperature change rate are saved together as the fish body temperature feature of the sturgeon.

[0043] Preferably, in the step S3, the swimming trajectory analysis includes:

[0044] Based on the position of the fish body center point at the previous time, the position of the fish body center point at the next time is predicted and updated in real time using the Kalman filtering algorithm, the motion trajectory of the fish body center point within a period of time is obtained, and the length, curvature and motion direction change frequency of the motion trajectory are further obtained, the length, curvature and motion direction change frequency of the motion trajectory are saved together as the swimming trajectory feature, and the swimming trajectory analysis is completed;

[0045] The swimming speed analysis includes:

[0046] The fish body displacement within a unit of time is calculated according to the fish body center point coordinates at a plurality of times, and the instantaneous speed of the sturgeon is further calculated, the average speed within a period of time is calculated according to the instantaneous speed of the sturgeon, and the average speed is saved as the swimming speed feature, and the swimming speed analysis is completed;

[0047] The posture analysis includes:

[0048] The principal component analysis method is used to calculate the principal axis direction of the fish body contour, specifically:

[0049] Calculate the covariance matrix Cov of the fish body contour points:

[0050]

[0051] Among them, (x j ,y j ) is the coordinate of the j-th pixel point in the fish body contour point; x and y are the mean abscissa and mean ordinate of the fish body contour point respectively;

[0052] Construct the characteristic equation: Cov·v=λ·v, where λ and v are the eigenvalue and eigenvector of the covariance matrix Cov respectively;

[0053] Solving the characteristic equation to obtain a first eigenvalue λ1 and its corresponding first eigenvector v1, and a second eigenvalue λ2 and its corresponding second eigenvector v2;

[0054] The eigenvector corresponding to the maximum eigenvalue is used as the main axis direction of the fish body contour, the main axis direction is used to calculate the inclination angle of the sturgeon swimming, and the inclination angle is saved as the posture feature to complete the posture analysis;

[0055] The group behavior analysis includes:

[0056] Motion consistency analysis: The instantaneous speed of each sturgeon is used to calculate the consistency index C of the fish school:

[0057]

[0058] Among them, u k is the instantaneous speed of the sturgeon at the kth moment; N is the number of sturgeons in the school; the larger the consistency index C is, the more consistent the movement of the school of fish is;

[0059] Spatial distribution analysis: The spatial distribution of fish schools was evaluated using the nearest neighbor distance method. The Euclidean distance between each sturgeon and its nearest neighbor was calculated to obtain the nearest neighbor distance of each sturgeon, and the average nearest neighbor distance of the fish school was calculated.

[0060] The nearest neighbor distance of each sturgeon is saved as a group behavior feature to complete the group behavior analysis;

[0061] The swimming trajectory characteristics, swimming speed characteristics, posture characteristics and group behavior characteristics are collectively saved as the sturgeon behavior characteristics.

[0062] Preferably, in step S4, the sturgeon behavior feature and the fish body temperature feature are fused according to the following formula to obtain the sturgeon fusion feature F fused :

[0063]

[0064] wherein f1~f s respectively represent the first~s characteristic index values of the sturgeon; f 1,max ~f s,max respectively represent the maximum values of the first~s characteristic index values of the sturgeon; f 1,min ~f s,min respectively represent the minimum values of the first~s characteristic index values of the sturgeon; w1~w s respectively represent the weight factors corresponding to the first~s characteristic index values of the sturgeon.

[0065] The characteristic index of the sturgeon includes any one of the length of the movement trajectory, the curvature, the movement direction change frequency, the average speed, the inclination angle, the nearest neighbor distance, the average body temperature, and the temperature change rate.

[0066] Preferably, in the step S5, the anomaly detection model is specifically a multi-layer LSTM neural network model.

[0067] The multi-layer LSTM neural network model comprises, in sequence, an input layer, a plurality of layers of LSTM layers connected in sequence, a full connection layer, and an output layer.

[0068] A sliding window is set, time series samples are generated using the sturgeon fusion features acquired in the historical time period, and the time series samples are divided into a training set, a validation set, and a test set. The multi-layer LSTM neural network model is supervised and trained using a binary cross-entropy function. After validation and testing, a trained anomaly detection model is obtained.

[0069] The sturgeon fusion features acquired in real time are input into the trained anomaly detection model, an anomaly detection result is obtained, and an anomaly alarm is performed according to the anomaly detection result. The anomaly detection result is an anomaly score of each characteristic index of the sturgeon in each time window.

[0070] Preferably, the trained anomaly detection model judges whether each characteristic index is abnormal based on the following rules:

[0071] If the length of the movement trajectory of the sturgeon is less than a first preset threshold, the sturgeon is abnormal in behavior.

[0072] If the curvature of the movement trajectory of the sturgeon is greater than a second preset threshold, the sturgeon is abnormal in behavior.

[0073] If the movement direction change frequency of the sturgeon is greater than a third preset threshold, the sturgeon is abnormal in behavior.

[0074] If the average speed of the sturgeon is less than a fourth preset threshold, the sturgeon is abnormal in behavior.

[0075] If the inclination angle of the sturgeon is greater than the fifth preset threshold, the sturgeon is abnormal;

[0076] If the absolute value of the difference between the nearest neighbor distance of the sturgeon and the average nearest neighbor distance of the fish school is greater than the sixth preset threshold, the sturgeon is abnormal;

[0077] If any of the following conditions occurs, the sturgeon is abnormal in temperature:

[0078] The average body temperature of the sturgeon is outside the preset normal temperature range, the absolute value of the difference between the average body temperature of the sturgeon and the average temperature of all sturgeons in the fish school is greater than twice the standard deviation, and the temperature change rate is greater than the seventh preset threshold.

[0079] The present application also provides a sturgeon health intelligent monitoring system based on multi-source data fusion, which applies the above-mentioned sturgeon health intelligent monitoring method based on multi-source data fusion, comprising:

[0080] A preprocessing unit is used to synchronously collect underwater image data and infrared thermal image data of a sturgeon breeding pond in real time and perform preprocessing respectively;

[0081] A fish body positioning unit is used to input the preprocessed underwater image data into a pre-trained fish body detection model, locate all sturgeons in each image, obtain sturgeon position information, and search for sturgeon fish bodies in the preprocessed infrared thermal image data according to the sturgeon position information;

[0082] A feature extraction unit is used to perform behavior analysis on sturgeons according to the sturgeon position information to obtain sturgeon behavior features; the behavior analysis includes swimming trajectory analysis, swimming speed analysis, posture analysis, and group behavior analysis;

[0083] and is used to extract fish body temperature features of each sturgeon in the preprocessed infrared thermal image data;

[0084] A feature fusion unit is used to fuse the sturgeon behavior features and fish body temperature features to obtain sturgeon fusion features;

[0085] An abnormality detection unit is used to input the sturgeon fusion features into a pre-trained abnormality detection model to obtain an abnormality detection result and complete intelligent monitoring of sturgeon health.

[0086] Compared with the prior art, the technical scheme of the present application has the following beneficial effects:

[0087] The application provides a sturgeon health intelligent monitoring method and system based on multi-source data fusion, which first synchronously collects underwater image data and infrared thermal image data of a sturgeon breeding pond in real time, and respectively pre-processes them; then inputs the pre-processed underwater image data into a pre-trained fish body detection model, locates all sturgeons in each image, and obtains sturgeon position information; searches for sturgeon bodies in the pre-processed infrared thermal image data according to the sturgeon position information, and extracts the body temperature features of each sturgeon; analyzes the behavior of the sturgeon according to the sturgeon position information, and obtains sturgeon behavior features; the behavior analysis includes swimming trajectory analysis, swimming speed analysis, posture analysis and group behavior analysis; the sturgeon behavior features and the body temperature features are fused to obtain sturgeon fusion features; finally, the sturgeon fusion features are input into a pre-trained anomaly detection model to obtain an anomaly detection result, and the intelligent monitoring of the sturgeon health is completed.

[0088] The beneficial effects of the application are as follows:

[0089] 1) Comprehensive monitoring: The underwater images and infrared thermal images obtained by underwater cameras and infrared thermal imaging sensors realize comprehensive monitoring of sturgeon behavior (swimming trajectory, swimming speed, posture analysis, group behavior) and body temperature, improving the accuracy and comprehensiveness of anomaly detection.

[0090] 2) Strong real-time performance: The application can monitor continuously for 24 hours, timely discover abnormal conditions, and greatly shorten the time interval from problem discovery to processing.

[0091] 3) High degree of intelligence: Combined with artificial intelligence technologies such as machine learning (target recognition model and anomaly detection model), the application can automatically identify complex abnormal patterns based on multi-source data, reduce human judgment errors, and improve detection efficiency and accuracy.

[0092] 4) Data fusion advantage: Through multi-source data fusion technology (time synchronization, spatial registration and feature fusion of multi-source data), the complementarity of different types of data is fully utilized, improving the reliability and robustness of the system.

[0093] 5) Convenient remote monitoring: The application can send monitoring data and alarm information to the terminal of the breeding management personnel, making it convenient to check at any time and anywhere, and improving management efficiency.

[0094] 6) Strong scalability: The overall architecture of the application is flexible and easy to integrate other types of sensors and analysis modules, adapting to future technological development and changes in demand.

[0095] 7) Significant economic benefits: By discovering and processing fish health problems early, the economic losses caused by diseases are effectively reduced, and the breeding efficiency is improved.

[0096] 8) Data value mining: The long-term accumulated data of the invention can be used for deep analysis and mining, providing scientific basis for optimizing breeding strategies and preventing diseases.

[0097] 9) Wide applicability: The basic principles and architecture of the invention can also be applied to other aquaculture species, with broad application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0098] Figure One A multi-source data fusion-based sturgeon health intelligent monitoring method flowchart provided in embodiment 1.

[0099] Figure Two A multi-source data fusion-based sturgeon health intelligent monitoring method flowchart provided in embodiment 2.

[0100] Figure Three A hardware architecture diagram based on which a multi-source data fusion-based sturgeon health intelligent monitoring method provided in embodiment 2 is based on.

[0101] Figure Four A central control platform function diagram provided in embodiment 2. DETAILED DESCRIPTION

[0102] The drawings are only used for illustrative purposes and cannot be understood as limiting the invention;

[0103] In order to better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the actual size of the product;

[0104] For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0105] The technical solutions of the present invention will be further described below in combination with the drawings and embodiments.

[0106] Embodiment 1

[0107] As Figure One shown, the present embodiment provides a multi-source data fusion-based sturgeon health intelligent monitoring method, comprising the following steps:

[0108] S1: Real-time synchronous acquisition of underwater image data and infrared thermal image data of sturgeon breeding pond, and pre-processing respectively;

[0109] S2: Input the pre-processed underwater image data into the pre-trained fish body detection model, locate all sturgeons in each image, obtain the sturgeon position information, and search for the sturgeon fish body in the pre-processed infrared thermal image data according to the sturgeon position information;

[0110] S3: performing behavior analysis on the sturgeons according to the sturgeon position information to obtain sturgeon behavior characteristics; the behavior analysis includes: swimming trajectory analysis, swimming speed analysis, posture analysis, and group behavior analysis;

[0111] extracting fish body temperature characteristics of each sturgeon in the pre-processed infrared thermal image data;

[0112] S4: performing feature fusion on the sturgeon behavior characteristics and the fish body temperature characteristics to obtain sturgeon fusion characteristics;

[0113] S5: inputting the sturgeon fusion characteristics into a pre-trained anomaly detection model to obtain an anomaly detection result, and completing intelligent monitoring of sturgeon health.

[0114] In the specific implementation process, first, underwater image data and infrared thermal image data of a sturgeon breeding pond are synchronously collected in real time and pre-processed respectively; then the pre-processed underwater image data is input into a pre-trained fish body detection model to locate all sturgeons in each image and obtain sturgeon position information; according to the sturgeon position information, sturgeon fish bodies in the pre-processed infrared thermal image data are searched, and fish body temperature characteristics of each sturgeon are extracted; according to the sturgeon position information, behavior analysis is performed on the sturgeons to obtain sturgeon behavior characteristics; the behavior analysis includes: swimming trajectory analysis, swimming speed analysis, posture analysis, and group behavior analysis; the sturgeon behavior characteristics and the fish body temperature characteristics are subjected to feature fusion to obtain sturgeon fusion characteristics; finally, the sturgeon fusion characteristics are input into a pre-trained anomaly detection model to obtain an anomaly detection result, and intelligent monitoring of sturgeon health is completed.

[0115] This method realizes comprehensive, real-time, and intelligent monitoring of the sturgeon breeding process by integrating multiple sensing technologies and advanced data analysis algorithms; not only can it timely discover and handle abnormal situations, but also can provide scientific basis and optimization suggestions for breeding management through long-term data accumulation and analysis.

[0116] Embodiment 2

[0117] The embodiment provides a sturgeon health intelligent monitoring method based on multi-source data fusion, including the following steps:

[0118] S1: synchronously collecting underwater image data and infrared thermal image data of a sturgeon breeding pond in real time, and pre-processing them respectively;

[0119] S2: inputting the pre-processed underwater image data into a pre-trained fish body detection model to locate all sturgeons in each image, obtaining sturgeon position information, and searching sturgeon fish bodies in the pre-processed infrared thermal image data according to the sturgeon position information;

[0120] S3: performing behavior analysis on the sturgeons according to the sturgeon position information to obtain sturgeon behavior characteristics; the behavior analysis comprises: swimming trajectory analysis, swimming speed analysis, posture analysis, and group behavior analysis;

[0121] extracting fish body temperature characteristics of each sturgeon in the preprocessed infrared thermal image data;

[0122] S4: performing feature fusion on the sturgeon behavior characteristics and the fish body temperature characteristics to obtain sturgeon fusion characteristics;

[0123] S5: inputting the sturgeon fusion characteristics into a pre-trained anomaly detection model to obtain an anomaly detection result, and completing intelligent monitoring of the health of the sturgeons;

[0124] In the step S1, the following operations are performed on the underwater image data to obtain preprocessed underwater image data:

[0125] Image denoising: Gaussian filtering or median filtering is used to remove noise caused by the underwater environment;

[0126] Contrast enhancement: a limited contrast adaptive histogram equalization algorithm is used to improve the clarity of the underwater image data;

[0127] Color correction: a reference white point is selected in the underwater image data, the average value of the reference white point on the red, green, and blue color channels is calculated, and the average value is compared with the pre-set standard values of the red, green, and blue color channels respectively to obtain gain coefficients of the respective color channels; the gain coefficients of the red, green, and blue color channels are used to perform white balance adjustment on the underwater image data;

[0128] The following operations are performed on the infrared thermal image data to obtain preprocessed infrared thermal image data:

[0129] Image segmentation: the infrared thermal image data is converted into a grayscale image, the OTUS algorithm is used to calculate the segmentation threshold of the water surface and fish body area, and the grayscale image is segmented according to the segmentation threshold to separate the water surface and fish body area; Figure Two

[0130] In the step S2, the fish body detection model is specifically a YOLOv8 target detection model, and the sturgeon position information comprises a fish body center point coordinate and a fish body contour point of the sturgeon;

[0131] In the step S2, searching for a sturgeon fish body in the preprocessed infrared thermal image data according to the sturgeon position information comprises the following steps:

[0132] An affine transformation is used to establish a coordinate system mapping relationship between the underwater image data and the infrared thermal image data, which is represented as:

[0133] ​

[0134] wherein (x i ,y i ) is the i-th pixel point coordinate in the underwater image data coordinate system, (x′ i ,y′ i ) is the corresponding pixel point coordinate in the infrared thermal image data coordinate system, (x i ,y i ) and (x′ i ,y′ i ) constitute a set of pixel point pairs; is a linear transformation matrix, is a translation matrix;

[0135] n sets of pixel point pairs are used to construct the following linear equation group:

[0136]

[0137] wherein n is a positive integer;

[0138] The least square method is used to solve the constructed linear equation group to obtain the calculation results of the linear transformation matrix and the translation matrix;

[0139] According to the calculation results, the sturgeon position information is mapped into the preprocessed infrared thermal image data, forming a plurality of mapping centers representing the sturgeon positions in the infrared thermal image data, and a search window is set with each mapping coordinate as the center, and a connected region with a temperature higher than a preset value is searched in the search window to obtain the corresponding sturgeon fish body;

[0140] In the step S3, the fish body temperature characteristics of each sturgeon include:

[0141] In the preprocessed infrared thermal image data, for each located sturgeon fish body region, the temperature values of all pixel points in the region are extracted, the temperature mean value of all pixel points in the region is calculated as the average temperature of the corresponding sturgeon fish body, and the average temperature and the standard deviation of all sturgeons in the fish school are further calculated;

[0142] According to the average temperature of the fish body at each time, the temperature change rate is calculated;

[0143] The average temperature of the fish body and the temperature change rate are saved together as the fish body temperature characteristics of the sturgeon;

[0144] In the step S3, the swimming trajectory analysis includes:

[0145] Based on the fish body center point position at the last moment, the Kalman filtering algorithm is used to predict and update the fish body center point position at the next moment in real time, to obtain the fish body center point motion trajectory in a period of time, and further to obtain the length, curvature and motion direction change frequency of the motion trajectory, to jointly save the length, curvature and motion direction change frequency of the motion trajectory as the swimming trajectory feature, to complete the swimming trajectory analysis;

[0146] The swimming speed analysis includes:

[0147] The fish body displacement in a unit of time is calculated according to the fish body center point coordinates at several moments, and the instantaneous speed of the sturgeon is further calculated, the average speed in a period of time is calculated according to the instantaneous speed of the sturgeon, and the average speed is saved as the swimming speed feature, to complete the swimming speed analysis;

[0148] The posture analysis includes:

[0149] The principal axis direction of the fish body contour is calculated using the principal component analysis method, specifically:

[0150] The covariance matrix Cov of the fish body contour points is calculated:

[0151]

[0152] Wherein, (x j ,y j ) is the coordinate of the jth pixel point in the fish body contour point; and are the horizontal coordinate mean and vertical coordinate mean of the fish body contour point, respectively;

[0153] The characteristic equation Cov·v=λ·v is constructed, wherein λ and v are the eigenvalue and eigenvector of the covariance matrix Cov, respectively;

[0154] The characteristic equation is solved to obtain the first eigenvalue λ1 and its corresponding first eigenvector v1, and the second eigenvalue λ2 and its corresponding second eigenvector v2;

[0155] The eigenvector corresponding to the maximum eigenvalue is taken as the principal axis direction of the fish body contour, and the inclination angle of the sturgeon swimming is calculated using the principal axis direction, and the inclination angle is saved as the posture feature, to complete the posture analysis;

[0156] The group behavior analysis includes:

[0157] Motion consistency analysis: the consistency index C of the fish school is calculated using the instantaneous speed of each sturgeon:

[0158]

[0159] Wherein, u kis the instantaneous speed of the sturgeon at the kth moment; N is the number of sturgeons in the fish school; the greater the consistency index C is, the more consistent the fish school moves;

[0160] Spatial distribution analysis: the spatial distribution of the fish school is evaluated using the nearest neighbor distance method, the Euclidean distance between each sturgeon and its nearest neighbor sturgeon is calculated to obtain the nearest neighbor distance of each sturgeon, and the average nearest neighbor distance of the fish school is calculated;

[0161] The nearest neighbor distance of each sturgeon is saved as a group behavior feature, and the group behavior analysis is completed;

[0162] The swimming trajectory feature, swimming speed feature, posture feature and group behavior feature are collectively saved as the sturgeon behavior feature;

[0163] In step S4, the sturgeon behavior feature and the fish body temperature feature are fused according to the following formula to obtain the sturgeon fusion feature F fused :

[0164]

[0165] wherein f1~fs respectively represent the 1st~s characteristic index value of the sturgeon; f s max1~fs respectively represent the maximum value of the 1st~s characteristic index value of the sturgeon; f 1,max min1~fs respectively represent the minimum value of the 1st~s characteristic index value of the sturgeon; w1~ws respectively represent the weight factor corresponding to the 1st~s characteristic index value of the sturgeon. s,max 1,min s,min s

[0166] The characteristic index of the sturgeon includes any one of the length of the movement trajectory, the curvature, the movement direction change frequency, the average speed, the inclination angle, the nearest neighbor distance, the average temperature of the fish body and the temperature change rate;

[0167] In step S5, the anomaly detection model is specifically a multi-layer LSTM neural network model;

[0168] The multi-layer LSTM neural network model includes an input layer, a plurality of layers of LSTM layers connected in sequence, a full connection layer and an output layer connected in sequence;

[0169] A sliding window is set, time sequence samples are generated using the sturgeon fusion features obtained in the historical time period, and are divided into a training set, a validation set and a test set, the multi-layer LSTM neural network model is supervised for training using a binary cross entropy function, and a trained anomaly detection model is obtained after verification and testing;

[0170] ​​​​Input the sturgeon fusion features obtained in real time into the trained anomaly detection model to obtain anomaly detection results, and issue an anomaly alarm based on the anomaly detection results; the anomaly detection results are the anomaly scores of each feature indicator of the sturgeon in each time window;

[0171] The trained anomaly detection model determines whether each feature indicator is abnormal based on the following rules:

[0172] If the length of the sturgeon's movement trajectory is less than a first preset threshold, the sturgeon's behavior is abnormal;

[0173] If the curvature of the sturgeon's trajectory is greater than a second preset threshold, the sturgeon's behavior is abnormal;

[0174] If the frequency of the sturgeon's movement direction changes is greater than a third preset threshold, the sturgeon's behavior is abnormal;

[0175] If the average speed of the sturgeon is less than a fourth preset threshold, the sturgeon's behavior is abnormal;

[0176] If the sturgeon's tilt angle is greater than a fifth preset threshold, the sturgeon's behavior is abnormal;

[0177] If the absolute value of the difference between the nearest neighbor distance of the sturgeon and the average nearest neighbor distance of the fish school is greater than a sixth preset threshold, the sturgeon's behavior is abnormal;

[0178] If any of the following conditions occur, the sturgeon's temperature is abnormal:

[0179] The average body temperature of the sturgeons exceeds the preset normal temperature range, the absolute value of the difference between the average body temperature of the sturgeons and the average temperature of all sturgeons in the school is greater than two standard deviations, and the temperature change rate is greater than the seventh preset threshold.

[0180] In the specific implementation process, Figure Two As shown in the flowchart, this method is mainly divided into four processes: data collection, data processing, analysis and decision-making, and result output, which will be described in detail below;

[0181] First, underwater image data and infrared thermal image data of sturgeon breeding ponds are collected synchronously in real time and preprocessed separately;

[0182] like Figure Three As shown in the figure, the hardware architecture of this method is based on underwater cameras, infrared thermal imagers, central control platforms and intelligent terminals;

[0183] In this embodiment, 3-5 high-definition cameras are installed in each breeding pond to cover different angles and depths and simultaneously obtain underwater image data; the high-definition underwater cameras are industrial-grade cameras with a waterproof level of IP68, a resolution of no less than 1080p, and a frame rate of no less than 30 fps; the cameras are connected to a waterproof junction box at the edge of the pond through a waterproof cable, and then transmit data to the central control platform through an optical fiber network; 1-2 infrared thermal imagers are installed above each breeding pond to collect infrared thermal image data; the infrared thermal imagers are fixed through adjustable supports to cover the entire water surface; the infrared thermal imagers are non-cooled thermal imagers with a resolution of no less than 640x480 and a temperature resolution of better than 0.05℃; the thermal imagers are connected to a data collector at the edge of the pond through a waterproof cable, and then transmit data to the central control platform through an Ethernet network;

[0184] As shown in Figure Four , the central control platform provides functions such as data reception, data storage, data fusion, intelligent analysis, and alarm generation, and sends the generated alarm information to the intelligent terminal (mobile phone, computer, etc.) for the manager to view, realizing real-time monitoring; in this embodiment, the central control platform uses the TCP / IP protocol to receive the data of the underwater cameras and the infrared thermal imagers, realizes a data buffering mechanism to cope with network fluctuations and data bursts, uses a time series database InfluxDB to store sensor data to optimize the storage and query efficiency of time series data, uses a relational database PostgreSQL to store structured data such as fish body information and device configuration, realizes data sharding and backup strategies, and ensures the reliability and scalability of the data; in addition, the received raw data is preliminarily cleaned and compressed to reduce space occupation;

[0185] In order to ensure that the data can be obtained synchronously, the central control platform uses the NTP (Network Time Protocol) protocol to ensure the timestamp consistency of different sensors, realizes a time window mechanism, and aligns the data from different sensors by time;

[0186] The preprocessing of the underwater image data includes:

[0187] Image denoising: using Gaussian filtering or median filtering to remove noise caused by the underwater environment;

[0188] For Gaussian filtering, the two-dimensional convolution operation of Gaussian filtering can be expressed as:

[0189]

[0190] where (x, y) represents the pixel coordinates, and σ is the standard deviation of the Gaussian distribution;

[0191] For median filtering, a 3x3 or 5x5 sliding window can be used to replace the center pixel with the median value of all pixels within the window, effectively removing salt and pepper noise;

[0192] Contrast enhancement: using the CLAHE algorithm to improve the clarity of underwater image data; the CLAHE algorithm divides the image into several small blocks, calculates the gray level histogram of each small block, and clips the pixel values that exceed the preset threshold to other gray levels to limit the contrast enhancement amplitude; then histogram equalization is performed on each small block, and the boundaries between adjacent small blocks are bilinearly interpolated to eliminate block effects and obtain a smooth enhanced image;

[0193] Color correction: selecting a reference white point in the underwater image data, calculating the average value of the reference white point on the red, green and blue (R, G and B) color channels, and comparing it with the preset standard value of the red, green and blue three color channels respectively to obtain the gain coefficient of each color channel; using the gain coefficients of the red, green and blue three color channels to perform white balance adjustment on the underwater image data, thereby restoring the true color of the image;

[0194] For infrared thermal image data preprocessing, including:

[0195] Image segmentation: converting the infrared thermal image data into a grayscale image, using the OTUS algorithm to calculate the segmentation threshold of the water surface and fish body area, the OTUS algorithm is an automatic threshold selection method based on image histogram, which can maximize the inter-class variance between target and background, thereby achieving the best image segmentation effect; the steps of the OTUS algorithm are as follows:

[0196] 1) Calculate the image histogram: count the number of pixels of each gray level in the image to obtain the gray level histogram of the image;

[0197] 2) Calculate the intra-class variance and inter-class variance: for each possible threshold t, divide the image into two classes: target (gray level ≤ t) and background (gray level > t);

[0198] Calculate the pixel proportion of the target and background:

[0199] ω0(t) = Σ p(i), (i = 0 ~ t)

[0200] ω1(t) = Σ p(i), (i = t + 1 ~ L - 1)

[0201] Where p(i) is the probability of gray level i appearing, and L is the total number of gray levels;

[0202] Calculate the average gray level of the target and the background:

[0203] μ0(t) = [∑i·p(i)] / ω0(t), (i = 0 ~ t)

[0204] μ1(t) = [∑i·p(i)] / ω1(t), (i = t + 1 ~ L - 1)

[0205] Calculate the total average gray level of the image:

[0206] μ T = ∑i·p(i), (i = 0 ~ L - 1)

[0207] Calculate the inter-class variance:

[0208] σ B 2 (t) = ω0(t)·[μ0(t) - μ T ] 2 + ω1(t)·[μ1(t) - μ T ] 2

[0209] Calculate the intra-class variance:

[0210] σ W 2 (t) = ω0(t)·σ0 2 (t) + ω1(t)·σ1 2 (t)

[0211] where σ0 2 (t) and σ1 2 (t) are the intra-class variances of the target and the background, respectively;

[0212] 3) Select the best threshold value: traverse all possible threshold values t, find the threshold value that makes the inter-class variance σ B 2 (t) maximum or the intra-class variance σ W 2 (t) minimum, which is the best threshold value;

[0213] Then, according to the best segmentation threshold value, the gray Figure Two value is quantized, the water surface and fish body area are separated, and the preprocessing of the infrared thermal image is completed;

[0214] Next, input the preprocessed underwater image data into the pre-trained fish body detection model to locate all sturgeons in each image and obtain the sturgeon position information;

[0215] In this embodiment, the fish body detection model is a YOLOv8 target detection algorithm, which can quickly and accurately locate sturgeon in the image after a large number of sample training for sturgeon. In the training process, the embodiment uses a data set containing 10,000 labeled sturgeon images, of which 80% is used for training and 20% is used for verification. The training parameters are set as follows: batch size: 64; learning rate: initial value 0.01, using cosine annealing strategy; optimizer: AdamW, weight decay coefficient is 0.0005; training rounds: 100; the detection result includes the position of the fish body (in the form of a bounding box [x, y, w, h], where x and y are the center coordinates of the bounding box, and w and h are the width and height), size (pixel area) and confidence (a floating point number between 0 and 1);

[0216] According to the sturgeon position information, search for the sturgeon fish body in the preprocessed infrared thermal image data, and combine the detection result of the underwater camera to accurately locate the fish body position in the thermal image;

[0217] Specifically, first use affine transformation to establish the coordinate system mapping relationship between the underwater image data and the infrared thermal image data, which is represented as:

[0218]

[0219] Where (x i ,y i ) is the coordinate of the i-th pixel point in the underwater image data coordinate system, (x′ i ,y′ i ) is the corresponding pixel point coordinate in the infrared thermal image data coordinate system, (x i ,y i ) and (x′ i ,y′ i ) form a set of pixel point pairs. is a linear transformation matrix, is a translation matrix;

[0220] Use n sets of pixel point pairs to construct the following linear equation set:

[0221]

[0222] Where n is a positive integer;

[0223] Use the least squares method to solve the constructed linear equation set to obtain the calculation results of the linear transformation matrix and the translation matrix. After obtaining the affine transformation matrix, any point coordinate in the underwater camera coordinate system can be converted into the corresponding point coordinate in the thermal imager coordinate system;

[0224] According to the calculation result, the sturgeon position information is mapped into the pre-processed infrared thermal image data, a plurality of mapping centers representing the sturgeon positions are formed in the infrared thermal image data, a search window is set with each mapping coordinate as a center, a connected region with a temperature higher than a preset value is searched in the search window, and a corresponding sturgeon fish body is obtained;

[0225] Then, a fish body temperature feature of each sturgeon is extracted, for each located fish body region, temperature values of all pixels in the region are extracted, an average value of the temperatures in the region is calculated as a fish body average temperature, a highest temperature value in the region is found as a fish body highest temperature, and further, an average temperature and a standard deviation of all sturgeons in the fish school are calculated, a temperature change rate is calculated according to the fish body average temperature at each time, and the fish body average temperature and the temperature change rate are saved together as the fish body temperature feature of the sturgeon;

[0226] Whether the fish body temperature is abnormal is judged based on the following principles:

[0227] 1) Absolute temperature threshold: according to a normal body temperature range of the sturgeon, upper and lower threshold values are set; in this embodiment, the normal body temperature range is set as 10℃-25℃ (which may be adjusted according to a specific sturgeon variety), a low temperature threshold value is T low =8℃, and a high temperature threshold value is T high =28℃, when the average temperature is lower than the low temperature threshold value or higher than the high temperature threshold value, the fish body temperature is abnormal;

[0228] 2) Relative temperature difference: temperature differences between different fish bodies in the same pool are compared; first, an average value μ and a standard deviation σ of the average temperatures of all fish bodies in the pool are calculated, an abnormal judgment criterion is set as |T-μ|>2σ, if the criterion is satisfied, the fish body temperature is abnormal, where T is the average body temperature of a single fish;

[0229] 3) Temperature change trend: a change trend of the body temperature of a single fish with time is analyzed; body temperature data of each fish in the past 24 hours is recorded, a temperature change slope k is calculated using linear regression, and a slope threshold value k threshold is set, when |k|>k threshold , it is determined that the change trend is abnormal;

[0230] When any of the above conditions is satisfied, the system marks the fish body as temperature abnormal, and triggers a corresponding alarm;

[0231] Based on the detected sturgeon position information, behavior analysis of the sturgeons is performed, and a sturgeon behavior feature is obtained;

[0232] In this embodiment, the behavior analysis includes: swimming trajectory analysis, swimming speed analysis, posture analysis, and group behavior analysis;

[0233] 1) Swimming trajectory analysis: based on the fish center point position at the last time, the Kalman filtering algorithm is used to predict and update the fish center point position at the next time, and the fish center point motion trajectory in a period of time is obtained; Kalman filtering is a recursive filter that can estimate dynamics from a series of noisy measurements, and its core formula is as follows:

[0234] Prediction step:

[0235]

[0236]

[0237] Update step:

[0238]

[0239] P[k / k]=(1-K[k]·H k )·P[k / k-1]

[0240] Where, is the state estimate, P is the error covariance matrix, F k is the state transition matrix, Q k is the process noise covariance, H k is the observation matrix, R k is the observation noise covariance, K is the Kalman gain, and z is the observation value;

[0241] The steps of calculating the swimming trajectory are as follows:

[0242] a. Initialization:

[0243] State vector Initialize the initial position and velocity of the fish center point, for example [x position , y position , x velocity , y velocity ];

[0244] Error covariance matrix P[0 / 0]: initialize the initial uncertainty of state estimation, usually set as a diagonal matrix, the diagonal elements represent the initial variance of the corresponding state variable;

[0245] State transition matrix F k : describes the law of state variable change over time, such as uniform linear motion model or uniform acceleration linear motion model;

[0246] Process noise covariance matrix Q k : represents the noise in the state transition process, such as water flow disturbance or uncertainty of fish body movement;

[0247] Observation matrix Hk : describes the relationship between state variables and observations, for example, if the observation is the coordinate of the fish center, the observation matrix can be set as [1 0 0 0; 0 1 0 0];

[0248] Observation noise covariance matrix R k : represents the noise in the observation process, for example, the error of the image recognition algorithm;

[0249] b. Prediction step (predict the state at the current time based on the state estimate at the last time):

[0250] State prediction Predict the state at the current time using the state transition matrix and the state estimate at the last time;

[0251] Error covariance prediction Predict the error covariance at the current time using the state transition matrix, the error covariance at the last time, and the process noise covariance;

[0252] c. Update step (correct the state prediction based on the observation at the current time):

[0253] Kalman gain Calculate the Kalman gain to balance the influence of the predicted value and the observation on the state estimate;

[0254] State update Update the state estimate at the current time using the Kalman gain, the observation, and the state prediction;

[0255] Error covariance update P[k / k] = (1-K[k]·H k )·P[k / k-1]: Update the error covariance at the current time using the Kalman gain and the observation matrix;

[0256] d. Iteration:

[0257] Repeat steps b and c to continuously update the state estimate using the observation at each time, and obtain the motion trajectory of the fish center;

[0258] e. Output:

[0259] The final output is a series of state estimates representing the position and velocity of the fish center at each time, and these points connected together constitute the swimming trajectory of the fish body;

[0260] For example, if the coordinates of the fish center point at three consecutive time points are (1, 1), (2, 2), and (3, 3), respectively, the Kalman filter algorithm can be used to estimate the position and velocity of the fish at each time point, and the swimming trajectory of the fish can be plotted. Then, the length, curvature, and direction change frequency of the motion trajectory are further obtained.

[0261] Abnormal behavior assessment:

[0262] Trajectory length: Healthy sturgeons usually swim continuously, and their trajectory length is relatively long. Sick or injured sturgeons may exhibit weak swimming, stagnation, and other symptoms, resulting in a significant reduction in trajectory length.

[0263] Trajectory curvature: The swimming trajectory of a healthy sturgeon is relatively smooth, with a small curvature. Sick sturgeons may exhibit unstable swimming direction, turning, and other abnormal behaviors, resulting in an increase in trajectory curvature.

[0264] Trajectory direction change frequency: The swimming direction of a healthy sturgeon is relatively stable, with a low direction change frequency. Sick sturgeons may frequently change their swimming direction, resulting in an increase in direction change frequency.

[0265] 2) Swimming speed analysis:

[0266] Calculate the fish displacement per unit time based on the coordinates of the fish center point at several time points, and further calculate the instantaneous speed of the sturgeon. Calculate the average speed over a period of time based on the instantaneous speed of the sturgeon, save the average speed as the swimming speed feature, and complete the swimming speed analysis.

[0267] Instantaneous speed calculation formula:

[0268] Average speed calculation formula:

[0269] where, is the i-th frame fish center coordinate, Δt is the time interval between adjacent frames (time points), and K is the total number of frames (time points).

[0270] Abnormal behavior judgment: According to the species, size, and breeding environment of sturgeons, set a reasonable swimming speed threshold. Compare the average speed of each fish with the pre-set threshold. If it is lower than the threshold, it is determined that the fish is swimming slowly, and there may be health problems.

[0271] 3) Posture analysis:

[0272] Determine whether there is a side flip or backstroke phenomenon by the inclination angle of the fish contour, which is one of the obvious characteristics of sturgeon disease. Use the principal component analysis method to calculate the principal axis direction of the fish contour, which is:

[0273] Covariance matrix Cov of fish body contour points is calculated:

[0274]

[0275] wherein (x j ,y j ) is the jth pixel point coordinate in the fish body contour points; x and y are the horizontal coordinate mean value and the vertical coordinate mean value of the fish body contour points respectively; the covariance matrix Cov of the fish body contour points is a 2x2 matrix, which is used to describe the relationship between the contour point coordinates x and y; the upper left element represents the variance of variable x; the lower right element represents the variance of variable y; the lower left element and the upper right element represent the covariance between variables x and y;

[0276] A characteristic equation Cov·v = λ·v is constructed, wherein λ and v are the eigenvalue and eigenvector of the covariance matrix Cov respectively;

[0277] The characteristic equation is solved to obtain the first eigenvalue λ1 and the corresponding first eigenvector v1, and the second eigenvalue λ2 and the corresponding second eigenvector v2;

[0278] The specific calculation process is as follows:

[0279] a. Calculate the characteristic polynomial:

[0280] det(Cov-λ·I)=0

[0281] wherein I is a 2x2 identity matrix;

[0282] b. Solve the characteristic equation: expand the characteristic polynomial to obtain a quadratic equation about λ, and solve the equation to obtain two eigenvalues λ1 and λ2;

[0283] c. Calculate the eigenvectors: substitute each eigenvalue into the characteristic equation to solve the corresponding eigenvectors v1 and v2;

[0284] The eigenvector corresponding to the maximum eigenvalue is taken as the principal axis direction of the fish body contour, and the tilt angle of the sturgeon swimming is calculated using the principal axis direction, the tilt angle is saved as a posture feature, and the posture analysis is completed;

[0285] For example, assuming that the covariance matrix Cov is: then the characteristic polynomial is:

[0286] det(Cov-λ·I)=(4-λ)(3-λ)-2×2=λ 2 -7λ+8=0

[0287] Solving the equation obtains two eigenvalues:

[0288]

[0289] Substitute λ1 into the characteristic equation, and obtain:

[0290]

[0291] Solve the equation set, and obtain the eigenvector v1≈[0.85,0.53];

[0292] Similarly, substitute λ2 into the characteristic equation, and obtain the eigenvector v2≈[-0.53,0.85];

[0293] The principal axis direction is the eigenvector corresponding to the maximum eigenvalue, and in this example, λ1>λ2, so the principal axis direction is v1≈[0.85,0.53];

[0294] The tilt angle θ calculation formula is: θ=arctan(v y / v x )

[0295] Where (v x ,v y ) is the unit vector of the principal axis direction; therefore, in this example, the sturgeon swimming tilt angle θ=arctan(0.53 / 0.85)≈31.9°;

[0296] Abnormal posture judgment:

[0297] Set the threshold angle θ threshold , when |θ|>θ threshold , it is determined to be an abnormal posture (such as a roll or backstroke);

[0298] 4) Group behavior analysis:

[0299] Analyze the relative positions and movement patterns of multiple fish to identify abnormal aggregation or dispersion behavior, using the following method:

[0300] Motion consistency analysis: calculate the average speed vector of the fish school and the included angle of each individual speed vector, and use the instantaneous speed of each sturgeon to calculate the consistency index C of the fish school:

[0301]

[0302] Where N is the number of sturgeons in the fish school; the greater the consistency index C, the more consistent the fish school motion, and the closer the C value to 0, the more dispersed the group motion; in this embodiment, a threshold value h of the consistency index is set, and h=0.3;

[0303] Spatial Distribution Analysis: Use the Nearest Neighbor Distance (NND) method to evaluate the spatial distribution of the fish school. Calculate the Euclidean distance between each sturgeon and its nearest neighbor sturgeon to obtain the nearest neighbor distance of each sturgeon, and calculate the average nearest neighbor distance of the fish school.

[0304] Specifically as follows:

[0305] a. Data Preparation:

[0306] First, obtain the coordinate information of each fish in space, such as (x, y) coordinates;

[0307] Store the coordinate information of each fish in a data structure, such as a list or array, for subsequent calculations;

[0308] b. Traverse each fish: use a loop to traverse each fish, taking it as the target fish;

[0309] c. Calculate the distance between the target fish and other fish:

[0310] Inside the loop, traverse all the fish again (including the target fish itself);

[0311] Calculate the distance between the target fish and the fish currently traversed, which can be calculated using the Euclidean distance formula:

[0312] Euclidean distance = sqrt[(x1-x2) 2 +(y1-y2) 2 ]

[0313] Where (x1, y1) is the coordinate of the target fish, and (x2, y2) is the coordinate of the fish currently traversed;

[0314] d. Find the nearest neighbor:

[0315] In the process of calculating the distance, record the minimum distance between the target fish and other fish, as well as the corresponding fish number or index;

[0316] Ignore the target fish itself, i.e. the minimum distance cannot be 0;

[0317] e. Store the nearest neighbor distance:

[0318] Store the minimum distance found in a list or array, which is the NND distribution;

[0319] f. Continue to traverse the next fish, repeat steps b-e until all fish are traversed;

[0320] Final result: get a list / array containing all the fish's nearest neighbor distances, i.e. the NND distribution; and, the fish swarm average nearest neighbor distance = sum of all fish's nearest neighbor distances / number of fish;

[0321] For example, suppose there are 4 fish with coordinates (1,2), (3,4), (5,1), (2,3).

[0322] The calculation process is as follows:

[0323] Fish 1 (1,2):

[0324] Calculate the distance between fish 1 and fish 1: sqrt((1-1) 2 +(2-2) 2 ) = 0;

[0325] Calculate the distance between fish 1 and fish 2: sqrt((1-3) 2 +(2-4) 2 ) = sqrt(8) ≈ 2.83;

[0326] Calculate the distance between fish 1 and fish 3: sqrt((1-5) 2 +(2-1) 2 ) = sqrt(17) ≈ 4.12;

[0327] Calculate the distance between fish 1 and fish 4: sqrt((1-2) 2 +(2-3) 2 ) = sqrt(2) ≈ 1.41;

[0328] The nearest neighbor fish is fish 4, and the nearest neighbor distance is 1.41;

[0329] Fish 2 (3,4):

[0330] Calculate the distance between fish 2 and fish 1: sqrt((3-1) 2 +(4-2) 2 ) = sqrt(8) ≈ 2.83;

[0331] Calculate the distance between fish 2 and fish 2: sqrt((3-3) 2 +(4-4) 2 ) = 0;

[0332] Calculate the distance between fish 2 and fish 3: sqrt((3-5) 2 +(4-1) 2 ) = sqrt(13) ≈ 3.61;

[0333] Calculate the distance between fish 2 and fish 4: sqrt((3-2) 2 +(4-3) 2 ) = sqrt(2) ≈ 1.41.) = sqrt(2) ~ 1.41;

[0334] Nearest neighbor fish is fish 4, nearest neighbor distance is 1.41;

[0335] Fish 3 (5, 1):

[0336] Compute distance of fish 3 from fish 1: sqrt((5-1) 2 +(1-2) 2 ) = sqrt(17) ~ 4.12;

[0337] Compute distance of fish 3 from fish 2: sqrt((5-3) 2 +(1-4) 2 ) = sqrt(13) ~ 3.61;

[0338] Compute distance of fish 3 from fish 3: sqrt((5-5) 2 +(1-1) 2 ) = 0;

[0339] Compute distance of fish 3 from fish 4: sqrt((5-2) 2 +(1-3) 2 ) = sqrt(13) ~ 3.61;

[0340] Nearest neighbor fish is fish 2 or fish 4, nearest neighbor distance is 3.61;

[0341] Fish 4 (2, 3):

[0342] Compute distance of fish 4 from fish 1: sqrt((2-1) 2 +(3-2) 2 ) = sqrt(2) ~ 1.41;

[0343] Compute distance of fish 4 from fish 2: sqrt((2-3) 2 +(3-4) 2 ) = sqrt(2) ~ 1.41;

[0344] Compute distance of fish 4 from fish 3: sqrt((2-5) 2 +(3-1) 2 ) = sqrt(13) ~ 3.61;

[0345] Compute distance of fish 4 from fish 4: sqrt((2-2) 2 +(3-3) 2 ) = 0;

[0346] Nearest neighbor fish is fish 1 or fish 2, nearest neighbor distance is 1.41;

[0347] The final NND distribution is [1.41, 1.41, 3.61, 1.41];

[0348] In this example, the sum of the nearest neighbor distances of all fish is 1.41 + 1.41 + 3.61 + 1.41 = 7.84, and the number of fish is 4. The sum of the nearest neighbor distances of all fish is divided by the number of fish to get the average nearest neighbor distance of the school: 7.84 / 4 = 1.96.

[0349] In the above example, the nearest neighbor distance of fish 3 (5, 1) is 3.61, which is greater than the average nearest neighbor distance of the fish group of 1.96, and is an abnormal object that needs attention;

[0350] Group behavior analysis can be used to identify abnormal dispersion of fish schools in a pond and the isolation of sick fish. Healthy sturgeons usually move in groups, while sick or injured sturgeons may break away from the school, move alone, or avoid other fish. Based on the above analysis, this embodiment sets the following abnormal behavior judgment criteria:

[0351] If the swimming direction and speed of a fish are significantly different from the average direction and speed of the school, it may be isolated and out of the group; when the consistency index C < 0.3, it is judged as abnormal dispersion; if the nearest neighbor distance of a fish is significantly greater than the average nearest neighbor distance of the school (for example, the former is more than three times the latter), it may be isolated and out of the group, and it is judged as abnormal spatial distribution;

[0352] Then, multi-source data fusion is performed, and the sturgeon behavior characteristics and fish body temperature characteristics are fused according to the following formula to obtain the sturgeon fusion feature F fused :

[0353]

[0354] Among them, f1~f s Respectively represent the 1st to sth characteristic index values ​​of sturgeon; f 1,max ~f s,max They represent the maximum values ​​of the 1st to sth characteristic index values ​​of sturgeon; f 1,min ~f s,min Respectively represent the minimum values ​​of the 1st to sth characteristic index values ​​of sturgeon; w1~w s They represent the weight factors corresponding to the 1st to sth characteristic index values ​​of sturgeon respectively. This fusion method comprehensively considers the two factors of weight distribution and data normalization, and fuses multiple variables together through linear combination. The specific values ​​of each feature are mapped to the interval [0,1] to achieve data normalization. This makes data from different ranges comparable and avoids the fusion results being dominated by data from a larger range.

[0355] The characteristic indexes of the sturgeon include any one of a length, a curvature, a frequency of change in moving direction, an average speed, an inclination angle, a nearest neighbor distance, an average temperature of the fish body, and a temperature change rate of a motion trajectory;

[0356] Finally, the sturgeon fusion features are input into a pre-trained anomaly detection model to obtain an anomaly detection result, and intelligent monitoring of the health of the sturgeon is completed.

[0357] In this embodiment, the anomaly detection model is specifically a multi-layer LSTM (Long Short-Term Memory) neural network model; the multi-layer LSTM can learn more complex time sequence features, thereby improving the accuracy and robustness of the model; the multi-layer LSTM stacks multiple LSTM layers together, and the output of each layer is used as the input of the next layer; this structure can learn time sequence features at different levels, in which the bottom layer learns short-term features and the high layer learns long-term features.

[0358] The multi-layer LSTM neural network model includes, which are connected in sequence: an input layer, a plurality of LSTM layers connected in sequence, a fully connected layer, and an output layer; the input layer is used to receive the sturgeon fusion features F fused The LSTM layers have the same structure and are each provided with 128 units, use a tanh activation function, and have a Dropout rate of 0.2; the fully connected layer includes 32 neurons and uses a ReLU activation function; the output layer includes one neuron and uses a Sigmoid activation function to output an anomaly score between 0 and 1;

[0359] A sliding window is set, the window size is 60 (corresponding to 1 minute of data), time sequence samples are generated using the sturgeon fusion features obtained in a historical time period, and the time sequence samples are divided into a training set, a validation set, and a test set at a ratio of 8:1:1; the multi-layer LSTM neural network model is supervised to perform training using a binary cross-entropy function, the optimizer is Adam, the initial value of the learning rate is 0.001, a learning rate decay strategy is used to prevent overfitting of the model; the training batch size is 64, the training round is 100, an early stopping strategy is used to avoid overfitting, and the training is stopped if the loss of the validation set does not decrease for 10 consecutive rounds; the model is tested using the test set, and after the test index meets the expectation, a trained anomaly detection model is obtained.

[0360] The sturgeon fusion features obtained in real time are input into the trained anomaly detection model to obtain an anomaly detection result, and an abnormality alarm is performed according to the anomaly detection result; the anomaly detection result is an anomaly score of each characteristic index of the sturgeon in each time window;

[0361] For each time window of data, the model outputs an anomaly score z between 0 and 1, representing the probability of the occurrence of an anomaly in the time window; set the anomaly score threshold to 0.7, when the anomaly score 0.7<z, determine the anomaly;

[0362] When the sturgeon state anomaly is detected, the following levels of alarm are generated and sent to the intelligent terminal of the management personnel:

[0363] 1) Mild alarm:

[0364] Triggering condition: single index slight anomaly, such as single fish behavior (slow swimming speed or short swimming trajectory or single fish nearest neighbor distance significantly greater than the average nearest neighbor distance of the fish school) or temperature slightly deviating from the normal range or a small number of intelligent analysis modules (comprehensive index) output anomaly score z greater than the set threshold;

[0365] Alarm mode: display yellow warning icon on the system interface and send push notification to the management personnel;

[0366] Suggested operation: pay close attention to the subsequent state of the fish;

[0367] 2) Moderate alarm:

[0368] Triggering condition: multiple index anomalies or single index severe anomalies, such as multiple fish simultaneously exhibiting abnormal behavior (slow swimming) or abnormal spatial distribution of the group (consistency index C<0.3), or single fish rolling or swimming on its back, or single fish body temperature significantly higher or multiple anomaly score z values greater than the set threshold;

[0369] Alarm mode: system interface displays an orange warning icon and sends a mobile phone push notification to the management personnel;

[0370] Suggested operation: check the relevant fish and water quality parameters in time, and if necessary, perform isolation observation and treatment;

[0371] 3) Serious alarm:

[0372] Triggering condition: continuous anomaly or multiple index severe anomaly, such as large area fish school exhibiting abnormal behavior (slow swimming and rolling or swimming on its back) or temperature anomaly, or a large number of anomaly scores z greater than the set threshold or detection of suspected infectious disease symptoms;

[0373] Alarm mode: system interface displays a red warning icon and sends SMS and phone notifications to all relevant personnel;

[0374] Suggested operation: immediately take emergency measures, such as isolating the affected fish school, adjusting water quality parameters, or starting the disease prevention and control process, etc.;

[0375] In addition, the model can be retrained periodically (e.g., weekly) using newly collected data to adapt to changes in the environment and fish population status, while implementing an incremental learning mechanism that allows the model to adapt to new patterns without losing previously learned knowledge.

[0376] It is worth mentioning that the long-term accumulated historical data can be mined and analyzed to discover potential patterns, such as disease prediction: analyzing the disease occurrence patterns in historical data to predict potential future disease risks; breeding strategy optimization: analyzing the effects of different breeding strategies on fish growth and health to optimize breeding strategies; water quality parameter analysis: analyzing the relationship between water quality parameters and fish health to develop more scientific water quality management plans;

[0377] The sturgeon health intelligent monitoring method of the present embodiment was tested in a large sturgeon farm for 6 months. The farm has 20 circular breeding ponds, each with a diameter of 15 meters and a water depth of 2 meters, with a total breeding capacity of about 100,000 sturgeons. The test results show that:

[0378] 1) Abnormal detection accuracy: behavior abnormality detection accuracy reached 94.3%, body temperature abnormality detection accuracy reached 96.7%, and comprehensive abnormality detection accuracy reached 95.8%;

[0379] 2) Response time: the average time from abnormality occurrence to system alarm was shortened to within 2 minutes, and the average time from alarm to the arrival of the breeding personnel at the scene was reduced to within 10 minutes;

[0380] 3) Disease prevention effect: early disease identification rate increased by 78%, greatly reducing the risk of large-scale disease outbreaks, and the overall mortality rate of the fish population decreased by 35%;

[0381] 4) Production efficiency improvement: feed utilization rate increased by 12%, mainly due to more precise feeding time and quantity control; fish population growth rate increased by an average of 8%, shortening the breeding cycle;

[0382] 5) Economic benefits: the use of this method increased the annual profit of the farm by about 20%, with an estimated investment return period of 1.5 years;

[0383] 6) Management efficiency: daily patrol time of breeding management personnel was reduced by 60%, allowing more focus on optimizing breeding strategies, and average processing time of abnormal events was shortened by 45%;

[0384] 7) Data value: the large amount of data accumulated by the system provides scientific basis for the farm to develop long-term development strategies, and based on historical data analysis, the farm successfully predicted and responded to several potential water quality crises;

[0385] 8) Environmental impact: Due to more accurate feeding and water quality management, the water quality of the breeding pond was significantly improved, with ammonia nitrogen and nitrite content reduced by 25% and 30% respectively; the water consumption of the breeding farm was reduced by 15%, reducing the pressure on surrounding water resources;

[0386] 9) Scalability: This method successfully adapts to different specifications of breeding ponds and different varieties of sturgeon, showing good universality. During the test period, sensors such as dissolved oxygen and pH value were successfully integrated, further improving the comprehensiveness of monitoring;

[0387] This method realizes comprehensive, real-time and intelligent monitoring of the sturgeon breeding process by integrating various sensing technologies and advanced data analysis algorithms. It not only can timely detect and handle abnormal situations, but also can provide scientific basis and optimization suggestions for breeding management through long-term data accumulation and analysis;

[0388] The implementation of this method significantly improves the efficiency and product quality of sturgeon breeding, reduces the risk of disease and economic loss, and also provides new ideas and methods for intelligent and fine management of the aquaculture industry. With continuous technological progress and system optimization, this invention is expected to play an important role in a wider range of aquaculture fields, promoting the entire industry towards more efficient and sustainable development.

[0389] The same or similar reference signs correspond to the same or similar components;

[0390] The positional relationship described in the drawings is only used for illustrative description, and cannot be understood as a limitation on the present application;

[0391] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. For those skilled in the art, on the basis of the above description, other different forms of changes or variations can also be made. Here, it is not necessary and impossible to exhaust all the embodiments. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A sturgeon health intelligent monitoring method based on multi-source data fusion, characterized in that: The following steps are involved: S1: Real-time synchronous acquisition of underwater image data and infrared thermal image data of sturgeon breeding ponds, and pre-processing of them respectively; S2: inputting the pre-processed underwater image data into a pre-trained fish detection model, locating all sturgeons in each image, obtaining sturgeon position information, and searching for sturgeons in the pre-processed infrared thermal image data based on the sturgeon position information; Searching for a sturgeon body in the pre-processed infrared thermal image data according to the sturgeon position information includes the following steps: Affine transformation is used to establish the coordinate system mapping relationship between underwater image data and infrared thermal image data, which can be expressed as: in, is the coordinate of the i-th pixel point in the underwater image data coordinate system, is the corresponding pixel coordinate in the infrared thermal image data coordinate system, and Constitute a set of pixel pairs; is the linear transformation matrix, is the translation matrix; Use n sets of pixel pairs to construct the following linear equations: Wherein, n is a positive integer; The constructed linear equations are solved using the least squares method to obtain the calculation results of the linear transformation matrix and the translation matrix; Mapping the sturgeon location information to the preprocessed infrared thermal imaging data according to the calculation result, forming a plurality of mapping centers representing the sturgeon location in the infrared thermal imaging data, setting a search window with each mapping coordinate as the center, searching for a connected area with a temperature higher than a preset value within the search window, and obtaining the corresponding sturgeon body; S3: performing behavioral analysis on the sturgeons according to the sturgeon location information to obtain behavioral characteristics of the sturgeons; the behavioral analysis includes: swimming trajectory analysis, swimming speed analysis, posture analysis and group behavior analysis; Extracting the body temperature characteristics of each sturgeon from the preprocessed infrared thermal imaging data; S4: fusing the sturgeon behavior characteristics and the fish body temperature characteristics to obtain sturgeon fusion characteristics; S5: Input the sturgeon fusion features into a pre-trained anomaly detection model to obtain anomaly detection results, thereby completing intelligent monitoring of sturgeon health.

2. The method for intelligent monitoring of sturgeon health based on multi-source data fusion according to claim 1, characterized in that: In step S1, the following operations are performed on the underwater image data to obtain pre-processed underwater image data: Image denoising: Use Gaussian filtering or median filtering to remove noise caused by the underwater environment; Contrast enhancement: A contrast-limited adaptive histogram equalization algorithm is used to improve the clarity of underwater image data; Color correction: Select a reference white point from the underwater image data, calculate the average value of the reference white point in the red, green, and blue color channels, and compare it with the preset standard values ​​of the three color channels to obtain the gain coefficient of each color channel; use the gain coefficients of the three color channels to perform white balance adjustment on the underwater image data; Perform the following operations on the infrared thermal imaging data to obtain pre-processed infrared thermal imaging data: Image segmentation: The infrared thermal image data is converted into a grayscale image, and the segmentation threshold of the water surface and fish body area is calculated using the OTUS algorithm. The grayscale image is binarized according to the segmentation threshold to separate the water surface and fish body area.

3. The method for intelligent monitoring of sturgeon health based on multi-source data fusion according to claim 1, characterized in that: In step S2, the fish body detection model is specifically a YOLOv8 target detection model, and the sturgeon position information includes the coordinates of the center point of the sturgeon body and the contour points of the sturgeon body.

4. The method for intelligent monitoring of sturgeon health based on multi-source data fusion according to claim 3, characterized in that: In step S3, extracting the body temperature characteristics of each sturgeon includes: In the preprocessed infrared thermal imaging data, for each located sturgeon body region, the temperature values ​​of all pixels in the region are extracted, the temperature mean of all pixels in the region is calculated as the average body temperature of the corresponding sturgeon, and the average temperature and standard deviation of all sturgeons in the fish school are further calculated; Calculate the temperature change rate based on the average temperature of the fish body at each moment; The fish body average temperature and the temperature change rate are saved together as the fish body temperature characteristics of the sturgeon.

5. The method for intelligent monitoring of sturgeon health based on multi-source data fusion according to claim 4, characterized in that: In step S3, the swimming trajectory analysis includes: Based on the fish center position at the previous moment, the Kalman filter algorithm is used to predict and update the fish center position at the next moment in real time. The fish center point motion trajectory over a period of time is obtained. The length, curvature, and frequency of movement direction changes of the motion trajectory are further obtained. The length, curvature, and frequency of movement direction changes of the motion trajectory are saved as the swimming trajectory features to complete the swimming trajectory analysis. The swimming speed analysis includes: The fish displacement per unit time is calculated based on the coordinates of the fish center point at several moments, and the sturgeon's instantaneous speed is further calculated. The average speed over a period of time is calculated based on the sturgeon's instantaneous speed, and the average speed is saved as the swimming speed feature to complete the swimming speed analysis. The posture analysis includes: The principal component analysis method is used to calculate the principal axis direction of the fish body contour, specifically: Calculate the covariance matrix of the fish body contour points : in, is the coordinate of the jth pixel point in the fish body contour point; and are the mean abscissa and mean ordinate of the fish body contour points respectively; Construct the characteristic equation: ,in, and The covariance matrices are The eigenvalues ​​and eigenvectors of Solve the characteristic equation to obtain the first eigenvalue and its corresponding first eigenvector , and the second eigenvalue and its corresponding second eigenvector ; The eigenvector corresponding to the maximum eigenvalue is used as the main axis direction of the fish body contour, the main axis direction is used to calculate the inclination angle of the sturgeon swimming, and the inclination angle is saved as the posture feature to complete the posture analysis; The group behavior analysis includes: Motion consistency analysis: The instantaneous speed of each sturgeon is used to calculate the consistency index C of the fish school: in, is the instantaneous speed of the sturgeon at the kth moment; is the number of sturgeons in the school of fish; the larger the consistency index C is, the more consistent the movement of the school of fish is; Spatial distribution analysis: The spatial distribution of fish schools was evaluated using the nearest neighbor distance method. The Euclidean distance between each sturgeon and its nearest neighbor was calculated to obtain the nearest neighbor distance of each sturgeon, and the average nearest neighbor distance of the fish school was calculated. The nearest neighbor distance of each sturgeon is saved as a group behavior feature to complete the group behavior analysis; The swimming trajectory characteristics, swimming speed characteristics, posture characteristics and group behavior characteristics are collectively saved as the sturgeon behavior characteristics.

6. The method for intelligent monitoring of sturgeon health based on multi-source data fusion according to claim 5, characterized in that: In step S4, the sturgeon behavior characteristics and the fish body temperature characteristics are fused according to the following formula to obtain the sturgeon fusion characteristics: : in, ~ represent the 1st to sth characteristic index values ​​of sturgeon respectively; ~ They represent the maximum values ​​of the 1st to sth characteristic index values ​​of sturgeon respectively; ~ They represent the minimum values ​​of the 1st to sth characteristic index values ​​of sturgeon respectively; ~ They represent the weight factors corresponding to the 1st to sth characteristic index values ​​of sturgeon respectively; The characteristic indicators of the sturgeon include: any one of the length of the movement trajectory, curvature, frequency of change of movement direction, average speed, tilt angle, nearest neighbor distance, average temperature of the fish body and temperature change rate.

7. The method for intelligent monitoring of sturgeon health based on multi-source data fusion according to claim 6, characterized in that: In step S5, the anomaly detection model is specifically a multi-layer LSTM neural network model; The multi-layer LSTM neural network model includes: an input layer, several LSTM layers connected in sequence, a fully connected layer, and an output layer; A sliding window is set, and sturgeon fusion features obtained in a historical time period are used to generate time series samples. The samples are divided into a training set, a validation set, and a test set. The multi-layer LSTM neural network model is supervised for training using a binary cross entropy function. After verification and testing, a trained anomaly detection model is obtained. The sturgeon fusion features obtained in real time are input into the trained anomaly detection model to obtain anomaly detection results, and an anomaly alarm is issued based on the anomaly detection results; the anomaly detection results are the anomaly scores of each feature indicator of the sturgeon in each time window.

8. The method for intelligent monitoring of sturgeon health based on multi-source data fusion according to claim 7, characterized in that: The trained anomaly detection model determines whether each feature indicator is abnormal based on the following rules: If the length of the sturgeon's movement trajectory is less than a first preset threshold, the sturgeon's behavior is abnormal; If the curvature of the sturgeon's trajectory is greater than a second preset threshold, the sturgeon's behavior is abnormal; If the frequency of the sturgeon's movement direction changes is greater than a third preset threshold, the sturgeon's behavior is abnormal; If the average speed of the sturgeon is less than a fourth preset threshold, the sturgeon's behavior is abnormal; If the sturgeon's tilt angle is greater than a fifth preset threshold, the sturgeon's behavior is abnormal; If the absolute value of the difference between the nearest neighbor distance of the sturgeon and the average nearest neighbor distance of the fish school is greater than a sixth preset threshold, the sturgeon's behavior is abnormal; If any of the following conditions occur, the sturgeon's temperature is abnormal: The average body temperature of the sturgeons exceeds the preset normal temperature range, the absolute value of the difference between the average body temperature of the sturgeons and the average temperature of all sturgeons in the school is greater than two standard deviations, and the temperature change rate is greater than the seventh preset threshold.

9. A sturgeon health intelligent monitoring system based on multi-source data fusion, applying the sturgeon health intelligent monitoring method based on multi-source data fusion described in any one of claims 1 to 8, characterized in that: include: Preprocessing unit: used to synchronously collect underwater image data and infrared thermal image data of sturgeon breeding ponds in real time and preprocess them separately; Fish body positioning unit: used for inputting the pre-processed underwater image data into a pre-trained fish body detection model, locating all sturgeons in each image, obtaining sturgeon position information, and searching for sturgeons in the pre-processed infrared thermal image data based on the sturgeon position information; Feature extraction unit: used for analyzing the behavior of the sturgeon according to the sturgeon position information to obtain the behavioral characteristics of the sturgeon; The behavior analysis includes: swimming trajectory analysis, swimming speed analysis, posture analysis and group behavior analysis; and for extracting the body temperature characteristics of each sturgeon from the pre-processed infrared thermal imaging data; Feature fusion unit: used for fusing the sturgeon behavior feature and the fish body temperature feature to obtain the sturgeon fusion feature; Anomaly detection unit: used to input the sturgeon fusion features into a pre-trained anomaly detection model, obtain anomaly detection results, and complete intelligent monitoring of sturgeon health.

Images