A fish school toxicity behavior analysis method and system based on deep learning

Abstract

The application discloses a fish group toxicity behavior analysis method and system based on deep learning, and the analysis method comprises the following steps: acquiring image data to be analyzed; inputting the image data to be analyzed into a fish group identification model to acquire fish group monitoring indexes, wherein the fish group identification model is acquired through training of a training set, and the training set is labeled image data containing fish groups; and acquiring a warning level of fish group comprehensive toxicity according to the fish group monitoring indexes. The application provides a fish group toxicity behavior analysis method and system based on deep learning, which has the functions of fish group detection, tracking and behavior identification, can automatically calculate the swimming speed, swimming distance and behavior classification indexes of fish groups under fixed sequence images, combines specific fish group swimming ability with relatively abstract fish group behavior indexes, and realizes the function of accurately calculating the water quality toxicity warning level of fish groups under fixed sequence images.

Description

Technical Field

[0001] This invention belongs to the field of aquatic ecological environment monitoring technology, and in particular relates to a method and system for analyzing the toxic behavior of fish populations based on deep learning. Background Technology

[0002] Fish are highly sensitive to changes in the aquatic environment and are important indicator organisms for aquatic environment assessment. They are also frequently used to evaluate the combined toxicity effects of single or multiple pollutants. By identifying the behavior of fish populations in response to abnormal changes in water quality, it is possible to achieve the function of monitoring and early warning of water toxicity.

[0003] Patent publication number CN112070799A proposes a method and system for fish trajectory tracking based on artificial neural networks. Patent publication number CN116363494A also proposes a method and system for monitoring fish populations and tracking migration. These methods achieve fish detection, tracking, and trajectory calculation, but they do not analyze the toxic behavior of fish schools, thus failing to achieve water quality toxicity monitoring and early warning functions. Therefore, there is an urgent need for a deep learning-based method for analyzing the toxic behavior of fish schools. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a deep learning-based method for analyzing the toxic behavior of fish schools. This method utilizes a deep learning detection model, a target tracking model, and a video classification model to identify the toxic behavior of fish schools, thereby enabling water quality toxicity monitoring and early warning functions.

[0005] This invention provides a deep learning-based method for analyzing the toxic behavior of fish populations, comprising:

[0006] Acquire the image data to be analyzed;

[0007] The image data to be analyzed is input into the fish school recognition model to obtain fish school monitoring indicators. The fish school recognition model is obtained by training a training set, which is labeled image data containing fish schools.

[0008] Based on the aforementioned fish population monitoring indicators, the overall toxicity level of the fish population is determined as an early warning level.

[0009] Optionally, obtaining the training set includes:

[0010] Obtain video data labeled with fish schools, perform frame extraction on the video data, and obtain the original training sample data;

[0011] The original training sample data is subjected to data augmentation processing to obtain an augmented dataset;

[0012] The original training sample data and the augmented dataset are merged to obtain the training set.

[0013] Optionally, the fish swarm recognition model includes: a fish swarm detection sub-model, a fish swarm tracking sub-model, and a fish swarm behavior recognition sub-model;

[0014] The fish detection sub-model is used to detect the coordinate position data of the fish in the image;

[0015] The fish swarm tracking sub-model is used to track the movement trajectory of the fish swarm based on the coordinate position data of the fish swarm;

[0016] The fish school behavior recognition sub-model is used to recognize fish school behavior.

[0017] Optionally, detecting the fish population includes:

[0018] Acquire image data, preprocess the image data, and obtain preprocessed image data;

[0019] The fish detection sub-model is used to detect the preprocessed image data and obtain the coordinate positions of the fish in the image.

[0020] Optionally, preprocessing the image data to obtain preprocessed image data includes:

[0021] The image data is scaled, and the scaled image data is then converted to a color space to obtain an HSV image;

[0022] The V channel in the HSV image is extracted, and the V channel is subjected to median filtering for noise reduction. An adaptive histogram equalization algorithm is used to perform contrast stretching on the processed V channel. A Gaussian blurring algorithm is used to reduce noise in the stretched V channel. The denoised V channel is then merged with the original H and S channel images to obtain a new HSV image.

[0023] The new HSV image is converted to RGB color space to obtain preprocessed image data.

[0024] Optionally, the fish behavior recognition includes:

[0025] The preprocessed image data is organized according to the data format input by the fish school behavior recognition model to obtain the organized image data;

[0026] The processed image data is input into the fish school behavior recognition sub-model to obtain the fish school behavior recognition result.

[0027] Optionally, obtaining the fish swarm monitoring indicators includes:

[0028] The coordinate location data of the detected and tracked fish schools are converted into trajectory data in units of fish IDs;

[0029] Based on the trajectory data, the monitoring indicators for a single fish species are calculated.

[0030] Based on the monitoring indicators of the individual fish, the monitoring indicators of the fish school are obtained, wherein the monitoring indicators of the fish school include: minimum swimming speed, maximum swimming speed, average swimming speed, maximum swimming distance, minimum swimming distance and average swimming distance.

[0031] Optionally, based on the fish school monitoring indicators, the warning level for the overall toxicity of the fish school can be obtained, including:

[0032] Calculate the fish toxicity warning level for a single sequence of image data;

[0033] The fish toxicity warning levels under several individual sequence images are integrated to obtain the overall fish toxicity warning level.

[0034] Optionally, calculating the fish toxicity warning level for a single sequence of image data includes:

[0035] The fish toxicity warning level is calculated based on the overall speed index of the fish school, and the first toxicity warning level is obtained.

[0036] If the overall speed index of the fish school does not meet the toxicity warning conditions, the fish school toxicity warning level is calculated based on the overall distance index of the fish school to obtain the second toxicity warning level.

[0037] If neither the comprehensive speed index nor the comprehensive distance index of the fish group meets the toxicity warning conditions, then the toxicity warning level of the fish group is calculated based on the individual monitoring index of each fish, and the third toxicity warning level is obtained.

[0038] The first toxicity warning level, the second toxicity warning level, and the third toxicity warning level are corrected using the fish behavior recognition results to obtain the fish toxicity warning level under the single sequence image data.

[0039] The present invention also provides a deep learning-based fish toxicity behavior analysis system, comprising: an image data acquisition module, a fish identification module, and an early warning level module;

[0040] The image data acquisition module is used to acquire image data of the fish school to be analyzed;

[0041] The fish school identification module is used to calculate the monitoring indicators of the fish school using a fish school detection, tracking and behavior recognition model.

[0042] The warning level module is used to calculate the warning level of the overall toxicity of the fish population based on fish population monitoring indicators.

[0043] Compared with the prior art, the present invention has the following advantages and technical effects:

[0044] This invention proposes a deep learning-based method and system for analyzing the toxic behavior of fish schools. It has the functions of detecting, tracking, and recognizing fish schools. It can automatically calculate the swimming speed, swimming distance, and behavioral classification indicators of fish schools under a fixed sequence of images. Then, it combines the specific swimming ability of fish schools with more abstract fish school behavioral indicators to accurately calculate the water quality toxicity warning level of fish schools under a fixed sequence of images.

[0045] The fish behavior recognition model developed in this invention can identify various behavioral patterns of fish, such as rapid swimming, normal swimming, slow swimming, near death / death, and grouping, providing more reference data for analyzing water quality toxicity warning levels.

[0046] This invention develops a method and system for analyzing fish toxicity behavior based on deep learning. By analyzing multiple individual fish toxicity warning levels over a period of time, a more accurate comprehensive toxicity warning level for fish water quality can be obtained. Attached Figure Description

[0047] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0048] Figure 1 This is a flowchart of a deep learning-based method for analyzing the toxic behavior of fish populations, according to an embodiment of the present invention.

[0049] Figure 2 This is an offline modeling flowchart of an embodiment of the present invention;

[0050] Figure 3 This is a flowchart of fish detection and tracking according to an embodiment of the present invention;

[0051] Figure 4 This is a flowchart of the image preprocessing process according to an embodiment of the present invention;

[0052] Figure 5 This is a flowchart of fish school behavior recognition according to an embodiment of the present invention;

[0053] Figure 6 This is a flowchart illustrating the calculation of fish school monitoring indicators according to an embodiment of the present invention;

[0054] Figure 7 This is a flowchart of fish swarm data processing according to an embodiment of the present invention;

[0055] Figure 8 This is a flowchart illustrating the calculation of monitoring indicators for a single fish species according to an embodiment of the present invention.

[0056] Figure 9This is a flowchart illustrating the calculation of fish school monitoring indicators according to an embodiment of the present invention;

[0057] Figure 10 This is a flowchart of the fish toxicity early warning system according to an embodiment of the present invention;

[0058] Figure 11 This is a flowchart of a single fish swarm toxicity warning according to an embodiment of the present invention;

[0059] Figure 12 This is a flowchart illustrating the calculation of the toxicity warning level of the comprehensive speed index of fish schools according to an embodiment of the present invention;

[0060] Figure 13 This is a flowchart illustrating the calculation of the toxicity warning level of the comprehensive distance index of fish schools according to an embodiment of the present invention;

[0061] Figure 14 This is a flowchart of the statistical fish swarm single-item speed early warning counting process according to an embodiment of the present invention;

[0062] Figure 15 This is a flowchart of the fish swarm single-item speed early warning calculation according to an embodiment of the present invention;

[0063] Figure 16 This is a flowchart of the fish school behavior recognition, early warning, and correction process according to an embodiment of the present invention;

[0064] Figure 17 This is a flowchart illustrating the calculation of toxicity warning levels based on fish behavior recognition, according to an embodiment of the present invention.

[0065] Figure 18 This is a flowchart of the fish swarm comprehensive toxicity early warning system according to an embodiment of the present invention;

[0066] Figure 19 This is a schematic diagram of a comprehensive fish toxicity early warning detection method according to an embodiment of the present invention;

[0067] Figure 20 This is a flowchart illustrating the statistical counting of single fish swarm toxicity warnings according to an embodiment of the present invention;

[0068] Figure 21 This is a flowchart of the fish swarm comprehensive toxicity early warning calculation according to an embodiment of the present invention. Detailed Implementation

[0069] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0070] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0071] This invention proposes a deep learning-based method for analyzing the toxic behavior of fish populations, such as... Figure 1 As shown, the specific steps include:

[0072] Acquire the image data to be analyzed;

[0073] The image data to be analyzed is input into the fish school recognition model to obtain fish school monitoring indicators. The fish school recognition model is obtained by training a training set, which consists of labeled image data containing fish schools.

[0074] Based on fish school monitoring indicators, obtain the early warning level of the overall toxicity of the fish school.

[0075] Specifically, the analytical method includes:

[0076] S1. Train deep learning-based fish target detection model, target tracking model, and fish school behavior recognition model.

[0077] S2. Collect video data of fish schools and perform detection and tracking processing on the fish schools in the images.

[0078] S3. Extract fixed frame image data from the video and perform fish swarm behavior recognition.

[0079] S4. Calculate fish monitoring indicators based on fish detection, tracking, and behavior recognition data.

[0080] S5. Calculate the fish toxicity warning level based on fish monitoring indicators.

[0081] Furthermore, obtaining the training set includes:

[0082] Obtain labeled video data containing fish schools, perform frame extraction on the video data, and obtain the original training sample data;

[0083] Perform data augmentation on the original training sample data to obtain the augmented dataset;

[0084] The original training sample data and the augmented dataset are merged to obtain the training set.

[0085] Specifically, S1 also includes: collecting and labeling video data containing fish schools, performing frame extraction on the data, and forming raw training sample data.

[0086] The original training sample data is automatically augmented to form an augmented dataset.

[0087] The original training samples are merged with the augmented dataset to form the training samples for the fish swarm detection, tracking, and behavior recognition model.

[0088] We selected and trained deep learning models for fish swarm detection, tracking, and behavior recognition.

[0089] Test the effectiveness of the fish swarm detection, tracking, and behavior recognition model, and adjust the training samples, annotation methods, or add new training samples based on the test results;

[0090] Iteratively optimize the training of fish swarm detection, tracking, and behavior recognition models until the accuracy meets the expected target.

[0091] Furthermore, the fish swarm recognition model includes: a fish swarm detection sub-model, a fish swarm tracking sub-model, and a fish swarm behavior recognition sub-model;

[0092] The fish school detection sub-model is used to detect the coordinate location data of fish schools in the image;

[0093] The fish swarm tracking sub-model is used to track the movement trajectory of the fish swarm based on the coordinate position data of the fish swarm;

[0094] A fish school behavior recognition sub-model is used to recognize fish school behavior.

[0095] Furthermore, the identification and tracking of fish schools includes:

[0096] Acquire image data, preprocess the image data, and obtain the preprocessed image data;

[0097] The fish detection sub-model is used to detect the preprocessed image data and obtain the coordinate position data of the fish in the image;

[0098] Based on the coordinate location data, the fish swarm is tracked using a fish swarm tracking sub-model, and the trajectory of the fish swarm is calculated.

[0099] Specifically, S2 also includes: preprocessing the acquired image data to improve the image contrast.

[0100] The fish detection model is used to infer the coordinates of the fish in the preprocessed image.

[0101] Using the coordinates of the fish school, we further perform fish school tracking model inference on the preprocessed image to calculate the position of the fish school in the next frame.

[0102] Furthermore, the image data is preprocessed to obtain preprocessed image data, including:

[0103] The image data is scaled, and the scaled image data is then converted to a color space to obtain an HSV image.

[0104] The V channel is extracted from the HSV image, and median filtering is applied to the V channel for noise reduction. An adaptive histogram equalization algorithm is used to perform contrast stretching on the processed V channel. Gaussian blurring algorithm is used to reduce noise in the stretched V channel. The denoised V channel is then merged with the original H and S channel images to obtain a new HSV image.

[0105] The new HSV image is converted to RGB color space to obtain preprocessed image data.

[0106] Furthermore, fish school behavior identification includes:

[0107] The preprocessed image data is organized according to the data format input to the fish school behavior recognition model to obtain the organized image data;

[0108] The processed image data is input into the fish school behavior recognition sub-model to obtain the fish school behavior recognition results.

[0109] Specifically, S3 also includes: collecting a fixed number of image data according to the model input requirements, and performing image enhancement processing on them.

[0110] The enhanced sequence image data is input into the fish behavior recognition model for inference, and the fish behavior classification result based on the current sequence image data is obtained.

[0111] Furthermore, S4 also includes converting the detected and tracked fish school coordinate data into trajectory data in units of fish ID.

[0112] Calculate the monitoring indicators for a single fish species using fish ID as the unit.

[0113] Collect individual fish monitoring indicators for all fish in the fish population and calculate the overall fish population monitoring indicators.

[0114] Furthermore, S5 also includes: calculating the water quality toxicity warning level for a single fish population based on fish population monitoring indicators from the current sequence image data.

[0115] The overall toxicity warning time for fish populations is set, and the overall toxicity warning level for fish populations is calculated by analyzing the single toxicity warning level of fish populations monitored multiple times within the time period.

[0116] Furthermore, calculating the fish toxicity warning level for a single sequence of image data includes:

[0117] The fish toxicity warning level is calculated based on the overall speed index of the fish school, and the first toxicity warning level is obtained.

[0118] If the overall speed index of the fish school does not meet the toxicity warning conditions, the toxicity warning level of the fish school is calculated based on the overall distance index of the fish school, and the second toxicity warning level is obtained.

[0119] If neither the overall speed index nor the overall distance index of the fish school meets the toxicity warning conditions, then the toxicity warning level of the fish school is calculated based on the individual monitoring index of a single fish, and the third toxicity warning level is obtained.

[0120] The first, second, and third toxicity warning levels were corrected using the results of fish school behavior recognition, and the fish school toxicity warning level was obtained under a single sequence of image data.

[0121] The present invention also provides a deep learning-based fish toxicity behavior analysis system, comprising: an image data acquisition module, a fish identification module, and an early warning level module;

[0122] The image data acquisition module is used to acquire image data of the fish school to be analyzed.

[0123] The fish school identification module is used to calculate the monitoring indicators of the fish school by utilizing fish school detection, tracking and behavior recognition models;

[0124] The early warning level module is used to calculate the early warning level of the overall toxicity of the fish population based on the monitoring indicators of the fish population.

[0125] The following is a detailed description of this embodiment with reference to the accompanying drawings:

[0126] This embodiment proposes a deep learning-based method for analyzing the toxic behavior of fish populations, including the following steps:

[0127] (1) Offline modeling: Training deep learning-based fish detection, tracking, and behavior recognition models.

[0128] (2) Fish Detection and Tracking: Collect video data of fish schools and perform detection and tracking processing on the fish schools in the images.

[0129] (3) Fish school behavior recognition: Extract fixed frame image data from the video and perform fish school behavior recognition.

[0130] (4) Calculation of fish school monitoring indicators: Calculate fish school monitoring indicators based on fish school detection, tracking and behavior identification data.

[0131] (5) Fish toxicity warning: Calculate the overall toxicity warning level of the fish population based on fish population monitoring indicators.

[0132] like Figure 2 As shown, the offline modeling process includes the following:

[0133] (1) Collect and label video data containing fish schools, and perform frame extraction to form raw training sample data.

[0134] (2) Perform automated data augmentation on the original training sample data to form an augmented dataset.

[0135] (3) The fish groups used for water toxicity early warning are all of the same species. Preferably, in this embodiment, the yolov10n model is used to train the fish group detection model, which improves the algorithm running efficiency while ensuring detection accuracy.

[0136] (4) Zebrafish are a model organism. Because zebrafish swim relatively fast, this embodiment uses zebrafish for water toxicity early warning and identification tasks. Therefore, preferably, the deepsort model is used to train the fish swarm tracking model.

[0137] (5) Through toxicity tests on zebrafish and analysis of relevant video data, it was found that zebrafish exhibit a relatively small variety of behavioral patterns under different toxic conditions. Therefore, preferably, this embodiment uses the PP-TSMv2 lightweight model to train the fish behavior recognition model.

[0138] ⑹ Test the effectiveness of the fish swarm detection, tracking and behavior recognition model, and adjust the training samples, annotation methods or add new training samples based on the test results.

[0139] (7) Iteratively optimize the training of the fish swarm detection, tracking, and behavior recognition model until the accuracy meets the expected target.

[0140] like Figure 3 As shown, the fish detection and tracking process includes the following:

[0141] (1) By setting parameters, a fixed amount of image data can be collected. The default is to collect 64 frames of sequential image data.

[0142] (2) Perform image preprocessing on the acquired sequence image data to improve image contrast.

[0143] (3) Perform fish detection model inference on the preprocessed sequence image data to detect the coordinate position of the fish in the image.

[0144] (4) Input the coordinates of the detected fish school, perform fish school tracking model inference on the preprocessed image, and calculate the position of the fish school on the next frame image and the ID of each fish in the fish school.

[0145] (5) Construct a vector <detectbox>The `fishBox` variable contains the following variables: `DetectBox`, a structure containing `x1`, `y1`, `x2`, `y2`, `confidence`, and `trackID`. `x1` and `y1` represent the x-coordinates of the top-left corner of the detected fish, respectively; `x2` and `y2` represent the x-coordinates of the bottom-right corner of the detected fish, respectively; `confidence` is the confidence level of the detected fish; and `trackID` is the tracking ID of the fish. The detected and tracked fish data from the preprocessed image sequence are stored in the `fishBox` variable.

[0146] like Figure 4 As shown, the image preprocessing workflow includes the following:

[0147] (1) First, the RGB image is scaled to improve overall operating efficiency.

[0148] (2) Convert the scaled RGB image to HSV color space to obtain an HSV image.

[0149] (3) Extract the V channel of the HSV image and perform median filtering to remove noise.

[0150] (4) The contrast stretching operation is performed on the V channel image after median filtering and denoising using the CLAHE (contrast-limited adaptive histogram equalization) algorithm.

[0151] (5) The image data stretched by the CLAHE algorithm will have a lot of noise, and Gaussian blurring algorithm is needed to further reduce noise.

[0152] (6) Merge the V channel image after Gaussian blurring and noise reduction with the H and S channels in the original HSV channel to obtain a new HSV_new image.

[0153] (7) Convert the HSV_new image to RGB color space to obtain the RGB_new image.

[0154] (8) Perform the operations of steps (1) to (7) on each frame of the sequence image to obtain the preprocessed sequence image data.

[0155] like Figure 5 As shown, the fish school behavior recognition process includes the following:

[0156] (1) Organize the preprocessed sequence image data into the input format of the PP-TSMv2 model.

[0157] (2) Reason the PP-TSMv2 model to obtain the behavior classification results.

[0158] (3) The classification result of the model with the highest confidence is used as the result of fish behavior recognition.

[0159] (4) Construct a variable `activityData` within a `fishActivityData` structure, which includes two variables: `confidence` and `type`. `confidence` represents the confidence level of the fish swarm behavior identification; `type` represents the category number of the fish swarm behavior. Save the results of the fish swarm behavior identification in step (3) into `activityData`.

[0160] like Figure 6 As shown, the calculation process for fish swarm detection indicators includes the following:

[0161] (1) Iterate through the trackIDs of fish in the fishBox, and store the detection data of fish with the same trackID together, forming fish trajectory data in units of trackID.

[0162] (2) Calculate the minimum swimming speed, maximum swimming speed, average swimming speed and swimming distance of fish with the same trackID.

[0163] (3) Statistical analysis was performed on the monitoring indicators of fish with different trackIDs to obtain the minimum swimming speed, maximum swimming speed, average swimming speed, minimum swimming distance, maximum swimming distance and average swimming distance of the fish under the current sequence image data.

[0164] like Figure 7 As shown, the fish swarm data processing process includes the following:

[0165] (1) Construct a map <int,vector <detectbox>The variable `fishBoxData` iterates through the `trackID` values ​​in each `fishBox`, using `trackID` as an index to store the fish detection data of `DetectBox`s with the same `trackID` together, resulting in fish swarm trajectory data `fishBoxData` in units of `trackID`.

[0166] (2) Count the number of fish tracks with the same trackID in fishBoxData to obtain the number of times that fish with trackID has been tracked.

[0167] (3) Because zebrafish swim fast, there may be tracking loss when the fish swims rapidly in a cross pattern. In order to reduce systematic error, when statistically analyzing fish monitoring indicators, it is necessary to remove fish detection data with fewer tracking times. Preferably, when the number of frames in the sequence image is 64, fish detection data with fewer than 8 tracking times will be removed.

[0168] like Figure 8 As shown, the calculation process for monitoring indicators of a single fish species includes the following:

[0169] (1) Construct a vector <fishdetectordata>The variable `fishData` contains a structure called `FishDetectorDATA`, which includes variables `m_minSpeed`, `m_maxSpeed`, `m_meanSpeed`, and `m_distance`. These represent the minimum swimming speed, maximum swimming speed, average swimming speed, and swimming distance of the fish, respectively.

[0170] (2) Assume that box1 and box2 are the coordinates of fish trajectories in adjacent frames with the same index in fishBoxData. First, calculate the center point coordinates (cx1, cy1) of box1, where cx1 = x1 + (x2 - x1) / 2, cy1 = y1 + (y2 - y1) / 2. Calculate the center point coordinates (cx2, cy2) of box2 in the same way. To simplify the calculation, the pixel distance dist between the two center point coordinates (cx1, cy1) and (cx2, cy2) is taken as the swimming speed and swimming distance of the fish in adjacent frames.

[0171] (3) The monitoring index structure for a single fish is FishDetectorDATA. Following step (2), calculate the pixel distance dist of the fish trajectories in all adjacent frames under the same fish index, find the minimum value and assign it to the m_minSpeed ​​variable, and the maximum value and assign it to the m_maxSpeed ​​variable. Then, sum up the pixel distance dist of all adjacent frames to obtain the total swimming distance of a single fish and assign it to the m_distance variable. Assuming that the pixel distance dist of the fish trajectories in adjacent frames is calculated N times, finally divide m_distance by N to obtain the average swimming distance of the fish and assign it to the m_meanSpeed ​​variable. Through the above method, the monitoring index of a single fish can be calculated.

[0172] (4) Traverse fishBoxData, calculate the individual monitoring indicators for each fish index according to step (3), and save them in fishData.

[0173] like Figure 9 As shown, the calculation process for fish school monitoring indicators includes the following:

[0174] (1) Construct a FishToxicityDATA structure variable fishTotalData, which includes the variables m_minSpeed, m_maxSpeed, m_meanSpeed, m_minDistance, m_maxDistance, and m_meanDistance. These represent the minimum swimming speed, maximum swimming speed, average swimming speed, minimum swimming distance, maximum swimming distance, and average swimming distance of all fish in the school, respectively.

[0175] (2) Iterate through the individual monitoring indicators of all fish in fishData, find the minimum swimming speed of all fish and assign it to the m_minSpeed ​​variable of fishTotalData; find the maximum swimming speed of all fish and assign it to the m_maxSpeed ​​variable of fishTotalData; find the minimum swimming distance of all fish and assign it to the m_minDistance variable of fishTotalData; find the maximum swimming distance of all fish and assign it to the m_maxDistance variable of fishTotalData; calculate the average swimming speed of all fish and assign it to the m_meanSpeed ​​variable of fishTotalData; calculate the average swimming distance of all fish and assign it to the m_meanDistance variable of fishTotalData.

[0176] like Figure 10 As shown, the fish toxicity early warning process includes the following:

[0177] (1) Single fish toxicity warning: Calculation of fish toxicity warning level under a single sequence of image data.

[0178] (2) Comprehensive toxicity warning for fish schools: Calculation of comprehensive toxicity warning level for fish schools based on multiple sequence image data.

[0179] like Figure 11 As shown, the single fish swarm toxicity warning process includes the following:

[0180] (1) Toxicity warning of comprehensive fish population indicators: Analyze fishTotalData to calculate the toxicity warning level of fish population monitoring indicators.

[0181] (2) Toxicity warning for individual fish indicators: The toxicity warning level is calculated by analyzing the swimming speed index of fish in fishData.

[0182] (3) Fish school behavior recognition warning correction: The fish school toxicity warning level calculated in steps (1) and (2) is corrected by using the fish school behavior recognition results in activityData.

[0183] like Figure 12 As shown, the calculation process for the toxicity warning level of the comprehensive speed index of fish schools includes the following:

[0184] (1) Analyze the speed indicators in the fish population monitoring metrics of fishTotalData, including the variables m_minSpeed, m_maxSpeed, and m_meanSpeed. If the minimum swimming speed m_minSpeed ​​or the average swimming speed m_meanSpeed ​​of the fish in the population is greater than the set threshold of 1 (preferably set to 12, and adjustable as needed), then the overall swimming speed of the fish in the population is considered to be relatively fast. This may be due to external environmental disturbances or trace amounts of toxins entering the water, causing a stress response in the fish population. Therefore, the calculated fish population toxicity warning level is yellow, indicating a low level of toxicity warning.

[0185] (2) If the maximum swimming speed m_maxSpeed ​​or the average swimming speed m_meanSpeed ​​of the fish in the school is less than a set threshold of 2 (preferably 1.2, and adjustable as needed), then it is assumed that the fish in the school are hardly moving, indicating a high level of toxins in the water, causing the fish to be in a near-death or dead state. Therefore, the calculated fish toxicity warning level is red, indicating a high level of toxicity.

[0186] (3) If the maximum swimming speed (m_maxSpeed) or average swimming speed (m_meanSpeed) of the fish in the school is less than a set threshold of 3 (preferably 5, and adjustable as needed), then the overall swimming speed of the fish in the school is considered slow, indicating a high level of toxins in the water. This suggests that the fish are exhibiting significant poisoning symptoms and a marked decrease in swimming ability. Therefore, the calculated fish toxicity warning level is orange, representing a moderate toxicity warning.

[0187] ⑷If steps (1), (2) and (3) are not satisfied, then the comprehensive distance index of the fish group will be further analyzed.

[0188] like Figure 13 As shown, the calculation process for the toxicity warning level of the fish school comprehensive distance index includes the following:

[0189] (1) Analyze the distance indicators in the fish population monitoring metrics of fishTotalData, including the variables m_minDistance, m_maxDistance, and m_meanDistance. If the minimum swimming distance m_minDistance or the average swimming distance m_meanDistance of the fish in the population is greater than the set threshold of 4 (preferably set to 190, and adjustable as needed), then it is considered that the overall swimming speed of the fish in the population is relatively fast. This may be due to interference from the external environment or the entry of trace toxins into the water, causing a stress response in the fish population. Therefore, the calculated fish population toxicity warning level is yellow, indicating a low toxicity warning.

[0190] (2) If the maximum swimming distance m_maxDistance or the average swimming distance m_meanDistance of the fish in the school is less than a set threshold of 5 (preferably 30, and adjustable as needed), then it is assumed that the fish in the school are hardly moving, indicating a high level of toxins in the water, causing the fish to be in a near-death or dead state. Therefore, the calculated fish toxicity warning level is red, indicating a high level of toxicity.

[0191] (3) If the maximum swimming distance m_maxDistance or the average swimming distance m_meanDistance of the fish in the school is less than the set threshold of 6 (preferably 70, and adjustable as needed) when the yellow and red warning levels for fish toxicity are not met, it is considered that the fish in the school are swimming slowly, indicating a high level of toxins in the water. This results in significant poisoning of the fish and a marked decrease in their swimming ability. Therefore, the calculated fish toxicity warning level is orange, representing a moderate toxicity warning.

[0192] (4) If steps (1), (2) and (3) are not satisfied, then we will further analyze the individual indicators of the fish population.

[0193] like Figure 14 As shown, the statistical counting process for single-item speed warnings in fish swarms includes the following:

[0194] (1) Assuming that fish_data is a monitoring index of a certain fish in fishData, analyze the minimum swimming speed m_minSpeed, maximum swimming speed m_maxSpeed ​​and average swimming speed m_meanSpeed ​​of fish_data, and determine whether the swimming speed index of the fish meets the toxicity warning conditions.

[0195] (2) Set up yellow warning counter variable YCount, red warning counter variable RCount, orange warning counter variable OCount, and normal counter variable NCount respectively. If a fish meets the corresponding warning condition, the corresponding counter will be incremented by 1.

[0196] (3) If the minimum swimming speed m_minSpeed ​​or the average swimming speed m_meanSpeed ​​of fish_data is greater than the set threshold 1, preferably 12, and can be adjusted as needed, then it is considered that the fish is swimming faster, possibly due to interference from the external environment or trace amounts of toxins entering the water, causing a stress response. Therefore, the YCount counter is incremented by 1.

[0197] (4) If the speed index in fish_data does not meet the yellow toxicity warning condition, and the fish's maximum swimming speed m_maxSpeed ​​or average swimming speed m_meanSpeed ​​is less than the set threshold 2 (preferably set to 1.2, and adjustable as needed), then it is considered that the fish is hardly swimming, indicating a high level of toxins in the water, causing it to be in a near-death or dead state. Therefore, the RCount counter is incremented by 1.

[0198] (5) If the speed index in fish_data does not meet the conditions for yellow and red toxicity warnings, and the fish's maximum swimming speed m_maxSpeed ​​or average swimming speed m_meanSpeed ​​is less than the set threshold of 3 (preferably set to 5, and adjustable as needed), then it is considered that the fish is swimming slowly, indicating a high level of toxins in the water, causing it to exhibit obvious poisoning symptoms and a significant reduction in swimming ability. Therefore, the OCount counter is incremented by 1.

[0199] ⑹ If the speed index in fish_data does not meet the yellow, red, and orange warning conditions for toxicity, it means that the fish has no obvious toxic reaction, and the NCount counter will be incremented by 1.

[0200] (7) Analyze the speed index of all fish in fishData, and count the toxicity warnings of all fish according to steps (3) to (6).

[0201] like Figure 15 As shown, the calculation process for single-item speed warning of fish swarms includes the following:

[0202] (1) Count the values ​​of all fish warning counters. If the value of the red warning counter is greater than the set threshold (preferably 10, but adjustable as needed), it is assumed that most fish in the school are barely moving, indicating a high level of toxins in the water, causing them to be near death or dying. Therefore, the calculated fish toxicity warning level is red, representing a high level of toxicity warning.

[0203] (2) If the red toxicity warning for fish populations is not met, but the orange warning counter value exceeds a set threshold (preferably 10, but adjustable as needed), it indicates that most fish in the school are swimming slowly, suggesting a high level of toxins in the water. This results in significant poisoning of the fish population and a marked decrease in their swimming ability. Therefore, the calculated toxicity warning level for the fish population is orange, representing a moderate toxicity warning.

[0204] (3) If the red and orange warning levels for fish toxicity are not met, and the value of the yellow warning counter exceeds a set threshold (preferably 10, but adjustable as needed), it is assumed that the faster swimming speed of most fish in the school is due to external environmental disturbances or trace amounts of toxins entering the water, causing a stress response in the fish. Therefore, the calculated fish toxicity warning level is yellow, indicating a low level of toxicity.

[0205] (4) If steps (1), (2) and (3) are not satisfied, then it is considered that most fish in the fish population do not show obvious toxic reactions. Therefore, the calculated fish toxicity warning level is green, indicating that no obvious toxic reactions were detected in the fish population in the water.

[0206] like Figure 16 As shown, the fish behavior recognition, early warning, and correction process includes the following:

[0207] (1) Count the values ​​of all fish warning counters. If the value of the red warning counter is greater than the set threshold (preferably 10, but adjustable as needed), it is assumed that most fish in the school are barely moving, indicating a high level of toxins in the water, causing them to be near death or dying. Therefore, the calculated fish toxicity warning level is red, representing a high level of toxicity warning.

[0208] (2) If the red toxicity warning for fish populations is not met, but the orange warning counter value exceeds a set threshold (preferably 10, but adjustable as needed), it indicates that most fish in the school are swimming slowly, suggesting a high level of toxins in the water. This results in significant poisoning of the fish population and a marked decrease in their swimming ability. Therefore, the calculated toxicity warning level for the fish population is orange, representing a moderate toxicity warning.

[0209] (3) If the red and orange warning levels for fish toxicity are not met, and the value of the yellow warning counter exceeds a set threshold (preferably 10, but adjustable as needed), it is assumed that the faster swimming speed of most fish in the school is due to external environmental disturbances or trace amounts of toxins entering the water, causing a stress response in the fish. Therefore, the calculated fish toxicity warning level is yellow, indicating a low level of toxicity.

[0210] (4) If steps (1), (2) and (3) are not satisfied, then it is considered that most fish in the fish population do not show obvious toxic reactions. Therefore, the calculated fish toxicity warning level is green, indicating that no obvious toxic reactions were detected in the fish population in the water.

[0211] like Figure 17 As shown, the calculation process for the toxicity warning level based on fish behavior includes the following:

[0212] (1) Analyze the `confidence` and `type` variables in `activityData`. If `type` = 4, it indicates that the fish behavior recognition model identifies fish mortality behavior. Further analyze the `confidence` value. If the `confidence` value is greater than the set threshold (preferably 0.8), it is considered that most fish in the school are hardly moving, and there should be a large amount of toxins in the water, causing them to be in a near-death or dead state. Therefore, the calculated fish toxicity warning level is red, indicating a high level of toxicity warning.

[0213] (2) If the red alert for fish toxicity is not met, analyze the `confidence` and `type` variables in `activityData`. If `type` = 3, it indicates that the fish behavior recognition model identifies slow swimming behavior. Further analyze the `confidence` value. If the `confidence` value is greater than the set threshold (preferably 0.8), it is considered that the overall swimming speed of the fish in the school is slow, indicating a high level of toxins in the water. This results in significant poisoning of the fish and a marked decrease in their swimming ability. Therefore, the calculated fish toxicity alert level is orange, indicating a moderate toxicity alert.

[0214] (3) If the red and orange warning levels for fish toxicity are not met, analyze the `confidence` and `type` variables in `activityData`. If `type` = 2, it indicates that the fish behavior recognition model identifies the fish as exhibiting rapid swimming behavior. Further analyze the `confidence` value. If the `confidence` value is greater than the set threshold (preferably set to 0.8), then it is considered that the fish in the school are swimming rapidly, possibly due to external environmental interference or trace amounts of toxins entering the water, causing a stress response in the fish. Therefore, the calculated fish toxicity warning level is yellow, indicating a low level of toxicity.

[0215] (4) If the red, orange, and yellow warning levels for fish toxicity are not met, analyze the confidence and type variables in activityData. If type = 1, it indicates that the fish behavior recognition model identifies normal swimming behavior. Further analyze the confidence value. If the confidence value is greater than the set threshold (preferably 0.8), then the swimming behavior of the fish in the school is considered normal, and no obvious toxic reaction is observed. Therefore, the calculated fish toxicity warning level is green, indicating that no obvious toxic reaction was detected in the fish in the water.

[0216] (5) If steps (1), (2), (3) and (4) are not satisfied, then it is considered that the fish school behavior recognition model has not recognized obvious fish school behavior. Therefore, the calculated fish school toxicity warning level is gray, indicating that the current fish school behavior recognition result is unreliable.

[0217] like Figure 18 As shown, the comprehensive toxicity early warning process for fish populations includes the following:

[0218] (1) Fish school behavior has a certain degree of randomness. In order to improve the accuracy of fish school toxicity warning, this embodiment will continuously observe multiple single fish school toxicity warning levels over a period of time and statistically analyze the data of multiple fish school toxicity warning levels.

[0219] (2) By statistically analyzing multiple fish toxicity warning level data, the comprehensive toxicity warning level of the fish population over a period of time is calculated to obtain more accurate warning results.

[0220] like Figure 19 The diagram shown illustrates a comprehensive toxicity early warning detection method for fish populations, which includes the following:

[0221] (1) The time interval for the comprehensive toxicity warning of fish schools is T, that is, every T minutes, the fish toxicity behavior analysis system will calculate a comprehensive toxicity warning level. This time T can be set, preferably 10, which means 10 minutes. Setting the time too short will affect the accuracy of the comprehensive toxicity warning level calculation; conversely, it will reduce the timeliness of the comprehensive toxicity warning, which is not conducive to early detection of problems when water pollution incidents occur.

[0222] (2) The analysis time for a single fish swarm toxicity warning is t1, in seconds. This time can be set, preferably 2 seconds. Since zebrafish swim relatively fast, this embodiment uses a frame rate of 30 frames per second to collect sequence image data when analyzing the fish swarm toxicity warning level within the t1 time period. When t1 is set to 2 seconds, the sequence image collected for a single fish swarm toxicity warning level analysis is 60 frames. Setting t1 too short will affect the accuracy of the comprehensive toxicity warning level calculation; conversely, although it will increase the accuracy of the toxicity warning level calculation, it will also greatly increase the computational load of the system. Through multiple experiments, it was found that when t1 is set to 2 to 5 seconds, there is no significant change in the output results.

[0223] (3) This embodiment aims to ensure the accuracy of the fish toxic behavior early warning system's warning results while minimizing the system's computational load. Assuming the presence of toxicity in the water, the toxicity concentration will not change significantly within a relatively short time period T, and the toxic behaviors exhibited by the fish will be largely similar. Therefore, to maintain the accuracy of the warning results, it is unnecessary to continuously analyze all sequence image data within the time period T. This function can be achieved by setting an interval time t2, where t2 is the interval between two fish toxic behavior early warning analyses within the time period T. Preferably, it is set to 10, representing a 10-second interval. Using this method, the fish toxic behavior analysis system will calculate a toxicity warning level every 12 seconds (t1 + t2). Within a time period T = 10 minutes, a total of 50 fish toxicity warning levels will be calculated. Statistical analysis of these 50 fish toxicity warning levels can significantly reduce the system's computational load while maintaining the accuracy of the toxicity warning.

[0224] like Figure 20 As shown, the statistical counting process for a single fish swarm toxicity warning includes the following:

[0225] (1) Construct a vector <int>The `alarmLevel` variable stores the data on the toxicity warning levels of fish populations within the time period T.

[0226] (2) Set up yellow warning counter variables YCount, red warning counter variables RCount, orange warning counter variables OCount, green warning counter variables GCount and gray warning counter variables GreyCount respectively. If the fish toxicity warning level meets the corresponding warning conditions, the corresponding counter will be incremented by 1.

[0227] (3) Iterate through and analyze the elements in alarmLevel, with each element corresponding to a fish swarm toxicity warning level. If the current element's toxicity warning level is red, then increment the RCount counter by 1.

[0228] (4) Iterate through and analyze the elements in alarmLevel, with each element corresponding to a fish swarm toxicity warning level. If the current element's toxicity warning level is orange, then increment the OCount counter by 1.

[0229] (5) Iterate through and analyze the elements in alarmLevel, with each element corresponding to a fish swarm toxicity warning level. If the current element's toxicity warning level is yellow, then increment the YCount counter by 1.

[0230] (6) Iterate through and analyze the elements in alarmLevel, with each element corresponding to a fish swarm toxicity warning level. If the current element's toxicity warning level is green, then increment the GCount counter by 1.

[0231] (7) Iterate through and analyze the elements in alarmLevel, with each element corresponding to a fish swarm toxicity warning level. If the current element's toxicity warning level is not red, orange, yellow, or green, then increment the GreyCount counter by 1.

[0232] like Figure 21 As shown, the calculation process for comprehensive fish toxicity early warning includes the following:

[0233] (1) Count the values ​​of all warning counters. If the value of the red warning counter RCount is greater than the set threshold (preferably 25, but adjustable as needed), then the overall toxicity warning level for the fish population during time period T is considered red, indicating a high level of toxicity.

[0234] (2) If the red alert for comprehensive fish toxicity is not met, but the value of the orange alert counter OCount is greater than the set threshold (preferably 25, but adjustable as needed), then the fish toxicity alert level for time period T is considered orange, indicating a moderate toxicity alert.

[0235] (3) If the red and orange alert levels for overall fish toxicity are not met, and the value of the yellow alert counter YCount is greater than the set threshold (preferably 25, but adjustable as needed), then the overall fish toxicity alert level for time period T is considered to be yellow, indicating a low level of toxicity.

[0236] (4) If the red, orange, and yellow alert levels for comprehensive fish toxicity are not met, and the value of the green alert counter GCount is greater than a set threshold (preferably 25, but adjustable as needed), then the comprehensive fish toxicity alert level for time period T is considered green, indicating that no significant toxic reaction was detected in the fish population.

[0237] (5) If steps (1), (2), (3) and (4) are not satisfied, then the overall toxicity warning level of the fish population is considered to be gray during time period T, indicating that no significant behavioral patterns of the fish population were detected and it is not used as a basis for warning.

[0238] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.< / int> < / fishdetectordata> < / detectbox> < / detectbox>

Claims

1. A method for analyzing the toxic behavior of fish populations based on deep learning, characterized in that, include: Acquire the image data to be analyzed; The image data to be analyzed is input into the fish school recognition model to obtain fish school monitoring indicators. The fish school recognition model is obtained by training a training set, which is labeled image data containing fish schools. Obtaining the training set includes: Obtain video data labeled with fish schools, perform frame extraction on the video data, and obtain the original training sample data; The original training sample data is subjected to data augmentation processing to obtain an augmented dataset; The original training sample data and the augmented dataset are merged to obtain the training set; The fish school identification model includes: a fish school detection sub-model, a fish school tracking sub-model, and a fish school behavior identification sub-model; The fish detection sub-model is used to detect the coordinate position data of the fish in the image; The fish swarm tracking sub-model is used to track the movement trajectory of the fish swarm based on the coordinate position data of the fish swarm; The fish school behavior recognition sub-model is used to recognize fish school behavior; The detection of the fish population includes: Acquire image data, preprocess the image data, and obtain preprocessed image data; The fish detection sub-model is used to detect the preprocessed image data and obtain the coordinate position data of the fish in the image. Preprocessing the image data to obtain preprocessed image data includes: The image data is scaled, and the scaled image data is then converted to a color space to obtain an HSV image; The V channel in the HSV image is extracted, and the V channel is subjected to median filtering for noise reduction. The processed V channel is then subjected to contrast stretching using an adaptive histogram equalization algorithm. The stretched V channel is then subjected to noise reduction using a Gaussian blurring algorithm. The denoised V channel is then merged with the original H and S channel images to obtain a new HSV image. The new HSV image is converted to RGB color space to obtain preprocessed image data. The fish school behavior recognition includes: The preprocessed image data is organized according to the data format input by the fish school behavior recognition model to obtain the organized image data; The processed image data is input into the fish school behavior recognition sub-model to obtain the fish school behavior recognition result; The fish swarm monitoring indicators include: The coordinate location data of the detected and tracked fish schools are converted into trajectory data in units of fish IDs; Based on the trajectory data, the monitoring indicators for a single fish species are calculated. Based on the monitoring indicators of the individual fish, the monitoring indicators of the fish school are obtained, wherein the monitoring indicators of the fish school include: minimum swimming speed, maximum swimming speed, average swimming speed, maximum swimming distance, minimum swimming distance, and average swimming distance; based on the monitoring indicators of the fish school, the warning level of the overall toxicity of the fish school is obtained; Based on the aforementioned fish school monitoring indicators, the overall toxicity warning level of the fish school is obtained, including: Calculate the fish toxicity warning level for a single sequence of image data; The fish toxicity warning levels under several individual sequence images are integrated to obtain the overall fish toxicity warning level; Calculating the fish toxicity warning level for a single sequence of image data includes: The fish toxicity warning level is calculated based on the overall speed index of the fish school, and the first toxicity warning level is obtained. If the overall speed index of the fish school does not meet the toxicity warning conditions, the fish school toxicity warning level is calculated based on the overall distance index of the fish school to obtain the second toxicity warning level. If neither the comprehensive speed index nor the comprehensive distance index of the fish group meets the toxicity warning conditions, then the toxicity warning level of the fish group is calculated based on the individual monitoring index of each fish, and the third toxicity warning level is obtained. The first toxicity warning level, the second toxicity warning level, and the third toxicity warning level are corrected using the fish behavior recognition results to obtain the fish toxicity warning level under the single sequence image data.

2. A deep learning-based fish toxicity behavior analysis system, used to implement the deep learning-based fish toxicity behavior analysis method as described in claim 1, characterized in that, include: Image data acquisition module, fish school identification module, and early warning level module; The image data acquisition module is used to acquire image data of the fish school to be analyzed; The fish school identification module is used to calculate the monitoring indicators of the fish school using a fish school detection, tracking and behavior recognition model. The warning level module is used to calculate the warning level of the overall toxicity of the fish population based on the monitoring indicators of the fish population.

Images