Deep learning-based fish school toxicity behavior analysis method and system

Abstract

The invention discloses a fish school toxicity behavior analysis method and system based on deep learning. The analysis method comprises the following steps: acquiring to-be-analyzed image data; the to-be-analyzed image data is input into a fish school recognition model, a fish school monitoring index is obtained, the fish school recognition model is obtained through training of a training set, and the training set is marked image data containing a fish school; and according to the fish school monitoring index, obtaining the early warning grade of the comprehensive toxicity of the fish school. The invention provides a fish school toxicity behavior analysis method and system based on deep learning, which has the functions of fish school detection, tracking and behavior recognition, and can automatically calculate the swimming speed, swimming distance and behavior classification index of a fish school under a fixed sequence image. And then the specific fish school swimming ability is combined with the abstract fish school behavior index, so that the function of accurately calculating the fish school water quality toxicity early warning level under the fixed sequence image is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of water ecological environment monitoring, and particularly relates to a method and system for analyzing the toxicity behavior of fish schools based on deep learning. Background Art

[0002] Fish are sensitive to changes in the water environment. They are important indicator organisms for water environment evaluation and are often used for the evaluation of the combined toxicity effects of single or multiple pollutants. By identifying the behavior of fish schools in response to abnormal changes in water quality, the toxicity monitoring and early warning functions of water quality can be achieved.

[0003] Patent Publication No. CN112070799A proposes a method and system for tracking fish trajectories based on an artificial neural network. Patent Publication No. CN116363494A also proposes a method and system for monitoring fish population and tracking migration. These methods achieve fish detection, tracking, and trajectory calculation, but do not analyze the toxicity behavior of fish schools and cannot realize the toxicity monitoring and early warning functions of water quality. Therefore, there is an urgent need for a method for analyzing the toxicity behavior of fish schools based on deep learning. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a method for analyzing the toxicity behavior of fish schools based on deep learning. This method utilizes deep learning detection models, target tracking models, and video classification models to have the ability to identify the toxicity behavior of fish schools, and realizes the toxicity monitoring and early warning functions of water quality.

[0005] The present invention provides a method for analyzing the toxicity behavior of fish schools based on deep learning, including:

[0006] Obtain the image data to be analyzed;

[0007] Input the image data to be analyzed into a fish school recognition model to obtain fish school monitoring indicators, where the fish school recognition model is obtained by training with a training set, and the training set is image data containing fish schools that have been labeled;

[0008] According to the fish school monitoring indicators, obtain the early warning level of the comprehensive toxicity of the fish school.

[0009] Optionally, obtaining the training set includes:

[0010] Obtain video data labeled with fish schools, perform frame extraction on the video data to obtain original training sample data;

[0011] Perform data augmentation processing on the original training sample data to obtain an augmented data set;

[0012] Merge the original training sample data and the augmented data set to obtain the training set.

[0013] Optionally, the fish school recognition model includes: a fish school detection sub-model, a fish school tracking sub-model, and a fish school behavior recognition sub-model;

[0014] The fish school detection sub-model is used to detect the coordinate position data of the fish school in the image;

[0015] The fish school tracking sub-model is used to track the movement trajectory of the fish school according to the coordinate position data of the fish school;

[0016] The fish school behavior recognition sub-model is used to recognize the behavior of the fish school.

[0017] Optionally, detecting the fish school includes:

[0018] Obtaining image data, preprocessing the image data, and obtaining the preprocessed image data;

[0019] Using the fish school detection sub-model to detect the preprocessed image data, and obtaining the coordinate position of the fish school on the image.

[0020] Optionally, preprocessing the image data to obtain the preprocessed image data includes:

[0021] Scaling the image data, performing color space conversion on the scaled image data, and obtaining an HSV image;

[0022] Extracting the V channel in the HSV image, performing median filtering denoising on the V channel, performing contrast stretching on the processed V channel using an adaptive histogram equalization algorithm, performing noise reduction on the stretched V channel using a Gaussian blurring algorithm, and merging the denoised V channel with the original H and S channel images to obtain a new HSV image;

[0023] Performing RGB color space conversion on the new HSV image to obtain the preprocessed image data.

[0024] Optionally, performing the fish school behavior recognition includes:

[0025] Sorting the preprocessed image data according to the data format required for input to the fish school behavior recognition model to obtain the sorted image data;

[0026] Inputting the sorted image data into the fish school behavior recognition sub-model to obtain the fish school behavior recognition result.

[0027] Optionally, obtaining the fish school monitoring indicators includes:

[0028] Converting the coordinate position data of the detected and tracked fish school into trajectory data in units of fish IDs;

[0029] Calculate the monitoring indicators of a single fish according to the trajectory data;

[0030] Obtain the fish school monitoring indicators according to the monitoring indicators of the single fish, wherein the fish school monitoring indicators include: minimum swimming speed, maximum swimming speed, average swimming speed, maximum swimming distance, minimum swimming distance, and average swimming distance.

[0031] Optionally, obtaining the early warning level of the comprehensive toxicity of the fish school according to the fish school monitoring indicators includes:

[0032] Calculate the early warning level of the fish school toxicity under a single sequence of image data;

[0033] Integrate the early warning levels of the fish school toxicity under several single sequences of images to obtain the early warning level of the comprehensive toxicity of the fish school.

[0034] Optionally, calculating the early warning level of the fish school toxicity under a single sequence of image data includes:

[0035] Calculate the early warning level of the fish school toxicity according to the comprehensive speed index of the fish school to obtain the first toxicity early warning level;

[0036] If the comprehensive speed index of the fish school does not meet the toxicity early warning condition, then calculate the early warning level of the fish school toxicity according to the comprehensive distance index of the fish school to obtain the second toxicity early warning level;

[0037] If both the comprehensive speed index and the comprehensive distance index of the fish school do not meet the toxicity early warning condition, then calculate the early warning level of the fish school toxicity according to the single-item monitoring indicators of a single fish to obtain the third toxicity early warning level;

[0038] Use the fish school behavior recognition result to correct the first toxicity early warning level, the second toxicity early warning level, and the third toxicity early warning level to obtain the early warning level of the fish school toxicity under the single sequence of image data.

[0039] The present invention also provides a fish school toxicity behavior analysis system based on deep learning, including: an image data acquisition module, a fish school recognition module, and an early warning level module;

[0040] The image data acquisition module is used to acquire the fish school image data to be analyzed;

[0041] The fish school recognition module is used to calculate the monitoring indicators of the fish school by using a fish school detection, tracking, and behavior recognition model;

[0042] The early warning level module is used to calculate the early warning level of the comprehensive toxicity of the fish school according to the fish school monitoring indicators.

[0043] Compared with the prior art, the present invention has the following advantages and technical effects:

[0044] The present invention proposes a method and system for analyzing the toxicity behavior of fish schools based on deep learning, which has the functions of detecting, tracking and behavior recognition of fish schools, can automatically calculate the swimming speed, swimming distance and behavior classification indexes of fish schools under a fixed sequence of images, and then combines the specific swimming ability of fish schools with relatively abstract fish school behavior indexes to realize the function of accurately calculating the water quality toxicity warning level of fish schools under a fixed sequence of images.

[0045] The fish behavior recognition model developed by the present invention can recognize various behavior patterns of fish schools, such as fast swimming, normal swimming, slow swimming, dying / dead, and aggregation, providing more reference data for analyzing the water quality toxicity warning level.

[0046] A method and system for analyzing the toxicity behavior of fish schools based on deep learning developed by the present invention can obtain a more accurate comprehensive water quality toxicity warning level of fish schools by analyzing the water quality toxicity warning levels of multiple single fish schools within a period of time. Brief Description of the Drawings

[0047] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application. In the drawings:

[0048] Figure 1 is a flowchart of a method for analyzing the toxicity behavior of fish schools based on deep learning according to an embodiment of the present invention;

[0049] Figure 2 is a flowchart of offline modeling according to an embodiment of the present invention;

[0050] Figure 3 is a flowchart of fish school detection and tracking according to an embodiment of the present invention;

[0051] Figure 4 is a flowchart of image preprocessing according to an embodiment of the present invention;

[0052] Figure 5 is a flowchart of fish school behavior recognition according to an embodiment of the present invention;

[0053] Figure 6 is a flowchart of calculating fish school monitoring indexes according to an embodiment of the present invention;

[0054] Figure 7 is a flowchart of fish school data arrangement according to an embodiment of the present invention;

[0055] Figure 8 is a flowchart of calculating monitoring indexes of a single fish according to an embodiment of the present invention;

[0056] Figure 9It is the flowchart for calculating fish population monitoring indicators in an embodiment of the present invention;

[0057] Figure 10 It is the flowchart for fish population toxicity early warning in an embodiment of the present invention;

[0058] Figure 11 It is the flowchart for single - time fish population toxicity early warning in an embodiment of the present invention;

[0059] Figure 12 It is the flowchart for calculating the toxicity early - warning level of the comprehensive speed index of the fish population in an embodiment of the present invention;

[0060] Figure 13 It is the flowchart for calculating the toxicity early - warning level of the comprehensive distance index of the fish population in an embodiment of the present invention;

[0061] Figure 14 It is the flowchart for counting the single - item speed early warning of the fish population in an embodiment of the present invention;

[0062] Figure 15 It is the flowchart for calculating the single - item speed early warning of the fish population in an embodiment of the present invention;

[0063] Figure 16 It is the flowchart for correcting the fish population behavior recognition early warning in an embodiment of the present invention;

[0064] Figure 17 It is the flowchart for calculating the toxicity early - warning level of the fish population behavior recognition in an embodiment of the present invention;

[0065] Figure 18 It is the flowchart for the comprehensive toxicity early warning of the fish population in an embodiment of the present invention;

[0066] Figure 19 It is the schematic diagram of the detection method for the comprehensive toxicity early warning of the fish population in an embodiment of the present invention;

[0067] Figure 20 It is the flowchart for counting the single - time fish population toxicity early warning in an embodiment of the present invention;

[0068] Figure 21 It is the flowchart for calculating the comprehensive toxicity early warning of the fish population in an embodiment of the present invention. Detailed implementation manners

[0069] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the accompanying drawings and combine with the embodiments to detail this application.

[0070] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer - executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0071] The present invention proposes a method for analyzing fish toxicity behavior based on deep learning, such as Figure 1 As shown, the specific steps include:

[0072] Acquire image data to be analyzed;

[0073] Input the image data to be analyzed into the fish school recognition model to obtain the fish school monitoring index, wherein the fish school recognition model is obtained by training the training set, and the training set is the labeled image data containing the fish school;

[0074] Based on the fish monitoring indicators, the early warning level of the comprehensive toxicity of the fish population is obtained.

[0075] Specifically, the analysis method includes:

[0076] S1. Train the fish target detection model, target tracking model and fish school behavior recognition model based on deep learning.

[0077] S2. Collect video data of fish schools, and detect and track the fish schools on the images.

[0078] S3, capturing fixed-frame image data from the video and performing fish school behavior recognition.

[0079] S4. Calculate fish school monitoring indicators based on fish school detection, tracking and behavior recognition data.

[0080] S5. Calculate the fish toxicity warning level based on fish monitoring indicators.

[0081] Furthermore, obtaining a training set includes:

[0082] Obtain video data labeled with fish schools, perform frame extraction on the video data, and obtain original training sample data;

[0083] Perform data augmentation processing on the original training sample data to obtain an augmented data set;

[0084] The original training sample data and the augmented data set are merged to obtain the training set.

[0085] Specifically, S1 also includes: collecting and labeling video data containing fish schools, and performing frame extraction processing on the video data to form original training sample data.

[0086] The original training sample data is automatically augmented to form an augmented data set.

[0087] The original training samples are combined with the augmented dataset to form training samples for the fish detection, tracking and behavior recognition models.

[0088] Select a deep learning fish school detection, tracking, and behavior recognition model and train it.

[0089] Test the performance of the fish school detection, tracking, and behavior recognition model, and adjust the training samples, annotation methods, or add new training samples according to the test results;

[0090] Iteratively optimize and train the fish school detection, tracking, and behavior recognition model until the accuracy meets the expected goal.

[0091] Furthermore, the fish school recognition model includes: a fish school detection sub-model, a fish school tracking sub-model, and a fish school behavior recognition sub-model;

[0092] The fish school detection sub-model is used to detect the coordinate position data of the fish school in the image;

[0093] The fish school tracking sub-model is used to track the movement trajectory of the fish school based on the coordinate position data of the fish school;

[0094] The fish school behavior recognition sub-model is used to recognize the behavior of the fish school.

[0095] Furthermore, the recognition and tracking of the fish school include:

[0096] Obtain image data, preprocess the image data, and obtain the preprocessed image data;

[0097] Use the fish school detection sub-model to detect the preprocessed image data and obtain the coordinate position data of the fish school on the image;

[0098] According to the coordinate position data, use the fish school tracking sub-model to track the fish school and calculate the trajectory of the fish school.

[0099] Specifically, S2 also includes: preprocessing the collected image data to improve the contrast of the image.

[0100] Perform fish school detection model inference on the preprocessed image to calculate the coordinate position of the fish school on the image.

[0101] Based on the fish school coordinate position, further perform fish school tracking model inference on the preprocessed image to calculate the position of the fish school in the next frame of the image.

[0102] Furthermore, preprocessing the image data to obtain the preprocessed image data includes:

[0103] Scale the image data, perform color space conversion on the scaled image data, and obtain the HSV image;

[0104] Extract the V channel from the HSV image, perform median filtering denoising on the V channel, use the adaptive histogram equalization algorithm to perform contrast stretching on the processed V channel, use the Gaussian blurring algorithm to reduce noise on the stretched V channel, and merge the denoised V channel with the original H and S channel images to obtain a new HSV image;

[0105] Convert the new HSV image to the RGB color space to obtain the preprocessed image data.

[0106] Furthermore, fish school behavior recognition includes:

[0107] Organize the preprocessed image data according to the data format required for input to the fish school behavior recognition model to obtain the organized image data;

[0108] Input the organized image data into the fish school behavior recognition sub-model to obtain the fish school behavior recognition result.

[0109] Specifically, S3 also includes: collecting image data of a fixed number of frames according to the model input requirements and performing image enhancement processing on it.

[0110] Input the enhanced sequential image data into the fish school behavior recognition model for inference to obtain the fish school behavior classification result based on the current sequential image data.

[0111] Furthermore, S4 also includes: converting the fish school coordinate data detected and tracked into trajectory data in units of fish IDs.

[0112] Calculate the monitoring indicators of a single fish in units of fish IDs.

[0113] Statistically calculate the monitoring indicators of a single fish for all fish in the fish school and calculate the fish school monitoring indicators.

[0114] Furthermore, S5 also includes: calculating the single-time fish school water quality toxicity warning level through the fish school monitoring indicators of the current sequential image data.

[0115] Set the fish school water quality comprehensive toxicity warning time, and calculate the fish school water quality comprehensive toxicity warning level by analyzing the single-time fish school water quality toxicity warning levels monitored multiple times within a time period.

[0116] Furthermore, calculating the fish school toxicity warning level for a single sequential image data includes:

[0117] Calculate the fish school toxicity warning level according to the fish school comprehensive speed index to obtain the first toxicity warning level;

[0118] If the fish school comprehensive speed index does not meet the toxicity warning conditions, then calculate the fish school toxicity warning level according to the fish school comprehensive distance index to obtain the second toxicity warning level;

[0119] If both the comprehensive speed index and the comprehensive distance index of the fish school do not meet the toxicity warning conditions, then calculate the toxicity warning level of the fish school based on the single - item monitoring index of each single fish, and obtain the third toxicity warning level;

[0120] Use the fish school behavior recognition result to correct the first toxicity warning level, the second toxicity warning level, and the third toxicity warning level, and obtain the toxicity warning level of the fish school under a single - sequence image data.

[0121] The present invention also provides a fish school toxicity behavior analysis system based on deep learning, including: an image data acquisition module, a fish school recognition module, and a warning level module;

[0122] The image data acquisition module is used to acquire the fish school image data to be analyzed;

[0123] The fish school recognition module is used to calculate the monitoring index of the fish school by using the fish school detection, tracking, and behavior recognition model;

[0124] The warning level module is used to calculate the warning level of the comprehensive toxicity of the fish school according to the monitoring index of the fish school.

[0125] The following elaborates on this embodiment in conjunction with the accompanying drawings:

[0126] This embodiment proposes a fish school toxicity behavior analysis method based on deep learning, including the following steps:

[0127] ⑴ Off - line modeling: Train a fish detection, tracking, and behavior recognition model based on deep learning.

[0128] ⑵ Fish school detection and tracking: Collect fish school video data, and perform detection and tracking processing on the fish school in the image.

[0129] ⑶ Fish school behavior recognition: Intercept the image data of fixed frames from the video, and perform fish school behavior recognition.

[0130] ⑷ Calculation of fish school monitoring index: Calculate the fish school monitoring index according to the fish school detection, tracking, and behavior recognition data.

[0131] ⑸ Fish school toxicity warning: Calculate the comprehensive toxicity warning level of the fish school according to the fish school monitoring index.

[0132] As Figure 2 shown, the off - line modeling process includes the following content:

[0133] ⑴ Collect and annotate the video data containing the fish school, perform frame extraction on it, and form the original training sample data.

[0134] ⑵ Perform automated data augmentation processing on the original training sample data to form an augmented data set.

[0135] (3) The fish populations used for water quality toxicity early warning are of the same species. Preferably, in this embodiment, the yolov10n model is used to train the fish population detection model, which improves the algorithm operation efficiency while ensuring the detection accuracy.

[0136] (4) Zebrafish are model organisms with a relatively fast swimming speed. In this embodiment, zebrafish are used for the water quality toxicity early warning and identification task. Therefore, preferably, the deepsort model is used to train the fish population tracking model.

[0137] (5) Through toxicity tests on zebrafish and collection and analysis of relevant video data, it is found that the types of behavioral patterns presented by zebrafish under different toxicity conditions are relatively few. Therefore, preferably, the PP-TSMv2 lightweight model is used to train the fish population behavior recognition model in this embodiment.

[0138] (6) Test the effects of the fish population detection, tracking, and behavior recognition models, and adjust the training samples, annotation methods, or add new training samples according to the test results.

[0139] (7) Iteratively optimize and train the fish population detection, tracking, and behavior recognition models until the accuracy meets the expected goal.

[0140] As Figure 3 shown, the fish population detection and tracking process includes the following:

[0141] (1) By setting parameters, a fixed number of image data can be collected. By default, 64 frames of sequential image data are collected.

[0142] (2) Perform image preprocessing operations on the collected sequential image data to improve the image contrast.

[0143] (3) Perform fish population detection model inference on the preprocessed sequential image data to detect the coordinate positions of the fish population in the image.

[0144] (4) Input the detected fish population coordinate positions, perform fish population tracking model inference on the preprocessed image, and calculate the positions of the fish population in the next frame image and the IDs of each fish in the fish population.

[0145] (5) Construct a vector <detectbox>Variables of fishBox, where DetectBox is a structure that includes variables x1, y1, x2, y2, confidence, and trackID. Among them, x1 and y1 represent the x-coordinate and y-coordinate of the upper-leftmost corner of the detected fish respectively; x2 and y2 represent the x-coordinate and y-coordinate of the lower-rightmost corner of the detected fish respectively; confidence is the confidence level of the detected fish, and trackID is the ID number for fish tracking. The data of the fish school detected and tracked on the preprocessed sequential image data is saved in the fishBox variable.

[0146] As Figure 4 shown, the image preprocessing process includes the following:

[0147] ⑴ First, perform scaling processing on the RGB image to improve the overall operation efficiency.

[0148] ⑵ Convert the scaled RGB image to the HSV color space to obtain the HSV image.

[0149] ⑶ Extract the V channel of the HSV image and perform median filtering denoising on it.

[0150] ⑷ Use the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm to perform contrast stretching on the V channel image after median filtering denoising.

[0151] ⑸ There will be more noise points in the image data after stretching by the CLAHE algorithm, and it is necessary to further reduce noise through the Gaussian blurring algorithm.

[0152] ⑹ Combine the V channel image after Gaussian blurring denoising with the H and S channels in the original HSV channels to obtain the new HSV_new image.

[0153] ⑺ Convert the HSV_new image to the RGB color space to obtain the RGB_new image.

[0154] ⑻ Perform the operations in steps (1) to (7) on each frame of the sequential image to obtain the preprocessed sequential image data.

[0155] As Figure 5 shown, the fish school behavior recognition process includes the following:

[0156] ⑴ Organize the preprocessed sequential image data into the input format of the PP-TSMv2 model.

[0157] ⑵ Infer the PP-TSMv2 model to obtain the result of behavior classification.

[0158] ⑶ Use the model classification result with the highest confidence as the result of fish school behavior recognition.

[0159] ⑷Construct a variable activityData of the fishActivityData structure, which includes two variables: confidence and type. Among them, confidence represents the confidence level of fish behavior recognition; type represents the category number of fish behavior. Save the result of fish behavior recognition in step (3) into activityData.

[0160] As Figure 6 shown, the fish school detection index calculation process includes the following:

[0161] ⑴Traverse the trackID of the fish in fishBox. Using trackID as the index, store the fish detection data with the same trackID together to form fish trajectory data in units of trackID.

[0162] ⑵Calculate monitoring indexes such as the minimum swimming speed, maximum swimming speed, average swimming speed, and swimming distance of fish with the same trackID.

[0163] ⑶Perform statistical analysis on the monitoring indexes of fish with different trackIDs to obtain monitoring indexes such as the minimum swimming speed, maximum swimming speed, average swimming speed, minimum swimming distance, maximum swimming distance, and average swimming distance of the fish school under the current sequence image data.

[0164] As Figure 7 shown, the fish school data sorting process includes the following:

[0165] ⑴Construct a map<int,vector <detectbox>>fishBoxData variable, traverses the trackID in fishBox, uses trackID as index, stores the DetectBox fish detection data with the same trackID together, and obtains the fish track data fishBoxData in units of trackID.

[0166] ⑵ Count the number of fish tracks with the same trackID in fishBoxData, and get the tracking times of the fish with this trackID.

[0167] ⑶ Since zebrafish swim very fast, there will be a problem of tracking loss when the fish school swims quickly and crosses each other. In order to reduce system errors, when counting fish monitoring indicators, it is necessary to eliminate fish detection data with fewer tracking times. Preferably, when the number of sequential image frames is 64 frames, fish detection data with tracking times less than 8 frames will be eliminated.

[0168] like Figure 8 As shown in the figure, the calculation process of single fish monitoring indicators includes the following:

[0169] ⑴Build a vector <fishdetectordata>The fishData variable, where FishDetectorDATA is a structure that includes variables m_minSpeed, m_maxSpeed, m_meanSpeed, and m_distance, representing the minimum swimming speed, maximum swimming speed, average swimming speed, and swimming distance of fish, respectively.

[0170] ⑵ Suppose box1 and box2 are the coordinates of the fish trajectories in adjacent frames with the same index in fishBoxData. First, calculate the center point coordinates (cx1, cy1) of box1, where cx1 = x1 + (x2 - x1) / 2 and cy1 = y1 + (y2 - y1) / 2. Calculate the center point coordinates (cx2, cy2) of box2 in the same way. For simplicity of calculation, take the pixel distance dist between the two center point coordinates (cx1, cy1) and (cx2, cy2) as the swimming speed and swimming distance of the fish in adjacent frames.

[0171] ⑶ The monitoring index structure for a single fish is FishDetectorDATA. Calculate the pixel distance dist of all adjacent frame fish trajectories with the same fish index according to step (2), find the minimum value and assign it to the m_minSpeed variable, and the maximum value and assign it to the m_maxSpeed variable. Then accumulate all the pixel distances dist of adjacent frames to get the total swimming distance of a single fish and assign it to the m_distance variable. Suppose the pixel distance dist of the adjacent frame fish trajectories of this fish is calculated N times. Finally, divide m_distance by N to get the average swimming distance of this fish and assign it to the m_meanSpeed variable. In this way, the monitoring indexes of a single fish can be calculated.

[0172] ⑷ Traverse fishBoxData, and for each fish index, calculate their individual monitoring indexes according to step (3) and save them in fishData.

[0173] As Figure 9 shown, the calculation process of the fish school monitoring indexes includes the following:

[0174] ⑴ Construct a FishToxicityDATA structure variable fishTotalData, which includes variables m_minSpeed, m_maxSpeed, m_meanSpeed, m_minDistance, m_maxDistance, and m_meanDistance, representing the minimum swimming speed, maximum swimming speed, average swimming speed, minimum swimming distance, maximum swimming distance, and average swimming distance of all fish in the fish school, respectively.

[0175] ⑵ Traverse the individual monitoring indicators of all fish in fishData, find the minimum swimming speed of all fish, and assign it to the m_minSpeed variable of fishTotalData; find the maximum swimming speed of all fish, and assign it to the m_maxSpeed variable of fishTotalData; find the minimum swimming distance of all fish, and assign it to the m_minDistance variable of fishTotalData; find the maximum swimming distance of all fish, and assign it to the m_maxDistance variable of fishTotalData; calculate the average swimming speed of all fish, and assign it to the m_meanSpeed variable of fishTotalData; calculate the average swimming distance of all fish, and assign it to the m_meanDistance variable of fishTotalData.

[0176] As Figure 10 shown, the fish population toxicity warning process includes the following:

[0177] ⑴ Single fish population toxicity warning: Calculate the fish population toxicity warning level under single sequence image data.

[0178] ⑵ Comprehensive fish population toxicity warning: Calculate the comprehensive fish population toxicity warning level under multiple sequence image data.

[0179] As Figure 11 shown, the single fish population toxicity warning process includes the following:

[0180] ⑴ Comprehensive fish population indicator toxicity warning: Analyze fishTotalData to calculate the toxicity warning level of fish population monitoring indicators.

[0181] ⑵ Single fish population indicator toxicity warning: Analyze the swimming speed indicator of fish in fishData to calculate the toxicity warning level.

[0182] ⑶ Fish population behavior recognition warning correction: Correct the fish population toxicity warning levels calculated in steps (1) and (2) based on the fish population behavior recognition results in activityData.

[0183] As Figure 12 shown, the process for calculating the fish population comprehensive speed indicator toxicity warning level includes the following:

[0184] ⑴ Analyze the speed indicators in the fishTotalData fish population monitoring indicators, including the m_minSpeed, m_maxSpeed, and m_meanSpeed variables. If the minimum swimming speed m_minSpeed or the average swimming speed m_meanSpeed of the fish in the fish population is greater than the set threshold 1. Preferably, this threshold is set to 12 and can be adjusted as needed. Then, it is considered that the overall swimming speed of the fish in the fish population is relatively fast, which may be due to interference from the external environment or trace toxins entering the water body, causing a stress response in the fish population. Therefore, the calculated fish population toxicity warning level is yellow, indicating a relatively low toxicity warning.

[0185] ⑵ In the case where the yellow warning for fish population toxicity is not met, if the maximum swimming speed m_maxSpeed or the average swimming speed m_meanSpeed of the fish in the fish population is less than the set threshold 2. Preferably, this threshold is set to 1.2 and can be adjusted as needed. Then, it is considered that the fish in the fish population hardly swim, and there should be a large amount of toxins in the water body, resulting in the overall dying or dead state of the fish in the fish population. Therefore, the calculated fish population toxicity warning level is red, indicating a relatively high toxicity warning.

[0186] ⑶ In the case where the yellow and red warnings for fish population toxicity are not met, if the maximum swimming speed m_maxSpeed or the average swimming speed m_meanSpeed of the fish in the fish population is less than the set threshold 3. Preferably, this threshold is set to 5 and can be adjusted as needed. Then, it is considered that the overall swimming speed of the fish in the fish population is slow, and there should be relatively many toxins in the water body, resulting in an obvious poisoning phenomenon in the overall fish in the fish population and an obvious decrease in swimming ability. Therefore, the calculated fish population toxicity warning level is orange, indicating a medium toxicity warning.

[0187] ⑷ If steps (1), (2), and (3) are not met, then the fish population comprehensive distance indicator will be further analyzed.

[0188] As Figure 13 shown, the calculation process of the fish population comprehensive distance indicator toxicity warning level includes the following:

[0189] ⑴ Analyze the distance indicators in the fishTotalData fish population monitoring indicators, including the m_minDistance, m_maxDistance, and m_meanDistance variables. If the minimum swimming distance m_minDistance or the average swimming distance m_meanDistance of the fish in the fish population is greater than the set threshold of 4, preferably, this threshold is set to 190 and can be adjusted as needed. Then, it is considered that the overall swimming speed of the fish in the fish population is relatively fast, which may be due to interference from the external environment or trace toxins entering the water body, causing a stress response in the fish population. Therefore, the calculated fish population toxicity warning level is yellow, indicating a relatively low toxicity warning.

[0190] ⑵ In the case where the yellow warning for fish population toxicity is not met, if the maximum swimming distance m_maxDistance or the average swimming distance m_meanDistance of the fish in the fish population is less than the set threshold of 5, preferably, this threshold is set to 30 and can be adjusted as needed. Then, it is considered that the fish in the fish population hardly swim, and there should be a large amount of toxins in the water body, resulting in the overall dying or dead state of the fish in the fish population. Therefore, the calculated fish population toxicity warning level is red, indicating a relatively high toxicity warning.

[0191] ⑶ In the case where the yellow and red warnings for fish population toxicity are not met, if the maximum swimming distance m_maxDistance or the average swimming distance m_meanDistance of the fish in the fish population is less than the set threshold of 6, preferably, this threshold is set to 70 and can be adjusted as needed. Then, it is considered that the overall swimming of the fish in the fish population is slow, and there should be more toxins in the water body, resulting in an obvious poisoning phenomenon in the overall fish in the fish population and an obvious decrease in swimming ability. Therefore, the calculated fish population toxicity warning level is orange, indicating a medium toxicity warning.

[0192] ⑷ If steps (1), (2), and (3) are not met, then the single-item indicators of the fish population will be further analyzed.

[0193] As Figure 14 shown, the statistical process of the single-item speed warning count of the fish population includes the following:

[0194] ⑴ Assume that fish_data is the monitoring indicator of a certain fish in fishData, analyze the minimum swimming speed m_minSpeed, the maximum swimming speed m_maxSpeed, and the average swimming speed m_meanSpeed of fish_data, and judge whether the swimming speed indicator of this fish meets the toxicity warning conditions.

[0195] ⑵ Set the yellow warning counter variable YCount, red warning counter variable RCount, orange warning counter variable OCount, and normal counter variable NCount respectively. If a certain fish meets the corresponding warning conditions, the corresponding counter will increment by 1.

[0196] ⑶ If the minimum swimming speed m_minSpeed or average swimming speed m_meanSpeed of fish_data is greater than the set threshold 1. Preferably, this threshold is set to 12 and can be adjusted as needed. Then, it is considered that the swimming speed of this fish is relatively fast, which may be due to interference from the external environment or trace toxins entering the water body, causing its stress response. Therefore, the YCount counter increments by 1.

[0197] ⑷ Under the condition that the speed index in fish_data does not meet the conditions for yellow toxicity warning, if the maximum swimming speed m_maxSpeed or average swimming speed m_meanSpeed of this fish is less than the set threshold 2. Preferably, this threshold is set to 1.2 and can be adjusted as needed. Then, it is considered that this fish hardly swims, and there should be a large amount of toxins in the water body, resulting in its near-death or dead state. Therefore, the RCount counter increments by 1.

[0198] ⑸ Under the condition that the speed index in fish_data does not meet the conditions for yellow and red toxicity warnings, if the maximum swimming speed m_maxSpeed or average swimming speed m_meanSpeed of this fish is less than the set threshold 3. Preferably, this threshold is set to 5 and can be adjusted as needed. Then, it is considered that the swimming speed of this fish is slow, and there should be more toxins in the water body, resulting in an obvious poisoning phenomenon and a significant decrease in its swimming ability. Therefore, the OCount counter increments by 1.

[0199] ⑹ If the speed index in fish_data does not meet the conditions for yellow, red, and orange toxicity warnings, it means that this fish has no obvious toxic reaction, and the NCount counter will increment by 1.

[0200] ⑺ Traverse and analyze the speed indexes of all fish in fishData, and count the toxicity warnings of all fish according to steps (3) to (6) respectively.

[0201] As Figure 15 shown, the single-speed warning calculation process for the fish population includes the following:

[0202] ⑴ Count the values of all fish warning counters. If the value of the red warning counter is greater than the set threshold, preferably, the threshold is 10 and can be adjusted according to the situation. Then it is considered that most fish in the fish school hardly swim, and there should be a large amount of toxins in the water body, causing them to be in a dying or dead state. Therefore, the calculated fish school toxicity warning level is red, indicating a high toxicity warning.

[0203] ⑵ In the case where the red warning of fish school toxicity is not met, if the value of the orange warning counter is greater than the set threshold, preferably, the threshold is 10 and can be adjusted according to the situation. Then, it is considered that most fish in the fish school swim slowly, and there should be more toxins in the water body, resulting in an obvious poisoning phenomenon in the overall fish in the fish school and an obvious decrease in swimming ability. Therefore, the calculated fish school toxicity warning level is orange, indicating a medium toxicity warning.

[0204] ⑶ In the case where the red and orange warnings of fish school toxicity are not met, if the value of the yellow warning counter is greater than the set threshold, preferably, the threshold is 10 and can be adjusted according to the situation. Then, it is considered that most fish in the fish school swim fast, which may be due to interference from the external environment or trace amounts of toxins entering the water body, causing a stress response in the fish school. Therefore, the calculated fish school toxicity warning level is yellow, indicating a low toxicity warning.

[0205] ⑷ If steps (1), (2), and (3) are not met, then it is considered that most fish in the fish school do not show obvious toxicity reactions. Therefore, the calculated fish school toxicity warning level is green, indicating that no obvious toxicity reaction of the fish school in the water body is detected.

[0206] As Figure 16 shown, the fish school behavior recognition warning correction process includes the following:

[0207] ⑴ Count the values of all fish warning counters. If the value of the red warning counter is greater than the set threshold, preferably, the threshold is 10 and can be adjusted according to the situation. Then it is considered that most fish in the fish school hardly swim, and there should be a large amount of toxins in the water body, causing them to be in a dying or dead state. Therefore, the calculated fish school toxicity warning level is red, indicating a high toxicity warning.

[0208] ⑵ In the case where the red warning of fish school toxicity is not met, if the value of the orange warning counter is greater than the set threshold, preferably, the threshold is 10 and can be adjusted according to the situation. Then, it is considered that most fish in the fish school swim slowly, and there should be more toxins in the water body, resulting in an obvious poisoning phenomenon in the overall fish in the fish school and an obvious decrease in swimming ability. Therefore, the calculated fish school toxicity warning level is orange, indicating a medium toxicity warning.

[0209] (3) When the red and orange warnings for fish group toxicity are not met, if the value of the yellow warning counter is greater than the set threshold, preferably, the threshold is 10 and can be adjusted according to the situation. Then, it is considered that the swimming speed of most fish in the fish group is relatively fast, which may be due to interference from the external environment or trace toxins entering the water body, causing a stress response in the fish group. Therefore, the calculated fish group toxicity warning level is yellow, indicating a lower toxicity warning.

[0210] (4) If steps (1), (2), and (3) are not met, then it is considered that most fish in the fish group do not show obvious toxicity reactions. Therefore, the calculated fish group toxicity warning level is green, indicating that no obvious toxicity reaction of the fish group in the water body is detected.

[0211] As Figure 17 shown, the calculation process of the fish group behavior recognition toxicity warning level includes the following:

[0212] (1) Analyze the confidence variable and type variable in activityData. If type = 4, it means that the recognition result of the fish group behavior recognition model is the fish group death behavior. Further analyze the value of confidence. If the confidence value is greater than the set threshold, preferably, the threshold is set to 0.8, then it is considered that most fish in the fish group hardly swim, and there should be a large amount of toxins in the water body, causing them to show a dying or dead state. Therefore, the calculated fish group toxicity warning level is red, indicating a higher toxicity warning.

[0213] (2) When the red warning for fish group toxicity is not met, analyze the confidence variable and type variable in activityData. If type = 3, it means that the recognition result of the fish group behavior recognition model is the fish group slow swimming behavior. Further analyze the value of confidence. If the confidence value is greater than the set threshold, preferably, the threshold is set to 0.8, then it is considered that the overall swimming speed of the fish in the fish group is slow, and there should be more toxins in the water body, causing an obvious poisoning phenomenon in the overall fish in the fish group and an obvious decrease in swimming ability. Therefore, the calculated fish group toxicity warning level is orange, indicating a medium toxicity warning.

[0214] ⑶ When the red and orange warnings for fish population toxicity are not met, analyze the confidence variable and type variable in activityData. If type = 2, it indicates that the recognition result of the fish population behavior recognition model is the fast swimming behavior of the fish population. Further analyze the value of confidence. If the confidence value is greater than the set threshold, preferably, the threshold is set to 0.8. Then, it is considered that the swimming speed of the fish in the fish population is relatively fast, which may be due to the interference of the external environment or the entry of trace toxins into the water body, causing a stress response in the fish population. Therefore, the calculated fish population toxicity warning level is yellow, indicating a lower toxicity warning.

[0215] ⑷ When the red, orange, and yellow warnings for fish population toxicity are not met, analyze the confidence variable and type variable in activityData. If type = 1, it indicates that the recognition result of the fish population behavior recognition model is the normal swimming behavior of the fish population. Further analyze the value of confidence. If the confidence value is greater than the set threshold, preferably, the threshold is set to 0.8. Then, it is considered that the swimming behavior of the fish in the fish population is normal and no obvious toxicity reaction is presented. Therefore, the calculated fish population toxicity warning level is green, indicating that no obvious toxicity reaction of the fish population in the water body is detected.

[0216] ⑸ If steps (1), (2), (3), and (4) are not met, then it is considered that the fish population behavior recognition model does not recognize obvious fish population behavior. Therefore, the calculated fish population toxicity warning level is gray, indicating that the current fish population behavior recognition result is not credible.

[0217] As Figure 18 shown, the comprehensive fish population toxicity warning process includes the following:

[0218] ⑴ The behavior of the fish population has a certain degree of randomness. To improve the accuracy of fish population toxicity warning, in this embodiment, the multiple single fish population toxicity warning levels within a period of time will be continuously observed, and the data of multiple fish population toxicity warning levels will be statistically analyzed.

[0219] ⑵ Through the statistically analyzed data of multiple fish population toxicity warning levels, calculate the comprehensive fish population toxicity warning level within a period of time to obtain a more accurate warning result.

[0220] As Figure 19 shown, the schematic diagram of the comprehensive fish population toxicity warning detection method, which includes the following:

[0221] ⑴ The time for the comprehensive toxicity warning of the fish school is T. That is, every T minutes, the fish school toxicity behavior analysis system will calculate a comprehensive toxicity warning level. This time T can be set. Preferably, it is set to 10, which means 10 minutes. Setting this time too short will affect the accuracy of calculating the comprehensive toxicity warning level; on the contrary, it will reduce the timeliness of the comprehensive toxicity warning. When a water body toxicity pollution event occurs, it is not conducive to detecting problems early.

[0222] ⑵ The time for a single fish school toxicity warning analysis is t1, with the unit of seconds. This time can be set. Preferably, it is set to 2, which means 2 seconds. Since the swimming speed of zebrafish is relatively fast, when analyzing the fish school toxicity warning level within the t1 time period, in this embodiment, the sequence image data is collected at a rate of 30 frames per second. When t1 is set to 2 seconds, the sequence images collected for a single fish school toxicity warning level analysis are 60 frames. Setting t1 too short will affect the accuracy of calculating the comprehensive toxicity warning level; on the contrary, although it will increase the calculation accuracy of the toxicity warning level, it will also greatly increase the computational load of the system. Through multiple experimental statistics, when t1 is set to 2 seconds to 5 seconds, there is no obvious change in the output results.

[0223] ⑶ This embodiment hopes that the warning result of the fish school toxicity behavior warning system is as accurate as possible, while minimizing the computational load of the system. Assuming that there is toxicity in the water body, then within a relatively short time period T, the toxicity concentration of the water body changes little, and the toxicity behaviors presented by the fish school are generally similar. Therefore, on the premise of ensuring the accuracy of the warning result of the fish school toxicity behavior warning system, it is not necessary to continuously analyze all the sequence image data within the T time period. The interval time t2 can be set to achieve this function. t2 is the interval time, with the unit of seconds, that is, the interval time between two fish school toxicity behavior warning analyses within the T time period. Preferably, it is set to 10, which means an interval of 10 seconds. Through the above method, every t1 + t2, a total of 12 seconds, the fish school toxicity behavior analysis system will calculate a toxicity warning level. Within the time period of T = 10 minutes, a total of 50 fish school toxicity warning levels are calculated. By statistically analyzing these 50 fish school toxicity warning levels, the computational load of the system can be greatly reduced while ensuring the toxicity warning accuracy of the system.

[0224] As Figure 20 shown, the statistical process of a single fish school toxicity warning count includes the following:

[0225] ⑴ Construct a vector <int>The alarmLevel variable stores the data of the fish population toxicity warning levels multiple times within the T time period in the alarmLevel variable.

[0226] ⑵ Respectively set the yellow warning counter variable YCount, the red warning counter variable RCount, the orange warning counter variable OCount, the green warning counter variable GCount, and the grey warning counter variable GreyCount. If the fish population toxicity warning level of a certain time meets the corresponding warning condition, then the corresponding counter will increment by 1.

[0227] ⑶ Traverse and analyze the elements in alarmLevel, and each element corresponds to a fish population toxicity warning level. If the toxicity warning level of the current element is red, then the RCount counter increments by 1.

[0228] ⑷ Traverse and analyze the elements in alarmLevel, and each element corresponds to a fish population toxicity warning level. If the toxicity warning level of the current element is orange, then the OCount counter increments by 1.

[0229] ⑸ Traverse and analyze the elements in alarmLevel, and each element corresponds to a fish population toxicity warning level. If the toxicity warning level of the current element is yellow, then the YCount counter increments by 1.

[0230] ⑹ Traverse and analyze the elements in alarmLevel, and each element corresponds to a fish population toxicity warning level. If the toxicity warning level of the current element is green, then the GCount counter increments by 1.

[0231] ⑺ Traverse and analyze the elements in alarmLevel, and each element corresponds to a fish population toxicity warning level. If the toxicity warning level of the current element is not red, orange, yellow, or green, then the GreyCount counter increments by 1.

[0232] As Figure 21 shown, the calculation process of the comprehensive fish population toxicity warning includes the following:

[0233] ⑴ Statistically analyze the values of all warning counters. If the value of the red warning counter RCount is greater than the set threshold, preferably, this threshold is 25 and can be adjusted according to the situation. Then it is considered that within the T time period, the comprehensive fish population toxicity warning level is red, indicating a relatively high toxicity warning.

[0234] ⑵ In the case where the comprehensive fish population toxicity red warning is not met, if the value of the orange warning counter OCount is greater than the set threshold, preferably, this threshold is 25 and can be adjusted according to the situation. Then it is considered that within the T time period, the comprehensive fish population toxicity warning level is orange, indicating a medium toxicity warning.

[0235] (3) When the combined toxicity of fish population does not meet the red and orange warnings, if the value of the yellow warning counter YCount is greater than the set threshold, preferably, the threshold is 25 and can be adjusted according to the situation. Then it is considered that within the T time period, the combined toxicity warning level of the fish population is yellow, indicating a lower toxicity warning.

[0236] (4) When the combined toxicity of fish population does not meet the red, orange and yellow warnings, if the value of the green warning counter GCount is greater than the set threshold, preferably, the threshold is 25 and can be adjusted according to the situation. Then it is considered that within the T time period, the combined toxicity warning level of the fish population is green, indicating that no obvious toxic reaction of the fish population in the water body is detected.

[0237] (5) If steps (1), (2), (3) and (4) are not satisfied, then it is considered that within the T time period, the combined toxicity warning level of the fish population is gray, indicating that no significant behavior pattern of the fish population is monitored and it is not used as a warning basis.

[0238] The above is only a preferred specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.< / int> < / fishdetectordata> < / detectbox> < / detectbox>

Claims

1. A method for analyzing fish toxicity behavior based on deep learning, characterized in that: include: Acquire image data to be analyzed; Inputting the image data to be analyzed into a fish school recognition model to obtain fish school monitoring indicators, wherein the fish school recognition model is obtained by training a training set, and the training set is labeled image data containing fish schools; According to the fish monitoring indicators, the early warning level of the comprehensive toxicity of the fish is obtained.

2. A method for analyzing fish toxicity behavior based on deep learning according to claim 1, characterized in that: Acquiring the training set includes: Obtain video data labeled with a school of fish, perform frame extraction on the video data, and obtain original training sample data; Performing data augmentation processing on the original training sample data to obtain an augmented data set; The original training sample data and the augmented data set are combined to obtain the training set.

3. The method for analyzing fish toxicity behavior based on deep learning according to claim 1, characterized in that: The fish school recognition model includes: a fish school detection sub-model, a fish school tracking sub-model and a fish school behavior recognition sub-model; The fish school detection sub-model is used to detect the coordinate position data of the fish school in the image; The fish school tracking sub-model is used to track the movement trajectory of the fish school according to the coordinate position data of the fish school; The fish school behavior identification sub-model is used for performing fish school behavior identification.

4. The method for analyzing fish toxicity behavior based on deep learning according to claim 3, characterized in that: Detecting the fish school includes: Acquiring image data, preprocessing the image data, and acquiring preprocessed image data; The fish school detection sub-model is used to detect the preprocessed image data to obtain the coordinate position data of the fish school on the image.

5. The method for analyzing fish toxicity behavior based on deep learning according to claim 4, characterized in that: Preprocessing the image data to obtain the preprocessed image data includes: Scaling the image data, and performing color space conversion on the scaled image data to obtain an HSV image; Extract the V channel in the HSV image, perform median filtering denoising on the V channel, use an adaptive histogram equalization algorithm to perform contrast stretching on the processed V channel, use a Gaussian blur algorithm to reduce noise on the stretched V channel, merge the denoised V channel with the original H and S channel images, and obtain a new HSV image; The new HSV image is converted into RGB color space to obtain preprocessed image data.

6. The method for analyzing fish toxic behavior based on deep learning according to claim 4, characterized in that: The fish school behavior identification includes: Arrange the preprocessed image data according to the data format input by the fish school behavior recognition model to obtain the arranged image data; The sorted image data is input into the fish school behavior recognition sub-model to obtain the fish school behavior recognition result.

7. The method for analyzing fish toxicity behavior based on deep learning according to claim 6, characterized in that: Obtaining the fish school monitoring index includes: Convert the detected and tracked fish school coordinate position data into trajectory data in units of fish ID; Calculating monitoring indicators of a single fish according to the trajectory data; The fish school monitoring indicators are obtained according to the monitoring indicators of the single fish, wherein the fish school monitoring indicators include: minimum swimming speed, maximum swimming speed, average swimming speed, maximum swimming distance, minimum swimming distance and average swimming distance.

8. The method for analyzing fish toxicity behavior based on deep learning according to claim 7, characterized in that: According to the fish monitoring indicators, the early warning levels of the comprehensive toxicity of fish are obtained, including: Calculate the fish toxicity warning level based on a single sequence of image data; The toxicity warning levels of the fish school under the plurality of single sequence images are integrated to obtain the comprehensive toxicity warning level of the fish school.

9. The method for analyzing fish toxicity behavior based on deep learning according to claim 8, characterized in that: Calculating the fish toxicity warning level under a single sequence of image data includes: Calculate the fish school toxicity warning level according to the comprehensive speed index of the fish school to obtain the first toxicity warning level; If the fish school comprehensive speed index does not meet the toxicity warning condition, the fish school toxicity warning level is calculated according to the fish school comprehensive distance index to obtain a second toxicity warning level; If the comprehensive speed index and the comprehensive distance index of the fish school do not meet the toxicity warning conditions, the toxicity warning level of the fish school is calculated according to the single monitoring index of a single fish to obtain the third toxicity warning level; The first toxicity warning level, the second toxicity warning level and the third toxicity warning level are corrected using the fish school behavior recognition result to obtain the fish school toxicity warning level under the single sequence image data.

10. A fish toxicity behavior analysis system based on deep learning, characterized in that: include: Image data acquisition module, fish school identification module and warning level module; The image data acquisition module is used to acquire the fish school image data to be analyzed; The fish school identification module is used to calculate the monitoring index of the fish school by using the fish school detection, tracking and behavior identification model; The warning level module is used to calculate the warning level of the comprehensive toxicity of the fish school based on the monitoring indicators of the fish school.

Images