A Deep Learning-Based System and Method for Detecting and Counting Dead Fish in Low-Light Environments

Abstract

This invention discloses a deep learning-based system and method for identifying and counting dead fish under low-light conditions. Addressing the issue of insufficient lighting in fish farms at night, the system utilizes drones to capture real-time images of the ponds and improves the YOLOv8 target detection model to adapt to dead fish identification scenarios in low-light aquaculture environments. Specifically, the system includes the following steps: S1: Using a drone, a dataset of dead fish under low-light conditions is collected and divided into training, validation, and test sets; S2: The dataset is input into the improved YOLOv8 model to train a target detection model for dead fish under low-light conditions; S3: At night, the drone is used to capture real-time images of the pond surface, identifying dead fish on the surface. The identification results can be used to outline the location of dead fish in real-time within the detection system. This invention can quickly detect dead fish at night and issue warnings, prompting manual water body inspections to prevent the number of dead fish from increasing or even polluting the water, thus reducing economic losses.

Description

Technical Field

[0001] This invention belongs to the field of image recognition, specifically relating to a system and method for identifying and counting dead fish in low-brightness environments based on deep learning. Background Technology

[0002] In the absence of light at night, the cessation of photosynthesis by aquatic plants leads to a drop in oxygen levels, easily causing fish to die from oxygen deprivation. Furthermore, the dark environment at night and poor lighting conditions in aquaculture areas often result in images taken by cameras with low brightness and contrast. Additionally, ordinary cameras have a small monitoring area, further hindering the detection of dead fish. Failure to detect and warn of fish deaths will lead to economic losses.

[0003] Traditional methods for identifying dead fish typically rely on manual labor, which is time-consuming and labor-intensive. Limited by human resources and time constraints, continuous monitoring is impossible, especially for large-scale fish farms where labor costs are even higher. With the rapid development of artificial intelligence and deep learning technologies, image target detection technology is increasingly widely used in agriculture. Classic recognition models such as Faster R-CNN and YOLO have emerged and become widely adopted techniques in this field. The paper "A Dead Fish Recognition Method and Early Warning System Based on Deep Convolutional Neural Networks" directly transfers the original Faster R-CNN model to underwater dead fish recognition. The paper "A Fish Farming Monitoring Method and System Based on Image Recognition" directly applies the YOLO model to fish recognition. Both methods directly apply existing models without targeted optimization, resulting in low recognition accuracy.

[0004] The prior art discloses a method, apparatus, device and medium for detecting dead fish in ponds based on deep learning, which uses an improved YOLOv5 dead fish detection model to detect whether there are dead fish in the first image; if so, a second image corresponding to the target area is collected; if not, the first image is re-detected.

[0005] The cited documents only consider using improved target detection algorithms to detect dead fish, without taking into account the impact of the actual nighttime aquaculture environment on target detection accuracy, thus affecting the accuracy of the entire device. Therefore, we propose a deep learning-based dead fish target detection and counting system and method in low-light environments to address the aforementioned problems. Summary of the Invention

[0006] The purpose of this invention is to provide a deep learning-based dead fish identification and counting system in low-light environments, in order to solve the problem mentioned in the background art that there is currently no system on the market for identifying and counting dead fish in low-light environments at night.

[0007] The technical solution of the present invention is as follows:

[0008] A deep learning-based system for identifying and counting dead fish in low-light environments, the system comprising:

[0009] The drone data acquisition module uses drones to continuously film the surface of the aquaculture pond at night, and collects video of dead fish that have already died and floated to the surface.

[0010] The dead fish image dataset module is used to store images collected by drones on aquaculture ponds;

[0011] The image brightness correction module is used to preprocess the acquired image and restore and correct the brightness within the dynamic range of the image to facilitate subsequent target detection.

[0012] The attention mechanism module assigns different weights to different parts of the input, extracts more critical and important information, and enhances the ability to resist interference in recognition.

[0013] The dead fish target detection module uses the YOLOv8 algorithm model as the initial starting point and integrates with the attention mechanism module to select the fish positions with detection boxes, thus completing the fish target detection.

[0014] Preferably, the drone data acquisition module includes a drone, a camera, and a computer. The drone, camera, and computer are connected. The camera collects images of dead fish on the surface of the aquaculture pond by shooting videos. A dead fish detection system in a low-light environment is built using deep learning-based target detection technology.

[0015] Preferably, the dead fish image dataset module includes two major categories of data: dead fish and live fish, and is divided into three datasets: training set, validation set, and test set.

[0016] Preferably, the image brightness correction module uses the Gamma algorithm to perform a non-linear operation on the grayscale values ​​of the input image, so that the grayscale values ​​of the output image are exponentially related to the grayscale values ​​of the input image, thereby correcting the brightness and contrast of the resulting image in a low-brightness environment and improving the accuracy of model recognition.

[0017] Preferably, the attention mechanism module adopts the CBAM attention mechanism, which is a combination of channel attention and spatial attention mechanisms. It can improve the performance and robustness of the model, enhance the expressive power and interpretability of features, improve the accuracy of the model, and accelerate convergence during training.

[0018] Preferably, the basic model framework of the YOLOv8 algorithm includes an input terminal, a backbone network, and a detection head;

[0019] The input terminal is an image with a resolution of 640*640 after image preprocessing;

[0020] The backbone network primarily utilizes two 3x3 convolutional kernels, a C2F module, and an SPP module. The two 3x3 convolutional kernels reduce the resolution by a factor of four, yielding two feature maps with 64 and 128 layers respectively. The C2F module employs three convolutional modules (Conv+BN+SiLU) and three BottleNeck modules. The SPP module transforms feature maps of arbitrary size into feature vectors of fixed size, dividing the input image into 1x1, 2x2, and 4x4 blocks. A total of 1*1 + 2*2 + 4*4 = 21 maximum values ​​are extracted from each block, and these 21 maximum values ​​are concatenated as feature values.

[0021] The detection head adopts a decoupled head structure, separating classification and detection.

[0022] This invention also provides a method for identifying and counting dead fish in low-light environments based on deep learning, comprising the following steps:

[0023] S1: Control the drone to hover above the aquaculture pond and take pictures of the water surface of the pond;

[0024] S2: The video of dead fish in low-light environment captured by the drone is used to generate a set of image database by frame extraction, and the dead fish image data in low-light environment is generated into a table file for storage.

[0025] S3: Construct a dataset of dead fish images under low brightness conditions, label the dead fish in the aquaculture environment with names and select their locations, and then divide it into training set, validation set and test set;

[0026] S4: Add the paths to the labeled training and validation sets to the dataset configuration file VOC.yaml. Set the training parameters of the object detection model in the configuration file default.py. Run the training.py file to train the model until the set number of training epochs is completed. Export the applied object detection model fish_best.pt and then apply it to predict.py to use the test set to verify the accuracy of the object detection model.

[0027] S5: Compare the target detection and recognition results with the actual data to verify the accuracy of the target detection model. If the accuracy is equal to or greater than the preset confidence threshold, then execute step S6; otherwise, proceed to step S2.

[0028] S6: Control the drone to hover above the aquaculture pond, collect real-time video of the pond's surface, and input the video into the trained low-brightness environment dead fish target detection model. If there are dead fish in the video, the model will output the number of dead fish and their location information in the video.

[0029] Preferably, if the requirements for fish target detection are not met, optimization needs to be done in three aspects. First, increase the number of training rounds. Second, collect more datasets of dead fish images in low-light environments. Finally, optimize or replace the target detection model algorithm, modify various convolutional layers in the algorithm, and then find the best model through ablation experiments.

[0030] Preferably, the method uses SIoU as a performance metric. SIoU allows the prediction box to quickly drift to the nearest axis, and then only needs to regress a coordinate X or Y to quickly and accurately detect dead fish floating on the water surface.

[0031] Technical effects of the present invention:

[0032] This invention can identify the location of dead fish in real time within the detection system, enabling rapid detection and early warning of dead fish, prompting manual inspection of the water body, preventing the number of dead fish from continuing to increase, and reducing economic losses.

[0033] This invention can be deployed in fishponds with poor lighting conditions at night. By promptly detecting dead fish, it can prevent the further spread of fish diseases, detect water abnormalities as early as possible, and improve the adult fish rate.

[0034] This invention is also applicable to other fields in agriculture where there is a need for target detection in low-light environments. It can detect other targets in low-light environments, such as thieves, and can promptly detect the theft of agricultural products, provide timely warnings, and achieve the purpose of protecting crops and reducing losses. Attached Figure Description

[0035] The accompanying drawings illustrate various embodiments generally by way of example rather than limitation, and are used, together with the specification and claims, to explain embodiments of the invention. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0036] Figure 1 The illustration shows a flowchart of a method for identifying and counting dead fish in a low-brightness environment based on deep learning, according to an embodiment of the present invention. Detailed Implementation

[0037] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0038] This invention provides a technical solution: see Figure 1 This disclosure provides a deep learning-based system for identifying and counting dead fish in low-light drones, comprising the following steps:

[0039] S1: Control the drone to hover above the aquaculture pond and take pictures of the water surface of the pond;

[0040] S2: The video of dead fish in low-light environment captured by the drone is used to generate a set of image database by frame extraction, and the dead fish image data in low-light environment is generated into a table file for storage.

[0041] S3: Construct a dataset of dead fish images under low brightness conditions, label the dead fish in the aquaculture environment with names and select their locations, and then divide it into training set, validation set and test set;

[0042] S4: Add the paths to the labeled training and validation sets to the dataset configuration file VOC.yaml. Set the training parameters of the object detection model in the configuration file default.py. Run the training.py file to train the model until the set number of training epochs is completed. Export the applied object detection model fish_best.pt and then apply it to predict.py to use the test set to verify the accuracy of the object detection model.

[0043] S5: Compare the target detection and recognition results with the actual data to verify the accuracy of the target detection model. If the accuracy is equal to or greater than the preset confidence threshold, then execute step S6; otherwise, proceed to step S2.

[0044] S6: Control the drone to hover above the aquaculture pond, collect real-time video of the pond's surface, and input the video into the trained low-brightness environment dead fish target detection model. If there are dead fish in the video, the model will output the number of dead fish and their location information in the video.

[0045] Step 3 includes the following steps:

[0046] S3-1 extracts frames from videos of dead fish on the water surface in low-light conditions captured by drones to obtain images of dead fish on the water surface. One image is obtained every frame, and 30 images can be obtained every second.

[0047] S3-2 uses labelimg annotation software to annotate dead fish / live fish images in low-brightness environments after video frame extraction, generating image files and corresponding .txt files;

[0048] S3-3 divides the dataset into training, validation, and test sets in a 7:2:1 ratio;

[0049] The training set consists of images input to the model for training; the validation set consists of images used to validate the confidence of the model in each training round; and the test set consists of images used to test the model after training is complete.

[0050] Step 4 includes the following steps:

[0051] S4-1 Before inputting the data into the object detection neural network, brightness correction is first applied to the data. Then, the training images are input into the YOLOv8 object detection model with the CBAM attention mechanism added. S4-2 After completing the set number of training epochs, the best-performing model, fish_best.ph, is output.

[0052] S4-3 Then run predict.py, input the test set in the dataset into the model fish_best.ph, obtain the detection results, compare the number of dead fish obtained from the target detection results with the number of dead fish manually measured in the image, and calculate the model accuracy;

[0053] If the requirements for fish target detection are not met, optimization needs to be done in three aspects. First, increase the number of training rounds. Second, collect more datasets of dead fish images in low-light environments. Finally, optimize or replace the target detection model algorithm, modify various convolutional layers in the algorithm, and then find the best model through ablation experiments.

[0054] The method of this invention uses SIoU as a performance metric, which has the following advantages: SCYLLA-IoU (SIoU) takes into account the angle between the vectors of expected regression and redefines the angle penalty, which allows the predicted box to quickly drift to the nearest axis, and then only one coordinate (X or Y) needs to be regressed, which effectively reduces the total number of degrees of freedom. This reduces the offset of the detection box, increases recognition accuracy, and can quickly and accurately detect dead fish floating on the water surface.

[0055] This invention also provides a deep learning-based system for identifying and counting dead fish in low-light environments, the system comprising:

[0056] The drone data acquisition module uses drones to continuously film the surface of the aquaculture pond at night, and collects video of dead fish that have already died and floated to the surface.

[0057] The dead fish image dataset module is used to store images collected by drones on aquaculture ponds;

[0058] The image brightness correction module is used to preprocess the acquired image and restore and correct the brightness within the dynamic range of the image to facilitate subsequent target detection.

[0059] The attention mechanism module assigns different weights to different parts of the input, extracts more critical and important information, and enhances the ability to resist interference in recognition.

[0060] The dead fish target detection module uses the YOLOv8 algorithm model as the initial starting point and integrates with the attention mechanism module to select the fish positions with detection boxes, thus completing the fish target detection.

[0061] The drone data acquisition module includes a drone, a camera, and a computer. The drone, camera, and computer are connected. The camera collects images of dead fish on the surface of the aquaculture pond by shooting videos. A dead fish detection system in low-light environments is built using deep learning-based object detection technology.

[0062] The dead fish image dataset module includes two main categories of data: dead fish and live fish, and is further divided into three datasets: training set, validation set, and test set.

[0063] The image brightness correction module uses the Gamma algorithm to perform a non-linear operation on the grayscale values ​​of the input image, so that the grayscale values ​​of the output image are exponentially related to the grayscale values ​​of the input image. This corrects the brightness and contrast of the image obtained by the top-mounted camera in low-light environments, thereby improving the accuracy of model recognition.

[0064] The attention mechanism module adopts the CBAM attention mechanism, which is a combination of channel attention and spatial attention mechanisms. It can improve the performance and robustness of the model, enhance the expressive power and interpretability of features, improve the accuracy of the model, and accelerate convergence during training.

[0065] The basic model framework of the YOLOv8 algorithm includes an input terminal, a backbone network, and a detection head;

[0066] The input terminal is an image with a resolution of 640*640 after image preprocessing;

[0067] The backbone network primarily utilizes two 3x3 convolutional kernels, a C2F module, and an SPP module. The two 3x3 convolutional kernels reduce the resolution by a factor of four, yielding two feature maps with 64 and 128 layers respectively. The C2F module employs three convolutional modules (Conv+BN+SiLU) and three BottleNeck modules. The SPP module transforms feature maps of arbitrary size into feature vectors of fixed size, dividing the input image into 1x1, 2x2, and 4x4 blocks. A total of 1*1 + 2*2 + 4*4 = 21 maximum values ​​are extracted from each block, and these 21 maximum values ​​are concatenated as feature values.

[0068] The detection head adopts a decoupled head structure, separating classification and detection.

[0069] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based dead fish recognition and counting system in a low-light environment, characterized by, The system comprises: The unmanned aerial vehicle data acquisition module uses the unmanned aerial vehicle to cyclically shoot the water surface of the culture pond at night to collect video of the dead fish that have floated up; The dead fish image dataset module is used to store the images collected by the unmanned aerial vehicle on the culture pond; The image brightness correction module is used to pre-process the collected images and correct the brightness in the dynamic range of the images, so as to facilitate subsequent target detection; The attention mechanism module gives different weights to different parts of the input, extracts more critical and important information, and enhances the anti-interference capability in identification; The dead fish target detection module takes the YOLOv8 algorithm model as the initial starting point, fuses with the attention mechanism module, frames the fish position with a detection box, and completes the fish target detection; The CBAM attention mechanism is used in the attention mechanism module, which is a combination of channel attention mechanism and spatial attention mechanism, can improve the performance and robustness of the model, enhance the expression ability and interpretability of the features, improve the precision of the model and speed up the convergence during training; The image brightness correction module uses Gamma algorithm to perform nonlinear operation on the input image gray value, so that the output image gray value is in exponential relationship with the input image gray value, so that the brightness and contrast of the picture in the low brightness environment obtained by the top-mounted camera are corrected to improve the precision of model recognition; The basic model framework of the YOLOv8 algorithm includes an input end, a backbone network and a detection head; The input end is a picture with a resolution of 640*640 after image preprocessing; The backbone network includes two 3*3 convolution kernels, a c2f module and a spp module; the two 3*3 convolution kernels make the input picture resolution drop by four times to obtain two feature maps of 64 and 128 layers respectively; the C2f module uses three convolution modules Conv+BN+SiLU and three BottleNeck; the spp module converts an arbitrary size feature map into a fixed size feature vector; The detection head adopts a decoupling head structure to separate classification and detection.

2. The dead fish recognition and counting system in a low brightness environment based on deep learning according to claim 1, wherein The unmanned aerial vehicle data acquisition module includes an unmanned aerial vehicle, a camera and a computer, the unmanned aerial vehicle, the camera and the computer are connected, the camera collects dead fish image on the water surface of the culture pond by shooting video, and the construction of the dead fish detection system in a low brightness environment is realized by using deep learning target detection technology.

3. The dead fish recognition and counting system in a low brightness environment based on deep learning according to claim 1, wherein The dead fish image dataset module includes two categories of data, dead fish and live fish, and is divided into three data sets, training set, validation set and test set.

4. A deep learning-based method for identifying and counting dead fish in a low-light environment, based on the system of any one of claims 1 to 3, characterized in that, The steps include: S1: control the unmanned aerial vehicle to hover above the culture pond to shoot the water surface of the culture pond; S2: generate a group of image databases by frame extraction from the dead fish video in a low brightness environment shot by the unmanned aerial vehicle, and store the low brightness environment dead fish image data in a table file; S3: Construct a dead fish image dataset in low brightness environment, label the name and position of the dead fish in the aquaculture environment, and then divide it into training set, validation set and test set; S4: Add the training set and validation set paths after labeling to the dataset configuration file VOC.yaml, set the training parameters of the target detection model in the configuration file default.py file, run the train.py file to train the model until the set epoch training round is completed, export the applied target detection model fish_best.pt, and then apply it to the predict.py to verify the accuracy of the target detection model using the test set; S5: Compare the target detection recognition result with the actual data to verify the accuracy of the target detection model, if the accuracy is equal to or greater than the preset confidence threshold, then execute step S6, otherwise jump to step S2; S6: Control the unmanned aerial vehicle to hover above the aquaculture pond, collect real-time video of the water surface of the aquaculture pond, input the water surface video into the trained dead fish target detection model in low brightness environment, if there is a dead fish in the video, the number of dead fish will be output, and the position information of the dead fish in the video will be output.

5. The deep learning-based dead fish recognition and counting method in a low-light environment according to claim 4, characterized in that, If the fish target detection requirement is not met, optimization needs to be done from three aspects, first is to increase the number of training rounds, then to collect more dead fish image dataset in low brightness environment, and finally to optimize or replace the target detection model algorithm, modify various convolution layers in the algorithm, and then find the best model through ablation experiment.

6. The deep learning-based dead fish recognition and counting method in a low-light environment according to claim 5, characterized in that, The method uses SIoU as a performance indicator for measurement, which can make the predicted box quickly drift to the nearest axis, and then only need to regress a coordinate X or Y, which can quickly and accurately detect the dead fish floating on the water surface.

Images