Grouper detection and tracking method and system based on target detection and information fusion

Abstract

The invention belongs to the technical field of grouper detection, and discloses a grouper detection and tracking method and system based on target detection and information fusion, and the method comprises the steps: taking a YOLOv8n target detection algorithm as a basis, achieving the rapid detection and tracking of grouper individuals in a grouper breeding process, and achieving the detection and tracking of the grouper individuals. Environment data in the grouper breeding process are collected through external multiple sensors, and detection and environment information are packaged and uploaded to a cloud end through a cloud communication protocol. According to the invention, the improved YOLOv8n target detection algorithm is utilized to realize rapid classification positioning and tracking of groupers, improvement is carried out on the basis of the original algorithm, the calculation amount of the detection process is reduced, the detection precision is improved, information fusion is carried out through an external sensor, and the detection precision is improved. And the cloud real-time grouper type detection and environment information monitoring functions are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of grouper farming, and particularly relates to an improved YOLOv8n object detection algorithm and an information fusion-based grouper detection and tracking method and system. Background Art

[0002] Fishery is one of the important economic industries globally. With the increasing global demand for high-quality protein, the proportion of protein supply from fishery farming is increasing year by year. In many countries around the world, the output of mariculture is huge. Especially in China, the output of mariculture ranks first in the world and shows an upward trend. In China, grouper is an important economic fish in mariculture with a large output. Pearl grouper is a new variety developed in Hainan, which has a fast growth rate and strong disease resistance and is an important commercial and economic fish in the southeast coast of China. Pearl gentian grouper is hybridized from male giant grouper and female tiger grouper and has highly similar appearance characteristics to tiger grouper. In actual farming, the two are often polycultured, and it is not easy for farmers to distinguish the quantity of the two. On the other hand, due to their different economic values, some merchants sell the pearl gentian grouper with lower economic value as the tiger grouper with higher economic value to deceive consumers in order to seek more profits. Therefore, in order to quickly distinguish pearl gentian grouper and tiger grouper during the fishing process of grouper, and at the same time, for consumers to accurately distinguish them when purchasing grouper, it is necessary to develop a system that can quickly and accurately identify individual groupers.

[0003] The individual sorting of groupers is a process that is extremely important for ecological stability but labor-consuming during the fishing process. Sorting different species helps to protect and maintain the ecological balance of different populations. Reasonable sorting can reduce overfishing of certain species and protect the stability and diversity of the marine ecosystem. Currently, in many developing countries, since the automated fishery farming model has not been widely popularized, the sorting of individual fish mainly relies on manual identification. Manual identification depends on the experience and knowledge of workers to distinguish different species of fish. This method is inefficient, and its accuracy is also limited by the subjectivity of manual identification. Some scholars have also proposed to distinguish different grouper individuals by changing the salt concentration in the living environment of groupers to cause changes in the grouper's body, but the groupers it can distinguish are limited, and the steps are complex. To solve the problems of excessive labor consumption and low accuracy rate in the manual identification process, some scholars have proposed to use computer vision technology to achieve the automated sorting task of fish individuals. Early research scholars proposed to classify fish based on multi-class support vector machines (MSVM). By using computer vision technology to extract the color and texture features of fish, they successfully classified 6 common species of fish. In Japan, research scholars successfully distinguished two types of mackerels with different economic values based on computer vision and neural networks and completed automatic sorting. With the progress of technology, deep learning provides a more efficient and accurate method to automate the fish sorting process. By using algorithms such as convolutional neural networks (CNN), fish images can be analyzed to identify different species of fish, and even in complex underwater environments. These advanced technologies have significantly improved the speed and accuracy of sorting, bringing revolutionary changes to the fishery and aquatic product processing industries. To further improve the speed and accuracy of fish individual detection, the object detection model based on the regression algorithm converts object detection into a regression task, truly achieving the effect of real-time detection. Among them, the YOLO algorithm is representative, which brings object detection into the one-stage detection process. As a classic one-stage object detection algorithm, the algorithms of the YOLO family have shown better results than other object detection algorithms in many fields. In the field of fishery farming, some research scholars have combined YOLO with deep neural networks to improve the detection and classification effects of fish in underwater videos. Through the fast detection ability of the YOLO algorithm, free-moving fish individuals can be detected in a short time. At the same time, the size of fish individuals can also be detected by the YOLO algorithm to capture the intrusion pictures of large fish, improving the efficiency of ecologists when counting species. There is a lack of a large amount of research on using deep learning and computer vision technology to quickly detect and distinguish groupers with different economic values, especially in distinguishing the huge-yield pearl gentian grouper and tiger grouper.Based on the above research, aiming at the problems that the YOLO algorithm still shows a relatively large model and slow running speed on non-GPU hardware during the actual production application process, the present invention proposes a grouper detection method improved based on the YOLOv8 algorithm. This method integrates the Distributed Shift Convolution (DSConv), which can effectively reduce the computational cost during the algorithm operation. In addition, it also combines the BiFPN network structure, which improves the accuracy of the YOLOv8 algorithm while greatly reducing the computational cost. The SIoU loss function is introduced to effectively improve the robustness of the model.

[0004] In the field of fishery aquaculture, in order to promote the YOLO-based fish individual detection technology, the FPGA with an acceleration operation function provides hardware support for fish individual detection. However, the YOLO model shows problems such as a large number of parameters during operation, which leads to unsatisfactory operation effects on the hardware platform. Some scholars have also tried to reduce the number of parameters during YOLO operation to improve the inference speed, but at the cost of sacrificing accuracy. In addition, due to the similar appearance of Epinephelus fuscoguttatus♀×Epinephelus lanceolatus♂ and Epinephelus tigrinus, YOLOv8 will make mistakes when analyzing and extracting skin features, resulting in low detection accuracy. Therefore, reducing the running parameters of YOLOv8 and improving the detection accuracy of YOLOv8 are the key points of the present invention. On this basis, the present invention proposes a scheme to distinguish Epinephelus fuscoguttatus♀×Epinephelus lanceolatus♂ and Epinephelus tigrinus. Considering the large number of parameters affecting the running speed of YOLOv8 on the hardware platform and the appearance characteristics of the research object of the present invention, a grouper detection network with high accuracy, few parameters, and strong anti-interference ability is designed, and the detection results of this network are fused with the monitoring information of the sensor and uploaded to the cloud, inventing a grouper detection and tracking system based on object detection and information fusion.

[0005] Developing a grouper detection and tracking system based on YOLOv8 can achieve accurate monitoring of individual groupers, including accurate identification of grouper species, location, quantity, and movement trajectory, which helps to solve the problems of low efficiency, poor accuracy, and limited range of traditional manual monitoring methods, and realize the refined management of grouper breeding process and improve breeding efficiency. By combining the real-time environmental data collection function, the comprehensive monitoring of the grouper breeding environment is realized, and the visual information is integrated with environmental data such as underwater temperature to provide a scientific basis for fishery breeding decisions, so as to reasonably plan the breeding density, optimize the breeding environment, reduce the impact on the marine ecological environment, promote the sustainable development of fisheries, and improve the automation and intelligence level of fishery management. In addition, due to the significant difference in economic value between the two, some merchants will make profits by pretending to be high-value tiger groupers, which poses a challenge to market fairness and consumer rights. Therefore, developing a detection system that can quickly and accurately distinguish between pearl gentian grouper and tiger grouper is of great significance to improving breeding efficiency and regulating market circulation. This paper proposes an improved detection model based on YOLOv8. Through deep learning and tracking technology, it can efficiently and accurately detect and distinguish fish in underwater environments, providing strong support for fishery management. At the same time, it also plays an important role in promoting the application of computer vision technology in the field of fisheries, expanding the application scenarios of target detection and tracking algorithms, and providing useful reference and reference for research in related fields. Summary of the invention

[0006] The present invention aims to provide a grouper detection and tracking method based on target detection and information fusion to solve the above-mentioned technical problems existing in the prior art.

[0007] In order to solve the above technical problems, the specific technical solution of a grouper detection and tracking system based on target detection and information fusion of the present invention is as follows: A grouper detection and tracking method based on target detection and information fusion, 1. Based on the YOLOv8n target detection algorithm, the YOLOv8n target detection algorithm is improved according to the image characteristics of two difficult-to-distinguish sub-species (pearl spot and tiger grouper) of grouper, and the two groupers are quickly detected by the YOLOv8n algorithm. The detection results are combined with the DeepSort algorithm to track underwater objects, and the breeding environment data of groupers is monitored by multiple sensors. The monitoring information is fused with the detection and tracking results of YOLOv8 and uploaded to the cloud server, so as to realize real-time monitoring of grouper breeding status in the cloud.

[0008] Furthermore, the improved algorithm structure 2. proposes a DC-YOLOv8 detection method, which can improve the detection accuracy of the network while reducing the computational cost. On this basis, the idea of multi-scale feature fusion is introduced, and the BiPFN network is integrated into the feature pyramid network to obtain multi-scale features of grouper texture. In addition, an angular loss factor is introduced to adjust the IOU loss function, enabling the model to accurately learn the position and size of the bounding box.

[0009] Furthermore, the system has established a grouper dataset under multiple scenarios and different scales (including different backgrounds and angles), providing a high-quality dataset for the research on distinguishing Epinephelus fuscoguttatus♀×Epinephelus lanceolatus♂ and Epinephelus tigrinus.

[0010] Furthermore, the improved YOLOv8n object detection algorithm improves some C2F modules in the original network, fuses the distribution shift convolution (DSConv) with the C2F module, and names it C2f_DSConv, further reducing the computational cost.

[0011] Furthermore, the improved YOLOv8n object detection algorithm network introduces a bidirectional feature pyramid network module BiFPN_Add, which enhances the comprehensive ability of different feature maps through weight normalization and weighted fusion, while maintaining a small number of parameters and optimizing the fusion effect of the feature maps.

[0012] Furthermore, the grouper detection and tracking system based on object detection and information fusion is characterized in that the DeepSort tracking algorithm identifies different species of groupers through the improved YOLOv8n object detection algorithm and extracts the bounding boxes, extracts coordinate information and classification information from the detected bounding boxes, and the DeepSort tracking algorithm performs Kalman filter prediction on the objects with bounding boxes extracted in the previous frame for the current frame to estimate their new positions, and transmits the detection and tracking information of different species of groupers to the cloud through the local area network.

[0013] Furthermore, the improved YOLOv8n object detection algorithm modifies the loss function in the original network, replaces CIOU with SIOU, and further improves the accuracy of the detector.

[0014] Furthermore, the grouper detection and tracking system based on object detection and information fusion is characterized in that the system can fuse the detection and tracking information with the environmental monitoring information of the sensor and upload it to the cloud for display, including the position information, species information and temperature information of the groupers.

[0015] The present invention also discloses a grouper detection and tracking system based on object detection and information fusion, including the following steps: Step 1: Run the improved YOLOv8n object detection algorithm to achieve fast and accurate detection of different species of groupers. Step 2: Use the detection results as the individual groupers to be tracked in the Kalman filter of the DeepSort tracking algorithm, and estimate the positions of all detected grouper individuals in the next frame. Step 3: The DeepSort tracking algorithm integrates detection and tracking information, detects environmental data through multiple sensors and performs information fusion. Step 4: Display the real-time detection and tracking video, showing the image of the current grouper farming environment and the sensor monitoring information in the video, and upload the data after information fusion to the cloud server to monitor the grouper farming situation in the cloud. Step 5: The improved YOLOv8n object detection and tracking algorithm repeats Steps 1-4 to realize real-time detection of the grouper farming monitoring system and update it.

[0016] A grouper detection and tracking system based on object detection and information fusion of the present invention has the following advantages: The present invention establishes and provides a grouper dataset in multiple scenarios and different scales (including different backgrounds and angles), providing a high-quality dataset for the research on distinguishing Epinephelus fuscoguttatus♀×Epinephelus lanceolatus♂ and Epinephelus tigrinus.

[0017] The DC-YOLOv8 algorithm proposed by the present invention significantly improves the performance of grouper detection. Compared with the unimproved YOLOv8 and the higher version YOLO algorithm (YOLOv9), this network has fewer parameters, faster detection speed and higher accuracy, which helps to achieve accurate and rapid identification of different species of groupers. Experiments show that this algorithm obtains 93.4% mAP@0.5-0.95 on the grouper dataset at 166 frames per second. This detection algorithm provides intelligent decision-making support for the sorting work in grouper farming.

[0018] The present invention combines the improved YOLOv8n detection algorithm with the DeepSort tracking algorithm to achieve fast detection and tracking of different species of groupers.

[0019] The present invention fuses the environmental monitoring information of the sensor with the grouper detection information, and provides real-time farming video display, realizing the high-quality development of grouper farming.

[0020] The present invention uses the cloud server to upload the information of the local detection and information fusion system of the system to the cloud and update it in real time, realizing remote monitoring of grouper farming. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is the overall system deployment diagram of the present invention; Figure 2 It is the overall structure diagram of the original YOLOv8 detection algorithm; Figure 3 It is the overall structure diagram of the improved YOLOv8 detection algorithm of the present invention, namely the DC-YOLOv8 detection algorithm; Figure 4 It is the loss comparison diagram of the ablation experiment of the improved YOLOv8 detection algorithm of the present invention, namely the DC-YOLOv8 detection algorithm, on the training set; Figure 5 It is the loss comparison diagram of the ablation experiment of the improved YOLOv8 detection algorithm of the present invention, namely the DC-YOLOv8 detection algorithm, on the validation set; Figure 6 It is the accuracy comparison diagram of the ablation experiment of the improved YOLOv8 detection algorithm of the present invention, namely the DC-YOLOv8 detection algorithm; Figure 7 It is the comparison diagram of the computational amount and accuracy (map50-95) of the improved YOLOv8 algorithm of the present invention, namely the DC-YOLOv8 detection algorithm, and various algorithms; Figure 8 It is the algorithm operation diagram of the BiFPN_Add module introduced for the bidirectional feature pyramid module of the present invention; Figure 9 It is the structure diagram of the C2f_DSConv module, which is the fusion of the distributed shift convolution (DSConv) and the C2F module of the present invention; Figure 10 It is the integrated detection and tracking flow chart of the present invention; Figure 11 It is the dataset production diagram before the operation of the present invention; Figure 12 It is the cloud information display diagram of the grouper detection and information fusion system of the present invention. Detailed implementation manners

[0022] In order to better understand the purpose, structure and function of the present invention, the following further describes in detail a grouper detection and tracking system based on object detection and information fusion of the present invention with reference to the accompanying drawings.

[0023] Such as Figure 1As shown in the figure, the grouper detection and tracking system based on object detection and information fusion of the present invention runs the improved YOLOv8n object detection algorithm to achieve fast and accurate detection of different species of groupers. The detection results are used as the grouper individuals to be tracked in the Kalman filter of the DeepSort tracking algorithm, and the positions of all detected objects are estimated in the next frame. The DeepSort tracking algorithm integrates the detection and tracking information, and monitors the aquaculture environment data of groupers through multiple sensors. The monitoring information is fused with the detection and tracking results of YOLOv8 and uploaded to the cloud server to achieve real-time monitoring of the grouper aquaculture status on the cloud.

[0024] As Figure 3 shown, aiming at the problems of the overly large network structure model and high computational cost of the YOLOv8 network, a network model suitable for the individual recognition and detection of grouper fish is proposed. The distributed shift convolution (DSConv) is used to replace all the convolutions in the backbone part of the original YOLOV8 network, which improves the inference speed of the model while reducing the computational cost. As Figure 9 shown, the C2F structure is improved by combining the distributed shift convolution (DSConv) to further reduce the computational cost. As Figure 8 shown, the BiFPN structure is introduced to enable the model to maintain high-performance object detection on small embedded devices with limited computational resources. Aiming at the deficiency of the loss function CIOU in the original YOLOv8 network, the SIOU loss function is introduced to improve the detection effect when fish individuals overlap during the positioning and tracking of grouper individuals. The loss comparison of the ablation experiment of the proposed new YOLOv8-based grouper detection network, namely the DC-YOLOv8 detection network, on the training set is as Figure 4 shown, and the loss comparison of the ablation experiment on the validation set is as Figure 5 shown, and the index comparison with other YOLO detection algorithms is as Figure 6 shown; the comparison of the computational amount and accuracy (map50-95) of various algorithms is as Figure 7 shown.

[0025] After the improved YOLOv8n detection algorithm detects the bionic robot fish and the underwater object to be tracked, the DeepSort tracking algorithm runs the Kalman filter prediction on the objects that have been extracted with bounding boxes in the previous frame in the current frame to estimate their new positions, and transmits the detection and tracking information of different species of groupers to the cloud through the local area network.

[0026] As Figure 10As shown in the figure, the object detection and tracking system mainly includes two parts: the detection process and the tracking process. In the detection process, the system first receives the video stream as input, uses the improved YOLOv8n algorithm to detect objects in the video frames, and then the algorithm determines whether a target is detected. If it is the first frame of the video, the system will be initialized; if not, the system will update the target information. After completing the detection of the current frame, the system proceeds to the tracking process. The results of the detection process are input into the tracking system to calculate the predicted positions and sizes of each tracking target, calculate the intersection over union (IOU) matrix between the detected targets and the tracking targets, determine whether the targets are matched according to the IOU threshold, update the tracking status and position information of the matched targets, and if a tracking target fails to match the detection result for a long time, the system will delete the tracker, and then process the next frame of video image. The improved YOLOv8n provides powerful detection capabilities, while DeepSort is responsible for tracking to ensure continuous tracking even when the object is occluded or temporarily disappears. The combination of this technology enables the system to operate stably in complex environments and provide coherent tracking results.

[0027] Before the operation of the present invention, a large number of pictures of different species of groupers need to be taken, such as Figure 11 As shown in the figure, use the LabelImg tool for marking, and divide the training set, test set and validation set. During operation, an external PC runs the object detection algorithm based on the improved YOLOv8n and the DeepSort tracking algorithm to obtain the grouper image information in the monitoring screen in real time, and perform species detection and environmental information monitoring on all grouper individuals in the screen. Set up a local area network communication network to fuse the monitoring information of the sensor and the grouper detection information, and divide the fused information flow into two parts. One part is used for parsing and displaying in the real-time detection screen, and the other part is uploaded to the cloud server to achieve simultaneous monitoring locally and in the cloud. The cloud display is as shown in Figure 12 the figure.

[0028] It can be understood that the present invention is described through some embodiments. Those skilled in the art know that without departing from the spirit and scope of the present invention, various changes or equivalent replacements can be made to these features and embodiments. In addition, under the teaching of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. A method for detecting and tracking grouper based on target detection and information fusion, characterized in that The steps include: Step 1: Run the improved YOLOv8n target detection algorithm to quickly and accurately detect different types of groupers; Step 2: Use the detection results as the grouper individuals to be tracked in the Kalman filter of the DeepSort tracking algorithm, and estimate the positions of all detected grouper individuals in the next frame; Step 3: The DeepSort tracking algorithm integrates detection and tracking information, detects environmental data through multiple sensors and performs information fusion; Step 4: Display the real-time detection and tracking screen, which displays the image and sensor monitoring information of the current grouper breeding environment, and upload the fused data to the cloud server to monitor the breeding status of grouper in the cloud; Step 5: Repeat steps 1-4 to detect the grouper breeding monitoring system in real time and update the information.

2. The method for detecting and tracking grouper based on target detection and information fusion according to claim 1, characterized in that: The improved YOLOv8n target detection algorithm introduces multi-scale feature fusion, integrates the BiPFN network into the bidirectional feature pyramid network, obtains the multi-scale features of the grouper texture, and introduces the angle loss factor to adjust the IOU loss function.

3. The method for detecting and tracking grouper based on target detection and information fusion according to claim 1, characterized in that: The improved YOLOv8n target detection algorithm improves some C2F modules in the original network, merges the distribution shift convolution (DSConv) with the C2F module, and names it C2f_DSConv.

4. The method for detecting and tracking grouper based on target detection and information fusion according to claim 1, characterized in that: The DeepSort tracking algorithm identifies different types of groupers and extracts bounding boxes through the improved YOLOv8n target detection algorithm, extracts coordinate information and classification information from the detected bounding boxes, performs Kalman filter prediction on the objects whose bounding boxes have been extracted in the previous frame in the current frame to estimate their new positions, and transmits the detection and tracking information of different types of groupers to the cloud through the local area network.

5. The method for detecting and tracking grouper based on target detection and information fusion according to claim 1, characterized in that: The data after the information fusion includes the location information, species information and temperature information of the grouper.

6. A grouper detection and tracking system based on target detection and information fusion, characterized in that: A grouper detection and tracking method based on target detection and information fusion as described in any one of claims 1 to 5 is adopted.

Images