A computer vision-based dynamic counting method and device for fish fry

Abstract

The application discloses a kind of fish fry dynamic counting method and device based on computer vision belonging to fish fry dynamic counting technical field.The method considers fish fry counting as the multi-target tracking problem based on detection tracking framework, to realize fish fry detection with light YOLOv5-Nano, and to the influence of tracking for different degrees of occlusion between fish fry, with multiple matching and trajectory recovery direction design multi-target tracking algorithm, improve the joint effect of detection model and tracking algorithm, finally realize dynamic fish fry counting with the intersection of tracking trajectory and counting line as the basis.The device includes data acquisition and data processing two parts, and the acquisition part can reduce the resistance of fish fry sliding, which is helpful for trajectory tracking, and the occlusion degree of gathered fish fry is reduced, which helps to improve the counting accuracy, and the processing part can ensure the real-time performance of fish fry counting.The fish fry counting method and equipment designed by the application have wide use value for activities that require a large number of fish fry counting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fry dynamic counting, in particular to a fry dynamic counting method and device based on computer vision. BACKGROUND

[0002] Manual counting not only requires a lot of manpower and material resources, but also easily causes physical damage to fish, resulting in economic losses. At the same time, long-time work inevitably causes fatigue, resulting in inaccurate counting results. In the past, some extensive researches are mainly based on machine learning techniques, including connected domain area method, endpoint thinning method, curve evolution method, classification detection method, etc. These methods rely on geometric feature information such as the contour, area, skeleton and endpoint of the fish body to count the number of fry, and require a large amount of feature extraction engineering and are not universal.

[0003] At present, most of the fish population counting algorithms based on deep learning are still in the laboratory research and innovation stage, and there is still a lack of relevant research on their specific use in actual life. The restrictions of fish population counting algorithms in different breeding environments are different, so the development of fish population counting hardware based on different breeding environments is essential for the use of fish population counting in actual life. In the development of fish population counting hardware, the portability, physical stability and adjustability of the equipment are all content that need to be considered.

[0004] Most of the research methods currently studied have the following limitations:

[0005] (1) Fry occlusion problem: the mutual occlusion between individual fry will lead to a decrease in counting accuracy. The occlusion problem is always a difficult point in the counting of fry.

[0006] (2) Data acquisition problem: most fish population counting algorithms are based on multispectral images that meet the lighting conditions in the laboratory. In actual life, multispectral cameras are easily affected by lighting intensity, fish population occlusion and complex background when collecting fish population information. Insufficient light cannot obtain high-quality image and video information.

[0007] (3) Static counting problem: most computer vision-based fry counting methods can only count a certain number of fry in a fixed container, and the fry needs to be operated in batches in advance. The operation is complicated and not flexible enough.

[0008] Therefore, a computer vision-based dynamic fry counting method and device are needed to solve the above limitations. SUMMARY

[0009] The purpose of the present application is to provide a computer vision-based dynamic fry counting method and device, characterized by the following steps:

[0010] Step 1: Collecting running water fish fry data; collecting video data for model training and counting test respectively, data cleaning is performed on the collected video data for model training to obtain effective images, and LabelImg tool is used to mark the boundary box of fish fry target in the original data set; the video data for counting test is manually counted for subsequent experiments;

[0011] Step 2: Constructing running water fish fry data set; performing data enhancement such as motion blur, brightness adjustment, noise increase and image flipping on the effective images obtained in step 1 to construct the data set, and after manual inspection confirms that the expansion result is correct, dividing the training set and the test set;

[0012] Step 3: Determining fish fry detection model; in order to ensure the lightness and real-time performance of the counting method, the detection accuracy of SSD, YOLOv5-Small, YOLOv5-Nano, YOLOv6-Tiny and YOLOv7-Tiny light one-stage detection models is compared, and finally YOLOv5-Nano is determined as the detection model for fish fry detection;

[0013] Step 4: Optimizing fish fry multi-target tracking algorithm; using the multi-target tracking method of the detection tracking framework, the fish fry motion tracking is realized in combination with the detection model of step 3; wherein different matching strategies are designed for detection boxes with different confidence levels to ensure the correct matching of the tracking target, and the latest detection information of each successful tracking target is saved to help restore the tracking trajectory of the occluded target;

[0014] Step 5: Determining dynamic counting basis; comparing the counting effects of three methods based on tracking ID, counting line and counting area, finally determining to set a counting line in the image area, when the fish fry tracking trajectory in the video image intersects with the counting line, it indicates that the fish fry has entered the packaging bag, and the fish fry entering the packaging bag is counted;

[0015] Step 6: Model training evaluation; YOLOv5-Nano is combined with ByteTrack, OC-SORT and multi-target tracking method to realize fish fry motion tracking respectively, and the dynamic counting test is realized by the intersection of fish fry tracking trajectory and counting line in step 5, and the dynamic counting accuracy is evaluated by comparing the counting effects of the above three tracking algorithms in the known fish fry number test video.

[0016] The step 4 specifically includes the following sub-steps:

[0017] Step 41: dividing the detection box into high-score detection box and low-score detection box according to the confidence threshold of 0.7, the high-score detection box is matched with the tracking trajectory for the first time, and the Euclidean distance D ij The first data association of high-score detection box and tracking information is realized, and the unmatched trajectory, unmatched detection box and matched trajectory are obtained;

[0018] Step 42: combine the intersection-over-union IoU of the high-score detection box and the low-score detection box with the DIOU of the center point distance to perform subsequent data association, and perform secondary matching of the low-score detection box and the unmatched track obtained in step 41;

[0019] Step 43: using the stored latest detection box information of the secondary matched unmatched track and the high-score detection box to perform DIOU matching, and recovering the target tracking track.

[0020] Euclidean distance D ij The calculation formula of DIOU is as follows:

[0021]

[0022]

[0023] DIOU=(IOU+D ij ) / 2 (3)

[0024] Wherein D, T respectively represent the detection box and the tracking box, x i 、x j represent the center points of the detection box and the tracking box in the x-axis direction, y i 、y j represent the center points of the detection box and the tracking box in the y-axis direction, and L represents the length of the diagonal line of the minimum circumscribed rectangle of the detection box and the tracking box.

[0025] A counting device of a fish fry dynamic counting method based on computer vision, characterized in that the counting device comprises a channel, a fish outlet, a camera, an upper water bucket, a lower water bucket and a support; wherein the channel is arranged above the upper water bucket and the lower water bucket, the fish outlet is arranged near the lower water bucket in the channel, the lower water bucket is sleeved into a packaging bag, the camera is connected with the support and the camera is arranged above the middle part of the channel.

[0026] The width of the channel is 32cm, and the length is 1m.

[0027] The lower water bucket is lower than the upper water bucket, forming an inclination angle of 14° of the channel.

[0028] A fish fry pipeline is built by using the channel, the fish outlet, the camera, the upper water bucket, the lower water bucket and the support, and a packaging bag is sleeved into the lower water bucket; a server runs a counting program, selects an online counting function, remotely connects the camera; a large number of fish fry are poured into the channel and slide into the lower water bucket; the camera shoots video data and transmits the video data to the server through a local area network via a base station; the counting program in the server calculates the number of fish fry sliding into the lower water bucket and displays the counting result.

[0029] The beneficial effects of the present application are:

[0030] 1. The fish fry pipeline designed in this invention makes the movement of fish fry more stable when shooting videos of flowing water fish fry, which can reduce the resistance of fish fry sliding down and help track the trajectory. In addition, the degree of occlusion between fish fry is reduced, which helps to improve the counting accuracy.

[0031] 2. Regarding the data collection problem, the dataset of this invention was collected under dim lighting in the breeding workshop, and the dataset has strong robustness to data collected under various complex environments.

[0032] 3. This invention uses video to collect data. The counting does not stop as long as the video is playing, so it can realize dynamic counting of multiple batches and multiple numbers of fish fry, which can avoid the operation of fish fry in batches.

[0033] 4. To address the impact of varying degrees of occlusion between fish fry on tracking, the SORT tracking algorithm was improved using two methods: multiple matching and trajectory recovery, resulting in higher multi-target tracking accuracy and counting accuracy.

[0034] 5. The video fish fry counting model designed in this invention can achieve a computing speed of 15-35 frames per second on the GPU, which has a certain real-time performance, laying the foundation for real-time effects in actual use. Moreover, the device is simple and flexible to operate, saving a lot of manpower and resources. Attached Figure Description

[0035] Figure 1 This is a flowchart of the dynamic fish fry counting method based on computer vision according to the present invention;

[0036] Figure 2 Roadmap for the practical application of dynamic fish fry counting;

[0037] Figure 3 Diagram of a data acquisition device for dynamic fish fry counting;

[0038] Figure 4 This is a diagram of the overall system of the dynamic fish fry counting device.

[0039] Figure 5 This is a diagram of the fish fry counting system interface.

[0040] Figure 6 A schematic diagram illustrating the application process of the dynamic fish fry counting method;

[0041] Figure 7 (a)(b)(c) are the consecutive frame counting effect diagrams for frames 40, 41 and 42, respectively;

[0042] Figure 8 (a), (b), and (c) are the counting results of three different tracking algorithms: ByteTrack, OC-SORT, and Improved SORT, respectively.

[0043] Figure: 1-channel, 2-fish outlet, 3-camera, 4-upper water bucket, 5-lower water bucket, 6-bracket, 7-base station, 8-server. DETAILED DESCRIPTION

[0044] The present application proposes a dynamic fry counting method and device based on computer vision, which will be further described below in combination with the drawings and specific embodiments.

[0045] Figure 1 The flowchart of the dynamic fry counting method based on computer vision of the present application is as follows, Figure 2 The actual application roadmap of dynamic fry counting is as follows. A set of data acquisition, data expansion and data optimization using equipment is designed to establish a data set, which provides a basis for the actual use of the counting method. The fry counting is regarded as a multi-target tracking problem based on a detection and tracking framework, and a dynamic fry counting method is designed. Finally, the training results are evaluated and compared. The specific implementation is as follows:

[0046] Equipment setup and data construction:

[0047] (1) Equipment design

[0048] In order to effectively collect the data of the fry flowing through the channel, the present embodiment sets up a counting device with a width of 32 cm and a length of 1 m, as shown in Figure 3 The counting device includes a channel 1, a fish outlet 2, a camera 3, an upper water bucket 4, a lower water bucket 5, and a bracket 6. The channel 1 is placed above the upper water bucket 4 and the lower water bucket 5, the fish outlet 2 is arranged near the lower water bucket 5 in the channel 1, the lower water bucket 5 is sleeved into a packaging bag, the camera 3 is connected with the bracket 6 and the camera 3 is arranged above the middle part of the channel 1. The fry with small size and different densities has sufficient movement range, and the 1 m pipeline length helps the individual separation of the gathered fry during the sliding process. The two water buckets with different heights are used as the upper and lower supports of the channel, which not only makes the channel inclined at a relatively gentle angle, but also directly makes the fry flowing through the channel enter the lower water bucket. Under the dim light in the breeding workshop, a network camera is used to shoot videos at a height of 45 cm above the water tank.

[0049] In the data collection process, in order to ensure that the fry passes through the pipeline smoothly, a water flow device is arranged at the tail of the channel 1 to flow water into the channel 1 at a constant speed, so as to reduce the friction of the fry sliding and solve the problem of fry gathering and shielding. Since the width of the channel 1 is 32 cm, the fry with small size and different densities has sufficient movement range, and the 1 m pipeline length helps the individual separation of the gathered fry during the sliding process. Therefore, when collecting data in the middle part of the pipeline, not only the movement of the fry tends to be stable, which is helpful for trajectory tracking, but also the shielding degree between the gathered fry is reduced, which is helpful for improving the counting accuracy.

[0050] Due to the real-time requirement of counting on the performance of the device, the counting process is divided into two aspects of video shooting and data processing. Only video shooting work is carried out in the actual breeding workshop, and the obtained data is transmitted in real time and safely by using the local area network, so as to realize data processing in a remote high-performance computing device, thereby ensuring the real-time effect of fry counting. Moreover, the high-performance computing device can realize the simultaneous operation of multiple models, which is helpful for the accurate counting of multiple types of fry. Among them, the counting effect is displayed by the related counting system in the computing device, and the specific process is as follows:

[0051] A fry pipeline is built by using channel 1, fish outlet 2, camera 3, upper water bucket 4, lower water bucket 5, support 6, and a packing bag is sleeved in the lower water bucket 5; the server 8 runs the counting program, selects the online counting function, and remotely connects the camera 3; a large number of fry is poured into the channel 1 and slides into the lower water bucket 5; the camera 3 shoots video data, which is transmitted to the server 8 through the base station 7 by using the local area network; the counting program in the server 8 calculates the number of fry sliding into the lower water bucket, and displays the counting result. The overall system and the display interface are shown in Figure 4 and Figure 5 .

[0052] (2) Data construction and counting method implementation

[0053] Since the still water counting method is difficult to obtain good counting results for flat fry, the flat striped snapper fry with a length of about 5-6 cm is taken as the target in this embodiment, 300 fry are repeatedly poured into the above-mentioned channel 1 by artificial, and 16 videos of fry flowing through the channel are collected at different fish pouring speeds. The specific parameters of the video are: resolution 1920*1080, 30 frames per second, and average duration 15s. In addition, in order to verify the counting accuracy, 49, 100, 142, 200, 257 and 300 fry are manually counted, and the counted fry with different quantities are respectively passed through the channel 1, so as to obtain three videos with different quantities of fry as test videos, which do not participate in model training.

[0054] According to the actual counting requirements, 1318 effective images are obtained for training and testing the detection model by data cleaning of the 16 collected videos, and the fish fry targets in the original data set are labeled by using the LabelImg tool. When labeling, the minimum bounding box of the fry is taken as the true box. Since the outlines of the fry that are severely blocked and at the image boundary in the video are incomplete, it will cause matching and counting difficulties, therefore, only the targets that can be manually recognized or the outlines of the fry can be speculated are labeled, and other difficult-to-identify fry are not labeled.

[0055] Data augmentation is a common method to enhance the robustness of deep learning models, that is, without changing the image category, expanding the training data set, making the data set as diverse as possible, which helps to reduce overfitting in supervised learning algorithms, improve the learning process and improve the model generalization ability. Considering the influence of water flow speed, shooting environment and fry flow direction in practical application, the initial data set is augmented in the application, including motion blur, brightness adjustment, noise increase and image flipping. Motion blur reduces the influence of water flow speed on fry detection, which helps to detect moving fry targets; brightness adjustment and noise increase enhance the brightness range and image quality of the original data set to reduce the influence of uneven lighting and large water mist in the breeding workshop on detection; image flipping rotates the original image by 90°, 180° and 270° to solve the detection error caused by different camera installation positions. Finally, 1318 original data are expanded to 10528, and after artificial inspection confirms that the expansion result is correct, the data set is randomly divided into training set and test set according to the ratio of 6:4.

[0056] The fry counting is regarded as a multi-target tracking problem based on a detection and tracking framework, and a fry dynamic counting method is proposed. Based on the phenomenon of high similarity of fry appearance, YOLOv5-Nano and SORT are used to realize fry track tracking, and the SORT algorithm is improved based on multiple matching and track recovery to solve the influence of fry mutual occlusion on tracking results, and realize 82.6% multi-target tracking accuracy. Finally, the tracking trajectory is used to realize fry counting through the counting line, and 96.4% counting accuracy is obtained. The counting method processing flow is as shown in Figure 6 The specific content is as follows:

[0057] Target detection module: The detection and tracking method depends on the result of target detection, but in actual use, the accuracy and processing speed of detection in the fry detection task have higher requirements. One-stage detection network is more suitable for this task scenario because it can output the anchor frame and probability of the category during the detection process. The current more classic one-stage detection network mainly has YOLO, and so far, the YOLO series has been updated to the seventh generation, and in recent years, the YOLO framework has been widely used due to its fast detection function. Among them, YOLOv5 was proposed in 2020, and the lightweight network YOLOv5-nano designed for low-end image processor devices was proposed in 2021. Various studies have shown that YOLOv5-nano can be used to develop a network model with excellent detection accuracy and speed. In order to ensure that the detection model used in the present application has both detection accuracy and speed advantages, comparative experiments were conducted on the lightweight network of the current newer YOLO series under the same test data set. The results show that the detection accuracy of YOLOv5s is the best, but it has high requirements for the performance of the computer to meet the calculation requirements, and the detection accuracy of YOLOv5-Nano is only 0.7 percentage points lower than that of YOLOv5s, with a detection accuracy of 85.5%, but the required calculation amount is the lowest, and the overall parameters are only 1.76M. Finally, YOLOv5-Nano is selected for fry detection.

[0058] Multi-target tracking module: The tracker is a key component of the proposed counting method. Most current detection and tracking-based methods use a combination of IOU distance features and the Hungarian algorithm to achieve data association, and use Kalman filters to predict the new location of the target in frame T+1 from the target in frame T, achieve target tracking, and representative algorithms include SORT, ByteTrack, etc. In order to ensure the simplicity and lightness of the proposed counting method, SORT algorithm is used to realize target tracking, but the missing of detection results caused by the light occlusion of fry and the long-term interruption of tracking caused by the heavy occlusion will all lead to tracking errors, which will reduce the counting accuracy. In order to improve the tracking accuracy, the SORT algorithm needs to be improved.

[0059] In view of the problem that fry has different degrees of occlusion, three matching is designed in the tracking process, and the overall process is as follows:

[0060] First, the problem of tracking target matching failure caused by YOLOv5-Nano missed detection. Currently, most multi-target tracking methods will select a detection threshold, only keep the detection results above the threshold to get the tracking results, and discard the detection results below the threshold. This is one of the reasons for detection missed detection in tracking target matching. Especially, the detection score of the fry will decrease due to occlusion, and the missed detection problem will generally occur. Therefore, in this embodiment, the detection frame is divided into high-score detection frame and low-score detection frame based on the confidence threshold of 0.7. The high-score detection frame is matched with the tracking trajectory for the first time. The IOU matching method used by the SORT algorithm cannot accurately reflect the degree of overlap between the two frames and has more calculation contents. Therefore, the Euclidean distance D ij The first data association of the high-score detection frame and the tracking information is realized, and three kinds of data are obtained: unmatched trajectories, unmatched detection frames and matched trajectories.

[0061] Since the low-score detection frame is generally the detection information of the occluded fry, the D ij The matching of a large number of detection frames and tracking trajectories is realized, which not only reduces the calculation amount but also lays a foundation for the tracking of the occluded fry. However, D ij is not enough to support the tracking matching of the occluded fry. Therefore, in order to ensure the correctness of the matching of the occluded fry, the present application uses DIOU which combines the intersection-over-union of the two frames and the center point distance for subsequent data association, realizes the secondary matching of the low-score detection frame and the unmatched trajectory obtained by the above first association, and slows down the target loss problem caused by occlusion.

[0062] If the fry is occluded for a long time, it will cause the tracking to be interrupted, making it difficult to match the tracking prediction result and the detection result. In view of the fact that the prediction result will increase the error as the interruption time increases, and with the development of target detection algorithm, the detection result is often more accurate than the estimated result of the Kalman filter, the present application discards the wrong estimated value and uses the detection result to assist in realizing the recovery of the interrupted trajectory. The design stores the latest detection value of the tracking trajectory when the tracking is updated by Kalman filter, so as to directly associate and match the last detection value of the interrupted tracking with the reappeared detection value to recover the tracking trajectory when the fry tracking is interrupted again. The main method is to use the latest detection frame information of the unmatched trajectory after the secondary matching and the high-score detection frame for DIOU matching, so as to realize the tracking trajectory recovery of the severely occluded target.

[0063] In the late tracking stage, the new trajectory generation and deletion methods are used to process the remaining high-score detection boxes and unmatched tracking, respectively. Since the proposed fry counting method relies heavily on the tracking information returned by the tracker, in order to improve the counting accuracy, the last detection box information of tracking is used for trajectory drawing instead of the tracking estimate value, so as to reduce the influence of the error existing in the estimate value on the counting accuracy.

[0064] IOU, Euclidean distance D ij and the formula of DIOU is as follows:

[0065]

[0066]

[0067] DIOU=(IOU+D ij ) / 2 (3)

[0068] where D, T represent the detection box and the tracking box respectively, x i , x j represent the center points of the detection box and the tracking box in the x-axis direction, y i , y j represent the center points of the detection box and the tracking box in the y-axis direction, and L represents the length of the diagonal line of the minimum circumscribed rectangle of the detection box and the tracking box.

[0069] Fry counting module: In order to realize accurate counting of fry, in the embodiment, a counting method based on a counting line is selected. A virtual counting line is drawn at the top of the field of view of the video, and when the latest tracking trajectory of the fry in the frame intersects with the counting line, it is considered that the fry has passed the counting line and the counting is once. The tracking trajectory here is drawn according to the different center point positions of the fry detection box in adjacent frames. It is worth noting that in order to reduce the counting error caused by tracking, the detection box information used for drawing the trajectory is not the tracking estimate value but the latest matched detection value of tracking, and the counting effect of consecutive frames is shown in Figure 7 (a)(b)(c), where the horizontal line above the image is the counting line, the vertical line at the tail of the fry is the tracked motion trajectory, and the counting result is displayed in the upper left corner of the image. Under the same experimental environment, the accuracy of the method based on the counting line is much higher than that of the commonly used method of using tracking ID to count the number, and the counting result based on the tracking ID is often greater than the actual result.

[0070] (3) Model effect evaluation

[0071] The embodiment combines YOLOv5-Nano, an improved SORT algorithm and a counting method based on a counting line, and proposes a dynamic fry counting method, which reduces the influence of fry occlusion on counting accuracy and can realize the counting of a large number of fry. By comparing multiple lightweight detection models, the YOLOv5-Nano used in the embodiment reduces the operation requirement of the calculation carrier while meeting the required detection accuracy. According to the reason for data association failure in tracking, the SORT algorithm is improved through multiple matching and trajectory recovery, so that the MOTA can reach 82.6%, and the MOTP is 17.0%, which is improved by 25.2% and reduced by 6.4% compared with the initial SORT algorithm, has better performance, and reduces the influence of tracking error on counting. Finally, the estimated value of tracking is discarded, the latest detection box information of tracking is used to draw the fry trajectory, and the fry counting is realized based on the intersection of the trajectory and the counting line. After testing, for videos with 49, 100, 142, 200, 257 and 300 fry respectively, 49, 100, 137, 186, 246 and 278 automatic counting results can be obtained respectively, and the counting accuracy can reach 96.4%. Since the proposed counting algorithm is simple, it can reach 15-35fps on GPU, which is helpful for the practical application of the algorithm.

[0072] In addition, YOLOv5 is combined with classic tracking algorithms such as ByteTrack and OC-SORT to do counting test, only the parameters of the two are adjusted to achieve better counting effect, and the algorithm structure is not changed. Figure 8 (a) (b) (c) The results show that ByteTrack and OC-SORT have poor effect, and cannot be applied to the counting task, and even the trajectory drawing error occurs due to the instability of the tracker.

[0073] The dynamic counting method adopted by the application can directly realize multi-batch and multi-target counting without the large amount of manpower and material resources investment required by traditional manual counting. Compared with the traditional method which can only count a certain number of fry in a fixed container, the method is more flexible and efficient. In practical application, the counting process is divided into video shooting and data processing. Only the designed use device is used for video shooting in the actual breeding workshop, the obtained data is transmitted in real time and safely through the local area network, and the designed counting model is embedded in the remote high-performance calculation device for data processing, so as to ensure the real-time effect of fry counting, and the application range is wide, which can play a good effect in multiple breeding environments and has strong practicality.

Claims

1. A computer vision-based dynamic counting method for fish larvae, characterized by, Comprise the following steps: Step 1: Collecting the data of the fish fry in the flowing water; Collecting the video data for model training and counting test respectively, and performing data cleaning on the collected video data for model training to obtain effective images, and using LabelImg tool to perform boundary box labeling on the fish fry targets in the original data set; Manually counting the video data for counting test for subsequent experiments; Step 2: Constructing the fish fry data set in the flowing water; performing data enhancement such as motion blur, brightness adjustment, noise increase and image flipping on the effective images obtained in step 1 to construct the data set, and after confirming that the expansion result is correct through manual inspection, dividing the training set and the test set; Step 3: Determining the fish fry detection model; in order to ensure the lightness and real-time performance of the counting method, comparing the detection accuracy of the light one-stage detection models of SSD, YOLOv5-Small, YOLOv5-Nano, YOLOv6-Tiny and YOLOv7-Tiny, and finally determining to use YOLOv5-Nano as the detection model to detect the fish fry targets; Step 4: Optimizing the fish fry multi-target tracking algorithm; using the multi-target tracking method of the detection tracking framework, and combining with the detection model of step 3 to realize the motion tracking of the fish fry; wherein different matching strategies are designed for detection boxes with different confidences to ensure the correct matching of the tracked targets, and the latest detection information of each successfully tracked target is saved to help restore the tracking trajectory of the occluded target; The step 4 specifically comprises the following sub-steps: Step 41: divide the detection frame into high-score detection frame and low-score detection frame based on the confidence threshold value 0.7, the high-score detection frame is matched with the tracking trajectory for the first time, and the Euclidean distance D ij The first data association of the high-score detection frame and the tracking information is realized, and the unmatched trajectory, the unmatched detection frame and the matched trajectory are obtained. Step 42: Combining the intersection-over-union IoU and the center point distance DIOU of the high-score detection box and the low-score detection box to perform subsequent data association, and performing secondary matching on the low-score detection box and the unmatched track obtained in step 41; Step 43: Using the stored latest detection box information of the unmatched track after secondary matching to perform DIOU matching with the high-score detection box to restore the target tracking trajectory; Euclidean distance D ij The calculation formula of DIOU is as follows: DIOU = (IOU + D ij ) / 2 (3) wherein D, T represent the detection box and the tracking box respectively, x i , x j represents the center point of the detection box and the tracking box in the x-axis direction, y i , y j represents the center point of the detection box and the tracking box in the y-axis direction, L represents the length of the diagonal line of the minimum circumscribed rectangle of the detection box and the tracking box; Step 5: Determining the basis for dynamic counting; comparing the counting effects of the three methods based on tracking ID, counting line and counting area, and finally determining to set a counting line in the image area, when the fish fry tracking trajectory in the video image intersects with the counting line, it indicates that the fish fry has entered the packaging bag, and the fish fry entering the packaging bag is counted; Step 6: Model training evaluation; combining YOLOv5-Nano with ByteTrack, OC-SORT and multi-target tracking method to realize fish fry motion tracking respectively, and realizing dynamic counting test by the intersection of the fish fry tracking trajectory of step 5 and the counting line, and evaluating the dynamic counting accuracy by comparing the counting effects of the above three tracking algorithms in the test video with known fish fry quantity.

2. A counting device for the computer vision based dynamic counting method of fish larvae according to claim 1, characterized in that, The counting device comprises a channel (1), an outlet (2), a camera (3), an upper water bucket (4), a lower water bucket (5) and a support (6); wherein the channel (1) is arranged above the upper water bucket (4) and the lower water bucket (5), the outlet (2) is arranged near the lower water bucket (5) in the channel (1), the lower water bucket (5) is sleeved into the packaging bag, the camera (3) is connected with the support (6) and the camera (3) is arranged above the middle part of the channel (1).

3. The counting device of the computer vision-based fish larvae dynamic counting method according to claim 2, wherein, The width of the channel (1) is 32 cm, and the length is 1 m.

4. The counting device of the computer vision-based fish larvae dynamic counting method according to claim 2, wherein, The lower water bucket (5) is lower than the upper water bucket (4), forming a 14° inclination angle of the channel (1).

5. The counting device of the computer vision-based fish larvae dynamic counting method according to claim 2, wherein, The fish fry pipeline is built by using the channel (1), the fish outlet (2), the camera (3), the upper water bucket (4), the lower water bucket (5), and the support (6), and the packing bag is sleeved into the lower water bucket (5); the server (8) runs the counting program, selects the online counting function, remotely connects the camera (3); a large number of fish fry are poured into the channel (1) and slide into the lower water bucket (5); the camera (3) shoots video data, and the video data are transmitted to the server (8) through the base station (7) through the local area network; the counting program in the server (8) calculates the number of fish fry sliding into the lower water bucket, and displays the counting result.

Images