A fish identification, counting, speed measuring, early warning and quantity statistics system

Abstract

The application discloses a fish identification, counting, speed measurement, early warning and in-out quantity statistical system. Users can combine their own actual conditions to customize a training data set, combine a trained weight file with a multi-target tracking algorithm based on Deepsort, a counting algorithm based on double-convex track measurement and a speed measurement algorithm based on visual inclination used in the application, and realize a series of functions such as fish multi-target tracking, counting of various fish quantities in a picture, calculation of fish quantities entering and leaving a monitoring picture and speed measurement of the swimming speed of each fish. The application can achieve intelligent management and control effects according to the ornamental fish to be managed. The application has good application expansibility, and can be developed secondarily in many ways by adding some algorithms according to user demands.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to a system for fish identification, counting, speed measurement and early warning, and counting of fish entering and leaving the fish population. Background Technology

[0002] In recent years, with the rise in the economic level of the Chinese people, art appreciation and leisure activities have become increasingly popular. Admiring and raising ornamental fish has become a delightful activity, and now these fish can be seen in many aquariums, shopping malls, hotels, tourist attractions, entertainment and exhibition venues, and even in private homes. In recent years, the scale of marine ornamental fish farming and exhibitions has gradually expanded, bringing with it a series of problems. Due to their relatively high economic value, their ability to satisfy the public's need for artistic appreciation, and the joy they bring, marine ornamental fish generally require meticulous management and scientific breeding. However, manual management is not only time-consuming and labor-intensive but also generally ineffective, thus urgently requiring artificial intelligence technology to alleviate this problem.

[0003] Currently, some people are only using artificial intelligence technology to identify fish species, without further research to monitor more fish indicators or to conduct more in-depth research on intelligent aquaculture. Summary of the Invention

[0004] This invention addresses the problems described in the background by proposing a fish identification, counting, speed measurement, early warning, and entry / exit quantity statistics system. This system can dynamically track fish, count the number of various fish in a video feed, calculate the number of fish entering and leaving the monitoring screen, and measure and warn about the swimming speed of each fish.

[0005] 1. The technical solution of the present invention includes the following steps:

[0006] S1: Collect videos or images of 5 ornamental fish species as a dataset. Use a sprite annotation assistant to annotate the coordinates of the fish in the images, and then generate an annotation file. Use a Python script to divide the annotated dataset into a training set and a validation set.

[0007] S2: Put the dataset into the YOLOv5 network structure for training. After training, save the trained weight file.

[0008] S3: The weight file is used to perform object detection on the video frames, detecting fish target boxes in the scene. After processing by the Deepsort-based multi-object tracking algorithm and the visual tilt-based velocity measurement algorithm, the index and velocity of the fish target boxes are obtained.

[0009] S4: Then, using a counting algorithm based on double convex rails, the number of fish entering and leaving is obtained.

[0010] S5: Output system test results.

[0011] 2. In some alternative implementations, step S1 includes:

[0012] S11: Record videos of ornamental fish using a camera, take photos of ornamental fish, and search for videos or photos of ornamental fish online. Use these videos and photos as the training dataset. The videos need to be converted to images for training, so the following steps are required. Press the Windows key and R key simultaneously, type cmd, open the command line window, and type pipinstall ffmpeg to install ffmpeg (a toolkit in the command line). After installing ffmpeg, use ffmpeg in the command line to convert the video into an image dataset at a certain sampling rate. Use commands such as: ffmpeg -itest.mp4 -r 5 -f image2.\output\frame_%05d.jpg, where -i is followed by the video file with the .mp4 extension; -r is followed by the sampling rate, i.e., how many images to split per second of video; -f is followed by the image type; and finally, the output image folder and naming format are specified, where %05d indicates a 5-digit sequence number, and the image names are recursively named frame_00000.jpg, frame_00001.jpg, frame_00002.jpg, etc.

[0013] S12: Use the Sprite Labeling Assistant (a software for creating dataset labels) to label the dataset. First, define the types of ornamental fish. This system uses five types of ornamental fish: clownfish, yellow boxfish, schooling banner fish, emperor angelfish, and yellow tang fish. Then, draw a box around each ornamental fish appearing in each image, selecting one species for each box, such as one of the five mentioned above. After all labels are completed, set the export path and export format. Here, the export format is selected as Pascal-VOC, generating 5228 labeled files with the .xml extension from 5228 images.

[0014] S13: Because the training uses annotation files with the .txt extension, the annotation files need to be converted. A Python script is used for batch conversion. The relevant library files are imported using `import os` and `import xml.etree.ElementTree` to convert the pixel coordinates in the .xml file to normalized coordinates in the .txt file. Fish species are represented by the five numbers 0, 1, 2, 3, and 4. The conversion formulas are shown in Formulas 1, 2, 3, and 4.

[0015]

[0016]

[0017]

[0018]

[0019] In the formula, (x, y) are the coordinates of the center point of the normalized box.

[0020] w, h — the normalized width and height of the box

[0021] (x min y min — Coordinates of the top left corner of the box

[0022] (x max y max — Coordinates of the bottom right corner of the box

[0023] width, height — the width and height of the entire image.

[0024] 3. In some alternative implementations, step S2 includes:

[0025] S21: In the Input part of the network structure. (1) First, perform Mosaic data augmentation. This algorithm scales, distributes, and crops four input images and randomly stitches them together. This method makes the input images more diverse, and the trained model is more helpful in detecting small objects. At the same time, it reduces the amount of computation by processing only one image instead of four. (2) Then, perform an adaptive image scaling algorithm. This algorithm is used when testing and using the model for inference, which greatly improves the inference speed of the algorithm. (3) Then, perform an adaptive anchor box calculation algorithm. The program corresponding to the initial anchor box value obtained from the test is put into the original code. The best anchor box value can be obtained each time the dataset is trained.

[0026] S22: In the Backbone part of the network structure, within the Focus structure, the image is sliced, then concatenated, and finally convolved to obtain a more easily processed feature map. This reduces floating-point operations and speeds up computation. For example, a 796×796×3 image becomes a 398×398×12 feature map after slicing, concatenation, and convolution. Then, it passes through the CSPNet structure, which primarily splits the feature map into two parts: one part undergoes convolution, and the other part is concatenated with the result of the convolution operation of the first part. This structure reduces computational cost, but the improvement in accuracy is minimal.

[0027] S23: In the Neck part of the network structure, the FPN-PAN structure is used, which performs multi-dimensional feature extraction and greatly increases the receptive field.

[0028] S24: In the head part of the network structure, the CIOU Loss loss function is used as the regression loss function for the bounding boxes. This loss function considers more comprehensive information and has better practical results. In the post-processing stage of object detection, non-maximum suppression is performed for filtering many object boxes. Based on... Figure 6 Formula 5 for calculating CIOU Loss is derived.

[0029]

[0030] In the formula, Distance_C represents the diagonal length of the smallest bounding rectangle.

[0031] Distance_2 — The length of the line connecting the center points of the two bounding boxes

[0032] IOU (Intersection over Union)

[0033] v — A parameter measuring aspect ratio consistency

[0034] 4. In some alternative implementations, step S3 includes:

[0035] S31: The main steps of the multi-target tracking algorithm based on Deepsort are as follows: (1) First, the fish targets in the video frame are detected using the trained fish classification weight file to obtain feature information. Here, our information includes the fish's location information, category information, and confidence information. (2) By calculating the matching degree of the image information of the two frames before and after, the target is matched, and then a sequence number is assigned to each tracked target to realize the dynamic tracking of various fish. The Deepsort algorithm adds cascade matching and new trajectory confirmation, which solves the problem of objects losing connection due to occlusion in the Sort algorithm. The trajectory is divided into confirmed trajectory and unconfirmed trajectory. The newly generated trajectory must be matched with the detector multiple times in a row before it can be converted from an unconfirmed trajectory to a confirmed trajectory; the confirmed trajectory will be deleted only after losing connection with the detector multiple times in a row.

[0036] S32: The core idea of ​​the speed measurement algorithm based on visual tilt is as follows. First, record the pixel coordinates (1920×1280 resolution): Record all fish ID numbers and center XY coordinates after detection and sorting by the Deepsort algorithm in a list. When tracking the same ID number at intervals of multiple video frames, calculate the pixel speed using formulas 6 and 7.

[0037] (xx -1 ) 2 +(yy -1 ) 2 =Distance 2 (Formula 6)

[0038]

[0039] In the formula, x and y are the coordinates of the position after the movement.

[0040] x -1 y -1 —Position coordinates before moving

[0041] Distance – Distance of movement

[0042] t — movement time

[0043] —Pixel speed

[0044] The speed at which a fish moves in a video frame, or the number of pixels it moves per unit time, is called pixel velocity. Pixel velocity differs from actual velocity, requiring analysis of the physical model and the tilt angle of the camera lens. In the same amount of time, fish swimming the same distance will pass through more pixels closer to the camera, while fish swimming closer to the camera will pass through fewer pixels. Therefore, the former needs a scaling factor to reduce its size, and the latter needs a scaling factor to enlarge it; however, this parameter is difficult to find manually. The best solution is to measure multiple sets of actual velocity and pixel velocity, and then use machine learning to solve the equations, as shown in Equation 8.

[0045]

[0046] In the formula, λ is a proportionality coefficient between the actual speed and the pixel speed.

[0047] s i —Based on multiple sets of actual speeds and pixel speeds, machine learning needs to solve for the parameters.

[0048] y - pixel coordinate on the vertical axis

[0049] n — An integer, which will be assigned a value based on the actual situation.

[0050] v—Actual speed

[0051] 5. In some alternative implementations, step S4 includes:

[0052] S41: The core idea of ​​the counting algorithm based on double convex track measurement is as follows: (1) Draw two convex detection lines on the video: green line and yellow line. (2) Continuously compare the fish position coordinates with the coordinates of the two lines. When the monitoring point coordinates intersect the two convex detection lines, count them. (3) If the fish position coordinates pass through the green line first and then through the yellow line, it is determined that it has entered the monitoring area, and the number of fish is increased by 1; if the fish position coordinates pass through the yellow line first and then through the green line, it is determined that it has left the monitoring area, and the number of fish is decreased by 1.

[0053] 6. In general, compared with the prior art, the above technical solutions of the present invention can achieve the following beneficial effects:

[0054] (1) This invention uses a multi-target tracking algorithm based on Deepsort, a counting algorithm based on double convex orbital calculation, and a speed measurement algorithm based on visual tilt to realize a series of functions such as multi-target tracking of fish, counting the number of various fish in the screen, calculating the number of fish entering and leaving the monitoring screen, and measuring the swimming speed of each fish.

[0055] (2) The present invention can collect datasets for training based on the ornamental fish species it wants to manage, thereby achieving intelligent management and control.

[0056] (3) The invention has good application scalability. By combining user needs and adding some algorithms, many secondary developments can be carried out, such as: fish speed measurement and early warning, fish leaving the tank detection, fish theft alarm, etc. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the implementation of the present invention.

[0058] Figure 2 This is the weighted file detection effect of the present invention.

[0059] Figure 3 This invention relates to Mosaic data enhancement.

[0060] Figure 4 This invention relates to adaptive image scaling.

[0061] Figure 5 This is the FPN-PAN structure of the present invention.

[0062] Figure 6 This is a schematic diagram of the CIOU Loss principle of the present invention.

[0063] Figure 7 This is the nonmaximum suppression operation of the present invention.

[0064] Figure 8 This describes the workflow of the Deepsort algorithm in this invention.

[0065] Figure 9 This is a schematic diagram of the double convex rail measurement of the present invention.

[0066] Figure 10 This is the system detection result of the present invention. Detailed Implementation

[0067] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0068] A fish identification, counting, speed measurement, early warning, and entry / exit quantity statistics system, the technical solution of which includes the following steps:

[0069] S1: Collect videos or images of 5 ornamental fish species as a dataset. Use a sprite annotation assistant to annotate the coordinates of the fish in the images, and then generate an annotation file. Use a Python script to divide the annotated dataset into a training set and a validation set.

[0070] In this embodiment, step S1 can be implemented in the following way:

[0071] S11: Record videos of ornamental fish using a camera, take photos of ornamental fish, and search for videos or photos of ornamental fish online. Use these videos and photos as the training dataset. The videos need to be converted to images for training, so the following steps are required. Press the Windows key and R key simultaneously, type cmd, open the command line window, and type pipinstall ffmpeg to install ffmpeg (a toolkit in the command line). After installing ffmpeg, use ffmpeg in the command line to convert the video into an image dataset at a certain sampling rate. Use commands such as: ffmpeg -itest.mp4 -r 5 -f image2.\output\frame_%05d.jpg, where -i is followed by the video file with the .mp4 extension; -r is followed by the sampling rate, i.e., how many images to split per second of video; -f is followed by the image type; and finally, the output image folder and naming format are specified, where %05d indicates a 5-digit sequence number, and the image names are recursively named frame_00000.jpg, frame_00001.jpg, frame_00002.jpg, etc.

[0072] S12: Use the Sprite Labeling Assistant (a software for creating dataset labels) to label the dataset. First, define the types of ornamental fish. This system uses five types of ornamental fish: clownfish, yellow boxfish, schooling banner fish, emperor angelfish, and yellow tang fish. Then, draw a box around each ornamental fish appearing in each image, selecting one species for each box, such as one of the five mentioned above. After all labels are completed, set the export path and export format. Here, the export format is selected as Pascal-VOC, generating 5228 labeled files with the .xml extension from 5228 images.

[0073] S13: Because the training uses annotation files with the .txt extension, the annotation files need to be converted. A Python script is used for batch conversion. The relevant library files are imported using `import os` and `import xml.etree.ElementTree` to convert the pixel coordinates in the .xml file to normalized coordinates in the .txt file. Fish species are represented by the five numbers 0, 1, 2, 3, and 4. The conversion formulas are shown in Formulas 1, 2, 3, and 4.

[0074]

[0075]

[0076]

[0077]

[0078] In the formula, (x, y) are the coordinates of the center point of the normalized box.

[0079] w, h — the normalized width and height of the box

[0080] (x min y min — Coordinates of the top left corner of the box

[0081] (x max y max — Coordinates of the bottom right corner of the box

[0082] width, height — the width and height of the entire image.

[0083] S2: Place the dataset into the YOL0v5 network structure for training. After training, save the trained weight file. The detection performance of the weight file is as follows: Figure 2 As shown.

[0084] In this embodiment, step S2 can be implemented in the following way:

[0085] S21: In the Input part of the network structure. (1) First, perform Mosaic data augmentation. This algorithm scales, distributes, crops, and randomly stitches together the four input images, such as... Figure 3 As shown. This method makes the input images more diverse, and the trained model is more helpful in detecting small objects. At the same time, it reduces the amount of computation by processing only one image instead of four. (2) Then, an adaptive image scaling algorithm is used. This algorithm is used in testing and model inference, which greatly improves the inference speed of the algorithm, such as Figure 4 As shown. (3) Then perform the adaptive anchor box calculation algorithm, put the program corresponding to the initial anchor box value obtained by the test into the original code, and obtain the best anchor box value each time the dataset is trained.

[0086] S22: In the Backbone part of the network structure, within the Focus structure, the image is sliced, then concatenated, and finally convolved to form a easily processed feature map. This reduces floating-point operations and speeds up computation. For example, a 796×796×3 image becomes a 398×398×12 feature map after slicing, concatenation, and convolution. Then, it passes through the CSPNet structure, which primarily splits the feature map into two parts: one part undergoes convolution, and the other part is concatenated with the result of the convolution operation of the first part. This structure reduces computational cost, but the improvement in accuracy is minimal.

[0087] S23: In the Neck part of the network structure, an FPN-PAN structure was used. This structure performs multi-dimensional feature extraction, which greatly increases the receptive field, such as... Figure 5 As shown.

[0088] S24: In the head part of the network structure, the CIOU Loss loss function is used as the regression loss function for the bounding boxes. This loss function considers more comprehensive information and has better practical results. In the post-processing stage of object detection, non-maximum suppression is performed for the selection of many object boxes, such as... Figure 7 As shown. According to Figure 6 Formula 5 for calculating CIOU Loss is derived.

[0089]

[0090] In the formula, Distance_C represents the diagonal length of the smallest bounding rectangle.

[0091] Distance_2 — The length of the line connecting the center points of the two bounding boxes

[0092] IOU (Intersection over Union)

[0093] v — A parameter measuring aspect ratio consistency

[0094] S3: The weight file is used to perform object detection on the video frames, detecting fish target boxes in the scene. After processing by the Deepsort-based multi-object tracking algorithm and the visual tilt-based velocity measurement algorithm, the index and velocity of the fish target boxes can be obtained.

[0095] In this embodiment, step S3 can be implemented in the following way:

[0096] S31: The main steps of the multi-target tracking algorithm based on Deepsort: (1) First, use the trained fish classification weight file to detect fish targets in the video frame and obtain feature information. Here, our information includes the fish's location information, category information, and confidence information. (2) By calculating the matching degree of the image information of the two frames before and after, the target is matched, and then a sequence number is assigned to each tracked target, realizing the dynamic tracking of various fish. The Deepsort algorithm adds cascade matching and new trajectory confirmation, solving the problem of objects losing connection due to occlusion in the Sort algorithm. The trajectory is divided into confirmed trajectory and unconfirmed trajectory. The newly generated trajectory must be matched with the detector multiple times in a row before it can be converted from an unconfirmed trajectory to a confirmed trajectory; the confirmed trajectory will be deleted after losing connection with the detector multiple times in a row. The Deepsort algorithm workflow is as follows Figure 8 As shown.

[0097] S32: The core idea of ​​the speed measurement algorithm based on visual tilt is as follows. First, record the pixel coordinates (1920×1280 resolution): Record all fish ID numbers and center XY coordinates after detection and sorting by the Deepsort algorithm in a list. When tracking the same ID number at intervals of multiple video frames, calculate the pixel speed using formulas 6 and 7.

[0098] (xx -1 ) 2 +(yy -1 ) 2 =Distance 2 (Formula 6)

[0099]

[0100] In the formula, x and y are the coordinates of the position after the movement.

[0101] x -1 y -1 —Position coordinates before moving

[0102] Distance – Distance of movement

[0103] t — movement time

[0104] —Pixel speed

[0105] The speed at which a fish moves in a video frame, or the number of pixels it moves per unit time, is called pixel velocity. Pixel velocity differs from actual velocity, requiring analysis of the physical model and the tilt angle of the camera lens. In the same amount of time, fish swimming the same distance will pass through more pixels closer to the camera, while fish swimming closer to the camera will pass through fewer pixels. Therefore, the former needs a scaling factor to reduce its size, and the latter needs a scaling factor to enlarge it; however, this parameter is difficult to find manually. The best solution is to measure multiple sets of actual velocity and pixel velocity, and then use machine learning to solve the equations, as shown in Equation 8.

[0106]

[0107] In the formula, λ is a proportionality coefficient between the actual speed and the pixel speed.

[0108] s i —Based on multiple sets of actual speeds and pixel speeds, machine learning needs to solve for the parameters.

[0109] y - pixel coordinate on the vertical axis

[0110] n — An integer, which will be assigned a value based on the actual situation.

[0111] v—Actual speed

[0112] S4: Then, using a counting algorithm based on double convex rails, the number of fish entering and leaving is obtained.

[0113] In this embodiment, step S4 can be implemented in the following way:

[0114] S41: The core idea of ​​the counting algorithm based on double convex track measurement is as follows. (1) Draw two convex detection lines on the video: green line and yellow line, as shown. Figure 9 As shown. (2) The fish position coordinates are continuously compared with the coordinates of the two lines. When the coordinates of the monitoring point intersect the two convex detection lines, a count is performed. (3) If the fish position coordinates first cross the green line and then cross the yellow line, it is determined that the fish has entered the monitoring area, and the number of fish is increased by 1; if the fish position coordinates first cross the yellow line and then cross the green line, it is determined that the fish has left the monitoring area, and the number of fish is decreased by 1.

[0115] After the above steps, the system test results are output, such as... Figure 10 As shown.

[0116] It should be stated that, depending on the needs of implementing the method, the various steps described in this application can be broken down into more steps, or two or more steps or parts of steps can be combined into new steps to achieve the purpose of this invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for fish identification, counting, speed measurement and early warning, and entry / exit quantity statistics, characterized in that, include: S1: Collect videos or pictures of 5 kinds of ornamental fish as a dataset, use the Sprite Annotation Assistant to annotate the coordinates of the fish in the pictures, and then generate an annotation file; Use a Python script to divide the labeled dataset into a training set and a validation set; S2: Put the dataset into the YOLOv5 network structure for training. After training, save the trained weight file. S3: Use the weight file to perform target detection on the video frames and detect the fish target boxes in the picture; after processing by the Deepsort-based multi-target tracking algorithm and the visual tilt-based velocity measurement algorithm, the sequence number and velocity of the fish target boxes are obtained. S4: Then, using a counting algorithm based on double convex rails, the number of fish entering and leaving is obtained; S5: Output system test results.

2. The method for fish identification, counting, speed measurement and early warning, and entry / exit quantity statistics according to claim 1, characterized in that, Step S1 includes: S11: Record videos and photos of ornamental fish using a camera, and search for videos or photos of ornamental fish online. Use these videos and photos as the training dataset. Press and hold the Windows key and R key on your keyboard, type cmd, open the command line window, and type pip install ffmpeg to install the command-line tool package ffmpeg. After installing ffmpeg, use ffmpeg in the command line to convert the video into an image dataset according to a certain sampling rate. Use commands such as: ffmpeg -i test.mp4 -r 5 -f image2 .\output\frame_%05d.jpg, where -i is followed by the video file with the .mp4 extension, -r is followed by the sampling rate (how many images to split per second of video), and -f is followed by the image type. Finally, specify the folder and naming format for the output images, where %05d indicates a 5-digit sequence number, and the image names are recursively named from frame_00000.jpg, frame_00001.jpg, frame_00002.jpg, and so on. S12: Use the dataset labeling software, Wizard Labeling Assistant, to label the dataset. First, define the types of ornamental fish. This system uses five types of ornamental fish: clownfish, yellow boxfish, schooling banner fish, emperor angelfish, and yellow tang fish. Then, draw a box around each ornamental fish appearing in each image, selecting one species for each box, such as one of the five mentioned above. After all the labels are completed, set the export path and export format. Here, the export format is selected as pascal-voc, generating 5228 labeled files with the .xml extension from 5228 images. S13: Use Python scripts for batch conversion; import the relevant library files using import os and importxml.etree.ElementTree to convert the pixel coordinates in the xml file to the normalized coordinates in the txt file. Fish species are represented by the five numbers 0, 1, 2, 3, and 4. The conversion formulas are shown in Formula 1, Formula 2, Formula 3, and Formula 4. (Official 1) (Official 2) (Official 3) (Official 4) In the formula — Coordinates of the center point of the normalized box —The width and height of the normalized box — Coordinates of the top left corner of the box — Coordinates of the bottom right corner of the box — Width and height of the entire image 3. The method according to claim 2, characterized in that, Step S2 includes: S21: In the Input part of the network structure: (1) Mosaic data augmentation is performed first. This algorithm scales, distributes, and crops four input images and splices them randomly. (2) Then an adaptive image scaling algorithm is performed. This algorithm is used when testing and using the model for inference. (3) Then an adaptive anchor box calculation algorithm is performed. The program corresponding to the initial anchor box value obtained from the test is put into the original code. S22: In the Backbone part of the network structure, in the Focus structure, the image is sliced, then concatenated, and finally transformed into a feature map that is easy to process. For example, a 796×796×3 image becomes a 398×398×12 feature map after slicing, stitching, and convolution. Then it goes through the CSPNet structure, which mainly splits the feature map into two parts. One part is convolved, and the other part is stitched together with the result of the convolution operation of the previous part. S23: In the Neck part of the network structure, an FPN-PAN structure is used; S24: In the Head part of the network structure, the CIOU Loss loss function is used as the regression loss function for the bounding boxes; in the post-processing stage of object detection, non-maximum suppression is performed for the selection of many object boxes; CIOU Loss is calculated using Equation 5. (Official 5) In the formula, Distance_C represents the diagonal length of the smallest bounding rectangle. Distance_2 — The length of the line connecting the center points of the two bounding boxes IOU (Intersection over Union) v — A parameter measuring aspect ratio consistency 4. The method according to claim 3, characterized in that, Step S3 includes: S31: The main steps of the multi-target tracking algorithm based on Deepsort are: (1) First, the fish targets in the video frame are detected using the trained fish classification weight file to obtain feature information, which includes the fish's position information, category information and confidence information, etc.; (2) The target is matched by calculating the matching degree of the image information of the two frames before and after, and then a sequence number is assigned to each tracked target; (3) The trajectory is divided into confirmed trajectory and unconfirmed trajectory: the newly generated trajectory is continuously matched with the detector multiple times, and the unconfirmed trajectory is converted into a confirmed trajectory; the confirmed trajectory will be deleted if it loses contact with the detector multiple times. S32: The core idea of ​​the speed measurement algorithm based on visual tilt is as follows: (1) Pixel coordinate record (1920×1280 resolution): The list of all records of fish ID numbers and center XY coordinates after detection and sorting by the Deepsort algorithm; (2) When multiple video frames are separated, the same ID number is tracked and the pixel velocity is calculated. The calculation formulas are Formula 6 and Formula 7. (Official 6) (Official 7) In the formula —Position coordinates after movement —Position coordinates before moving —Distance of movement ——Movement Time —Pixel speed (3) The pixel speed is the number of pixels that move per unit time. The pixel speed is different from the actual speed. It is necessary to analyze the tilt angle between the physical model and the camera lens. (4) Measure multiple sets of actual speeds and pixel speeds, and use machine learning to solve the equations, as shown in Formula 8. (Official 8) In the formula — A scaling factor between actual speed and pixel speed —Based on multiple sets of actual speeds and pixel speeds, machine learning needs to solve for the parameters. — Vertical axis pixel coordinates — An integer, which will be assigned a value based on the actual situation. ——Actual speed 5. The method according to claim 4, characterized in that, Step S4 includes: S41: The core idea of ​​the counting algorithm based on double convex track measurement is as follows: (1) Draw two convex detection lines on the video: green line and yellow line; (2) Continuously compare the fish position coordinates with the coordinates of the two lines. When the monitoring point coordinates intersect with the two convex detection lines, count them; (3) If the fish position coordinates pass through the green line first and then through the yellow line, it is determined that the fish has entered the monitoring area, and the number of fish is increased by 1; if the fish position coordinates pass through the yellow line first and then through the green line, it is determined that the fish has left the monitoring area, and the number of fish is decreased by 1.

Images