A machine vision-based migratory fish school counting and identifying method

Abstract

The application discloses a kind of machine vision-based migratory fish school counting identification method, utilize yolov8 as backbone, create dve-yolo structure, modify the sample allocation strategy and loss function of yolov8 to adapt to dve-yolo structure.The structure uses two continuous pictures as input, and the anchor frame combination of the corresponding object before and after as output, which can better obtain the dynamic characteristics of moving objects in the picture, clearly reflect the relationship between the objects in the two pictures, and make the network have dynamic vision.As it can obtain the state information before and after simultaneously, it does not need the traditional Kalman filter for state estimation, so it can create a simpler tracking counting logic.The application can accurately identify whether the fish school is migrating, has low cost, saves labor, can work 24 hours without interruption, and has high application value.

Description

Technical Field

[0001] This invention relates to the field of methods for counting and identifying migratory fish schools, specifically a machine vision-based method for counting and identifying migratory fish schools. Background Technology

[0002] Counting and identifying fish during migration reflects changes in fish populations and is of great significance for agricultural production and environmental management. Traditional fish counting methods primarily rely on the mark-and-recapture method, which involves marking, counting, and releasing fish caught manually. A second catch is conducted after a period of time to count the number and proportion of marked fish, thus roughly estimating the total number of fish in the school. This method can directly calculate the total number of fish; and when the second catch is sufficiently large, according to Chebyshev's law of large numbers, the proportion of marked fish will approach the true value infinitely, thus theoretically providing a highly accurate total number of fish. However, traditional methods require extensive manual labor, incurring significant time and financial costs. They are unsuitable for complex environments and cannot perform continuous counting over a period of time. Furthermore, the marking method can cause some harm to the fish. Summary of the Invention

[0003] This invention provides a machine vision-based method for counting and identifying migratory fish schools, which solves the problems of existing migratory fish school tag-recapture counting methods being unable to adapt to complex environments and unable to continuously achieve migratory counting.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A machine vision-based method for counting and identifying migratory fish schools includes the following steps:

[0006] Step 1: Obtain raw video data of the river surface monitoring over a period of time, and extract multiple frames of image data based on the raw video data;

[0007] Step 2: Use a Siamese DVE-YOLO network. The DVE-YOLO network takes two images from consecutive frames as input and uses the Siamese DVE-YOLO network to detect, extract features and match fish targets.

[0008] The image data obtained in step 1 is divided into a training set and a test set. The Siamese DVE-YOLO network is trained using the image data in the training set. The trained Siamese DVE-YOLO network is then used to process the image data in the test set to complete the fish target detection.

[0009] Step 3: Based on the output of the trained Siamese DVE-YOLO network in Step 2, fish target tracking is achieved, thereby obtaining the trajectory of the fish target from appearance to disappearance in the original video data, and the trajectory of the fish target is cumulatively counted, which is the number of fish swimming through the river in the corresponding time period of the original video data.

[0010] Step 4: Divide the original video data into multiple time periods, calculate the number of fish in each time period, and compare the number of fish in each time period with a preset threshold. If the number of fish in a certain time period is greater than the preset threshold, it is determined that there is migration in that time period; otherwise, it is determined that there is no migration in that time period.

[0011] In the further step 2, the structure of the twin DVE-YOLO network is a detection network YOLOv8, which includes two image feature extractors, YOLOv8's backbone and DVE head. The two image feature extractors, YOLOv8's backbone, are twin networks and share weight parameters.

[0012] The basic features of two adjacent frames in the YOLOv8 detection network are extracted by the two image feature extractors in the YOLOv8 backbone. The extracted feature information is fed into the DVE head in the YOLOv8 detection network for fusion. Finally, the DVE head combines the anchor boxes of the corresponding targets in the two images according to the output format as the output, thereby completing the detection, feature extraction and matching of the fish target.

[0013] Furthermore, in the detection network YOLOv8, the Conv module of each image feature extractor YOLOv8's backbone uses the SiLU activation function, as shown in Equation (1).

[0014]

[0015] In formula (1): x represents the numerical input of each element of the neural network tensor.

[0016] Furthermore, when the DVE head of the YOLOv8 detection network performs feature fusion, it concatenates the feature information of the two images at different scales, and performs object category inference and anchor box information inference on the information at the same scale, thereby obtaining the anchor boxes of the corresponding targets in the two images, and obtaining the output format of the anchor box combination.

[0017] Furthermore, to match the output format of the anchor box combination obtained from the DVE head, the sample allocation strategy of the Siamese DVE-YOLO network is designed as follows:

[0018] Before sample allocation, the bounding box information output by the dve head needs to be decoded. The decoding process is shown in formula (2):

[0019]

[0020] The 128-bit anchor frame information is divided into 8 groups, with each group containing 16 bits corresponding to a coordinate value; δx is the deviation of the detected fish's position coordinates from the anchor point; encoding(δx) T δx is the undecoded vector, a 16×1 vector; stride is the anchor spacing; softmax represents the softmax function commonly used in machine learning.

[0021] The final coordinate values ​​are obtained by formula (3), which is as follows:

[0022] x=δx+anchor (3)

[0023] Here, anchor is a set of coordinates evenly distributed on the pixel plane, plus offset δx to calculate the actual coordinate position of the detected fish;

[0024] After decoding according to formula (2), substitute it into formula (3) to output the position information of two anchor frames with corresponding relationship;

[0025] When allocating samples, since the DVE-YOLO network will output 8400 predicted anchor boxes, and an image contains n target anchor boxes, n×8400 corresponding score values ​​are calculated. The top k predicted boxes with the highest scores are regarded as positive samples, and the other predicted boxes with low scores are regarded as negative samples. The score calculation method is shown in formula (4):

[0026] score = class_score α ·(0.5*CIou1 β +0.5*CIou2 β (4)

[0027] In formula (4), score, class_score, CIou1, CIou2∈R n×8400 ; class_score refers to the confidence level of the corresponding target class; CIou1 and CIou2 represent the overlap between the two predicted anchor boxes and the target; α and β are adjustable hyperparameters; R refers to the real number domain;

[0028] As can be seen from formula (4), the higher the confidence level of the target type in the predicted value and the greater the overlap of the anchor frames, the easier it is to be considered a positive sample.

[0029] Furthermore, in order to match the output format of the anchor box combination obtained from the DVE head, the loss function of the Siamese DVE-YOLO network is designed as shown in Equation (5):

[0030] L=w1·iou_loss+w2·cls_loss+w3·dfl_loss (5)

[0031] In formula (5), L is a multi-objective optimization of anchor frame overlap, type, and anchor frame coordinates, which requires multiplying the three loss functions by the weights w1, w2, and w3 respectively; iou_loss is the loss caused by insufficient overlap between the predicted anchor frame and the target anchor frame; cls_loss is the loss caused by the mismatch between the predicted object type and the target object type; and dfl_loss is specifically for the loss caused by the coordinate error of the upper left and lower right corners of the anchor frame.

[0032] Among them, cls_loss is used to calculate class loss in the form of cross-entropy;

[0033] The expression for iou_loss is shown in formula (6):

[0034] iou_loss=sum(1-0.5Ciou1-0.5Ciou2) (6)

[0035] In formula (6), iou_loss∈R 1 ;Ciou1∈R 8400×1 Ciou2∈R 8400×1 Ciou1 and Ciou2 are the Ciou values ​​between the 8400 predicted boxes and their assigned samples, respectively; sum represents the summation of Ciou1 and Ciou2 over the first dimension.

[0036] The expression for dfl_loss is shown in equation (7):

[0037]

[0038] In formula (7), dfl_loss∈R 1 ;x i ∈R 8400×16 target_x i ∈R 8400×1 ;y i ∈R 8400×16 target_y i ∈R 8400×16 ;x i and y i These are the direct outputs of the DVE head before decoding;

[0039] The DFL function in formula (7) is in the form of formula (8):

[0040]

[0041] In formula (8), y k It is the largest integer tensor less than y; y k+1 It is the smallest integer tensor greater than y, with the same dimension as y; CrossEntropy represents the cross-entropy function commonly used in machine learning.

[0042] In the further step 3, whenever a new detected object appears, the dve-yolo tracking strategy needs to perform two steps: First, match the newly detected position at time t with the position at time t of the existing trajectory. If the distance between the two is less than a threshold, then update the existing trajectory with the position at time t+1 of the new trajectory so that the existing trajectory includes the position at time t+1. If there is no existing trajectory that matches the new detected object, then create a new trajectory.

[0043] After the above matching process, the trajectories that have not been matched with new objects need to be updated. If the number of unmatched times exceeds the threshold, the object corresponding to the trajectory is considered to have disappeared and the trajectory is deleted.

[0044] Compared with the prior art, the advantages of the present invention are:

[0045] This invention employs computer vision and deep learning algorithms to propose a machine vision-based method for counting and identifying migratory fish schools. Based on fishermen's experience, video monitoring is conducted at specific time periods and locations where fish have historically migrated. The method counts fish schools exhibiting abnormally large clustering patterns at those times and locations, and determines the duration of such clustering in the vicinity. A DVE-YLO architecture is created using YOLOv8 as the backbone, and the sample allocation strategy and loss function of YOLOv8 are modified to adapt to the DVE-YLO architecture. This architecture uses two consecutive images as input and the anchor boxes of corresponding objects in the preceding and following images as output. This allows for better capture of the dynamic features of moving objects in the images, clearly reflecting the relationship between objects in the two images, thus giving the network dynamic vision.

[0046] Because it can simultaneously acquire both preceding and following state information, it eliminates the need for traditional Kalman filters for state estimation, thus enabling the creation of simpler tracking and counting logic. This counting method can accurately identify whether a school of fish is migrating. It boasts advantages such as low cost, labor savings, 24 / 7 operation, and high speed, making it highly valuable for application. Attached Figure Description

[0047] Figure 1This is a schematic diagram of the fish migration counting and identification method according to an embodiment of the present invention.

[0048] Figure 2 This is the overall structure of the twin DVE-YOLO in the embodiments of the present invention.

[0049] Figure 3 This is the yolov8 backbone structure in an embodiment of the present invention, where b represents the batch_size.

[0050] Figure 4 The diagram shows the dve head structure in this embodiment of the invention. In the diagram, c represents the number of tracked objects. conv1x1 uses a 1x1 size convolution kernel and has no activation function.

[0051] Figure 5 This is the output format after decoding in this embodiment of the invention.

[0052] Figure 6 The following is a flowchart of the tracking strategy in this embodiment of the invention: (1) is the update of the existing trajectory by the newly detected object, and (2) is the deletion of the trajectory that has not been updated for a long time. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] like Figure 1 As shown, this embodiment discloses a machine vision-based method for counting and identifying migratory fish schools, including the following steps:

[0055] Step 1: Obtain raw video data of the river surface monitoring over a period of time, and extract multiple frames of image data based on the raw video data.

[0056] In this embodiment, a fixed camera at a dam in Anji County, Zhejiang Province, is used as an example. This camera is aimed at an area near the annual fish migration locations, thus obtaining raw video data. Based on the raw video data, a dataset of 1350 images captured by the camera is extracted. Each set of images contains two frames, resulting in a total of 2700 images. 80% (1080 sets) of these 1350 images are used for training, and 20% (270 sets) are used for testing. The dataset contains both positive samples with fish images and negative samples without fish images.

[0057] Step 2: Employ a Siamese DVE-YOLO network. The DVE-YOLO network takes two adjacent frames as input and uses them to detect, extract features, and match fish targets. The process is as follows:

[0058] First, construct the twin DVE-YOLO network structure, as shown in the DVE-YOLO network structure. Figure 2 As shown, the state-of-the-art detection network YOLOv8 is used. The YOLOv8 detection network consists of two image feature extractors: the YOLOv8 backbone and the DVE head. The two YOLOv8 backbones are twin networks that share weight parameters.

[0059] The YOLOv8 detection network uses two image feature extractors, the YOLOv8 backbone, to extract the basic features of two adjacent frames. The extracted feature information is fed into the DVE head of the YOLOv8 detection network for fusion. Finally, the DVE head combines the anchor boxes of the corresponding targets in the two images into the output according to the output format. This output format not only realizes the detection of the target object, but also reflects the correspondence between objects in the preceding and following frames, thus completing the integrated detection, feature extraction and matching of the fish target.

[0060] The advantage of using the Siamese DVE-YOLO network in this embodiment is that it does not require generating additional training parameters and has consistency in the extraction of image features, so that feature fusion can be performed in deep networks.

[0061] The dve head is the core part of dve-yolo. The specific structure of the dve head is as follows: Figure 4 As shown, when the DVE head performs feature fusion, it first concatenates the feature information of the two images at different scales, and then performs object category inference and anchor box information inference on the information at the same scale, thereby obtaining the anchor boxes of the corresponding targets in the two images, and obtaining the output format of the anchor box combination. In the fish counting application, there is only one type of object, fish, so c is 1; the anchor box information is 128 bits in total. Because the combination of the front and rear anchor boxes needs to be output, the number of bits of the anchor box information here is twice that of the original YOLOv8.

[0062] The Siamese DVE-YOLO network is trained using the training set image data from step 1. Before training, the pre-trained parameters of the YOLOv8 backbone for each image feature extractor are loaded. The specific structure of the YOLOv8 backbone used by DVE-YOLO is as follows: Figure 3As shown in the figure, the tensor dimensions marked in the figure are those of the YOLOv8n version. The depth and breadth of the Conv and C2f modules are different in different YOLOv8 versions, resulting in changes in the output tensor dimensions, and different trainable parameters and learning capabilities of the network. YOLOv8's backbone is a single-input, three-output structure. After inputting an image, it outputs object feature information and location information at three scales: large, medium, and small. Conv, C2f, and SPPF are all custom modules of YOLOv8. The Conv module of the image feature extractor YOLOv8's backbone uses the SiLU activation function. The expression of the SiLU activation function is shown in formula (1).

[0063]

[0064] In formula (1): x represents the numerical input of each element of the neural network tensor.

[0065] The training process uses stochastic gradient descent (SGD) with a learning rate of 0.001 and a maximum training duration of 500 epochs. Training is considered complete and stopped if the loss function does not decrease within 50 epochs. After training, the model with the best test performance (highest AP50:95) will be saved.

[0066] In this embodiment, to match the output format of the anchor box combination obtained from the DVE head, the sample allocation strategy process of the Siamese DVE-YOLO network is designed as follows:

[0067] Before sample allocation, it is necessary to first... Figure 4 The bounding box information output by dve-yolo is decoded, and the decoding process is shown in formula (2):

[0068]

[0069] The 128-bit anchor frame information is divided into 8 groups, with each group containing 16 bits corresponding to a coordinate value; δx is the deviation of the detected fish's position coordinates from the anchor point; encoding(δx) T δx is the undecoded vector, a 16×1 vector; stride is the anchor spacing; softmax represents the softmax function commonly used in machine learning.

[0070] The final coordinate values ​​are obtained by formula (3), which is as follows:

[0071] x=δx+anchor (3)

[0072] Here, anchor is a set of coordinates evenly distributed on the pixel plane, and the offset δx is used to calculate the actual coordinate position of the detected fish.

[0073] After decoding according to formula (2), substitute the result into formula (3). The output format after decoding is as follows: Figure 5 As shown, the output shows the position information of two anchor boxes with a corresponding relationship (Yolov8 itself only outputs one anchor box).

[0074] refer to Figure 4 In the output tensor dimension, when allocating samples, since the DVE-YLO network will output 8400 predicted anchor boxes for a set of images (2 images), and an image may contain n target anchor boxes, n×8400 corresponding score values ​​are calculated. The top k predicted boxes with the highest scores are regarded as positive samples, and the other predicted boxes with low scores are regarded as negative samples. The score calculation method is shown in formula (4):

[0075] score = class_score α ·(0.5*CIou1 β +0.5*CIou2 β (4)

[0076] In formula (4), score, class_score, CIou1, CIou2∈R n×8400 R is the real number field. `class_score` refers to the confidence level for the corresponding target class.

[0077] CIou1 and CIou2 represent the overlap between the two predicted anchor boxes and the target. CIou is a commonly used method for calculating anchor box overlap in machine learning.

[0078] α and β are adjustable hyperparameters. In this embodiment, α = 1.0 and β = 6.0.

[0079] As can be seen from formula (4), the higher the confidence level of the target type in the predicted value and the greater the overlap of the anchor frames, the easier it is to be considered a positive sample.

[0080] In this embodiment, in order to match the output format of the anchor box combination obtained by the DVE head, the loss function of the Siamese DVE-YOLO network is designed as shown in formula (5):

[0081] L=w1·iou_loss+w2·cls_loss+w3·dfl_loss (5)

[0082] In formula (5), L is a multi-objective optimization of anchor frame overlap, type, and anchor frame coordinates, which requires multiplying the three loss functions by the weights w1, w2, and w3 respectively; iou_loss is the loss caused by insufficient overlap between the predicted anchor frame and the target anchor frame; cls_loss is the loss caused by the mismatch between the predicted object type and the target object type; and dfl_loss is specifically designed to address the loss caused by the coordinate errors of the upper left and lower right corners of the anchor frame, thereby improving the accuracy of the anchor frame position.

[0083] Among them, cls_loss calculates the class loss in the most commonly used form of cross-entropy, and the dve-yolo network is not modified.

[0084] The expression for iou_loss is shown in formula (6):

[0085] iou_loss=sum(1-0.5Ciou1-0.5Ciou2) (6)

[0086] In formula (6), iou_loss∈R 1 ;Ciou1∈R 8400×1 Ciou1∈R 8400×1 Ciou1 and Ciou2 are the Ciou values ​​between the 8400 prediction boxes and their assigned samples, respectively; sum represents the summation of Ciou1 and Ciou2 over the first dimension.

[0087] The expression for dfl_loss is shown in equation (7):

[0088]

[0089] In formula (7), dfl_loss∈R 1 ;x i ∈R 8400×16 target_x i ∈R 8400×1 ;y i ∈R 8400×16 target_y i ∈R 8400×16 ;x i and y i These are the direct outputs of the dve head before decoding.

[0090] The DFL function in formula (7) is in the form of formula (8):

[0091]

[0092] In formula (8), y k It is the largest integer tensor less than y; y k+1It is the smallest integer tensor greater than y, with the same dimension as y; GrossEntropy represents the cross-entropy function commonly used in machine learning.

[0093] In this embodiment, after training, the trained Siamese DVE-YOLO network is used to process image data in the test set to complete fish target detection. Detection performance is combined using a comprehensive index, AP, which measures precision (P) and recall (R). Since different confidence thresholds are selected for predicting positive samples, there will be different sets of P and R values. The AP value is obtained by integrating the curve on the PR axis with the region enclosed by the coordinate axis. Changing the IOU threshold for predicting positive samples will produce different AP values: AP50 represents an IOU threshold of 0.5 for predicted positive samples; AP75 represents an IOU threshold of 0.75; and AP50-95 is a weighted average of AP values ​​with IOU thresholds ranging from 0.5 to 0.95.

[0094] Step 3: Based on the output of the trained Siamese DVE-YOLO network from Step 2, fish target tracking is achieved. This yields the trajectory of the fish target from its appearance to its disappearance in the original video data. The DVE-YOLO network and its tracking strategy are then used to cumulatively count the fish target's trajectory, which represents the number of fish that swam across the river during the corresponding time period in the original video data. Since the DVE-YOLO network can output the positions of two objects simultaneously, this embodiment does not require a Kalman filter for state estimation.

[0095] like Figure 6 As shown, whenever a new detected object appears, the dve-yolo tracking strategy needs to perform two steps: First, match the newly detected position at time t with the position at time t of the existing trajectory. If the distance between the two is less than a threshold, then update the existing trajectory with the position at time t+1 of the new trajectory so that the existing trajectory includes the position at time t+1. If there is no existing trajectory that matches the newly detected object, then create a new trajectory.

[0096] After the above matching process, the trajectories that have not been matched with new objects need to be updated. If the number of unmatched times exceeds the threshold, the object corresponding to the trajectory is considered to have disappeared and the trajectory is deleted.

[0097] Step 4: Divide the original video data into multiple time periods, calculate the number of fish in each time period, and compare the number of fish in each time period with a preset threshold. If the number of fish in a certain time period is greater than the preset threshold, it is determined that there is migration in that time period; otherwise, it is determined that there is no migration in that time period.

[0098] Although dve-yolo can distinguish between floating objects and fish, and its detection performance is stronger than yolov8, false positives still occur (due to limitations in detection performance). On the one hand, floating aquatic plants and water surface reflections may be falsely identified as positive samples (false positives); on the other hand, fish may not be detected when they swim by (false negatives). Therefore, in this embodiment, the comparison between the number of fish counted within a set time period and a preset threshold is used to determine whether migration has occurred. By updating the migration status at a set time period, the migration behavior of fish schools can be effectively identified.

[0099] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. These embodiments are merely descriptions of preferred embodiments and are not intended to limit the scope or concept of the invention. The specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. Such combinations, as long as they do not violate the spirit of the present invention, should also be considered as part of this disclosure. To avoid unnecessary repetition, the present invention will not further describe the various possible combinations.

[0100] This invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this invention and without departing from the design idea of ​​this invention, all modifications and improvements made by those skilled in the art to the technical solutions of this invention should fall within the protection scope of this invention. The technical content for which protection is sought in this invention has been fully described in the claims.

Claims

1. A machine vision-based method for counting and identifying migratory fish schools, characterized in that, Includes the following steps: Step 1: Obtain raw video data of the river surface monitoring over a period of time, and extract multiple frames of image data based on the raw video data; Step 2: Use a Siamese DVE-YOLO network. The DVE-YOLO network takes two images from consecutive frames as input and uses the Siamese DVE-YOLO network to detect, extract features and match fish targets. The image data obtained in step 1 is divided into a training set and a test set. The Siamese DVE-YOLO network is trained using the image data in the training set. The trained Siamese DVE-YOLO network is then used to process the image data in the test set to complete the fish target detection. Step 3: Based on the output of the trained Siamese DVE-YOLO network in Step 2, fish target tracking is achieved, thereby obtaining the trajectory of the fish target from appearance to disappearance in the original video data, and the trajectory of the fish target is cumulatively counted, which is the number of fish swimming through the river in the corresponding time period of the original video data. Step 4: Divide the original video data into multiple time periods, calculate the number of fish in each time period, and compare the number of fish in each time period with a preset threshold. If the number of fish in a certain time period is greater than the preset threshold, it is determined that there is migration in that time period; otherwise, it is determined that there is no migration in that time period. In step 2, the structure of the twin DVE-YOLO network is a detection network YOLOv8. The detection network YOLOv8 includes two image feature extractors, YOLOv8's backbone and DVE head. The two image feature extractors YOLOv8's backbone are twin networks and share weight parameters. The basic features of two adjacent frames in the YOLOv8 detection network are extracted by the two image feature extractors in the YOLOv8 backbone. The extracted feature information is fed into the DVE head in the YOLOv8 detection network for fusion. Finally, the DVE head combines the anchor boxes of the corresponding targets in the two images according to the output format as the output, thereby completing the detection, feature extraction and matching of the fish target. When performing feature fusion in the DVE head of the YOLOv8 detection network, the feature information of the two images at different scales is concatenated, and object type inference and anchor box information inference are performed on the information at the same scale respectively. This yields the anchor boxes of the corresponding targets in the two images, and the output format of the anchor box combination is obtained.

2. The method for counting and identifying migratory fish schools based on machine vision according to claim 1, characterized in that, In the YOLOv8 detection network, the Conv module of each image feature extractor in YOLOv8's backbone uses... The activation function is expressed as shown in formula (1); ， In formula (1): x represents the numerical input of each element of the neural network tensor.

3. The method for counting and identifying migratory fish schools based on machine vision according to claim 1, characterized in that, To match the output format of the anchor box combination obtained from the DVE head, the sample allocation strategy of the Siamese DVE-YOLO network is designed as follows: Before sample allocation, the bounding box information output by the dve head needs to be decoded. The decoding process is shown in formula (2): , The 128-bit anchor frame information is divided into 8 groups, with each group consisting of 16 bits corresponding to a coordinate value. It is the deviation of the detected fish's position coordinates from the anchor point; It is undecoded , is a vector of shape 16×1; It refers to the anchor point interval; This represents the softmax function, which is commonly used in machine learning. The final coordinate values ​​are obtained by formula (3), which is as follows: ， in, It is a set of coordinates evenly distributed on the pixel plane, plus an offset. Used to calculate the actual coordinates of the detected fish; After decoding according to formula (2), substitute it into formula (3) to output the position information of two anchor frames with corresponding relationship; During sample allocation, since the dve-yolo network will output 8400 predicted anchor boxes, and a single image contains... There are several target anchor frames, so a one-to-one correspondence is calculated. Each score value will be assigned to the highest score value. One predicted box is considered a positive sample, and other predicted boxes with low scores are considered negative samples. The score is calculated as shown in formula (4): ， In formula (4) ; This refers to the confidence level for the corresponding target category; and This represents the degree of overlap between the two predicted anchor boxes and the target. and These are adjustable hyperparameters; R refers to the real number field. As can be seen from formula (4), the higher the confidence level of the target type in the predicted value and the greater the overlap of the anchor frames, the easier it is to be considered a positive sample.

4. The method for counting and identifying migratory fish schools based on machine vision according to claim 1, characterized in that, To match the output format of the anchor box combination obtained from the DVE head, the loss function of the Siamese DVE-YOLO network is designed as shown in Equation (5): ， In formula (5), It involves multi-objective optimization of anchor frame overlap, type, and anchor frame coordinates, requiring the three loss functions to be multiplied by... Weight; The loss is due to insufficient overlap between the predicted anchor frame and the target anchor frame. This refers to the loss caused by the mismatch between the predicted object type and the target object type. The loss function is specifically designed to address the losses caused by the coordinate errors of the upper left and lower right corners of the anchor frame; in, Calculate the class loss in the form of cross-entropy; The expression is shown in formula (6): ， In formula (6), , and These refer to the 8400 predicted bounding boxes and the samples they are assigned to. Value; sum represents the sum of values ​​along the first dimension. and Perform summation; The expression is shown in formula (7): ， In formula (7), ; , ; , ; and These are the direct outputs of the DVE head before decoding; In formula (7) The function is in the form of formula (8): ， In formula (8), It is less than The largest integer tensor; It is greater than The smallest integer tensor with and Same dimensions; This represents the cross-entropy function commonly used in machine learning.

5. The method for counting and identifying migratory fish schools based on machine vision according to claim 1, characterized in that, In step 3, whenever a new detected object appears, the dve-yolo tracking strategy needs to perform two steps: First, match the newly detected position at time t with the position at time t of the existing trajectory. If the distance between the two is less than a threshold, then update the existing trajectory with the position at time t+1 of the new trajectory so that the existing trajectory includes the position at time t+1. If there is no existing trajectory that matches the new detected object, then create a new trajectory. After the above matching process, the trajectories that have not been matched with new objects need to be updated. If the number of unmatched times exceeds the threshold, the object corresponding to the trajectory is considered to have disappeared and the trajectory is deleted.

Images