[0055] The present invention will be further described below in conjunction with the drawings.
[0056] Reference Figure 1 ~ Figure 6 , A fish school target tracking method based on selective attention, including the following steps:
[0057] 1) Taking crucian carp school (4 to 8) as the monitoring object, real-time video of fish school behavior is obtained through the camera, so as to realize real-time monitoring of each individual in the group ( figure 1 ). First, take tracking one of the target fish in the fish school as an example to describe the algorithm flow.
[0058] 2) The algorithm is initialized, and the initial contour of the target fish is manually obtained and the field of view of the fish is tracked.
[0059] 2.1) Get the initial contour f of the target fish 0. First, pause the current video stream to obtain the outline of the target fish in the current frame. To this end, the target fish of the current frame is manually found, and any point of the target fish body is clicked by the mouse to obtain the coordinates of the click. According to the color (RGB, red, green, and blue) value of the coordinate point, the nearest neighbor method is used to find the outline of the entire target fish (Figure 4-a). In the present invention, the difference between all pixels and the color R of the mouse point is within (-30, 30), and the color difference of G and B is also within (-30, 30), then the mouse point and this pixel The points are connected to find the outline of the target fish.
[0060] 2.2) Via f 0 Get the initial attention area r of the target fish 0. First, get f 0 , And then calculate the center point of each boundary point (c 00 +c 01 +...c 0n )/n, where c 0n Means f 0 The nth boundary pixel of. Then, use the dilate function in opencv to expand the contour of the target fish to four times the original contour with the center point as the center, and use this size as the boundary point of the attention area. The range formed by the boundary points of each attention area is defined as the attention area r 0. The position of the target fish in the next frame is determined from the attention area.
[0061] 3) Since the fish is swimming non-stop, tracking the contour of the target fish at the current moment requires the use of the position information of the target fish at the previous moment. In addition, as the school of fish swims, the fish bodies will inevitably cross each other. Therefore, the identification of the target fish in the current frame may be misaligned. In other words, other fish in the school will be mistaken for the target fish. For this reason, the present invention proposes a tracking method based on a visual attention model, so that the target fish can be accurately tracked when a school of fish is staggered.
[0062] According to the visual selective attention mechanism, the present invention abstracts target tracking into two key calculations. The first calculation is to determine the range of selective attention, and the second calculation is the tracking target of the selected attention range. The flow of these two calculations is:
[0063] B1) Initialize the initial contour f of the target fish 0 And the initial attention area r 0.
[0064] B2) Calculate the attention area r of the target fish at the current moment t , R t =U(f t-1 ), where f t-1 Is the contour of the target fish at the previous moment, U is the function used in the first calculation, the calculation process of this function is described in 3.1), and the flow is shown in Figure 5.
[0065] B3) Calculate the contour f of the target fish at the current moment t ,f t =S(r t ), where r t Is the target fish's attention area calculated at time t, S is the function used in the second calculation, see 3.2) for the description of the function calculation process, see the flow Figure 5.
[0066] B4) Loop B2) and B3) to realize the tracking of the target fish.
[0067] 3.1) U function is to get a new attention area according to the current fish contour. Because the fish's swimming speed is constant, the displacement between each frame generally does not exceed 10 pixels. Therefore, using the dilate function in opencv, the target fish contour is expanded to four times the original contour with the center point of the target fish as the center, and this size is used as the boundary point of the attention area.
[0068] 3.2) The S function is based on the attention area r of the target fish t , The outline of the target fish is found f t. First in r t The nearest neighbor algorithm is used to find all the fish contours in the area (B t ). It should be noted that area(B t ) In addition to target fish, non-target fish may also be included. Therefore, the present invention proposes that the target fish profile f t-1 Exclude all the contours that do not belong to the target fish, and what remains is the contour of the fish at the current moment f t.
[0069] 3.2.1) In the attention area r t The nearest neighbor algorithm is used for clustering, and the fish contour area (B t ), the process is as follows:
[0070] C1) Randomly select k (=5) center points, that is, the number of clustering categories;
[0071] C2) Traverse the color of all pixels, that is, the RGB value. Divide each pixel to the nearest center point;
[0072] C3) Calculate the average value of each category after clustering and use it as the new center point;
[0073] C4) Repeat the C2-C3 process until the k center points no longer change (converge), or enough iteration steps are performed.
[0074] Because the fish are similar in color. Therefore, among the k classes after clustering, one class must belong to fish (Figure 3, the white area is the effective class).
[0075] However, in these k categories, which category belongs to the fish area still needs to be further calculated (Figure 4-b). The calculation process is as follows:
[0076] D1) Obtain the binary image of the target fish at t-1, the pixel value of the target fish is 1, and the others are 0;
[0077] D2) Recalculate the similarity between the binary image of each category and the binary image of the target fish at t-1, and take the most similar category as the fish contour.
[0078] 3.2.2) As fish schools are swimming, they inevitably cross each other. Therefore, area(B t In addition to the target fish being tracked in ), non-target fish may also be included. Therefore, the target fish profile f t-1 Exclude all contours that do not belong to the target fish. The process is as follows:
[0079] E1) Get fish f at time t-1 t-1 The center point c 1;
[0080] E2) Get fish outline B t The center point c 2;
[0081] E3) Connection center point c 1 And center point c 2 Get direct L;
[0082] E4) It will move along L, and stop moving when its area and area overlap the most;
[0083] E5) The area with the largest overlap area is the target fish contour range at the current time t.
[0084] 4) Tracking of multiple fishes: First, manually obtain the initial contours of multiple target fishes, and then track each target fish separately according to steps 2) and 3) to achieve multiple fish tracking at the same time.
