High-density fish fry counting method and system based on image edge curvature

Abstract

The present invention belongs to the field of metrology and relates to a fish fry counting method and counting system, comprising: 1) placing an unknown number of fish fry into a fry box; 2) collecting image information of the fish fry; 3) performing background removal processing on the image information; 4) obtaining a single connected domain, extracting edge features of the single connected domain, and obtaining a closed edge image with a width of a single pixel; skeletonizing the single connected domain to obtain a skeleton graph of the single connected domain; 5) obtaining a position sequence of the closed edge image; 6) performing filtering and denoising processing on the position sequence; 7) obtaining a curvature signal; 8) filtering the curvature signal, marking and extracting the positions of concave point clusters; 9) drawing a segmentation line within the single connected domain based on the skeleton graph of the single connected domain and the positions of the concave point clusters; and 10) counting the number of fish fry. The present invention effectively solves the problems of high density and multiple overlap when counting fish fry, ensuring accurate and reliable counting results.

Description

Technical Field

[0001] The invention belongs to the field of measurement, and relates to a fish fry counting method and a counting system, and in particular to a high-density fish fry counting method and a counting system based on image edge curvature. Background Art

[0002] In aquaculture, accurate counting of fry is a crucial requirement for fry marketing. Traditional counting methods, such as cupping, sieving, and weighing, are often manual. These methods are inefficient, inaccurate, and can cause significant damage to fry.

[0003] Many researchers are currently working on faster and more accurate methods for counting fry. For example, patent application publication number CN113240650A discloses a fry counting system and method based on deep learning density map regression. This system combines the model with embedded devices to ensure over 90% accuracy. Another example is patent application publication number CN110973036A, which discloses a machine vision-based fry counting device and method. By controlling a revolving door, fry are counted in batches, effectively avoiding duplicate or missed counts caused by continuous image capture. Furthermore, patent application publication number CN109509175A discloses a portable fry counter based on machine vision and deep learning, enabling rapid, lossless, and accurate fry counting with an error margin of less than 5%. However, these technologies neither fundamentally address the problem of accurately counting small fry nor are they applicable to situations where dense fry images overlap. Summary of the Invention

[0004] In order to solve the above technical problems existing in the background technology, the present invention provides a high-density fish fry counting method and system based on image edge curvature, which can effectively solve the problems of high density and multiple overlap when counting fish fry, and can ensure accurate and reliable counting results.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A high-density fish fry counting method based on image edge curvature is characterized in that the high-density fish fry counting method based on image edge curvature comprises the following steps:

[0007] 1) Place an unknown number of fry into the fry box;

[0008] 2) using a camera to collect image information of the fry in the fry box;

[0009] 3) performing background removal processing on the image information obtained in step 2) to obtain a processed fry image;

[0010] 4) obtaining a single connected domain in the processed fry image, extracting edge features of the single connected domain, and obtaining a closed edge image with a width of a single pixel; skeletonizing the single connected domain to obtain a skeleton graph of the single connected domain; the single connected domain is a region in the processed fry image having the same pixel value and consisting of pixels adjacent to each other; the skeletonization is a process of thinning image features into one-pixel-wide lines by continuously stripping peripheral pixels of the image to obtain a central skeleton of the image; the skeleton of the connected domain runs through the interior of the connected domain, and branch points are often formed in the skeleton graph at positions where the fry overlap, and the branch points of the skeleton and the absolute distance between the skeleton and the edge can reflect the relative position information, overlapping situation, etc. of the fry;

[0011] 5) Using edge tracking method to obtain the position sequence of closed edge images;

[0012] 6) performing filtering and denoising on the position sequence obtained in step 5);

[0013] 7) Calculate the curvature of each closed edge image of a single connected domain and perform bilateral filtering on the curvature calculation results to finally obtain the curvature signal;

[0014] 8) Filtering the curvature signal in step 7), treating points with curvature greater than 0 as concave points, marking and extracting the positions of the concave point clusters;

[0015] 9) combining the skeleton graph of the single connected domain and the position of the concave point group obtained in step 8) to draw a segmentation line within the single connected domain;

[0016] 10) According to the segmentation result, the number of fry in a single connected domain is counted; Steps 4) to 9) are repeated until the number of fry in all single connected domains in the image information of the fry in the fry box collected by the camera is counted, and the number of fry in the image information is finally obtained.

[0017] Preferably, the specific division method of the dividing line in step 9) adopted by the present invention is:

[0018] 9.1) Determine whether there is overlap of fry images in a single connected domain. If so, proceed to step 9.2); if not, do not segment;

[0019] 9.2) Determine whether the fry image contains paired concave point clusters. If so, proceed to step 9.3); if not, the fry image contains a single concave point cluster overlap, and proceed to step 9.6); the concave point cluster is a continuous concave point identified at a concave position on the edge of a single connected domain;

[0020] 9.3) Determine whether the branch points of the skeleton graph of a single connected domain of the fry image are greater than zero. If so, the fry image is a cross-over overlap, and step 9.4) is performed simultaneously. If equal to zero, the fry image is a connection-overlap overlap, and step 9.5) is performed simultaneously. The cross-over overlap is an overlap generated when the fish bodies cross-contact; the connection-overlap is an overlap where the fry are head-to-head, head-to-tail, or tail-to-tail connected.

[0021] 9.4) Using the branch point in the skeleton graph of a single connected domain as a reference point, the overlapping fish bodies are segmented by finding and connecting the two concave points closest to the branch point in the two concave point clusters.

[0022] 9.5) First, match the remaining concave point clusters one by one, then draw the dividing line; then match the concave point clusters in pairs according to the relative distance between them from near to far;

[0023] 9.6) Select the point with the greatest degree of concavity from the same concave point group. That is, find the concave point with the smallest angle with the previous and next points within the concave point group as the segmentation point. Segment the fish body inward along the diagonal of this angle. The intersection of the segmentation line and the edge of the fish body is used as the other segmentation point.

[0024] Preferably, in step 9.4) of the present invention, when multiple branch points appear in the skeleton graph of a single connected domain, the priority is first determined when segmenting the branch points until all branch points are used as reference points to complete an image segmentation.

[0025] Preferably, in step 9.4) adopted by the present invention, when multiple branch points appear in the skeleton graph of a single connected domain, the priority is determined by respectively calculating the absolute distance between each branch point and each remaining concave point group in the concave point group set, and the branch points closer to the concave point group are preferentially used as reference points for image segmentation.

[0026] Preferably, the calculation formula of the curvature k(t) in step 7) adopted in the present invention is:

[0027]

[0028] in:

[0029] The x(t) and y(t) are the row coordinates and column coordinates of the edge position sequence after filtering, respectively;

[0030] Said t is the sequence number in the position sequence;

[0031] The x'(t) and y'(t) are the approximate first derivatives of x(t) and y(t) with respect to t, respectively;

[0032] Said x”(t) and y”(t) are respectively the approximate quadratic derivatives of x(t) and y(t) with respect to t;

[0033] in:

[0034] The calculation method of x'(t), y'(t), x"(t) and y"(t) is:

[0035]

[0036]

[0037] in:

[0038] Said Δt is the increment of the variable t, and here Δt=3;

[0039] The specific implementation method of performing bilateral filtering on the curvature calculation result in step 7) is:

[0040] The curvature calculation result is subjected to bilateral filtering with T1 = -0.025 and T2 = 0.05 as thresholds. The formula is as follows:

[0041]

[0042] in:

[0043] The K(t) is the curvature signal after bilateral filtering.

[0044] Preferably, the background removal process in step 3) adopted in the present invention includes binarization processing and morphological image processing; the morphological processing is dilation processing and erosion processing.

[0045] Preferably, the specific implementation method of using the edge tracing method in step 5) of the present invention to obtain the position sequence of the closed edge image is: starting from the first contour point on the leftmost side of a single connected domain, moving along the pixel points within the 8-neighborhood that have not been traversed until returning to the first contour point to obtain the position sequence of the closed edge.

[0046] Preferably, the filtering process in step 6) of the present invention is to perform mean filtering on the row coordinates X and column coordinates Y of the position sequence obtained in step 5).

[0047] Preferably, the camera in step 2) used in the present invention is a monocular camera.

[0048] A counting system based on the high-density fry counting method based on image edge curvature as described above is characterized in that: the counting system includes a black box, a monocular camera, a fry box, a translucent white background plate, an industrial LED high-brightness light source, a USB interface, and a computer; the monocular camera, the fry box, the translucent white background plate, and the industrial LED high-brightness light source are arranged in the black box in sequence from top to bottom; and the monocular camera is connected to the computer via the USB interface.

[0049] The advantages of the present invention are:

[0050] The present invention discloses a high-density fish fry counting method and system based on image edge curvature. The method comprises: turning on an auxiliary light source and placing an unknown number of fish fry into a fish fry box; using a monocular camera to capture images of the fish fry and storing them in a computer; performing binarization and morphological operations on the captured images to remove image background and redundant noise. Edge features of connected domains in the binary image are extracted to obtain a closed edge image with a width of a single pixel, and edge tracing is used to obtain a sequence of closed edge positions. The edge signals of the fish fry images are filtered and denoised, the curvature of each edge is calculated, and the curvature signals are bilaterally filtered. Concave point clusters are marked on the image edges, i.e., continuous concave points identified at concave positions on the edges, and segmentation lines are drawn within the connected domains based on the connected domain skeleton map and the positions of the concave point clusters. The total number of connected domains is counted as the final count result and stored in a database. This method, combining monocular vision with image segmentation technology based on edge curvature, can effectively solve the problems of high density and multiple overlaps in fish fry counting and ensure accurate counting results. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Schematic diagram of a high-density fry counting system based on image edge curvature;

[0052] Figure 2 Flowchart of the fry counting method based on image edge curvature.

[0053] Figure 3 Example graph of labeled concave point clusters in the skeleton and edges of a single connected component.

[0054] Figure 4 An example diagram of image segmentation based on three different overlap types. Figure 4 (a) is the result of cross-overlapping image segmentation. Figure 4 (b) is the result of connection-type overlapping image segmentation. Figure 4 (c) is the result of single concave point group overlapping image segmentation.

[0055] Figure 5 The interface that displays the results after the fry counting is completed.

[0056] In the picture:

[0057] 1- Black box, 2- Monocular camera, 3- Fry box, 4- Transparent white background, 5- Industrial LED high brightness light source, 6- USB port and 7- Computer. DETAILED DESCRIPTION

[0058] The present invention proposes a high-density fish fry counting method and system based on image edge curvature. The present invention is further described below with reference to the accompanying drawings and specific embodiments. Figure 2 The high-density fry counting method based on image edge curvature provided by the present invention comprises the following steps:

[0059] 1) Place an unknown number of fry into the fry box;

[0060] 2) using a camera to collect image information of the fry in the fry box, the camera is preferably a monocular camera;

[0061] 3) Perform background removal on the image information obtained in step 2) to obtain a processed fish fry image. Background removal includes binarization and morphological image processing. Morphological processing includes dilation and erosion. Dilation and erosion, collectively referred to as morphological operations, are a set of operations that process images based on shape. They are typically performed on binary images, similar to contour detection. Dilation increases the size of bright areas by adding pixels to the perceived boundaries of objects in the image, expanding and amplifying the bright white areas in the image. Erosion, on the other hand, increases the size of dark areas by removing pixels along object boundaries and reducing the size of the object.

[0062] 4) obtaining a single connected domain in the processed fry image, extracting edge features of the single connected domain, and obtaining a closed edge image with a width of a single pixel; a single connected domain is a region in the processed fry image having the same pixel value and consisting of adjacent pixels;

[0063] 5) The position sequence of the closed edge image is obtained by edge tracing. The specific implementation method is: starting from the first contour point on the leftmost side of a single connected domain, moving along the untraversed pixel points in the 8-neighborhood until returning to the first contour point to obtain the position sequence of the closed edge.

[0064] 6) performing filtering and denoising on the position sequence obtained in step 5); the filtering process is to perform mean filtering on the row coordinates X and column coordinates Y of the position sequence obtained in step 5);

[0065] 7) Calculate the curvature of each edge image of a single connected domain and perform bilateral filtering on the curvature calculation results to finally obtain the curvature signal; wherein, the calculation formula of the curvature k(t) is:

[0066]

[0067] in:

[0068] x(t) and y(t) are the row and column coordinates of the filtered edge position sequence, respectively;

[0069] t is the sequence number in the position sequence;

[0070] The x'(t) and y'(t) are the approximate first derivatives of x(t) and y(t) with respect to t, respectively;

[0071] Said x”(t) and y”(t) are respectively the approximate quadratic derivatives of x(t) and y(t) with respect to t;

[0072] in:

[0073] x'(t), y'(t), x"(t), and y"(t) are calculated as follows:

[0074]

[0075]

[0076] Wherein: Δt is the increment of variable t, and Δt=3 is taken here.

[0077] Table 1 shows an example of the curvature k(t) calculation results. When the k(t) value is positive, it means that the curve is concave; when the k(t) value is negative, it means that the curve is convex; when the k(t) value is 0, it means that the curve is a straight line.

[0078] Table 1 Example of calculation results of curvature k(t)

[0079]

[0080] The specific implementation method of performing bilateral filtering on the curvature calculation result in step 7) is:

[0081] The curvature calculation result is subjected to bilateral filtering with T1 = -0.025 and T2 = 0.05 as thresholds. The formula is as follows:

[0082]

[0083] Where: K(t) is the curvature signal after bilateral filtering.

[0084] 8) Filtering the curvature signal in step 7), treating points with curvature greater than 0 as concave points, marking and extracting the positions of the concave point clusters;

[0085] 9) Based on the connected domain skeleton graph and the position of the concave point group obtained in step 8), a segmentation line is drawn within a single connected domain. The specific division method of the segmentation line is:

[0086] 9.1) Determine whether there is overlap of fry images in a single connected domain. If so, proceed to step 9.2); if not, do not segment;

[0087] 9.2) Determine whether the fry image contains paired concave point clusters. If so, proceed to step 9.3); if not, the fry image contains a single concave point cluster overlap, and proceed to step 9.6); a concave point cluster is a continuous concave point identified at a concave position on the edge of a single connected domain;

[0088] 9.3) Determine whether the branch point of the fry image is greater than zero. If so, the fry image is a cross-overlap, and step 9.4) is performed simultaneously. If equal to zero, the fry image is a connection-overlap, and step 9.5) is performed simultaneously. Cross-overlap is an overlap caused by cross-contact between fish bodies. Connection-overlap is an overlap where the fry are connected head-to-head, head-to-tail, or tail-to-tail.

[0089] 9.4) Use the branch points in the skeleton as reference points, and find and connect the two concave points closest to the branch points in the two concave point groups to achieve the effect of segmenting the overlapping fish bodies; when there are multiple branch points in the skeleton image, determine the priority of the branch points when segmenting, until all branch points are used as reference points to complete the image segmentation; the priority is determined by calculating the absolute distance from each branch point to each remaining concave point group in the concave point group set, and the branch points closer to the concave point group are given priority as reference points for image segmentation. Figure 3 In the example shown, branch point 1 of the connected domain skeleton is flanked by concave point groups 1 and 2, and branch point 2 is flanked by concave point groups 3 and 4. By connecting the two concave points closest to branch point 1 in concave point groups 1 and 2, and by connecting the two concave points closest to branch point 2 in concave point group 3 and 4, the fry can be accurately segmented at the overlapping locations.

[0090] 9.5) First, match the remaining concave point clusters one by one, then draw the dividing line; then match the concave point clusters in pairs according to the relative distance between them from near to far;

[0091] 9.6) Select the point with the largest degree of concavity from the same concave point group. That is, find the concave point with the smallest angle with the preceding and succeeding points within the concave point group as the segmentation point. Segment the fish body inward along the diagonal of this angle. The intersection of the segmentation line and the edge of the fish body is used as the other segmentation point.

[0092] After processing, the example Figure 4 shown. Figure 4(a) is the segmentation result of the fish fry image under cross-overlapping conditions. Figure 4 (b) is a connection-type overlap, Figure 4 (c) is the single concave point group overlap.

[0093] 10) After drawing the segmentation line, the original single connected domain can be segmented into a plurality of independent connected domains that are not connected to each other. Therefore, the number of fry in the segmented independent connected domains can be counted based on the segmentation results. Steps 4) to 9) are repeated until the number of fry in all the single connected domains in the image information of the fry in the fry box captured by the camera is counted, and the number of fry in the image information is finally obtained.

[0094] See also Figure 1 While providing a high-density fish fry counting method based on image edge curvature, the present invention also provides a counting system for implementing the method. The counting system includes a black box 1, a monocular camera 2, a fish fry box 3, a translucent white background board 4, an industrial LED high-brightness light source 5, a USB interface 6, and a computer 7. The monocular camera 2, the fish fry box 3, the translucent white background board 4, and the industrial LED high-brightness light source 5 are arranged in the black box 1 from top to bottom. The monocular camera 2 is connected to the computer 7 via the USB interface 6. In this embodiment, the entire shooting process is carried out in the black box to prevent external light sources from interfering with the fish fry images. An industrial LED high-brightness light source is selected and placed at the bottom of the device to provide a stable and uniform light source. The monocular camera is placed directly above the device with the lens facing vertically downward, and images of the fish fry are captured from top to bottom. A white light is placed between the light source and the camera. The fish fry are placed in a square fish fry box, and black opaque stickers are affixed to all four sides of the fish fry box to prevent reflections around the box when the light source is applied, thereby affecting the image quality. The water depth in the fry box was controlled between 0.8 cm and 1.0 cm to reduce the cross-overlap of the fry images.

[0095] After the image captured by the monocular camera is input into the computer, the algorithm can be used to directly process the image to complete the calculation of the number of fry. Figure 5 As shown, you can view the results in the pop-up window after the algorithm is completed. Click the "OK" button to process the next image. Select the fry density as 0.25 / cm 2 , 0.33 / cm 2 , 0.42 / cm 2 and 0.5 / cm 2 Four groups of images with a resolution of 1280×720 pixels, 1000 each, were used for testing.

[0096] The mean absolute error (MAE) and mean absolute percentage error (MAPE) are used to measure the accuracy of the algorithm calculation. The formula for calculating the mean absolute error (MAE) is as follows:

[0097]

[0098] The mean absolute percentage error (MAPE) is calculated as follows:

[0099]

[0100] In the formula, n represents the total number of samples, A t is the true value, F t The calculation results are as follows. When the MAPE is less than 5%, the algorithm is considered to meet the accuracy requirements for the given fry density. Furthermore, the average number of fry calculated within 1 second is used as a metric to measure the algorithm's calculation speed under different fry densities. The unit is fry / s. The computer used for counting is an RTX 4050 with an i5-12450H processor.

[0101] The results show that the counting method achieves good accuracy and speed, and the detection results are shown in Table 2. The plane fry density is 0.25 to 0.58 tails / cm 2 The average absolute error percentage does not exceed 1.37%, and the average number of fry counted per second exceeds 100.

[0102] Table 2 Statistical results of fry counting

[0103]

[0104] From the perspective of the fry themselves, on the one hand, the size of the error is mainly related to the amount of fry overlap in the image; on the other hand, the complexity of the overlap in a single connected domain directly affects the accuracy of the segmentation position. The more overlaps in a single connected domain and the more complex the type, the greater the difficulty of the algorithm and the larger the counting error. Although the greater the planar density of fry in the experiment, the higher the counting efficiency, it is still necessary to balance the relationship between the planar density of fry and the counting efficiency. According to Table 2, the planar density of fry is controlled at 0.25 to 0.58 tails / cm 2 Within the range, the counting accuracy can be maintained as much as possible.

Claims

1. A high-density fry counting method based on image edge curvature, characterized by: The high-density fry counting method based on image edge curvature comprises the following steps: 1) Place an unknown number of fry into the fry box; 2) using a camera to collect image information of the fry in the fry box; 3) performing background removal processing on the image information obtained in step 2) to obtain a processed fry image; 4) obtaining a single connected domain in the processed fry image, extracting edge features of the single connected domain to obtain a closed edge image with a width of a single pixel; skeletonizing the single connected domain to obtain a skeleton graph of the single connected domain; the single connected domain is a region in the processed fry image having the same pixel value and consisting of adjacent pixels; 5) Using edge tracking method to obtain the position sequence of closed edge images; 6) performing filtering and denoising on the position sequence obtained in step 5); 7) Calculate the curvature of each closed edge image of a single connected domain and perform bilateral filtering on the curvature calculation results to finally obtain the curvature signal; 8) Filtering the curvature signal in step 7), treating points with curvature greater than 0 as concave points, marking and extracting the positions of the concave point clusters; 9) Combining the skeleton graph of a single connected domain and the position of the concave point group obtained in step 8) to draw a segmentation line within the single connected domain, specifically: 9.1) Determine whether there is overlap of fry images in a single connected domain. If so, proceed to step 9.2); if not, do not segment; 9.2) Determine whether the fry image contains paired concave point clusters. If so, proceed to step 9.3); if not, the fry image contains a single concave point cluster overlap, and proceed to step 9.6); the concave point cluster is a continuous concave point identified at a concave position on the edge of a single connected domain; 9.3) Determine whether the branch points of the skeleton graph of a single connected domain of the fry image are greater than zero. If so, the fry image is a cross-over overlap, and step 9.4) is performed simultaneously. If equal to zero, the fry image is a connection-overlap overlap, and step 9.5) is performed simultaneously. The cross-over overlap is an overlap generated when the fish bodies cross-contact; the connection-overlap is an overlap where the fry are head-to-head, head-to-tail, or tail-to-tail connected. 9.4) Using the branch point in the skeleton graph of a single connected domain as a reference point, the overlapping fish bodies are segmented by finding and connecting the two concave points closest to the branch point in the two concave point clusters. 9.5) First, match the remaining concave point clusters one by one, then draw the dividing line; then match the concave point clusters in pairs according to the relative distance between them from near to far; 9.6) Select the point with the largest degree of concavity from the same concave point group. That is, find the concave point with the smallest angle with the preceding and succeeding points within the concave point group as the segmentation point. Segment the fish body inward along the diagonal of this angle. The intersection of the segmentation line and the edge of the fish body is used as the other segmentation point. 10) According to the segmentation result, the number of fry in a single connected domain is counted; Steps 4) to 9) are repeated until the number of fry in all single connected domains in the image information of the fry in the fry box collected by the camera is counted, and the number of fry in the image information is finally obtained.

2. The high-density fry counting method based on image edge curvature according to claim 1, wherein: In step 9.4), when multiple branch points appear in the skeleton graph of a single connected domain, the priorities are first determined when segmenting the branch points, and image segmentation is completed once all the branch points are used as reference points.

3. The high-density fry counting method based on image edge curvature according to claim 2, wherein: In step 9.4), when multiple branch points appear in the skeleton graph of a single connected domain, the priority is determined by calculating the absolute distance between each branch point and each remaining concave point group in the concave point group set, and the branch point closer to the concave point group is preferentially used as a reference point for image segmentation.

4. The high-density fry counting method based on image edge curvature according to claim 3, wherein: The calculation formula of curvature k(t) in step 7) is: in: The x(t) and y(t) are the row coordinates and column coordinates of the edge position sequence after filtering, respectively; Said t is the sequence number in the position sequence; The x'(t) and y'(t) are the approximate first derivatives of x(t) and y(t) with respect to t, respectively; Said x”(t) and y”(t) are respectively the approximate quadratic derivatives of x(t) and y(t) with respect to t; in: The calculation method of x'(t), y'(t), x"(t) and y"(t) is: in: Said Δt is the increment of the variable t, and here Δt=3; The specific implementation method of performing bilateral filtering on the curvature calculation result in step 7) is: The curvature calculation result is subjected to bilateral filtering with T1 = -0.025 and T2 = 0.05 as thresholds. The formula is as follows: in: The K(t) is the curvature signal after bilateral filtering.

5. The high-density fry counting method based on image edge curvature according to claim 1, 2, 3 or 4, wherein: The background removal process in step 3) includes binarization and morphological image processing; the morphological processing is dilation and erosion.

6. The high-density fry counting method based on image edge curvature according to claim 5, wherein: The specific implementation method of using the edge tracing method in step 5) to obtain the position sequence of the closed edge image is: starting from the first contour point on the leftmost side of a single connected domain, moving along the pixel points in the 8-neighborhood that have not been traversed until returning to the first contour point to obtain the position sequence of the closed edge.

7. The high-density fry counting method based on image edge curvature according to claim 6, wherein: The filtering process in step 6) is to perform mean filtering on the row coordinates X and column coordinates Y of the position sequence obtained in step 5).

8. The high-density fry counting method based on image edge curvature according to claim 7, wherein: The camera in step 2) is a monocular camera.

Images