Automatic fish body height measuring method

Abstract

The invention relates to the technical field of aquaculture automatic measurement, in particular to an automatic fish body height measurement method, which comprises the following steps of: acquiring images of a fish and a platform through an image acquisition device, performing instance segmentation on the acquired images by adopting an instance segmentation algorithm to respectively obtain mask images of the fish and the platform, and calibrating a straight line with a known length in the platform to obtain the height of the fish body. Constructing an external rectangle between the upper contour and the lower contour of the fish body to predict the height of the fish body, eliminating the influence of different shooting positions and angles on a detection result through a perspective transformation technology, and finally outputting the data of the height of the fish body. According to the invention, the automatic measurement of the fish body height is realized, the problem of inaccurate measurement of the body height in the traditional image recognition technology is solved, the objectivity and accuracy of data are improved, the applicability in actual production is greatly improved, and the repeatability and credibility of the data in production guidance and scientific research are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of aquaculture automation measurement, and particularly relates to an automated method for measuring the height of fish bodies. Background Art

[0002] With the improvement of residents' living standards, consumers' demand for high-quality fish is increasing day by day. High-quality fish have higher market value, and aquaculture farmers also pay more attention to the quality control of farmed fish. Therefore, during the farming process, it is necessary to regularly monitor the quality of fish to optimize farming plans such as feed feeding, ensure that the quality of the farmed fish is excellent, and thus improve the economic benefits of aquaculture. The height of the fish body is a key indicator for evaluating the quality of the fish's shape. Traditional measurement of the fish body height relies on manual measurement one by one with a ruler or vernier caliper. Not only is the measurement efficiency low, but it is also difficult to define the fish body height with the naked eye, and it is easily affected by subjective factors, resulting in inaccurate measurement values.

[0003] In recent years, with the development of artificial intelligence technology, computer vision technology integrating image analysis and machine learning algorithms has been widely used in the field of automated measurement. This technology breaks through the physical limitations of traditional sensing technologies and realizes non-contact, high-precision, and high-efficiency intelligent measurement. It is widely used in industrial quality inspection, biomedical imaging, agricultural automation and other fields, but it is less used in fish research. Summary of the Invention

[0004] In view of the above deficiencies of the prior art, the present invention proposes an automated method for measuring the height of fish bodies, which applies computer image recognition and processing technology to the aquaculture industry and can efficiently and quickly measure the height data of fish.

[0005] The automated method for measuring the height of fish bodies according to the present invention is carried out according to the following steps:

[0006] (1) Place the fish on an image acquisition platform marked with a reference straight line of known length, and acquire images of the fish and the platform through an image acquisition device;

[0007] (2) Use an instance segmentation algorithm to perform instance segmentation on the acquired image, respectively obtain the mask images of the fish and the platform, and adjust the size of the obtained mask images to be the same as the size of the original image;

[0008] (3) Use the Hough transform algorithm to detect the reference straight line in the image acquired in step (1), and determine the coordinates and pixel length of the reference straight line;

[0009] (4) Based on the fish mask image obtained in step (2), extract a set of pixel points, perform function fitting, and construct a minimum circumscribed rectangle for fish body height detection;

[0010] (5) Detect the coordinates of the four corner points of the platform mask image obtained in step (2), then perform perspective transformation on the mask images of the fish and the platform, project the fish body in the three-dimensional space onto the two-dimensional plane, re-determine the coordinates and pixel length of the reference line, so as to update the coordinates and pixel length of the minimum bounding rectangle constructed in step (4), and use the wide side of the minimum bounding rectangle as the predicted data of the fish body height;

[0011] (6) Determine the correction coefficient and correct the predicted data of the fish body height.

[0012] As a preferred solution, the specific process of the instance segmentation described in step (2) is as follows:

[0013] Locate the target object in the image collected in step (1) according to the text prompt provided by the user to obtain the bounding box;

[0014] Transfer the bounding box information to the general image segmentation model, and the general image segmentation model performs instance segmentation within the bounding box;

[0015] Use the image generation tool to generate an image according to the text description, and further fuse it with the instance segmentation result to generate the final visual output.

[0016] As a preferred solution, the specific process of the reference line detection described in step (3) is as follows:

[0017] Convert the image collected in step (1) from the BGR color space to the HSV color space, and perform dilation and erosion operations on the image;

[0018] Set the distance resolution i, the angle resolution j, and specify that the minimum detection length of the line is p pixels;

[0019] Set the maximum gap between lines to q pixels. If the gap between two lines is less than q pixels, they are regarded as belonging to the same line;

[0020] Set the accumulator threshold to w. Only when the accumulator value of the line exceeds w, the line will be confirmed as a valid line.

[0021] As a preferred solution, the specific process of step (4) is as follows:

[0022] According to the fish mask image obtained in step (2), obtain the coordinates of all pixel points where the fish is located, and use the nonlinear least squares fitting method to fit the quadratic function model with the pixel point coordinates to obtain the preliminary fitting curve;

[0023] Evaluate the preliminary fitting curve, reassign weights to the pixel points deviating from the fitting curve, and use the nonlinear least squares fitting method to obtain the quadratic fitting curve of the lower contour of the fish body;

[0024] The upper boundary of the fish body is detected using a mask as the upper contour of the fish. A minimum circumscribed rectangle is constructed between the upper and lower contours with the center position of the fish body. The wide side of the minimum circumscribed rectangle is used for detecting the body height of the large yellow croaker.

[0025] As a preferred solution, the correction coefficient described in step (6) is the median of the proportional coefficient values between several groups of true measurement data of the fish body height and the predicted data of the fish body height. Further, the correction coefficient is preferably 0.85.

[0026] The advantages of the present invention are as follows:

[0027] (1) By calibrating a straight line of known length in the calibration platform and constructing a circumscribed rectangle between the upper and lower contours of the fish body to measure the fish body height, and then using the perspective transformation technology to eliminate the influence of different shooting positions and angles on the detection results, the problem of inaccurate body height measurement in traditional image recognition technology is solved, the objectivity and accuracy of the data are improved, and the applicability of this method in actual production is greatly enhanced.

[0028] (2) The application of computer vision technology realizes the automatic measurement of the fish body height, optimizes the cumbersome measurement steps such as the human eye identifying the measurement site of the fish body height, manually operating the scale and reading the values, etc., and greatly improves the detection efficiency.

[0029] (3) A circumscribed rectangle and a correction coefficient are standardizedly constructed for predicting the fish body height, avoiding the problem of inconsistent detection results caused by different standards of different detection personnel, improving the standardization level of data detection, eliminating the influence of other factors on data detection, and ensuring the repeatability and credibility of the data in production guidance and scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1 It is the working principle diagram of Embodiment 1;

[0032] Figure 2 It is the image collected by the image acquisition device in Embodiment 1;

[0033] Figure 3 It is the fish mask image in Embodiment 1;

[0034] Figure 4 It is the platform mask image in Embodiment 1;

[0035] Figure 5 Schematic diagram of the marker in the fish mask image after the reference line detection in Example 1;

[0036] Figure 6 Schematic diagram of the quadratic curve of the lower contour of the fish body after fitting in Example 1;

[0037] Figure 7 Schematic diagram of the minimum circumscribed rectangle constructed in Example 1;

[0038] Figure 8 Schematic diagram of the corner detection of the platform mask image in Example 1;

[0039] Figure 9 Box plot of the ratio relationship between the predicted length and the actual measured value of the fish body height in Example 1. Specific implementation manners

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] Example 1:

[0042] As Figure 1 shown, this embodiment discloses an automated method for measuring the fish body height, which is carried out according to the following steps:

[0043] (1) Image acquisition: Select large yellow croaker as the fish for this experiment, place the large yellow croaker on the image acquisition platform marked with a reference line of known length, and completely acquire the images of the large yellow croaker and the platform through the image acquisition device, as Figure 2 shown.

[0044] In this step, the large yellow croaker image acquisition platform is a white rectangular plate, and a 5-cm reference line is marked on the platform. Place the large yellow croaker at a suitable position on the image acquisition platform, and adjust the position and angle of the shooting lens according to the shape and size of the large yellow croaker to ensure that the entire fish body and the four corner points of the platform are within the field of view of the lens. The above process is crucial for the subsequent measurement results. First, a complete image of the fish body needs to be taken to ensure the accuracy of the subsequent measurement; second, the four corner points of the platform need to be taken for perspective transformation in the subsequent image processing process. In addition, the light source needs to be adjusted to an appropriate intensity to ensure appropriate illumination during image acquisition, avoiding overexposure or underexposure. At the same time, avoid direct light on the reference line to reduce the reflection phenomenon of the reference line. Prevent overexposure of the reference line due to strong light source, which will affect the calibration of the reference line and the accuracy of the subsequent body height measurement.

[0045] Start the camera to capture an image of the large yellow croaker. It is necessary to control the image quality to ensure that the image is clear and has no obvious noise. The inspection of image quality includes multiple aspects such as contrast, brightness, and clarity. Any non-compliance in any item may lead to inaccurate measurement results.

[0046] (2) Image preprocessing: Use the instance segmentation algorithm to perform instance segmentation on the image collected in step (1), respectively obtain the mask images of the fish and the platform, and adjust the size of the obtained mask images to be the same as the size of the original image.

[0047] In this step, the specific process of the instance segmentation algorithm is as follows: Use the Grounded-Segment-Anything segmentation algorithm to perform instance segmentation on the image according to the prompt words, so as to achieve precise segmentation of the large yellow croaker and the platform. The model can capture the features of the fish body and the platform, and thus identify their contours in the image. First, use GroundingDINO to locate the target object in the image according to the text prompt provided by the user to obtain the bounding box. Then, transfer the bounding box information to the general image segmentation model SAM model, and SAM will perform precise instance segmentation within these regions. Use the StableDiffusion image generation tool to generate images according to the text description. Finally, the system fuses the detection and instance segmentation results to generate the final visual output. Through the segmentation results output by the algorithm, the masks of the large yellow croaker and the platform can be obtained. These masks clearly identify the contours of the fish body and the platform, providing accurate regions for subsequent perspective transformation and data analysis. Figure 2 Perform instance segmentation, and the obtained fish mask image is as Figure 3 shown, and the platform mask image is as Figure 4 shown.

[0048] (3) Calibrate the reference line: Use the Hough transform algorithm to detect the reference line in the image collected in step (1), and determine the coordinates and pixel length of the reference line;

[0049] In this step, the Hough transform algorithm is used to calibrate the reference line in the image and determine its position coordinates and pixel length. The calibration of the reference line is a key step in body height measurement, and its accurate calibration is crucial for subsequent calculation of the fish body height. Therefore, it is necessary to adjust and optimize the parameters of the Hough transform algorithm to improve the accuracy of detection.

[0050] First, adjust the threshold in the HSV color space to capture specific black color tones and saturation ranges. Since the HSV color space is more efficient in color recognition and processing, the input image is converted from the BGR color space to the HSV color space. Subsequently, perform a dilation operation on the image, which helps to bridge the broken black areas in the image. Perform an erosion operation to eliminate noise and small black areas.

[0051] In the Hough transform, a distance resolution of 1 and an angular resolution of Π / 180 are set. The minimum detection length of a straight line is specified as 130 pixels, and only straight lines longer than this length will be recognized. At the same time, the maximum gap between straight lines is set to 8 pixels. If the gap between two straight lines is less than this value, they will be considered as belonging to the same straight line. In addition, an accumulator threshold of 50 is set, which means that only when the accumulator value of a straight line exceeds this threshold will the straight line be confirmed as a valid straight line. Take Figure 5 as an example. The green line is the reference straight line, and the coordinates of the two endpoints of the line segment are (1167, 1162) and (1425, 1149) respectively. Based on this, calculate the pixel length

[0052] (4) Construct the minimum bounding rectangle: Based on the fish mask image obtained in step (2), extract the set of pixel points, perform function fitting, and construct the minimum bounding rectangle for the body height detection of large yellow croaker;

[0053] This step specifically includes, based on the fish mask image obtained in step (2), using image processing techniques to detect the outer contour of the fish and obtain the set of all pixel points inside the fish. These pixel points represent the edge features of the fish body and are the basis for shape analysis.

[0054] Using the set of pixel points inside the fish, perform quadratic function fitting to describe the horizontal position and overall trend of the fish. According to the fish mask, the coordinates of all pixel points at the position of the fish can be obtained. Using the nonlinear least squares fitting method, the curve_fit function, fit the quadratic function model with the data points. During the fitting process, the curve_fit function will try to adjust the coefficients required for the quadratic function: the values of a, b, and c, to minimize the difference between the predicted y value of the model and the actual data point y value. After the fitting is completed, the curve_fit function will return an array of the best fitting parameters, thereby obtaining a preliminary fitting curve. Evaluate the fitting result, reassign weights to the pixel points that deviate from the fitting curve, and enhance the influence of these points during the fitting process. After reweighting, use the nonlinear least squares fitting method, the curve_fit function, to refit the quadratic function to obtain a more accurate fitting result. Figure 6 The arc in is the quadratic curve of the lower contour of the fitted fish body, which is used as a reference for measuring the fish body height. The quadratic function y = ax2 The coefficients of +bx + c are respectively:

[0055] a = -0.00030787,

[0056] b = 0.59795136,

[0057] c = 527.31464557

[0058] Then use the mask to detect the upper boundary as the upper contour of the fish. Construct the minimum bounding rectangle between the upper and lower contours based on the central position of the fish body, that is, find the rectangle with the smallest area that circumscribes the polygon. The width of the rectangle is the height of the fish body. To construct the minimum bounding rectangle, it is necessary to first find the simple bounding rectangle of the polygon, then rotate the polygon from -90° to 90°, calculate the areas of the bounding rectangles at each rotation angle, screen and record the rectangle with the smallest area and its rotation angle, and finally rotate the rectangle in the reverse direction to obtain the minimum bounding rectangle. The minimum bounding rectangle is as Figure 7 shown by the rectangle wireframe in, and the coordinates of the four points are respectively: (874, 550), (1258, 550), (1258, 813), (874, 813), and the pixel width is: 813 - 550 = 263.

[0059] (5) Prediction of the body height of large yellow croaker: Detect the coordinates of the four corner points of the platform mask image obtained in step (2), then perform perspective transformation on the mask images of the fish and the platform, project the fish body in three-dimensional space onto a two-dimensional plane, re-determine the coordinates and pixel lengths of the reference line, so as to update the coordinates and pixel lengths of the minimum bounding rectangle constructed in step (4), and use the wide side of the minimum bounding rectangle as the prediction data of the fish body height;

[0060] Before the perspective transformation, it is necessary to use the corner detection algorithm to detect the coordinates of the four corner points of the platform. This algorithm can identify the intersection points of the lines in the image, and these intersection points are the corner points of the plate. The corner points are as Figure 8 shown by the dots in. The accuracy of corner detection is crucial for subsequent perspective transformation because perspective transformation requires accurate corner point coordinates to construct the mapping from three-dimensional space to two-dimensional plane. Once the four corner points are accurately identified, the coordinates can be used for the preparation work of perspective transformation.

[0061] Perspective transformation is a geometric transformation that can map the three-dimensional structure in an image onto a two-dimensional plane while maintaining the straightness of lines. This step involves constructing a perspective transformation matrix, which is based on the internal and external parameters of the camera and the positions of the plate corner points in three-dimensional space. After determining the length and position of the reference line, the perspective transformation of the picture is performed through the four right-angle points of the rectangular plate. The fish body in three-dimensional space is projected onto the two-dimensional plane. The purpose of perspective transformation is to eliminate perspective distortion in the image to obtain accurate measurement results. This step involves the technique of orthogonally projecting points in a three-dimensional coordinate system onto a two-dimensional plane, which is achieved through matrix operations. Precise calculation of matrix parameters is required to ensure the accuracy of the transformation. This may include calculating the internal and external parameters of the camera and the position of the fish body in three-dimensional space.

[0062] The calculated perspective transformation matrix is:

[0063]

[0064] The new coordinates of the reference line after perspective transformation are: (1146.9, 995.1), (1471.8, 981.0), the new pixel length is 325.2, and the corresponding actual length is 5 cm.

[0065] The new coordinates of the wide side of the minimum bounding rectangle after perspective transformation are: (1279.8, 47.5), (1271.7, 475.9), the new pixel length is 428.4, and the corresponding actual length is: 5 * 428.4 / 325.2 = 6.586 cm. That is, the predicted value of the fish body height is 6.586 cm.

[0066] (6) Correction and output of large yellow croaker body height data: Determine the correction coefficient and correct the predicted fish body height data.

[0067] After measuring the fish body height length according to step (5), the proportional coefficient relationship between the true measurement data and the predicted data of the body height of 70 large yellow croakers was further analyzed. Through statistical analysis, it was found that the median of this proportional coefficient is 0.85 and the average value is 0.86. Given that the median can better reflect the central tendency of the data and is not easily affected by extreme values, the median is selected as the body height correction coefficient. Therefore, multiply the body height length predicted by image recognition by the correction coefficient to obtain a more accurate predicted value of the body height. This method improves the accuracy and reliability of the prediction.

[0068] Figure 9 It is a box plot of the proportional relationship between the predicted length of the fish body height and the true measurement value. Most of the values are concentrated between 0.79 - 0.93, and the median is 0.85.

[0069] Therefore, the final body height of the large yellow croaker shown in this figure is 6.586 * 0.85 = 5.59 cm

[0070] The above is a detailed introduction of the present invention. Specific examples are used in this article to expound the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method and core idea of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. An automated method for measuring the height of a fish body, characterized in that, The following steps are carried out: (1) Place the fish on the image acquisition platform marked with a reference straight line of known length, and acquire the images of the fish and the platform through the image acquisition device; (2) Use the instance segmentation algorithm to perform instance segmentation on the acquired images, respectively obtain the mask images of the fish and the platform, and adjust the size of the obtained mask images to be the same as the size of the original images; (3) Use the Hough transform algorithm to detect the reference straight line in the image acquired in step (1), and determine the coordinates and pixel length of the reference straight line; (4) Based on the fish mask image obtained in step (2), extract the set of pixel points, perform function fitting, and construct the minimum bounding rectangle for fish body height detection; (5) Detect the coordinates of the four corner points of the platform mask image obtained in step (2), then perform perspective transformation on the mask images of the fish and the platform, project the fish body in the three-dimensional space onto the two-dimensional plane, re-determine the coordinates and pixel length of the reference straight line, so as to update the coordinates and pixel length of the minimum bounding rectangle constructed in step (4), and use the wide side of the minimum bounding rectangle as the fish body height prediction data; (6) Determine the correction coefficient and correct the fish body height prediction data.

2. The automated fish body height measurement method according to claim 1, wherein The specific process of the instance segmentation described in step (2) is as follows: Locate the target object in the image acquired in step (1) according to the text prompt provided by the user to obtain the bounding box; Transmit the bounding box information to the general image segmentation model, and the general image segmentation model performs instance segmentation within the bounding box; Use the image generation tool to generate images according to the text description, and further fuse them with the instance segmentation results to generate the final visual output.

3. An automated method for measuring the height of a fish body according to claim 1, characterized in that, The specific process of the reference straight line detection described in step (3) is as follows: Convert the image acquired in step (1) from the BGR color space to the HSV color space, and perform dilation and erosion operations on the image; Set the distance resolution i and the angle resolution j, and stipulate that the minimum detection length of the straight line is p pixels; Set the maximum gap between straight lines to q pixels. If the gap between two straight lines is less than q pixels, they are regarded as belonging to the same straight line; Set the accumulator threshold to w. Only when the accumulator value of the straight line exceeds w, the straight line will be confirmed as a valid straight line.

4. An automated method for measuring the height of a fish body according to claim 1, characterized in that, The specific process of step (4) is as follows: According to the fish mask image obtained in step (2), obtain the coordinates of all pixel points where the fish is located, use the nonlinear least squares fitting method to fit the quadratic function model with the pixel point coordinates to obtain the preliminary fitting curve; Evaluate the preliminary fitting curve, reassign weights to the pixel points deviating from the fitting curve, and use the nonlinear least squares fitting method to obtain the quadratic fitting curve of the lower contour of the fish body; Use the mask to detect the upper boundary of the fish body as the upper contour of the fish, and construct the minimum bounding rectangle between the upper and lower contours at the center position of the fish body. The wide side of the minimum bounding rectangle is used for large yellow croaker body height detection.

5. An automated method for measuring the height of a fish body according to claim 1, characterized in that, The correction coefficient described in step (6) is the median of the proportional coefficient values between a number of groups of actual measurement data of fish body height and fish body height prediction data.

6. An automated fish body height measurement method according to claim 5, characterized in that, The correction coefficient described in step (6) is 0.85.

Images