Method for measuring phenotype and judging gender of rainbow trout based on deep learning

Abstract

The present application belongs to the technical field of agricultural aquaculture, and relates to a rainbow trout phenotype measurement and gender determination method based on deep learning. Each part of the rainbow trout in the photo is labeled and preprocessed as a training set and a verification set of the model. An Unet model of a full convolutional neural network is used for training to obtain an optimal model to complete segmentation of each part of the rainbow trout. Color thresholds of each part of the segmented rainbow trout are defined respectively, contour searching is performed, and a minimum bounding matrix is obtained. The actual length and width of each part of the rainbow trout are calculated from the ratio relationship between the physical distance and the image pixels, and the body weight of the fish body is calculated from the function relationship between the body length and the body weight. The gender of the rainbow trout is determined based on the support vector machine according to the length-width ratio of the head. The present application can quickly obtain phenotype information such as the body length, total length, body width and body weight of the rainbow trout, improve the efficiency, and minimize the damage to the fish body. The gender of the rainbow trout is identified through image processing, which meets the needs of the production and processing of the rainbow trout.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of agricultural aquaculture, and particularly relates to a rainbow trout phenotype measurement and gender determination method based on deep learning. BACKGROUND

[0002] Rainbow trout (Oncorhynchus mykiss) has a high economic impact worldwide, and its production is increasing due to the increasing market demand. In the process of breeding, the phenotypes and morphological indicators of rainbow trout, such as body length, body width and body weight, can not only be used for yield evaluation, but also provide a reference for genetic breeding work. Rainbow trout has obvious sexual dimorphism, and female individuals are larger than male individuals after sexual maturation, with a short and wide head and good muscle quality. These differences in phenotypic characteristics affect product quality and consumer choice, and female rainbow trout is more popular in the market. The traditional method of obtaining phenotypic and morphological indicators is to measure the rainbow trout body with a ruler, but this method is not only low in efficiency, but also harmful; and the previous method of using an ultrasonic instrument to identify the gender of rainbow trout is costly.

[0003] Therefore, developing and optimizing the related gender identification technology of rainbow trout is an important basis for good seed selection and increasing the efficiency of aquaculture production; and how to obtain morphological parameters while minimizing damage to the fish body has become a problem to be solved in rainbow trout breeding. SUMMARY

[0004] The purpose of the present application is to overcome the difficulty of fish morphological measurement and gender determination in the prior art, and to provide a rainbow trout phenotype measurement and gender determination method based on deep learning. The method uses semantic segmentation of deep learning combined with Open cv image processing to obtain morphological indicators such as body length, body width and body weight of rainbow trout, and realizes gender determination of rainbow trout through machine learning support vector machine according to the head proportion relationship.

[0005] The technical scheme of the present application is as follows:

[0006] A rainbow trout phenotype measurement and gender determination method based on deep learning, comprising the following steps:

[0007] Step 1: Collecting several images containing the whole body of rainbow trout, and labeling and preprocessing each part of the rainbow trout, marking the fish body pixels in the images as four categories of head, trunk, tail and tail fin, as a data set for model training;

[0008] Step 2: Establishing an initial U-net network model and training it with the data set obtained by preprocessing to obtain an optimal model file that can effectively segment each part of the fish body;

[0009] Step three, using the trained optimal model to perform semantic segmentation on the image, and completing the segmentation of each part of the rainbow trout;

[0010] Step four, according to the prediction results of each part of the rainbow trout body after semantic segmentation, define the HSV color threshold range of different parts respectively, realize HSV-based color classification of each part, and finally merge and apply the mask;

[0011] Step five, contour finding is performed on each part of the classified fish body respectively, and the minimum circumscribed matrix of each part is obtained. The function can return the length and width pixel values of the minimum circumscribed matrix.

[0012] Step six, determine the ratio relationship between length and pixel value, calculate the actual length of rainbow trout according to the ratio relationship, and calculate the body weight of fish body according to the functional relationship between body length and body weight;

[0013] Step seven, according to the length-width ratio of the minimum circumscribed matrix of the head, use the support vector machine of machine learning to predict the linear kernel function of gender binary classification, and judge the gender of rainbow trout based on the linear kernel function.

[0014] Further, in the image acquisition of step one, a white PVC plastic plate is used, and a ruler is fixed at the bottom of the white background, which is used to compare the pixel measurement value with the physical distance and determine the scale value.

[0015] Further, in step one, hundreds of whole body images of rainbow trout are taken, and the fish body pixels in the images are manually labeled into four categories in Labelme software, which are head, trunk, tail and tail fin. The labeled images are used as a data set, which is divided into training set images and validation set images according to the ratio of 9:1. The training set images are input into the initial network model after training, and the loss function is calculated.

[0016] Further, in step two, the model training is 50 epochs, the batch size is set to 2, the maximum learning rate is 1e-4, NVIDIA A30 GPU is used as the accelerator, and the loss function is returned after model training as the evaluation index.

[0017] Further, in step three, under the premise of ensuring that the image is as clear as possible, the image acquisition device can be used to collect images of any size and resolution, and the trained Unet model can be used to perform semantic segmentation on the images to predict each part of the rainbow trout body according to the image information.

[0018] Further, in step four, four different colors are used to classify different parts of the fish body in the HSV-based color classification, wherein the green color range is defined as lower=np.array([40,50.50]), upper=np.array([90,255,255]), the red color range is defined as lower=np.array([0,50,50]), upper=np.array([10,255,255]), the blue color range is defined as lower=np.array([100,50.50]), upper=np.array([130,255,255]), and the yellow color range is defined as lower=np.array([15,50,50]), upper=np.array([40,255,255]).

[0019] Further, in step five, the cv2.findCounters() function of Open cv is used to find the contours of each part of the fish body after color classification, and the cv2.minAreaRect() function is used to obtain the minimum bounding rectangle of each part, and the return value contains five predicted values, i.e., (x, y, w, h, θ), wherein the center coordinates of the matrix are (x, y), the width and height of the matrix are (w, h), and the rotation angle is θ. The pixel values of the width and height of the matrix are used as the pixel values of the width and height of the detected part.

[0020] Further, in step six, the longer part is defined as the length of each part of the fish body, and the actual length of the rainbow trout is obtained by a ruler according to the formula: length=pixel number / scale value, wherein the unit of length is cm.

[0021] The scale value is obtained in advance, which is obtained by pixel number / cm, and then the actual length of the rainbow trout is predicted according to the scale value.

[0022] Further, in step six, the length-weight relationship between the fish body and the weight conforms to the power function W=bL a The power function relationship curve is fitted according to the relationship between the actual length and the weight of the rainbow trout, and the weight of the fish body can be predicted from the length of the rainbow trout fish body through the relationship curve.

[0023] Wherein, W is the weight of the fish body, L is the length of the fish body, a and b are parameters, and the specific values of a and b can be obtained by fitting the relationship curve.

[0024] Further, the gender judgment in the step seven is based on the obvious gender dimorphism of rainbow trout, compared with the male, the female rainbow trout individual is larger, and the head is short and wide; therefore, the gender judgment is carried out by using the minimum circumscribed matrix of the head of the rainbow trout, and the classification linear kernel function y=cx-d is obtained by processing the head data of a plurality of fish individuals with different genders through the machine learning support vector machine according to the difference between the length-width ratios of the heads of the male and female rainbow trout, and the gender of the rainbow trout is judged according to the relationship. Wherein, c and d are parameters, and the specific values can be obtained by the length-width values of the head of the rainbow trout individual and the two-class linear graph.

[0025] The beneficial effects of the present application are as follows:

[0026] The rainbow trout phenotype measurement and gender judgment method based on deep learning provided by the present application can quickly obtain phenotype information such as body length, total length, body width and body weight of the rainbow trout, solve the problem of low efficiency of obtaining rainbow trout phenotype data by traditional manual method, and also minimize the damage to the fish body; at the same time, the gender of the rainbow trout is identified through image processing, which improves the production efficiency in the breeding process and meets the subsequent production and processing, genetic breeding and other needs of the rainbow trout. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The flowchart of the method provided by the present application is shown in the figure;

[0028] Figure 2 The loss graph of model training is shown in the figure;

[0029] Figure 3 The model segmentation schematic diagram is shown in the figure;

[0030] Figure 4 The head length-width relationship graph of male and female rainbow trout individuals is shown in the figure. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0032] In order to further understand the present application, the present application will be further described in conjunction with the drawings and embodiments.

[0033] As Figure 1 shown, the embodiments of the present application provide a rainbow trout phenotype measurement and gender judgment method based on deep learning, which comprises the following steps:

[0034] Step one, data set acquisition and image processing:

[0035] During the rearing process, hundreds of images containing the whole body of rainbow trout were taken using a digital camera, Canon 70D, with a white PVC plastic board to reduce background noise and a ruler fixed to the bottom of the white background to calculate the relationship between pixel measurements and physical distance, i.e., to compare pixel measurements with physical distance to determine the scale value.

[0036] The fish body pixels in the image were manually labeled into four categories, head, trunk, tail, and tail fin, in Labelme software. The labeled images were used as the data set for model training, and were divided into training set images and validation set images in a ratio of 9:1. The training set images were input into the initial network model after training and the loss function was calculated.

[0037] Step 2: Model training

[0038] An initial Unet network model was established, and the pre-processed training set was used to train the Unet model. The model was trained for 50 rounds, with a batch size of 2 and a maximum learning rate of 1e-4. An NVIDIA A30 GPU was used as an accelerator. The loss function was returned as an evaluation indicator after model training, and the results are shown in Figure 2 As the number of model training rounds increases, the loss value gradually decreases.

[0039] After 50 rounds of model training, the best model file that can effectively segment the parts of the fish body was obtained, which was used for subsequent segmentation of the parts of the fish body.

[0040] Step 3: Rainbow trout image acquisition and semantic segmentation

[0041] Under the premise of ensuring that the image is as clear as possible, an image acquisition device can be used to collect images of any size and resolution, and the trained Unet model can be used for semantic segmentation of the images to predict the parts of the rainbow trout body based on image information.

[0042] Step 4: Define color threshold

[0043] Based on the prediction results of the parts of the rainbow trout body after semantic segmentation, different part HSV color threshold ranges were defined in Python, color classification based on HSV was implemented for each part, and finally the mask was applied.

[0044] In HSV color classification, four different colors were used to classify different parts of the fish body. Among them:

[0045] The green color range was defined as lower = np.array([40, 50, 50]), upper = np.array([90, 255, 255]),

[0046] Define the red color range as lower = np.array([0, 50, 50]), upper = np.array([10, 255, 255]),

[0047] Define the blue color range as lower = np.array([100, 50, 50]), upper = np.array([130, 255, 255]),

[0048] Define the yellow color range as lower = np.array([15, 50, 50]), upper = np.array([40, 255, 255]);

[0049] Different colors show different parts of the fish body.

[0050] Step five: contour finding and minimum bounding box acquisition:

[0051] Use the cv2.findCounters() function of Open cv to find the contours of each part of the fish body after color classification, and acquire the minimum bounding box of each part. Use the cv2.minAreaRect() function, and the return value contains 5 predicted values, i.e. (x, y, w, h, θ), where the center coordinates of the matrix are (x, y), the width and height of the matrix are (w, h), and the rotation angle is θ. Take the pixel value of the width and height of the matrix as the pixel value of the width and height of the detected part. The effect is shown in the accompanying Figure 3 .

[0052] Step six: pixel value conversion:

[0053] Since the actual length of each part of the rainbow trout is greater than the width, the length and width need to be compared, and the longer one is defined as the length of each part of the fish body. By referring to the scale, the scale value is obtained in advance according to the formula: length (cm) = number of pixels (pixel number) / scale value (pixel number / cm). Then the actual length of the rainbow trout can be predicted according to this scale value.

[0054] In a specific embodiment, the actual length of the rainbow trout is measured in advance as a reference according to the scale, and then the pixel value is calculated from the minimum bounding box. Then the characteristic size per centimeter of the actual length can be converted into the number of pixels in the computer to determine the scale value. In this embodiment, the scale value can be determined by referring to the reference scale, which is 12.5. Therefore, the actual length of the rainbow trout = pixel value / 12.5.

[0055] Body weight prediction:

[0056] A previous study (see the literature: Huang Zhenli, Chang Jianbo. Fractal characteristics in the relationship between fish body length and body weight [J]. Acta Hydrobiologica Sinica, 1999, 23(4): 330-336.) shows that the relationship between fish body length and body weight conforms to the power function W = bL a Equation, according to the power function relationship curve fitted by the relationship between the actual body length and body weight of 50 rainbow trout, the body weight of rainbow trout can be predicted by the body length of the fish.

[0057] In a specific embodiment, the function relationship between the body length and body weight of rainbow trout is actually calculated as: W = 0.12 x L 2.41 , the correlation coefficient R 2 = 0.93, which represents a high fitting degree between the body length and body weight of rainbow trout, and the fish body weight can be predicted by the fish body length according to the function. The system calculation results are shown in the table:

[0058] Table 1: Estimation results of rainbow trout body length

[0059]

[0060] Note: In Table 1, P: predicted value, R: true value.

[0061] Step seven: gender judgment:

[0062] Rainbow trout has obvious gender dimorphism, compared with male, female rainbow trout individuals are larger, and the head is short and wide. In a specific embodiment, the minimum circumscribed matrix of the head of rainbow trout is used for gender judgment, and the classification linear kernel function y = 1.57x-8.55 is obtained by processing the head data of 60 fish of different genders by machine learning support vector machine, as shown in Figure 4 , wherein male rainbow trout is more concentrated below the classification line, and female rainbow trout is more concentrated above the classification line, and the gender of rainbow trout can be judged according to the relationship.

[0063] The above description is only a preferred embodiment of the present application, and is not a limitation of the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions described in the foregoing embodiments, or make equivalent replacements to some technical features. Any modification, equivalent replacement, modification, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for deep learning-based rainbow trout phenotype measurement and gender determination, characterized in that, It comprises the following steps: Step one, collect several images containing the whole rainbow trout, and label and preprocess each part of the rainbow trout, label the fish pixels in the image as four categories of head, trunk, tail and tail fin, as the data set for model training; Step two, establish an initial U-net network model and train it with the data set obtained by preprocessing to obtain an optimal model file that can effectively segment each part of the fish body; Step three, use the trained optimal model to perform semantic segmentation on the image to complete the segmentation of each part of the rainbow trout; Step four, according to the prediction results of each part of the rainbow trout after semantic segmentation, define the HSV color threshold range of different parts respectively, realize the HSV-based color classification of each part, and finally merge and apply the mask; Step five, find the contour of each part of the classified fish body respectively, and obtain the minimum bounding matrix of each part. The function can return the length and width pixel values of the minimum bounding matrix; Step six, determine the length to pixel value ratio, calculate the actual length of the rainbow trout according to the ratio, and calculate the body weight of the fish body according to the functional relationship between the body length and the body weight; Step seven, according to the length-width ratio of the minimum bounding matrix of the head, use the support vector machine of machine learning to obtain the linear kernel function of gender binary classification, and judge the gender of the rainbow trout based on the linear kernel function.

2. The method of claim 1, wherein, When the image of step one is collected, a white PVC plastic plate is used, and a ruler is fixed at the bottom of the white background to compare the pixel measurement value with the physical distance and determine the scale value.

3. The method of claim 1, wherein, In step one, hundreds of whole rainbow trout images are taken, and the fish pixels in the images are manually labeled into four categories in Labelme software, which are head, trunk, tail and tail fin. The labeled images are used as data sets, which are divided into training set images and validation set images according to the ratio of 9:

1. The training set images are input into the initial network model after training, and the loss function is calculated.

4. The method of claim 1, wherein, In step two, the model training is 50 epochs, the batch size is set to 2, the maximum learning rate is 1e-4, NVIDIA A30 GPU is used as the accelerator, and the loss function is returned as the evaluation index after model training.

5. The method of claim 1, wherein, In step three, under the premise of ensuring that the image is as clear as possible, the image acquisition device can be used to collect images of any size and resolution, and the trained Unet model can be used to perform semantic segmentation on the image to predict each part of the rainbow trout body according to the image information.

6. The method of claim 1, wherein, In step four, in the HSV-based color classification, four different colors are used to classify different parts of the fish body, wherein the green color range is defined as lower=np.array([40,50.50]), upper=np.array([90,255,255]), the red color range is defined as lower=np.array([0,50,50]), upper=np.array([10,255,255]), the blue color range is defined as lower=np.array([100,50.50]), upper=np.array([130,255,255]), and the yellow color range is defined as lower=np.array([15,50,50]), upper=np.array([40,255,255]).

7. The method of claim 1, wherein, In step five, the cv2.findCounters() function of Open cv is used to find the contours of each part of the fish body after color classification, and the cv2.minAreaRect() function is used to obtain the minimum bounding rectangle of each part, and the return value contains five predicted values, i.e. (x, y, w, h, θ), wherein the center coordinates of the matrix are (x, y), the width and height of the matrix are (w, h), and the rotation angle is θ. The pixel values of the width and height of the matrix are used as the pixel values of the width and height of the detected part.

8. The method of claim 2, wherein, In step six, the longer part is defined as the length of each part of the fish body, and the actual length of the rainbow trout is obtained by using a ruler according to the formula: length=pixel number / scale value, wherein the unit of length is cm. The scale value is obtained in advance, which is the number of pixels per centimeter, and then the actual length of the rainbow trout is predicted according to the scale value.

9. The method of claim 1, wherein, In step six, the fish body length-weight conforms to the power function W = bL a Equation, according to the rainbow trout actual body length-weight relationship fitting power function curve, through the relationship curve can be predicted by rainbow trout fish body length fish body weight; Wherein, W is the body weight of the fish, L is the body length of the fish, and a, b are parameters.

10. The method of claim 1, wherein, In step seven, the gender judgment is based on the fact that rainbow trout has a clear gender dimorphism, and compared with male rainbow trout, female rainbow trout is larger and has a short and wide head. Therefore, the minimum bounding rectangle of the head of rainbow trout is used for gender judgment, and according to the difference in the length-width ratio of the head of male and female rainbow trout, the head data of several fish of different genders are processed by machine learning support vector machine to obtain the classification linear kernel function y=cx-d, and the gender of rainbow trout is judged according to this relationship; wherein, x is the head length of the fish, y is the head width of the fish, and c, d are parameters.

Images