Rainbow trout phenotype measurement and gender judgment method based on deep learning

Abstract

The invention belongs to the technical field of agricultural aquaculture, and relates to a rainbow trout phenotype measurement and gender judgment method based on deep learning, which comprises the following steps of: performing labeling preprocessing on each part of rainbow trout in a picture to serve as a training set and a verification set of a model; a Unet model of a full convolutional neural network is used for training, and an optimal model is obtained to complete segmentation of each part of the rainbow trout; color threshold values of all parts of the segmented rainbow trout are defined, contour searching is carried out, and a minimum external matrix is obtained; calculating the actual length and width of each part of the rainbow trout according to the ratio relationship between the physical distance and the image pixels, and calculating the body weight of the rainbow trout according to the function relationship between the body length and the body weight; and judging the gender of the rainbow trout based on a support vector machine according to the length-width ratio relation of the head. According to the method, phenotype information such as body length, overall length, body width and weight of oncorhynchus mykiss can be rapidly obtained, and damage to oncorhynchus mykiss is reduced to the maximum extent while the efficiency is improved; the rainbow trout is subjected to sex identification through image processing, and the production and processing requirements of the rainbow trout are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural aquaculture, and specifically relates to a method for measuring rainbow trout phenotype and determining sex based on deep learning. Background Art

[0002] Rainbow trout (Oncorhynchus mykiss) has a high economic impact worldwide. Due to the increasing market demand, the production of rainbow trout has also been increasing. In the breeding process, the phenotypic and morphological indicators of rainbow trout, such as body length, body width and weight, can not only be used to evaluate the production, but also provide a reference for genetic breeding. Rainbow trout has obvious sexual dimorphism. Sexually mature female rainbow trout are larger than male individuals, with a short and wide head and good muscle quality. The differences in these phenotypic characteristics will affect product quality and consumer choice. Female rainbow trout is very popular in the market. The traditional method of obtaining phenotypic and morphological indicators is generally to measure the body of rainbow trout with a ruler, but this method is not only inefficient but also harmful; in addition, the previous method of using an ultrasonic instrument to identify the sex of rainbow trout is expensive.

[0003] Therefore, the development and optimization of relevant sex identification technology for rainbow trout is an important basis for the selection of good varieties and increasing aquaculture production efficiency; and how to obtain its morphological parameters while minimizing damage to the fish body has also become an urgent problem to be solved in rainbow trout farming. Summary of the invention

[0004] The purpose of the present invention is to overcome the difficulties in fish morphological measurement and sex determination in the prior art, and proposes a rainbow trout phenotype measurement and sex determination method based on deep learning. The semantic segmentation of deep learning is combined with Open cv image processing to obtain morphological indicators such as body length, body width and weight of rainbow trout, and the sex of rainbow trout is determined according to the head proportion relationship through a machine learning support vector machine.

[0005] The technical solution of the present invention is:

[0006] A method for measuring rainbow trout phenotype and determining sex based on deep learning, comprising the following steps:

[0007] Step 1: Collect several images of the whole body of rainbow trout, and perform labeling preprocessing on each part of the rainbow trout, labeling the fish body pixels in the image into four categories: head, trunk, tail and caudal fin, as a data set for model training;

[0008] Step 2: Establish an initial U-net network model and train it with the preprocessed data set to obtain the optimal model file that can effectively segment the various parts of the fish body;

[0009] Step 3: Use the trained optimal model to perform semantic segmentation on the image and complete the segmentation of the various parts of the rainbow trout;

[0010] Step 4: According to the prediction results of each part of the rainbow trout body after semantic segmentation, the HSV color threshold range of different parts is defined respectively, and the HSV-based color classification is implemented for each part, and finally the mask is combined and applied;

[0011] Step 5: Search the contours of each part of the classified fish body and obtain the minimum circumscribed matrix of each part. The function can return the length and width pixel values ​​of the minimum circumscribed matrix.

[0012] Step 6: determine the ratio between the length and the pixel value, calculate the actual length of the rainbow trout according to the ratio, and calculate the weight of the fish according to the functional relationship between the length and weight of the fish;

[0013] Step 7: According to the length-width ratio of the head minimum circumscribed matrix, the linear kernel function for gender binary classification is predicted using the machine learning support vector machine, and the gender of rainbow trout is determined based on the linear kernel function.

[0014] Furthermore, during the image acquisition in step 1, a white PVC plastic plate is used, and a ruler is fixed to the bottom of the white background to compare the pixel measurement value with the physical distance to determine the scale value.

[0015] Furthermore, in the step one, hundreds of full-body images of rainbow trout are taken, and the fish body pixels in the images are manually labeled into four categories using Labelme software, namely, head, trunk, tail, and caudal fin. The labeled images are used as data sets and divided into training set images and validation set images in a ratio of 9:1. The training set images are trained and then input into the initial network model, and the loss function is calculated.

[0016] Furthermore, in step 2, the model is trained for 50 epochs, the batch size is set to 2, the maximum learning rate is 1e-4, and the NVIDIA A30 GPU is used as an accelerator. After the model is trained, the loss function is returned as an evaluation indicator.

[0017] Furthermore, in step three, while ensuring that the image is as clear as possible, an image acquisition device can be used to capture images of any size and resolution, and a trained Unet model can be used to perform semantic segmentation on the image to predict the various parts of the rainbow trout's body based on the image information.

[0018] Furthermore, in the step 4, in the HSV-based color classification, four different colors are used to classify different parts of the fish body, wherein the green range is defined as lower = np.array([40,50.50]), upper = np.array([90,255,255]), the red range is defined as lower = np.array([0,50,50]), upper = np.array([10,255,255]), the blue range is lower = np.array([100,50.50]), upper = np.array([130,255,255]), and the yellow range is lower = np.array([15,50,50]), upper = np.array([40,255,255]).

[0019] Furthermore, in the step five, the cv2.findCounters() function of Open cv is used to search the contours of each part of the fish body after color classification, and the cv2.minAreaRect() function is used to obtain the minimum external matrix of each part. The return value contains 5 predicted quantities, namely (x, y, w, h, θ), where the center point coordinates of the matrix are (x, y), the width and height of the matrix are (w, h), the rotation angle is θ, and the pixel values ​​of the width and height of the matrix are used as the pixel values ​​of the width and height of the detection part.

[0020] Furthermore, in step 6, the longer one is defined as the length of each part of the fish body, and the actual length of the rainbow trout is obtained by using a ruler according to the formula: length = number of pixels / scale value; wherein the unit of length is cm;

[0021] The scale value is calculated in advance, which is obtained by the number of pixels / cm, and then the actual length of rainbow trout is predicted based on the scale value.

[0022] Furthermore, in step 6, the relationship between the length and weight of the fish body conforms to the power function W=bL a The equation is used to fit a power function relationship curve based on the actual length-weight relationship of rainbow trout. The body weight of rainbow trout can be predicted from the body length through this relationship curve.

[0023] Wherein, W is the weight of the fish, L is the length of the fish, a and b are parameters, and the specific values ​​of a and b can be obtained by fitting the relationship curve.

[0024] Furthermore, the sex determination in step 7 is based on the obvious sex dimorphism of rainbow trout. Compared with males, female rainbow trout are larger and have shorter and wider heads. Therefore, the minimum circumscribed matrix of the rainbow trout head is used for sex determination. According to the difference in the length-width ratio of the male and female rainbow trout heads, the head data of several fish of different sexes are processed by a machine learning support vector machine to obtain the classification linear kernel function y=cx-d, and the sex of the rainbow trout is determined based on this relationship. Among them, c and d are parameters, and their specific values ​​can be obtained through the length-width values ​​of the individual rainbow trout heads and the binary classification line graph.

[0025] Beneficial effects of the present invention:

[0026] The deep learning-based rainbow trout phenotypic measurement and sex determination method proposed in the present invention can quickly obtain phenotypic information such as body length, total length, body width and weight of rainbow trout, solving the problem of low efficiency in traditional manual acquisition of rainbow trout phenotypic data, and can also minimize damage to the fish body; at the same time, the sex of rainbow trout is identified through image processing, which improves the production efficiency in the breeding process and meets the subsequent production and processing, genetic breeding and other needs of rainbow trout. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flow chart of the method provided by the present invention;

[0028] Figure 2 The loss graph for model training;

[0029] Figure 3 This is a schematic diagram of model segmentation;

[0030] Figure 4 This is the relationship between the length and width of the head of male and female rainbow trout. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In order to further understand the present invention, the present invention will be further described in conjunction with the accompanying drawings and embodiments.

[0033] like Figure 1 As shown, an embodiment of the present invention provides a method for measuring rainbow trout phenotype and determining sex based on deep learning, comprising the following steps:

[0034] Step 1: Dataset collection and image processing:

[0035] During the breeding process, hundreds of images containing the whole body of rainbow trout were taken using a digital camera Canon 70D. White PVC plastic plates were used to reduce background noise. A ruler was fixed at the bottom of the white background to calculate the relationship between pixel measurement values ​​and physical distances, that is, to compare pixel measurement values ​​with physical distances to determine the scale value.

[0036] The fish body pixels in the image were manually labeled into four categories using Labelme software, namely head, trunk, tail and caudal fin. The labeled images were used as data sets for model training and divided into training set images and validation set images in a ratio of 9:1. The training set images were input into the initial network model after training, and the loss function was calculated.

[0037] Step 2: Model training:

[0038] The initial Unet network model was established and the pre-processed training set was used for training on the Unet model. The model was trained for 50 rounds, the batch size was set to 2, the maximum learning rate was 1e-4, and the NVIDIA A30 GPU was used as an accelerator. After the model training, the loss function was returned as the evaluation indicator. The results are shown in the figure below. Figure 2 As shown in the figure, as the number of model training rounds increases, the loss value gradually decreases.

[0039] After 50 rounds of model training, the best model file that can effectively segment the various parts of the fish body is obtained, which is used for the subsequent segmentation of various parts of the fish body.

[0040] Step 3: Rainbow trout image collection and semantic segmentation:

[0041] On the premise of ensuring that the image is as clear as possible, image acquisition equipment can be used to collect images of any size and resolution, and the trained Unet model can be used to perform semantic segmentation on the image to predict the various parts of the rainbow trout's body based on the image information.

[0042] Step 4: Define color threshold:

[0043] According to the prediction results of various parts of the rainbow trout body after semantic segmentation, the HSV color threshold range of different parts was defined in Python, and HSV-based color classification was implemented for each part, and finally the masks were merged and applied.

[0044] In HSV color classification, four different colors are used to classify different parts of the fish body. Among them:

[0045] Define the green range as lower = np.array([40,50.50]), upper = np.array([90,255,255]),

[0046] Define the red range as lower = np.array([0,50,50]), upper = np.array([10,255,255]),

[0047] Define the blue range as lower = np.array([100,50.50]), upper = np.array([130,255,255]),

[0048] Define the yellow range as lower = np.array([15,50,50]), upper = np.array([40,255,255]);

[0049] Different colors show different parts of the fish.

[0050] Step 5: Find the contour and obtain the minimum circumscribed matrix:

[0051] Use Open cv's cv2.findCounters() function to find the contours of each part of the fish body after color classification, and obtain the minimum external matrix of each part. Use cv2.minAreaRect() function, and the return value contains 5 predicted quantities, namely (x, y, w, h, θ), where the center point coordinates of the matrix are (x, y), the width and height of the matrix are (w, h), the rotation angle is θ, and the pixel values ​​of the width and height of the matrix are used as the pixel values ​​of the width and height of the detection part. The effect is as shown in the attached figure. Figure 3 shown.

[0052] Step 6: Pixel value conversion:

[0053] Since the actual length of each part of the rainbow trout is greater than the width, it is necessary to compare the length with the width and define the longer one as the length of each part of the fish body. By using the reference object scale, according to the formula: length (cm) = number of pixels (number of pixels) / scale value (number of pixels / cm), the scale value is obtained in advance. Then the actual length of the rainbow trout can be predicted based on this scale value.

[0054] In a specific embodiment, the actual length of the rainbow trout is measured in advance according to the ruler as a reference, and then the pixel value is calculated by the minimum circumscribed matrix, and then the characteristic size per centimeter of the actual length is converted into the number of pixels in the computer to determine the scale value. In this embodiment, the scale value can be determined to be 12.5 by the reference ruler, so the actual length of the rainbow trout = pixel value / 12.5.

[0055] Weight prediction:

[0056] Previous studies (see reference: Huang Zhenli, Chang Jianbo. Fractal characteristics of the relationship between fish body length and weight [J]. Acta Hydrobiologica Sinica, 1999, 23(4): 330-336.) showed that the relationship between fish body length and weight conforms to the power function W = bL a The equation was used to fit a power function curve based on the actual length-weight relationship of 50 rainbow trout. The body weight of rainbow trout can be predicted by this curve.

[0057] In a specific embodiment, the actual calculated functional relationship between the length and weight of rainbow trout is: W = 0.12 × L 2.41 , correlation coefficient R 2 =0.93, indicating that the length and weight of rainbow trout have a high degree of fit. Based on this function, the weight of fish can be predicted from the length of the fish. The calculation results of this system are shown in the table:

[0058] Table 1 Rainbow trout body length estimation results

[0059]

[0060] Note: In Table 1, P: predicted value, R: true value.

[0061] Step 7: Gender determination:

[0062] Rainbow trout has obvious sexual dimorphism. Compared with males, female rainbow trout are larger and have shorter and wider heads. In a specific embodiment, the minimum circumscribed matrix of the rainbow trout head is used for gender determination. The head data of 60 fish of different genders are processed by machine learning support vector machine to obtain the classification linear kernel function y=1.57x-8.55, such as Figure 4 As shown, male rainbow trout are more concentrated below the classification line, while female rainbow trout are more concentrated above the classification line. The gender of rainbow trout can be judged based on this relationship.

[0063] The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, modification, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for measuring rainbow trout phenotype and determining sex based on deep learning, characterized in that: The following steps are involved: Step 1: Collect several images of the whole body of rainbow trout, and perform labeling preprocessing on each part of the rainbow trout, labeling the fish body pixels in the image into four categories: head, trunk, tail and caudal fin, as a data set for model training; Step 2: Establish an initial U-net network model and train it with the preprocessed data set to obtain the optimal model file that can effectively segment the various parts of the fish body; Step 3: Use the trained optimal model to perform semantic segmentation on the image and complete the segmentation of the various parts of the rainbow trout; Step 4: According to the prediction results of each part of the rainbow trout body after semantic segmentation, the HSV color threshold range of different parts is defined respectively, and the HSV-based color classification is implemented for each part, and finally the mask is combined and applied; Step 5: Search the contours of each part of the classified fish body and obtain the minimum circumscribed matrix of each part. The function can return the length and width pixel values ​​of the minimum circumscribed matrix. Step 6: determine the ratio between the length and the pixel value, calculate the actual length of the rainbow trout according to the ratio, and calculate the weight of the fish according to the functional relationship between the length and weight of the fish; Step 7: According to the length-to-width ratio of the head minimum circumscribed matrix, the linear kernel function for gender binary classification is predicted using the machine learning support vector machine, and the gender of rainbow trout is determined based on the linear kernel function.

2. The method according to claim 1, characterized in that When collecting the image in step 1, a white PVC plastic plate is used, and a ruler is fixed on the bottom of the white background to compare the pixel measurement value with the physical distance to determine the scale value.

3. The method according to claim 1, characterized in that In the step 1, hundreds of full-body images of rainbow trout are taken, and the fish body pixels in the images are manually labeled into four categories using Labelme software, namely, the head, trunk, tail, and caudal fin. The labeled images are used as data sets and divided into training set images and validation set images in a ratio of 9:

1. The training set images are trained and input into the initial network model, and the loss function is calculated.

4. The method according to claim 1, characterized in that: In step 2, the model is trained for 50 epochs, the batch size is set to 2, the maximum learning rate is 1e-4, and the NVIDIA A30 GPU is used as an accelerator. After the model is trained, the loss function is returned as the evaluation indicator.

5. The method according to claim 1, characterized in that In the step three, under the premise of ensuring that the image is as clear as possible, an image acquisition device can be selected to acquire images of any size and resolution, and a trained Unet model can be used to perform semantic segmentation on the image to predict the various parts of the rainbow trout's body based on the image information.

6. The method according to claim 1, characterized in that In the step 4, in the HSV-based color classification, four different colors are used to classify different parts of the fish body, wherein the green range is defined as lower=np.array([40,50.50]), upper=np.array([90,255,255]), the red range is defined as lower=np.array([0,50,50]), upper=np.array([10,255,255]), the blue range is lower=np.array([100,50.50]), upper=np.array([130,255,255]), and the yellow range is lower=np.array([15,50,50]), upper=np.array([40,255,255]).

7. The method according to claim 1, characterized in that In the step 5, the cv2.findCounters() function of Open cv is used to search the contours of each part of the fish body after color classification, and the cv2.minAreaRect() function is used to obtain the minimum external matrix of each part. The return value contains 5 prediction quantities, namely (x, y, w, h, θ), where the center point coordinates of the matrix are (x, y), the width and height of the matrix are (w, h), the rotation angle is θ, and the pixel values ​​of the width and height of the matrix are used as the pixel values ​​of the width and height of the detection part.

8. The method according to claim 2, characterized in that: In step 6, the longer one is defined as the length of each part of the fish body, and the actual length of the rainbow trout is obtained by using a ruler according to the formula: length = number of pixels / scale value; wherein the unit of length is cm; The scale value is calculated in advance as the number of pixels per centimeter, and then the actual length of the rainbow trout is predicted based on the scale value.

9. The method according to claim 1, characterized in that: In step 6, the relationship between the length and weight of the fish body conforms to the power function W=bL a The equation is used to fit a power function relationship curve based on the actual body length-weight relationship of rainbow trout. The body weight of rainbow trout can be predicted from the body length through this relationship curve. Wherein, W is the weight of the fish and L is the length of the fish.

10. The method according to claim 1, characterized in that The gender judgment in step seven is based on the fact that rainbow trout has obvious sexual dimorphism. Compared with males, female rainbow trout are larger and have shorter and wider heads. Therefore, the minimum circumscribed matrix of the rainbow trout head is used for gender judgment. According to the difference in the length-to-width ratio of the heads of male and female rainbow trout, the head data of several fish of different genders are processed by a machine learning support vector machine to obtain the classification linear kernel function y=cx-d, and the gender of the rainbow trout is judged based on this relationship.

Images