A method for determining phenotypic data of Penaeus vannamei
By employing computer vision and mathematical modeling methods, combined with GCN image segmentation and stereoscopic correction, the problem of non-destructive, rapid, and high-precision measurement of Litopenaeus vannamei phenotypic characteristics was solved, enabling accurate measurement of features such as body length and weight.
Patent Information
- Application Number
- CN202210298060.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-24
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-03-24
AI Technical Summary
Existing methods for phenotyping of Litopenaeus vannamei rely on manual measurement, which is labor-intensive, inefficient, and susceptible to subjective influence. Traditional image processing methods are not accurate enough in complex scenarios, and dehydration measurement can easily damage the shrimp, making it impossible to measure weight quickly and non-destructively.
By combining computer vision and mathematical modeling, and utilizing the GCN image segmentation algorithm and stereo spatial correction model, along with the SVR model, non-destructive measurement of Litopenaeus vannamei phenotypic data, including the calculation of features such as body length and weight, can be achieved.
It enables high-precision phenotypic measurement of Litopenaeus vannamei in a free state in water, reduces damage from human intervention, and improves measurement speed and accuracy. It can accurately calculate multiple phenotypic characteristics of shrimp, such as body length and weight.
Smart Images

Figure CN114612495B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for determining phenotypic data of Penaeus vannamei, which is based on computer vision and combines mathematical modeling to realize phenotypic measurement, and belongs to the field of artificial intelligence. Background Art
[0002] With its short growth cycle and high nutritional value, Penaeus vannamei is one of the aquatic animals with the highest global production and economic value. At the same time, in the face of huge social demand, the breeding technology of Penaeus vannamei has also attracted more and more attention. The phenotypic characteristics of Penaeus vannamei, such as body length and body width, are closely related to the weight of the shrimp. In aquaculture, these phenotypic characteristics are usually collected to judge the growth of the shrimp, and this information is used as an important reference for feeding, fishing and breeding.
[0003] At present, the phenotypic characteristics of Litopenaeus vannamei are mainly measured by manual measurement, which is labor-intensive, inefficient, and the measurement results are easily affected by subjective factors, which may even cause irreversible effects on the growth of shrimp. Therefore, accurate, rapid, and non-destructive measurement of phenotypic signs is an important basis for the identification of germplasm resources and the cultivation of improved varieties.
[0004] Among the known studies, some have used the traditional threshold-based image segmentation method, measured with a vernier caliper, and combined with the segmented pixel area and other features to propose a linear model for calculating the length of the cephalothorax. The image is segmented into background and foreground using image binarization, and multivariate features including shrimp area, perimeter, height and width are established for the segmented results to predict the weight of the shrimp. After using the target detection algorithm to detect the position of the shrimp in the image, the contour area of the shrimp is extracted in combination with the method of digital image processing, and the body length is studied. The advantage is that there is no need to intervene in the shrimp, and the image of the shrimp in the water is directly captured to calculate the body length of the shrimp from a single image. However, it does not combine the morphological characteristics of the shrimp that will curl up, and there are limitations in the calculation of body length.
[0005] Traditional methods based on image processing are difficult to show advantages in complex scenes. With the development of artificial intelligence, some effective methods have emerged in the field of computer vision. They have broken through the limitations of designing feature extractors in traditional image processing and have shown amazing results in complex application scenarios.
[0006] In image segmentation, network architectures such as FCN, SegNet, GCN, and DeepLab series have emerged in a spurt and are used in different scenarios. For these models, some applications apply them to fish for image segmentation. For example, the Mask R-CNN algorithm is used for image segmentation to calculate the size of different phenotypic features such as the length, width, and eyes of the fish. The SegNet network structure is used to segment the Nile tilapia image, and the effects of network structures with different numbers of layers on the segmentation effect are compared, and different features such as weight are calculated. In addition, after the obtained fish image is segmented using the GrabCut algorithm, a variety of methods are used to extract the features of the fish body, and the weight of the fish is predicted using methods such as support regression.
[0007] Different types of shrimps exhibit different behavioral characteristics in a free state. At present, it is rare to use neural network-based image segmentation methods to achieve fast, efficient and non-destructive measurements of Penaeus vannamei.
[0008] The existing problems in the phenotypic measurement of Penaeus vannamei mainly exist in the following aspects:
[0009] 1. Some methods in traditional image processing, such as threshold-based image segmentation methods, are far inferior to today's neural network methods in achieving segmentation accuracy.
[0010] 2. Most phenotypic measurement methods are performed under dehydration conditions, which can easily cause irreversible damage to shrimp. In addition, the number of phenotypic indicators measured is relatively small, and the calculation of body weight is not achieved.
[0011] Therefore, the study of how to quickly and non-destructively calculate the phenotypic characteristics of Penaeus vannamei and predict its weight plays an important role in the development of aquaculture. Summary of the invention
[0012] The technical problem to be solved by the present invention is to provide a method for measuring phenotypic data of Penaeus vannamei by combining computer vision and mathematical modeling methods in order to achieve non-destructive measurement of the phenotype of Penaeus vannamei in view of the deficiencies in the prior art.
[0013] In order to achieve the above object, the present invention adopts the following technical solution:
[0014] A method for determining phenotypic data of Penaeus vannamei comprises the following steps:
[0015] (1) using a camera to collect side view images and top view images of Penaeus vannamei;
[0016] (2) using the GCN image segmentation algorithm to perform image segmentation on the top view and side view images obtained in step (1), segmenting different parts of the body of the Penaeus vannamei, and using the sizes of pixels occupied by the different parts segmented from the Penaeus vannamei image as input data for steps (3) and (4);
[0017] (3) calculating partial phenotypic data of Penaeus vannamei from the pixel lengths occupied by the segmented different parts according to the proportionality factor of the unit pixel length of the reference object calculated in advance in the container;
[0018] (4) In response to the different postures of the shrimp, such as tilt and bending, in a container with water, a three-dimensional space correction model is used to calculate the corrected abdominal length of the shrimp; the size of the pixels occupied by different parts is used as input data, and the network structure in the neural network is used to predict the body length;
[0019] (5) Using the SVR model, different phenotypic data and the pixel sizes of different parts are combined as input data to measure the weight data.
[0020] In the above technical solution, in step (1), the vannamei shrimp is placed in a container filled with water, and then the container is placed in an image acquisition device for the vannamei shrimp, and two cameras are used to respectively acquire images of the vannamei shrimp to obtain a top view image and a side view image of the vannamei shrimp. A certain volume of water can ensure the swimming space of the shrimp.
[0021] In the above technical solution, in step (2), the different parts segmented from the side view image are: eyes, tail, abdomen, frontal horns and cephalothorax; the different parts segmented from the top view image are: tactile whip, tail limbs, frontal horns, eyes, cephalothorax, abdomen.
[0022] In the above technical solution, in step (3), when calculating the partial phenotypic data of Penaeus vannamei, the calculation formula is: unit pixel length (proportional factor) × pixel length occupied by the part = the size of the phenotypic data of the part.
[0023] In the above technical solution, in step (3), the phenotypic data calculated according to the proportional factors include: the length of the frontal horn, the length, width and height of the cephalothorax, and the width and height of the six abdominal segments.
[0024] In the above technical solution, in step (4), spatial curve integral is used to build a model using the least squares method to calculate the corrected abdominal length; and the MLP network structure in the neural network is combined to use the pixel sizes of different segmented parts to predict the body length.
[0025] In the above technical solution, in step (4), when using a three-dimensional space correction model, a plane perpendicular to the XOY plane is obtained through the top view image, and a curved surface perpendicular to the XOZ plane is obtained through the side view image. The intersection of the two surfaces is the abdominal length calculated by spatial curve integration; the plane equation and the surface equation are obtained by modeling the image coordinate points in the segmented abdominal position using the least squares method to obtain the corresponding equations, and finally these equations are spatially integrated to obtain the abdominal length.
[0026] In the above technical solution, in step (4), the MLP network structure in the neural network is used to realize the prediction of body length by using the phenotypic data size of different parts segmented: the number of pixels occupied by the different parts segmented is input into the MLP network structure to realize the prediction of body length, and the different parts using the number of pixels are: abdomen in top view and side view, cephalothorax and frontal horn in side view, eyes in top view and side view, cephalothorax in top view, tail limb in side view, and frontal horn in top view. The mean absolute error MAE on the validation set is 0.27cm.
[0027] In the above technical solution, in step (5), the SVR model is used to realize the determination of weight data by utilizing the size of phenotypic data of different parts segmented out: the number of pixels occupied by the different parts segmented out and the size of phenotypic data of different parts are input into the SVR model to realize the prediction of weight; the different parts using the number of pixels are: eyes in top view and side view, abdomen in top view and side view, cephalothorax in top view, cephalothorax and frontal angle in side view; the different parts using the phenotypic size data are: AL (abdomen length), AW1-6 (width of the first to sixth abdominal segments), AH1-6 (height of the first to sixth abdominal segments). The mean absolute error MAE finally obtained on the test data set is 0.91g.
[0028] In the present invention, the phenotypic data finally measured include: the phenotypic data of Penaeus vannamei include body length, total length, frontal horn length, cephalothorax length, width and height, abdomen length, width and height of six abdominal segments in the abdomen, and body weight.
[0029] Compared with the prior art, the method of the present invention has the following advantages:
[0030] 1. The phenotypic data of Penaeus vannamei is measured in a free state in water, without intervention of the shrimp, thus reducing the damage caused by manual measurement.
[0031] 2. The present invention measures the body length, total length, frontal horn length, cephalothorax length, width and height, abdomen length, width and height of six abdominal segments, and body weight of Penaeus vannamei. Compared with previous inventions, this invention measures the most phenotypic characteristics.
[0032] 3. The present invention introduces the image segmentation technology in deep learning into the determination of the phenotypic data of Penaeus vannamei, realizes high-precision segmentation of different parts of the shrimp, and provides speed and accuracy guarantees for phenotypic determination.
[0033] 4. Compared with previous methods, the present invention realizes the complex situation of shrimp bodies in water, performs spatial modeling correction, and increases the robustness of phenotypic measurement algorithms. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a schematic diagram of top view and side view image acquisition in Example 1 of the present invention;
[0035] Figure 2-1 It is the phenotypic data index of the cephalothorax height (a) and the abdominal segment height (b) provided in Example 1 of the present invention;
[0036] Figure 2-2 The phenotypic data indicators of the frontal horn length (c), cephalothorax length (d), cephalothorax width (e), abdomen length (f), abdominal segment width (g), and body length (h) provided in Example 1 of the present invention are measured;
[0037] Figure 3-1 are different parts segmented by the top view image segmentation in Example 1 of the present invention;
[0038] Figure 3-2 are different parts segmented out by the side view image segmentation in Embodiment 1 of the present invention;
[0039] Figure 4-1 1 is a schematic diagram of calculating the pixel size using the tilt correction AL2 of the abdomen length in Example 1 of the present invention (c represents the number of pixel columns occupied in the vertical direction, r represents the number of pixels occupied in the horizontal direction, AL2 represents the abdomen length, Y represents the vertical direction, and X represents the horizontal direction);
[0040] Figures 4-2 to 4-6 is a schematic diagram of calculating pixel sizes at different locations in Example 1 of the present invention (c represents the number of pixel columns in the vertical direction, r represents the number of pixels in the horizontal direction, c represents the number of pixels in the horizontal direction, n The vertical positions of the different segmented parts in the image matrix are listed in the c1-cn interval, r n Represents that the horizontal position of the segmented different parts in the image matrix is in the interval r1-rn, where Y represents the vertical direction and X represents the horizontal direction);
[0041] Figure 5 is a spatial curve model for the abdominal length correction in Example 1 of the present invention;
[0042] Figure 6This is a schematic diagram of the network structure of the present invention using MLP to predict body length (in the middle...some network layers are omitted because there are too many network layers in the middle and they cannot be drawn, so only a few layers are drawn as a schematic). DETAILED DESCRIPTION
[0043] The specific implementation of the technical solution of the present invention is described in detail below, but the present invention is not limited to the following description:
[0044] The present invention provides a method for determining phenotypic data of Penaeus vannamei, comprising the following steps:
[0045] (1) using a camera to collect side view images and top view images of Penaeus vannamei;
[0046] (2) using the GCN image segmentation algorithm to perform image segmentation on the top view and side view images obtained in step (1), segmenting different parts of the body of the Penaeus vannamei, and using the sizes of pixels occupied by the different parts segmented from the Penaeus vannamei image as input data for steps (3) and (4);
[0047] (3) calculating partial phenotypic data of Penaeus vannamei from the pixel lengths occupied by the segmented different parts according to the proportionality factor of the unit pixel length of the reference object calculated in advance in the container;
[0048] (4) In response to the different postures of the shrimp, such as tilt and bending, in a container with water, a three-dimensional space correction model is used to calculate the corrected abdominal length of the shrimp; the size of the pixels occupied by different parts is used as input data, and the network structure in the neural network is used to predict the body length;
[0049] (5) The SVR model is used to combine different phenotypic data and the size of pixels occupied by different parts as input data to measure weight data.
[0050] The method of the present invention is described below in conjunction with specific embodiments:
[0051] Embodiment 1:
[0052] A method for determining phenotypic data of Penaeus vannamei comprises the following steps:
[0053] (1) Using a camera to collect side and top view images of Penaeus vannamei:
[0054] In order to obtain the top view and side view images of the vannamei shrimp without loss, a container with a certain amount of water is placed in a fixed position and two cameras are used to capture these two images, such as Figure 1 As shown; a certain volume of water can ensure space for the shrimp to swim.
[0055] (2) Use the GCN image segmentation algorithm to segment the top view and side view images, such as Figure 3-1 and 3-2 As shown in the figure, different parts of the body of Penaeus vannamei are segmented. The different parts segmented from the side view image are: eyes, tail, abdomen, frontal horns and cephalothorax; the different parts segmented from the top view image are: tactile whip, tail limbs, frontal horns, eyes, cephalothorax, abdomen; the pixel sizes occupied by different parts of the segmented image of Penaeus vannamei (such as Figure 2-1 and 2-2 as input for the next step.
[0056] In order to ensure the accuracy of the different parts segmented, this example uses a top view dataset with 1,100 images, including 700 training sets, 100 validation sets, and 210 test sets. The side view dataset has 1,003 images, including 698 training sets, 100 validation sets, and 205 test sets. The image resolution during training is 1600x480. The segmentation MIoU obtained on the top view test set is 86.6%, and the segmentation MIoU obtained on the side view test set is 88.32%.
[0057] Among them, MIoU is the average value of the intersection and union ratio of the two sets of true labels and predicted labels for each category in each category.
[0058] (3) According to the proportionality factor of the unit pixel length of the reference object calculated in the container, the pixel length occupied by the segmented different parts × the unit pixel length (proportional factor) = the size of the phenotypic data of the part, the pixel length occupied by different parts needs to be calculated in combination with different angles, and the partial phenotypic data of the vannamei shrimp are calculated from different angles. Finally, the result with the best calculation angle is used as the method for calculating the phenotypic data. The specific calculation angles used are shown in Table 1 below;
[0059] Table 1 shows the calculation angles used in different parts when calculating the pixel length of phenotypic data.
[0060]
[0061] The pixel sizes of different parts are calculated by combining different calculation angles. Take the calculation of the tilt correction AL2 of the abdomen length as an example. Figure 4-1 As shown; the calculation formulas for other parts are as follows Figures 4-2 to 4-6 As shown; the calculation method of AL1 and FL is the same as that of CL1, and the calculation method of AW1-6 and AH1-6 is the same as that of CBH; The pixel sizes of different parts are obtained according to the formula and then multiplied by the unit pixel length (proportional factor) to obtain the phenotypic data of different parts.
[0062] (4) In order to deal with the different postures of the shrimp such as tilt and bending in the water container, modeling correction is performed to calculate the corrected body length of the shrimp. The correction is performed by spatial curve integration, as shown in Figure 4. Specifically, the plane perpendicular to the XOY plane is obtained from the top view image, and the surface perpendicular to the XOZ plane is obtained from the side view image. The intersection of the two surfaces is the abdomen length to be calculated by spatial curve integration; the plane equation and the surface equation are obtained by modeling the image coordinate points in the segmented abdomen position using the least squares method to obtain the corresponding equations, and finally these equations are spatially integrated to obtain the abdomen length.
[0063] The calculation formula of AL3 is:
[0064] In the formula for calculating the abdominal length AL3 by spatial curve integral, w(x), The plane equation and surface equation are respectively the coordinate points of the abdomen after image segmentation, and the abdomen length is finally obtained by integration. The specific solution is as follows;
[0065] At w(x): w(x)=a0+a1x
[0066]
[0067] XA=Y
[0068] A=(X T X) -1 X T Y
[0069] At
[0070]
[0071] XB=Z
[0072] B=(X T X) -1 X T Z
[0073] According to different calculation angles and correction methods, the final partial evaluation results on the data set are shown in Table 2 below:
[0074] Table 2: Some evaluation results
[0075]
[0076] The evaluation indicators in the table are R: correlation coefficient, SSR: regression sum of squares, MAE: mean absolute error, and the units of MAE and SSR are cm. The abdominal length AL3 calculated by the spatial correction model and the cephalothorax length CL1 calculated by the number of pixel columns in the vertical direction are the calculation schemes adopted by the present invention.
[0077] In addition, the body length was measured in our protocol using Figure 6 The MLP structure shown in the figure is used for prediction. The MLP network structure in the neural network is adopted. The number of pixels occupied by different segmented parts is input into the MLP network structure to realize the prediction of body length. The different segmented parts used are: abdomen in top view and side view, cephalothorax and frontal horn in side view, eyes in top view and side view, cephalothorax in top view, tail limb in side view, and frontal horn in top view.
[0078] Before inputting these feature data into the network structure, we first use standard deviation normalization to preprocess the data, and then use the MLPRegressor method in sklearn.neural_network, with the specific parameters of hidden_layer_sizes = (30, 40, 50, 60, 70, 100, 70, 40, 20, 10, 10, 10, 10, 10, 10) activation = 'relu', solver = 'sgd', alpha = 1e-4, max_iter = 5000. After cross-validation, the trained model is evaluated on the test set, and the mean absolute error MAE on the validation set is 0.27cm.
[0079] (5) Using the SVR model, the phenotypic data size of different segmented parts and the number of pixels occupied by different parts are used to measure the weight data: the number of pixels occupied by different segmented parts and the phenotypic size data of different parts are input into the SVR model to predict the weight; the different parts segmented using the number of pixels are: eyes in top view and side view, abdomen in top view and side view, cephalothorax in top view, cephalothorax and frontal horn in side view; the different parts using the phenotypic size data are: AL (abdomen length), AW1-6 (width of the first to sixth abdominal segments), AH1-6 (height of the first to sixth abdominal segments).
[0080] Before inputting these feature data into the SVR model, the data is first preprocessed using standard deviation normalization, and then the SVR method provided by the sklearn library is used, with the specific parameters: kernel = 'rbf'. After training with the cross-validation method, the trained model is evaluated on the test set, and the final mean absolute error MAE obtained on the test data set is 0.91g.
[0081] In summary, this embodiment is a method for measuring phenotypic data of Penaeus vannamei based on image segmentation and mathematical model establishment, which realizes the measurement of body length, total length, frontal horn length, cephalothorax length, width and height, abdomen length, width and height of six abdominal segments in the abdomen, and body weight. All of them are within a very small error range on the validation set. It can meet the needs in future practical use.
[0082] The above examples are only for illustrating the technical concept and technical features of the present invention, and cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the essence of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for determining phenotypic data of Penaeus vannamei, characterized in that: The following steps are involved: (1) using a camera to collect side view images and top view images of Penaeus vannamei; (2) using the GCN image segmentation algorithm to perform image segmentation on the top view and side view images obtained in step (1), segmenting different parts of the body of the Penaeus vannamei, and using the sizes of pixels occupied by the different parts segmented from the Penaeus vannamei image as input data for steps (3) and (4); (3) according to the ratio factor of the unit pixel length of the reference object calculated in advance in the container, the partial phenotypic data of Penaeus vannamei is calculated from the pixel lengths occupied by the different parts segmented; when calculating the partial phenotypic data of Penaeus vannamei, the calculation formula is: unit pixel length (ratio factor) × pixel length occupied by the part = the size of the phenotypic data of the part; (4) In response to the different tilting and bending postures of Penaeus vannamei in a water container, spatial curve integration and least squares method are used to build a model to calculate the corrected abdominal length of Penaeus vannamei; and combined with the MLP network structure in the neural network, the size of the pixels occupied by different parts of the segmentation is used as input data to use the network structure in the neural network to predict the body length; (5) Using the SVR model, the phenotypic data size of different segmented parts is used to measure the weight data: the number of pixels occupied by the different segmented parts and the size of the phenotypic data of the different parts are input into the SVR model to predict the weight; the different parts using the number of pixels are: eyes in top view and side view, abdomen in top view and side view, cephalothorax in top view, cephalothorax and frontal horn in side view; the different parts using the phenotypic size data are: AL (abdomen length), AW1-6 (width of the first to sixth abdominal segments), AH1-6 (height of the first to sixth abdominal segments).
2. The measuring method according to claim 1, characterized in that In step (1), the vannamei shrimp is placed in a container filled with water, and then the container is placed in an image acquisition device for the vannamei shrimp. Two cameras are used to respectively acquire images of the vannamei shrimp to obtain a top view image and a side view image of the vannamei shrimp.
3. The measuring method according to claim 1, characterized in that In step (2), the different parts segmented from the side view image are: eyes, tail, abdomen, frontal horns and cephalothorax; the different parts segmented from the top view image are: tactile whip, tail limbs, frontal horns, eyes, cephalothorax and abdomen.
4. The measuring method according to claim 1, characterized in that In step (3), the phenotypic data calculated based on the proportional factors include: the length of the frontal horn, the length, width and height of the cephalothorax, and the width and height of the six abdominal segments.
5. The measuring method according to claim 1, characterized in that In step (4), when using a three-dimensional space correction model, a plane perpendicular to the XOY plane is obtained through the top view image, and a curved surface perpendicular to the XOZ plane is obtained through the side view image. The intersection of the two surfaces is the abdominal length calculated by spatial curve integration; the plane equation and the surface equation are obtained by modeling the image coordinate points in the segmented abdominal position using the least squares method to obtain the corresponding equations, and finally these equations are integrated by spatial curves to obtain the abdominal length.
6. The measuring method according to claim 5, characterized in that In step (4), the MLP network structure in the neural network is used to realize the prediction of body length by utilizing the size of phenotypic data of different segmented parts: the number of pixels occupied by different segmented parts is input into the MLP network structure to realize the prediction of body length, and the different parts using the number of pixels are: abdomen in top view and side view, cephalothorax and frontal horn in side view, eyes in top view and side view, cephalothorax in top view, tail limb in side view, and frontal horn in top view.
Citation Information
Cited By
Macrobrachium rosenbergii small-size shrimp high-throughput phenotype measurement method and measurement system
CN121582324A
A method and system for high-throughput phenotyping of small-sized macrobrachium rosenbergii shrimps
CN121582324B
