High-throughput tracking measurement method, medium, device and application of individual growth phenotype of scophthalmus maximus

Abstract

The application provides a high-throughput tracking measurement method for growth phenotypes of individual Scophthalmus maximus, a medium, a device and an application thereof, and belongs to the technical field of biological identification. The method uses an image acquisition system to obtain complete images of the ventral surface and the dorsal surface of the Scophthalmus maximus, and performs pretreatment, and uses a machine learning algorithm to identify fish body ventral images that are easy to distinguish; then, a fish fin image of the Scophthalmus maximus is extracted from the dorsal image of an un-identified individual by using an image segmentation model, a deep learning algorithm is used to extract features, and a high-complexity fin image is identified; thereby, all Scophthalmus maximus individuals are identified; then, a data generation model is trained for the dorsal image of the Scophthalmus maximus individual, growth phenotype key point detection is completed, and growth traits of the Scophthalmus maximus are calculated according to key point data. The application also provides a medium, a device and an application for running the method. The method can track growth phenotype information of the whole growth cycle of the Scophthalmus maximus individual, automatically complete growth index detection, and has high precision.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of aquatic products, and particularly relates to a high-throughput tracking measurement method for individual growth phenotypes of Scophthalmus maximus, a medium, a device and application thereof. BACKGROUND

[0002] As important breeding reference data, the growth phenotype indicators of Scophthalmus maximus need a large amount of frequent measurement work. However, the existing tracking measurement methods have the following shortcomings:

[0003] (1) Low flux and large measurement error of traditional measurement tools: the traditional measurement is manually measured by using a ruler, an electronic scale and the like, and one record is observed for one indicator, which is low in flux and time-consuming; due to different measurement personnel, the observation angle and measurement habit are different, and there is a large error in the tracking measurement process, resulting in reduced data accuracy.

[0004] (2) High cost and great trauma of individual tracking method: in breeding, the electronic tag method is often used, and the individual can be tracked in the case of group culture, but this marking method is high in cost; and the fish body surface is easily damaged, and is susceptible to bacteria, and death occurs;

[0005] (3) Difficulty in completing tracking measurement of the entire growth cycle: in order to avoid damage, the electronic tag tracking cannot be carried out on fry and juvenile fish, which often affects the trait monitoring and evaluation of the entire growth cycle, and the performance of growth traits at different growth stages and different germplasms cannot be analyzed.

[0006] (4) The existing fish high-throughput measurement method and device, and the individual tracking measurement method still use the traditional electronic tag method for individual identification, and cannot solve the problems in the above points (2) and (3).

[0007] Therefore, a growth phenotype tracking measurement method with high-throughput and high automation degree without damage to individual tracking identification is needed, which is a key technical breakthrough for precision breeding. SUMMARY

[0008] In view of the above technical problems, the present application provides a high-throughput tracking measurement method for individual growth phenotypes of Scophthalmus maximus, a device and application thereof. In the previous study, it is found that the fish belly patterns of Scophthalmus maximus from juvenile fish (about 15 g) to adult fish (about 500 g) have individual differences and are unchangeable, and the textures of fins (including dorsal fin, tail fin, belly fin and anal fin) also have individual differences, although the color depth changes with growth, but the texture is basically unchanged; which provides a basic theory for feature extraction of Scophthalmus maximus individuals.

[0009] A high-throughput tracking measurement method for individual growth phenotypes of Scophthalmus maximus, characterized in that the method is as follows:

[0010] The first step is to use an image acquisition system to take complete images (top view) of the ventral and dorsal surfaces of Scophthalmus maximus respectively, and to pre-process the collected Scophthalmus maximus images, including mask annotation and image enhancement;

[0011] The second step is to use a machine learning algorithm to identify the fish body ventral image which is easy to distinguish from the pre-processed Scophthalmus maximus image in the first step; then use an image segmentation model to extract the fin image of Scophthalmus maximus from the dorsal image, use the ability of deep learning algorithm to extract features, and identify the fin image with high complexity; thereby realizing the identification of Scophthalmus maximus individuals;

[0012] The third step is to train a data generation model for the dorsal image features of Scophthalmus maximus individuals, complete the key point detection of growth phenotypes, and obtain the required key point data; the key point data is the data that can calculate the growth traits of Scophthalmus maximus;

[0013] The fourth step is to use a calibration board for single target calibration, and obtain the homography matrix, the image internal and external parameters, and the actual coordinates of the key point positions on the collection plane;

[0014] The fifth step is to calculate the distortion coefficient of the image by using the internal and external parameters obtained in the fourth step, correct the distortion of the original image, train the key point positioning network of the dorsal image, and improve the data accuracy of the model output.

[0015] As a more specific embodiment, in the second step, the individual identification of Scophthalmus maximus is carried out in the pre-processed Scophthalmus maximus data set, and the specific implementation process is as follows:

[0016] Step 2.1: For the continuously tracked Scophthalmus maximus individuals, at least 5 effective ventral sample images are selected for each individual, the image size is adjusted to 512*512 during training, and the ventral sample image data set is divided into a training data set and a test data set in a ratio of 4:1.

[0017] Step 2.2: For the samples of the training data set, convert them into 1-dimensional 1-column data, and form a matrix with the images of the training data set, calculate the average value of the matrix based on the column, and obtain the average ventral image;

[0018] Step 2.3: Expand the training data set image to form a matrix, subtract the average value of the matrix to obtain a new data matrix, and calculate the covariance matrix of the matrix to obtain its eigenvalues and eigenvectors;

[0019] Step 2.4: Based on the size of the eigenvalue obtained in step 2.3, the matrix of the eigenvector is sorted, the dimension of the eigenvector is selected as 120, the corresponding projection matrix is obtained, the distance of the image in the test data set in the projection matrix is calculated, and the large turbot images with a distance value greater than 35000 are discarded, and finally the test image is obtained;

[0020] Step 2.5: Test the picture using a multivariate logistic regression algorithm to complete the classification recognition; the correct recognition rate is 100%, and the recognition model verification is completed;

[0021] For individuals who do not use fish belly images for identification, perform fin texture feature extraction for identification, and the specific process is as follows:

[0022] Step 2.6: First, label the dorsal fin, tail fin, pelvic fin or anal fin of the turbot dorsal image;

[0023] Step 2.7: Divide the labeled image data set into training set and test set, and divide each image into background part and fin part;

[0024] Step 2.8: Select the deep learning network of deeplabv3 semantic segmentation, which includes resnet 50 backbone network, ASPP module (Atrous Spatial Pyramid Pooling Module) and FCN head part;

[0025] Step 2.9: Input the image of the training set, adjust the image size to 2048*512, set the learning rate from 0.0001 to 0.01, and train the network for 2000 rounds to complete the segmentation of the fin part of the image, with an accuracy of 93.99%;

[0026] For the turbot individual recognition model based on fin (dorsal fin, tail fin, pelvic fin and anal fin) image, the specific process is as follows:

[0027] Step 2.10: Based on the fin image data set obtained by the segmentation algorithm, divide the training set and test set in the ratio of 4:1;

[0028] Step 2.11: Use a deep learning network, the backbone network uses resnet18 structure, the neck uses global pooling average module, the head uses linear classifier, and the classification function uses softmax; the deep learning network optimizer selects stochastic gradient descent algorithm, the cross entropy is the loss function, and the class is 40; the deep learning network is trained for 200 rounds to obtain the weight of the deep learning network, and the test set data is input for testing to complete the individual recognition of all turbot image samples.

[0029] As a more specific embodiment, in the third step, the data generation model is trained for the image features of the individual Scophthalmus maximus back, the growth phenotype key point detection is completed, and the required key point data is obtained. The specific implementation process is as follows:

[0030] Step 3.1: Prepare the data set; for the collected Scophthalmus maximus back images, label the key points, and after all the back images are labeled, the json data (json refers to a file format) file generated by the labeling tool and the picture file are sorted into different folders respectively;

[0031] Step 3.2: Train the data generation model; after preparing the data set, train the data, and use the HRnet model to complete the key point detection; put the prepared data set into the previously built neural network structure for training to obtain the model weight required in actual use; then the image to be measured is obtained through the neural network loaded with the model weight The required key point data; the neural network structure is HRnet network structure, HRNet (High-Resolution Network) is a deep learning model mainly used for computer vision tasks such as image classification, object detection and semantic segmentation. The feature of HRNet is that it can directly process high-resolution images without gradually reducing the resolution in the processing process like traditional convolutional neural networks (CNN). This makes HRNet have obvious advantages in processing tasks with rich details.

[0032] As a more specific embodiment, in the fourth step, single target calibration is performed using a calibration board to obtain a homography matrix, obtain camera internal and external parameters, and obtain the actual coordinates of the key point positions on the collection plane; the specific implementation process is as follows:

[0033] Step 4.1: In the image acquisition system, place the calibration board on the collection platform and collect images of the calibration board at different positions.

[0034] Step 4.2: Perform corner detection on the images obtained in 4.1, that is, find the position of each corner point on the corresponding image (here the findChessboardCorners function in the opencv library is used for scanning and acquisition).

[0035] Step 4.3: Perform internal parameter evaluation, that is, first assume the camera model (usually a pinhole camera model) and the initial internal parameters, including focal length, optical center coordinates and distortion coefficient. Then use the corner point positions in multiple images to adjust the internal parameters through a nonlinear optimization algorithm (here the Levenberg-Marquardt algorithm is selected) to minimize the projection error. The obtained internal parameters include focal length (f x and f y ), optical center (c xand c y ) and distortion coefficient.

[0036] Step 4.4: Perform extrinsic evaluation, that is, for each image, estimate the pose of the camera relative to the calibration board, that is, the rotation matrix and translation vector, and then further optimize the rotation matrix and translation vector of each image by minimizing the projection error on the basis of the known intrinsic parameters.

[0037] Step 4.5: Obtain the homography matrix. The homography matrix H is a 3x3 matrix used to represent the projection transformation between two image planes. For a point (x, y) in the first image plane, its corresponding point (x', y') in the second image plane can be represented by the matrix as follows: Next we assume that we have a planar object and know the point p w = (X, Y, Z) in the world coordinate system of the plane and that the plane can be represented by the equation Z = 0. The coordinates of the point P C in the camera coordinate system can be calculated by the extrinsic parameters of the camera (rotation matrix R and translation vector T): Under the action of the camera intrinsic parameter matrix K, the image coordinates (x, y) can be represented as By substituting, we finally obtain H = K * [r1 r2 t] where H is the required homography matrix, K is the intrinsic matrix, r1 and r2 are the first two columns of the rotation matrix, t is the translation vector, and v represents the transformed coordinate values.

[0038] As a more specific implementation, in the fifth step, the distortion coefficient is calculated using the intrinsic and extrinsic parameters obtained in the fourth step, the original image is rectified for distortion, the key point positioning network of the back image is trained, and the data accuracy of the model output is improved. The specific implementation process is as follows:

[0039] Step 5.1: Rectify the original image for distortion by the distortion coefficient obtained in the fourth step. The rectification formula is x_distorted = x * (1 + k1 * r 2 + k2 * r 4 + k3 * r 6 )(Here we list the rectification formula for x-direction coordinates, and the y-direction is the same, k1, k2, and k3 are the values in the distortion coefficient D = [k1, k2, k3]). The rectified image is sent into the yolo8 model to obtain the detection frame, and the rectified image is cropped by the detection frame to obtain the cropped image and retain the cropping relationship.

[0040] Step 5.2: the cropped image is input into the neural network of key point detection loaded with the weight of the model trained in step 3.2, the key point positions of the input cropped image are obtained, and then the corrected image key point positions are obtained through the previously backed up cropped relationship.

[0041] Step 5.3: the actual coordinates of the key points on the acquisition plane are obtained through the corrected image key point positions and the homography matrix obtained in step 4.5, and the distances of the key points are obtained in sequence through the relationship of the key points in the json data file in step 3.1 to obtain the required total length, head length, body length, body height, tail handle height, tail handle length, eye diameter, snout length and chest fin length.

[0042] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is suitable for being loaded and executed by a processor to perform the individual growth phenotype high-throughput tracking measurement method of Scophthalmus maximus.

[0043] The application further provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the individual growth phenotype high-throughput tracking measurement method of Scophthalmus maximus.

[0044] An individual growth phenotype high-throughput tracking measurement device of Scophthalmus maximus, which comprises an image acquisition system, an image processing system and an individual identification system, the image acquisition system executes the first step of the method, the image processing system executes the second step of the method, and the individual identification system executes the third to fifth steps of the method.

[0045] The application further provides application of the method in individual growth phenotype high-throughput tracking measurement in Scophthalmus maximus breeding, and the method can be used to identify the source information, genetic information and growth phenotype tracking information in the whole growth cycle of Scophthalmus maximus individuals, is beneficial to screening of a core breeding population, breeding of a parent, development of a reasonable mating scheme, avoidance of inbreeding, protection of genetic diversity of germplasm and more accurate breeding.

[0046] Compared with the prior art, the application has the following beneficial effects:

[0047] 1、The present application is based on the previous research, it is found that Scophthalmus maximus from juvenile fish (about 15g) to adult fish (about 500g), the fish belly pattern has individual difference and is invariable, and the texture of fin (including dorsal fin, tail fin, ventral fin and anal fin) also has individual difference, although the color depth changes with growth, but the texture is basically unchanged, based on this discovery, a Scophthalmus maximus individual identification model based on cascade features is proposed, which has the advantages that: the traditional machine learning algorithm is used to identify the fish belly image which is easy to distinguish; the image segmentation model is used to extract the fin image of Scophthalmus maximus, and the deep learning algorithm is used to extract the features, so as to identify the fin image with high complexity; thereby the identification accuracy of Scophthalmus maximus individual is improved as a whole; the identification of multiple feature parts realizes the monitoring of Scophthalmus maximus in a growth cycle, completes the monitoring task from the seedling stage to adult fish, and has innovation.

[0048] 2、The present application can be applied in Scophthalmus maximus breeding, can identify the parent source information, genetic information and growth phenotype information of Scophthalmus maximus individual in the whole growth cycle; and the detection of growth index can be automatically completed, the data information acquisition standard is unified, the human error is extremely small, and the precision is improved.

[0049] 3、The present application can realize non-invasive tracking of fish body, that is, the individual is tracked without damaging the fish body surface by electronic tag, fluorescent marker and other damaging methods, the risk of pathogenic infection caused by fish body surface damage is reduced, energy saving and environmental protection are realized, and the breeding risk is reduced.

[0050] 4、The present application can reduce the tracking cost of Scophthalmus maximus individual growth phenotype, the market price of electronic tag is 12-15 yuan per tag, a batch of breeding trait tracking and monitoring generally has more than 10000, the cost is greatly reduced; the traditional growth phenotype tracking and measuring method needs 3-5 people each time, and takes 5-7 days, while the present application only needs one person to operate the equipment to measure, 1-2 people to fish, and 2-3 days to complete, so that the labor cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is a Scophthalmus maximus individual identification flowchart;

[0052] Figure 2 is a data set graph of Scophthalmus maximus individual images in different growth stages;

[0053] Figure 3 is a Scophthalmus maximus belly average graph;

[0054] Figure 4 is a segmentation method of Scophthalmus maximus image data;

[0055] Figure 5 is a deepLabV3 network structure diagram;

[0056] Figure 6 is a split dorsal fin, tail fin, ventral fin and anal fin diagram of turbot;

[0057] Figure 7 is a structure diagram of a deep learning network for individual identification based on turbot fin characteristics;

[0058] Figure 8 is a labeling diagram of 16 key points in a turbot image, wherein 1 head: a point is marked at the most forward end of the fish head, and the label is head; 2 tail_up: the topmost point of the tail; 3 tail: the point on the middle axis of the fish side, which is in the middle of the fish tail depression; 4 head_fin: the point farthest from the fish mouth along the head-tail axis of the gill cover trace; 5 eye_d_l: the left edge of the fish's eye near the mouth; 6 eye_d_r: the right edge of the fish's eye near the mouth; 7 eye_d: the upper edge of the fish's eye near the mouth; 8 eye_u: the lower edge of the fish's eye far from the mouth; 9 head_up: the point at the highest part of the upper fish width; 10 head_down: the point corresponding to the bottom of the fish body vertical line about the highest part of the upper fish width; 11 f_front_u: the point at the upper end of the fish tail handle starting point; 12 f_front_d: the point at the lower end of the fish tail handle starting point; 13 f_small_u: the point corresponding to the bottom of the vertical head-tail axis at the narrowest part of the fish tail handle upper end; 14 f_small_d: the point corresponding to the bottom of the vertical head-tail axis at the narrowest part of the fish tail handle lower end; 15 fins_begin: the starting point of the fish pectoral fin; 16 fins_end: the ending point of the fish pectoral fin;

[0059] Figure 9 is a detection frame diagram obtained by a YOLO model;

[0060] Figure 10 is a key point position acquisition result diagram of turbot;

[0061] Figure 11 is a flowchart of obtaining growth traits of turbot after individual identification. DETAILED DESCRIPTION

[0062] The technical solutions of the present application will be further explained by examples below, but the protection scope of the present application is not limited in any form by the examples.

[0063] Example 1

[0064] In this example, 40 turbot fish were tracked and monitored for 2 years, and each individual was provided with a physical identification mark. Image samples of turbot fish were collected once every three months, and abdominal and dorsal images were collected each time to construct a training data set for the present application. It was found through continuous tracking that the abdominal pattern of turbot fish had obvious individual characteristics, and therefore a turbot individual identification method combining dorsal and abdominal characteristics was designed. The individual identification method is as followsFigure 1 The specific method steps are as follows:

[0065] The first step is to collect pictures of the ventral and dorsal surfaces of Scophthalmus maximus by using a smart phone, and to pre-process the collected pictures of Scophthalmus maximus, mainly including mask annotation and image enhancement. Figure 2 )。

[0066] The second step is to recognize the fish body ventral image which is easy to distinguish by using a traditional machine learning algorithm, to extract the fin image of Scophthalmus maximus by using an image segmentation model, to recognize the fin image with high complexity by using the deep learning algorithm to extract features, and to realize the individual recognition of Scophthalmus maximus.

[0067] Step 2.1: For the continuously tracked Scophthalmus maximus fish individuals, the similarity of the photographed ventral images is relatively high, and there are only 5 effective ventral sample images for each individual at the same collection time, and there are 200 images for 40 individuals. When training, the image size is adjusted to 512*512, and the image dataset is divided into a training dataset and a test dataset in a ratio of 4:1, that is, 5 images for each individual, 4 images into the training dataset and 1 image into the test dataset.

[0068] Step 2.2: For the samples in the training set, they are converted into 1-dimensional 1-column data, and the 160 images of the training set are composed into a 160-column matrix X=[x1 x2 … x 160 ], and x i in the matrix is a 512*512=262144-dimensional vector. The average value of the matrix can be obtained based on the following formula Thus, the average ventral image of Scophthalmus maximus fish is obtained. Figure 3 )。

[0069]

[0070] Step 2.3: The training image is unfolded to form a matrix X, and the average matrix A new 262144*160 data matrix is obtained, and the covariance matrix is calculated, and the eigenvalues and eigenvectors are obtained, and the 160 eigenvalues are as follows:

[0071]

[0072] Step 2.4: Based on the size of the eigenvalues obtained in step 2.3, the matrix of the eigenvectors is obtained, the dimension of the eigenvectors is selected to be 120, the corresponding projection matrix is obtained, the distance of the images in the test dataset in the projection matrix is calculated, the Scophthalmus maximus fish test pictures with a distance value greater than 35000 are discarded, and finally 20 test pictures are obtained.

[0073] Step 2.5: 20 test images were obtained, and the logical regression algorithm was used to complete the identification of the categories, with a correct identification rate of 100%, and the identification model verification was completed.

[0074] For individuals who did not use the fish belly image for identification, the fin (dorsal fin, tail fin, ventral fin and anal fin) texture was used to segment the fin based on the deep learning model algorithm. The specific steps are as follows:

[0075] Step 2.6: First, use the 5000 pictures taken to label the fin (dorsal fin, tail fin, ventral fin and anal fin) using the labelme software.

[0076] Step 2.7: Divide the labeled image dataset into training set and test set, and divide each image into background and fin 2 parts, such as Figure 4 The closed area marked by the dot is the fin, and the rest is the background.

[0077] Step 2.8: Considering the sample size and complexity of the image data, the deep learning network deeplabv3 for semantic segmentation is selected Figure 5 ), which mainly includes resnet 50 backbone network, ASPP module (Atrous SpatialPyramid Pooling Module) and FCN head part.

[0078] Step 2.9: Input the training set image, adjust the image size to 2048*512, set the learning rate from 0.0001 to 0.01, and train the network for 2000 rounds to complete the image fin segmentation Figure 6 ), with a segmentation accuracy of 93.99%.

[0079] For the fin (dorsal fin, tail fin, ventral fin and anal fin) image dataset obtained, a deep learning network is designed to complete the identification of individuals. The specific steps are as follows:

[0080] Step 2.10: Based on the fin image dataset obtained by the segmentation algorithm, divide the training set and test set according to the image quantity ratio of 4:1;

[0081] Step 2.11: Since there is only fin in the fish sample image, the image complexity is not high, so the deep learning network as shown in Figure 7 is designed in this embodiment. The backbone network of the network uses resnet18 structure, the neck uses global pooling average module, the head uses linear classifier, and the classification function selects softmax. The deep learning network optimizer selects the stochastic gradient descent algorithm, the cross entropy is the loss function, and the class is 40.

[0082] Step 2.12: After 200 rounds of training, the network learns the weights of the deep learning network, the weight file size is 520M, and the test accuracy is 92%.

[0083] Step 3: For the image features of individual turbot back, train the data generation model, complete the growth phenotype key point detection, and obtain the required key point data.

[0084] Step 3.1: Prepare the data set, for the collected back images, use the labelimg software to label the images for 16 key points of head, tail_up, tail, head_fin, eye_d_l, eye_d_r, eye_d, eye_u, head_up, head_down, f_front_u, f_front_d, f_small_u, f_small_d, fins_begin, fins_end. After all the images are labeled, the json data file generated by the labeling tool and the picture file are sorted into different folders. At this time, the preparation of the data set is completed, as shown in Figure 8 .

[0085] Step 3.2: Train the data generation model, after preparing the data set, the data can be trained, and the HRnet model can be used to complete the key point detection. The prepared data set is sent to the previously built neural network structure for training to obtain the model weight required for actual use. Then the pictures to be measured can obtain the required key point data through the neural network loaded with the model weight. The neural network structure is HRnet network structure. HRNet (High-Resolution Network) is a deep learning model mainly used for computer vision tasks such as image classification, object detection and semantic segmentation. The feature of HRNet is that it can directly process high-resolution images without gradually reducing the resolution in the processing process like traditional convolutional neural networks (CNN). This makes HRNet have obvious advantages in processing tasks with rich details.

[0086] Step 4: Select the appropriate calibration board, perform single target calibration, and obtain the homography matrix to obtain the camera internal and external parameters and the actual coordinates of the key points on the collection plane.

[0087] Step 4.1: In the image acquisition system, select the appropriate calibration board. In this embodiment, a 11x9 checkerboard calibration board is selected. The coordinates of the points on the calibration board must be known and accurate. Place the calibration board on the collection platform and collect images containing the entire calibration board at different positions.

[0088] Step 4.2: Corner detection is performed on the images obtained in step 4.1, i.e., finding the position of each corner point on the corresponding image (here, the findChessboardCorners function in the opencv library is used for scanning and obtaining).

[0089] Step 4.3: Intrinsic parameter evaluation is performed, i.e., assuming a camera model (usually a pinhole camera model) and initial intrinsic parameters, including focal length, optical center coordinates, and distortion coefficients. Then, using the corner point positions in multiple images, the intrinsic parameters are adjusted through a nonlinear optimization algorithm (Levenberg-Marquardt algorithm is selected in this embodiment) to minimize the projection error. The obtained intrinsic parameters include focal length (f x and f y ), optical center (c x and c y ), and distortion coefficients.

[0090] Step 4.4: Extrinsic parameter evaluation is performed, i.e., for each image, the camera's pose relative to the calibration board is estimated, i.e., the rotation matrix and translation vector. Then, based on the known intrinsic parameters, the rotation matrix and translation vector of each image are further optimized by minimizing the projection error.

[0091] Step 4.5: Homography matrix is obtained. The homography matrix H is a 3x3 matrix that represents the projection transformation between two image planes. For a point (x, y) in the first image plane, its corresponding point (x', y') in the second image plane can be represented as: Next, we assume that we have a planar object and know the point p w =(X, Y, Z) in the world coordinate system of the plane and that the plane can be represented by the equation Z=0. The coordinates of this point in the camera coordinate system P C can be calculated by the camera's extrinsic parameters (rotation matrix R and translation vector T): Under the action of the camera intrinsic matrix K, the image coordinates (x, y) can be represented as By substitution, we finally get H=K*[r1r2t] where H is the required homography matrix, K, r1, r2, and t are the intrinsic matrix, the first two columns of the rotation matrix, and the translation vector, respectively.

[0092] Step 5: Using the intrinsic and extrinsic parameters obtained in step 4, the distortion coefficients are calculated, the original images are rectified, and the key point positioning network of the back image is trained to improve the data accuracy of the model output.

[0093] Step 5.1: Distortion correction is performed on the original image by the distortion coefficient obtained in the fourth step, and the correction formula is x_distorted = x * (1 + k1 * r^2 + k2 * r^4 + k3 * r^6) (the correction formula for the y direction is the same as that for the x direction coordinate). The corrected image is sent into the yolo8 model to obtain the detection frame, and the cropped image is obtained by cropping the corrected image through the detection frame, and the cropping relationship is retained.

[0094] Step 5.2: The cropped image is sent into the key point detection neural network loaded with the model weight trained in step 3.2 to obtain the key point positions of the cropped image, and then the key point positions of the corrected image are obtained by restoring the previously backed up cropping relationship. The point position acquisition result is as shown in Figure 10

[0095] Step 5.3: The actual coordinates of each key point position on the collection plane can be obtained through the key point positions of the corrected image and the homography matrix obtained in step 3.3. The distance can be obtained by calculating the relationship of each point position in the json data document (the annotation document refers to the document created by the data annotation personnel for the convenience of data annotation in the early preparation, which contains the name, position and detailed description of all the point positions to be annotated, and also contains the relationship between each specific index and point position.) in sequence. The full length, head length, body length, body height, tail handle height, tail handle length, eye diameter, snout length, and chest fin length indexes can be obtained in sequence.

[0096] The complete inference process is as shown in Figure 11

[0097] The above method is used in the breeding process of Scophthalmus maximus. After the back and ventral images of Scophthalmus maximus individuals are collected by using the growth phenotype image collection device for the first time, the individual recognition model is built and the growth phenotype is determined. In the later breeding process, the back and ventral images of Scophthalmus maximus can be collected at any time, the individual recognition is performed, and the growth traits are calculated, so as to establish a growth trait tracking database for each Scophthalmus maximus throughout the growth cycle, and finally realize the growth phenotype tracking measurement goal with high throughput, high automation degree, high precision and low cost.​​

Claims

1. A method for high-throughput tracking of individual growth phenotypes in Scophthalmus maximus, characterized by, The method is as follows: First, use an image acquisition system to take complete images of the ventral and dorsal surfaces of Scophthalmus maximus, and preprocess the collected images of Scophthalmus maximus; Second, use a machine learning algorithm to identify the fish body ventral surface image which is easy to distinguish from the preprocessed images of Scophthalmus maximus in the first step; then use an image segmentation model to extract the fin image of Scophthalmus maximus from the dorsal surface image, use the deep learning algorithm to extract the features, identify the fin image with high complexity, and realize the individual identification of Scophthalmus maximus; Third, train the data generation model according to the dorsal surface image features of Scophthalmus maximus, complete the growth phenotype key point detection, and obtain the required key point data; the key point data is the data that can calculate the growth traits of Scophthalmus maximus; the specific steps are as follows: Step 3.1: Prepare the data set; label the key points for the collected dorsal surface images of Scophthalmus maximus, and after all the dorsal surface images are labeled, arrange the json data file generated by the labeling tool and the picture file into different folders respectively; Step 3.2: Train the data generation model; after preparing the data set, train the data, use the HRnet model to complete the key point detection; put the prepared data set into the previously built neural network structure for training to obtain the model weight required in actual use; then the images to be measured are obtained through the neural network loaded with the model weight to obtain the required key point data; Fourth, use a calibration board for single target calibration, obtain a homography matrix, obtain the image internal and external parameters, and the actual coordinates of the key point positions on the collection plane; Fifth, calculate the distortion coefficient of the image by using the internal and external parameters obtained in the fourth step, correct the distortion of the original image, train the key point positioning network of the dorsal surface image, and improve the data accuracy of the model output; In the second step, the individual identification of Scophthalmus maximus is carried out in the preprocessed image data set of Scophthalmus maximus, and the specific implementation process is as follows: Step 2.1: For continuously tracked Scophthalmus maximus individuals, select at least 5 effective ventral sample images for each individual, adjust the image size to 512*512 during training, and divide the ventral sample image data set into a training data set and a test data set in a ratio of 4:1; Step 2.2: For the samples in the training data set, convert them into 1-dimensional 1-column data, and form a matrix with the images in the training data set, calculate the average value of the matrix based on the column to obtain the average ventral image; Step 2.3: Expand the training data set image to form a matrix, subtract the average value of the matrix to obtain a new data matrix, and calculate the covariance matrix of the matrix to obtain its eigenvalues and eigenvectors; Step 2.4: Sort the eigenvalues obtained in step 2.3 based on their size to obtain the matrix of eigenvectors, select the dimension of the eigenvectors as 120, obtain the corresponding projection matrix, calculate the distance of the images in the test data set in the projection matrix, discard the Scophthalmus maximus images with a distance greater than 35000, and finally obtain the test images; Step 2.5: Use a multivariate logistic regression algorithm to complete the classification identification of the test images in step 2.4; For individuals who do not use fish belly image for identification, fin texture feature extraction and identification are performed, and the specific process is as follows: Step 2.6: First, the back image of Scophthalmus maximus is labeled with dorsal fin, caudal fin, pelvic fin or anal fin; Step 2.7: The labeled image dataset is divided into a training set and a test set, and each image is divided into a background part and a fin part; Step 2.8: A deep learning network of deeplabv3 semantic segmentation is selected, and the structure of the deep learning network includes a resnet 50 backbone network, an ASPP module and an FCN head part; Step 2.9: The image in the training set is input, the image size is adjusted to 2048*512, the learning rate is set from 0.0001 to 0.01, and the deep learning network is trained to complete the segmentation of the fin part of the image; Step 2.10: Based on the fin image dataset obtained by the segmentation algorithm, the training set and the test set are divided in a ratio of 4:1; Step 2.11: A deep learning network is used, the backbone network uses a resnet18 structure, the neck uses a global pooling average module, the head uses a linear classifier, and the classification function uses softmax; the deep learning network optimizer selects a stochastic gradient descent algorithm, the cross-entropy is the loss function, and the class is 40; Step 2.12: The deep learning network is trained and learns to obtain the weight of the deep learning network, and the test set data is input for detection, thereby completing the individual identification of all images.

2. The method of claim 1, wherein the individual growth phenotypes of turbot are tracked by high-throughput measurement. In the fourth step, a single target is calibrated using a calibration board to obtain a homography matrix, camera internal and external parameters, and the actual coordinates of the key points on the acquisition plane; the specific implementation process is as follows: Step 4.1: In the image acquisition system, place the calibration board on the acquisition platform and acquire images of the calibration board at different positions; Step 4.2: Perform corner detection on the images obtained in 4.1, i.e., find the position of each corner point on the corresponding image; Step 4.3: Perform internal parameter evaluation, i.e., first assume the camera model and initial internal parameters, including focal length, optical center coordinates and distortion coefficients, then use the corner point positions in multiple images to adjust the internal parameters through a nonlinear optimization algorithm to minimize the projection error, and obtain the internal parameters including focal length, optical center and distortion coefficients; Step 4.4: Perform external parameter evaluation, i.e., for each image, estimate the pose of the camera relative to the calibration board, i.e., the rotation matrix and the translation vector, and then further optimize the rotation matrix and the translation vector of each image by minimizing the projection error based on the known internal parameters; Step 4.5: Obtain the homography matrix, which is a 3x3 matrix used to represent the projection transformation between two image planes.

3. The method of claim 2, wherein the method is used to track and measure the growth phenotype of individual turbot (Scophthalmus maximus). In the fifth step, the internal and external parameters obtained in the fourth step are used to calculate the distortion coefficient, the original image is corrected for distortion, the key point positioning network of the back image is trained, and the data accuracy of the model output is improved, and the specific implementation process is as follows: Step 5.1: Distort the original image by the distortion coefficient obtained in the fourth step, and send the corrected image into the yolo8 model to obtain the detection frame, and then crop the corrected image by the detection frame to obtain the cropped image and retain the cropping relationship; Step 5.2: Send the cropped image into the key point detection neural network loaded with the model weight trained in step 3.2 to obtain the key point positions of the cropped image, and then restore the key point positions of the corrected image through the previously backed up cropping relationship; Step 5.3: Obtain the actual coordinates of the key point positions on the collection plane by the key point positions of the corrected image and the homography matrix obtained in step 4.5, and then obtain the required full length, head length, body length, body height, tail handle height, tail handle length, eye diameter, snout length, and chest fin length by calculating the distances of the key point positions in the json data file in step 3.

1. 4.A computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being adapted to be loaded and executed by a processor to perform the individual growth phenotype high-throughput tracking measurement method of Scophthalmus maximus according to claim 1. 5.A computer device, the device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to make the processor perform the individual growth phenotype high-throughput tracking measurement method of Scophthalmus maximus according to claim 1. 6.A device for individual growth phenotype high-throughput tracking measurement of Scophthalmus maximus, the device comprising an image acquisition system, an image processing system, and an individual identification system, the image acquisition system performing the first step of the individual growth phenotype high-throughput tracking measurement method of Scophthalmus maximus according to any one of claims 1-3, the image processing system performing the second step of the individual growth phenotype high-throughput tracking measurement method of Scophthalmus maximus according to any one of claims 1-3, and the individual identification system performing the third to fifth steps of the individual growth phenotype high-throughput tracking measurement method of Scophthalmus maximus according to any one of claims 1-3.

7. Use of the method for high-throughput tracking of individual growth phenotypes of Scophthalmus maximus according to any of claims 1 to 3 for high-throughput tracking of individual growth phenotypes in Scophthalmus maximus breeding, characterized in that, The method is used to identify the source information, genetic information, and growth phenotype tracking information of Scophthalmus maximus in the whole growth cycle, so as to screen the core breeding population, cultivate the parents, and develop a reasonable mating scheme.

Images