Crayfish phenotype detection method and system based on key point detection and image segmentation

Abstract

The invention provides a crayfish phenotype detection method and system based on key point detection and image segmentation, and the method comprises the steps: obtaining a crayfish image, and carrying out the marking processing of the crayfish image, and forming a crayfish image data set; the crayfish image data set is preprocessed; constructing a crayfish form recognition model, and training the crayfish form recognition model based on the preprocessed crayfish image data set; and identifying the crayfish image through the trained crayfish form identification model, and outputting phenotype data of the crayfish. According to the method, the phenotype data of the crayfish can be quickly and accurately analyzed.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a crayfish phenotype detection method and system based on key point detection and image segmentation. Background Art

[0002] With the continuous development of computer information technology and image processing technology, machine vision technology has been widely used in the field of aquaculture animal attribute data collection and recognition. It can quickly, economically and non-destructively detect the appearance characteristics of aquaculture animals such as size, shape, and color, which are of great significance for evaluating the growth status of aquaculture animals and are one of the key factors in the development of the aquaculture industry.

[0003] When studying the relationship between the genotype, environment and phenotype of crayfish, it is crucial to obtain accurate phenotype data, and the accurate identification of phenotype data such as gender characteristics is particularly critical. However, at present, the size sorting of commercial crayfish mainly relies on manual selection. This method depends on the experience accumulated by workers to judge and classify the integrity, vitality and size of crayfish. However, this process has obvious defects. The subjectivity and differences of individual workers will lead to large errors in the classification results; moreover, the manual classification takes a long time, during which the vitality of crayfish decreases and the stress mortality rate increases, causing great economic losses to the industry. The traditional method of manually obtaining crayfish phenotype data is not only time-consuming and laborious, but also difficult to meet the requirements of the efficient and precise development of modern aquaculture. Therefore, there is an urgent need for an efficient and precise image morphology recognition method and system to solve these problems. Summary of the Invention

[0004] The present invention aims to provide a crayfish phenotype detection method and system based on key point detection and image segmentation to solve the problem of time-consuming and laborious manual observation and measurement to obtain crayfish phenotype data. By introducing computer vision technology, the rapid and precise analysis of crayfish phenotype data is realized, thereby saving the time and labor costs of size classification in actual production.

[0005] The technical solution of the present invention is as follows:

[0006] A crayfish phenotype detection method based on key point detection and image segmentation, the method comprising:

[0007] Obtain a crayfish image, and perform annotation processing on the crayfish image to form a crayfish image data set;

[0008] Perform preprocessing on the crayfish image data set;

[0009] Construct a crayfish morphology recognition model, and train the crayfish morphology recognition model based on the preprocessed crayfish image data set;

[0010] The trained crayfish morphology recognition model is used to recognize the crayfish image, and the phenotypic data of the crayfish is output.

[0011] Furthermore, the preprocessing of the crayfish image dataset includes resampling and normalizing the crayfish image, and performing image data enhancement by using one or more of the operations of cropping, expanding, randomly rotating, and flipping.

[0012] Furthermore, the crayfish morphology recognition model includes a detection module, a segmentation module, and a measurement module; when training the crayfish morphology recognition model based on the preprocessed crayfish image dataset, the detection module, the segmentation module, and the measurement module are trained in sequence.

[0013] Furthermore, the training of the detection module is specifically as follows:

[0014] Design the backbone structure of the detection module. The backbone structure includes a Conv module for feature extraction, a C2f module for optimizing gradient flow and reducing redundant parameters, and an SPPF module for splicing feature maps of different scales to enhance the detection ability for targets of different sizes;

[0015] Design the head structure of the detection module. The head structure includes a detection head and a classification head. The detection head is used to generate detection results. The classification head uses global average pooling to classify the feature map and output the class probability distribution. The final detection result including the bounding box, class, and confidence is converted through the Detect module from the fused feature map;

[0016] Calculate the loss function of the detection module. The loss function includes the cross-entropy loss for measuring the difference between the prediction and the true probability distribution, the DFL loss for correcting the bounding box regression error to improve the detection accuracy of blurred or unfocused pictures, and the CIOU loss for adding the center point distance and relative ratio on the basis of the IOU loss to enhance the convergence speed and detection effect;

[0017] Through multiple iterative trainings, continuously adjust the model parameters until the loss function converges to a preset threshold or reaches a preset number of training epochs to complete the training of the detection module.

[0018] Furthermore, the training of the segmentation module is specifically as follows:

[0019] Adopt the semi-supervised semantic segmentation method to perform perturbation operations on the labeled images and unlabeled images respectively. Among them, weak perturbations are performed on the labeled images, such as cropping, rotating, and flipping; strong perturbations are performed on the unlabeled images, such as changing color contrast, brightness, cropping and covering, and adding noise;

[0020] The perturbed labeled images and unlabeled images are respectively input into the segmentation module to obtain corresponding prediction results, and the cross-entropy loss between the prediction results and the true labels or pseudo-labels is calculated;

[0021] Implement a one-way replacement strategy from labeled to unlabeled, calculate the confidence scores of the labeled images and unlabeled images, the block average confidence scores, find the block with the lowest average confidence score in the unlabeled images, replace it with the image block at the corresponding position in the labeled images, and update the pseudo-labels of the unlabeled images;

[0022] The loss is fed back into the segmentation module through the backpropagation algorithm to adjust the parameters of the segmentation module, continuously optimize the network, evaluate the segmentation accuracy on the validation dataset, and save the parameters of the segmentation module when the segmentation accuracy reaches the best.

[0023] Furthermore, the training of the measurement module is specifically as follows:

[0024] Using the training results of the detection module and the segmentation module, perform gray conversion processing on the image to convert the color image into a grayscale image containing only luminance information;

[0025] Perform dilation processing on the grayscale image, and use a preset dilation kernel to thicken the thin edges;

[0026] Perform erosion processing on the dilated image, and use a preset erosion kernel to eliminate noise and some boundary values;

[0027] Use the Canny operator to perform edge detection on the eroded image to obtain the edge information of the image;

[0028] Add a minimum bounding rectangle to the contour map after edge detection, and approximate the length and width of the minimum bounding rectangle as the pixel values of the phenotypic features;

[0029] Calculate the actual length corresponding to each pixel, so as to obtain the actual lengths of various phenotypic features of the crayfish and complete the training of the measurement module.

[0030] Furthermore, the recognition of crayfish images by the trained crayfish morphology recognition model and the output of the phenotypic data of the crayfish are specifically as follows:

[0031] Use the trained detection module to recognize the input crayfish image, determine the positions of the crayfish and the calibration object in the image, and perform fixed-size slicing on the original image;

[0032] Input the sliced image into the trained segmentation module to perform image segmentation on different parts of the crayfish to obtain segmentation masks of different parts of the crayfish;

[0033] The trained measurement module identifies the length of the calibration object based on the segmentation mask output by the segmentation module, and calculates the proportional relationship between the pixel length and the actual length of the calibration object;

[0034] Based on the above proportional relationship, the measurement module calculates the actual lengths of different parts of the crayfish, and finally outputs the phenotypic data of the crayfish.

[0035] The present invention also provides a crayfish phenotypic detection system based on key point detection and image segmentation, including:

[0036] An image acquisition and annotation unit, configured to acquire a crayfish image and perform annotation processing on the crayfish image to form a crayfish image data set;

[0037] A data preprocessing unit, configured to preprocess the crayfish image data set, including resampling, normalization processing, and performing one or more of operations such as cropping, expansion, random rotation, and flipping for image data enhancement;

[0038] A model construction and training unit, configured to construct a crayfish morphology recognition model including a detection module, a segmentation module, and a measurement module, and sequentially train the detection module, the segmentation module, and the measurement module based on the preprocessed crayfish image data set;

[0039] An identification and output unit, configured to identify a crayfish image through the trained crayfish morphology recognition model and output the phenotypic data of the crayfish.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] Through preprocessing methods such as resampling, normalization, and image data enhancement, the present invention strengthens the original data samples, effectively alleviates the overfitting problem of the system, improves the robustness of the overall system, can capture image detail information more effectively, and enhances the ability to deeply understand and process image content.

[0042] The present invention is trained based on a small amount of labeled data, enhances the generalization of the model by adding weak perturbations and strong perturbations to the data, and uses a one-way replacement strategy of labeled images for unlabeled images, making the generated pseudo-label results more accurate and the network training process more stable, and finally obtaining a more accurate segmentation result. Description of the Drawings

[0043] The drawings generally illustrate various embodiments by way of example and not limitation, and are used together with the description and the claims to illustrate the embodiments of the invention. When appropriate, the same reference numerals are used in all the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be an exhaustive or exclusive embodiment of the device or method.

[0044] Figure 1 It is a schematic flowchart of the crayfish phenotype detection method of the present invention;

[0045] Figure 2 It is the overall structure diagram of the detection module designed by the present invention;

[0046] Figure 3 It is the training process diagram of the semi-supervised segmentation algorithm with labeled data designed by the present invention;

[0047] Figure 4 It is the training process diagram of the semi-supervised segmentation algorithm without labeled data designed by the present invention;

[0048] Figure 5 It is the unidirectional replacement process diagram from labeled data to unlabeled data designed by the present invention. Specific implementation manners

[0049] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will describe this application in detail with reference to the drawings and in combination with the embodiments.

[0050] As Figure 1 shown, the present invention provides a crayfish phenotype detection method based on key point detection and image segmentation, and the method includes:

[0051] Step S1: Obtain the required image dataset according to the requirements of the crayfish image recognition task;

[0052] Step S2: Preprocess the dataset;

[0053] Step S3: Divide the dataset into a training set and a test set;

[0054] Step S4: Design the overall network according to the task and train the detection module with the training set images;

[0055] Step S5: Train the segmentation module with the training set images;

[0056] Step S6: Train the measurement module with the results of the detection module and the segmentation module.

[0057] Step S7: Test the overall network trained with the training set images with the images of the test set;

[0058] Step S8: Input the image data into the trained overall network to obtain the corresponding output result.

[0059] For step S1: Obtain the required image dataset according to the requirements of the crayfish image recognition task.

[0060] Specifically, the data collection is to capture crayfish images at different angles, different growth stages, and different environments according to the task requirements, and label the positions, key parts, and gender characteristics of the crayfish in the images to form a new required image dataset.

[0061] For step S2: Preprocess the dataset.

[0062] Specifically, the preprocessing methods include resampling, normalization, and image data augmentation, where data augmentation includes operations such as cropping, expansion, random rotation, and flipping.

[0063] Normalization: Normalize the images, scale the pixel or voxel intensities in the dataset to [0, 1], and set the pixel or voxel intensities outside the interval to 0 and 1 respectively.

[0064] Data augmentation: Its purpose is to generate more training samples by transforming the original images, thereby enhancing the generalization ability of the model.

[0065] For step S3: Divide the images into a training set and a test set.

[0066] Specifically, this step ensures that the model can learn on one subset (the training set) and then be evaluated on another independent subset (the test set) to verify the generalization ability of the model. A common division ratio is 80% for the training set and 20% for the test set.

[0067] For step S4, training the detection module with the training set images specifically includes the following steps:

[0068] Step S41: Design the backbone structure of the detection module; as Figure 2 shown, its AP stage is the backbone structure. The backbone structure is mainly responsible for feature extraction and multi-scale feature fusion. Therefore, a series of convolutional and deconvolutional layers are adopted, residual connections and bottleneck structures are used to reduce the size of the network and improve performance. At the same time, an SPPF module is used to splice feature maps of different scales together to improve the detection ability for targets of different sizes.

[0069] The Conv module is used in the backbone structure; the Conv module is responsible for extracting the features of the input image. By sliding a small matrix (called a kernel or filter) over the input data to perform element-wise multiplication and summation, a feature map is generated. The Conv module is the basic module of the entire model.

[0070] In addition, a lightweight C2f module is used; by optimizing gradient flow and reducing redundant parameters, the C2f module achieves higher feature extraction efficiency and a lighter network structure. Therefore, it has fewer parameters and better feature extraction ability compared to other structures, reducing redundant parameters.

[0071] At the same time, to achieve multi-scale feature fusion, an SPPF module is used; the SPPF module is used for pooling operations at different scales. It concatenates feature maps of different scales to improve the detection ability for targets of different sizes.

[0072] Step S42: Design the head structure of the detection module; as Figure 2 shown, its PB stage is the head structure. The head structure is mainly responsible for the final object detection and classification tasks, including a detection head and a classification head. The detection head contains a series of convolutional layers and transposed convolutional layers for generating detection results; the classification head uses global average pooling to classify each feature map and outputs the probability distribution of each class.

[0073] In the backbone structure, in addition to using the basic Conv module and C2f module, a Detect module is also used; the Detect module is responsible for converting the fused feature map into the final detection results, including predicting bounding boxes, classes, and confidences, to achieve the object detection and classification tasks that the head structure needs to complete.

[0074] Step S43: Calculate the loss function; specifically, three losses are used in the detection module: cross-entropy loss, DFL loss, and CIOU loss.

[0075] The cross-entropy loss measures the difference between the predicted probability distribution and the true probability distribution. In image classification tasks, the cross-entropy loss function is usually used together with the softmax activation function to convert the original output of the model into a probability distribution.

[0076]

[0077] N represents the number of samples; K represents the number of classes; y ic represents the one-hot encoding of the sample target value. If the true class of sample p i is equal to c, it takes 1, otherwise it takes 0; (h θ (x i ) c represents the probability that sample x i belongs to class c. When the predicted probability distribution of the model is more consistent with the true label probability distribution, the value of the cross-entropy loss function is smaller, and the cross-entropy loss function can be directly used in the gradient descent algorithm to update the weights of the model.

[0078] DFL loss is mainly applied in the bounding box regression process of object detection tasks. The main role of DFL loss is to correct the errors of the model when predicting object bounding boxes. The optimized effect can improve the accuracy of object detection to a certain extent for some blurred or unfocused pictures.

[0079] DFL(y i ,y i+1 ) = -(i + 1 - y)log(y i ) - (y - i)log(y i+1 )

[0080] where y is the actual label, yi and yi+1 represent the two predicted labels closest to the actual label y, and the variables in the formula satisfy During the training process, the lower the DFL loss, the better the performance of the model in predicting bounding boxes.

[0081] CIOU loss is developed on the basis of IOU loss. IOU loss is mainly used in object detection tasks to measure the overlap degree between the anchor box and the target box. It effectively shields the interference of the bounding box size in the form of a ratio, enabling the model to balance the learning of large and small objects well when using 1 - IoU as the BBR loss. CIOU loss adds the center point distance and relative ratio on the basis of IOU loss, further enhancing the convergence speed and detection effect.

[0082]

[0083] In this formula:

[0084]

[0085] where v is used to measure the consistency of the relative ratio of two rectangular boxes, and α is the weight coefficient; where b and b gt represent the center points of two rectangular boxes, ρ represents the Euclidean distance between two rectangular boxes, c represents the diagonal distance of the closure region of two rectangular boxes; A represents the predicted image, B represents the true label, w and h represent the width and height of the predicted image, w gt and h gt represent the width and height of the true label. The optimization goal of CIoU loss is to directly reduce the Euclidean distance between the center points of two rectangular boxes. The role of c is to prevent the value of the loss function from being too large, improve the convergence speed, and at the same time add the relative ratio to punish the results where the predicted shape is inconsistent with the ground truth.

[0086] Step S44: Complete the training of the detection module.

[0087] For step S5, training the segmentation module with the training set images specifically includes the following steps:

[0088] S51 Obtain the required image dataset, divide the dataset into a training set, a validation set, and a test set according to different ratios. In the semi-supervised semantic segmentation scenario, the training set consists of a set of labeled images and a set of unlabeled images where represents the i-th labeled image, represents the segmentation mask corresponding to the i-th labeled image, represents the i-th unlabeled image. The unlabeled image only has the image itself without a corresponding segmentation mask, and the number of unlabeled images is much larger than the number of labeled images.

[0089] S52 For the labeled image First, perform weak perturbation a(·) and strong perturbation A(·) operations on the labeled image respectively. The weak perturbation operation includes cropping, rotating, and flipping the image; the strong perturbation operation includes changing attributes such as the color contrast and brightness of the image, performing cropping and covering on the image, and adding noise. The images after adding perturbations are divided into the weakly perturbed labeled image and the strongly perturbed labeled image Pass them through the segmentation model F(·) to obtain the corresponding prediction results and where represents the prediction result of the weakly perturbed labeled image, represents the prediction result of the strongly perturbed labeled image, as shown in the following formula.

[0090]

[0091] Calculate the cross-entropy loss between the obtained prediction results and the true labels to measure the difference between the predicted probability distribution and the true probability distribution, and the specific operation is as shown in the following formula.

[0092]

[0093] where CE represents the cross-entropy loss.

[0094] For the unlabeled image Perform the same weak perturbation a(·) and strong perturbation operation A(·) on the unlabeled image respectively, and generate the weakly perturbed unlabeled image and the strongly perturbed unlabeled image Pass them through the segmentation model F(·) to obtain the corresponding prediction results and Among them represents the prediction result of the weakly perturbed image without labels, represents the prediction result of the strongly perturbed image without labels, as shown in the following formula.

[0095]

[0096] Among them H and W are the image size, and C is the number of categories. Following the FixMatch method, is used as the pseudo-label of the unlabeled image for supervision, and the cross-entropy loss is calculated for both, as shown in the following formula.

[0097]

[0098] Among them, L semi represents the unlabeled loss; argmax c represents the index of taking the maximum value in the category dimension, indicating the category to which each pixel belongs with the highest probability; max c represents taking the maximum value in the category dimension; τ is the set hyperparameter, representing the confidence threshold; 1(·) retains the cross-entropy loss value at that position if the maximum prediction probability is greater than the set confidence threshold, otherwise it is set to 0 and is not considered when calculating the average value finally.

[0099] For the unlabeled image, on the basis of completing the above operations, the focus of the present invention is to provide a new additional operation for the utilization and training of the unlabeled image. In order to narrow the distribution gap between the labeled image and the unlabeled image and ensure the stable training of the network, the present invention provides a one-way replacement strategy from labeled to unlabeled, and the specific operation is as follows.

[0100] For the prediction result of the weakly perturbed labeled image and the prediction result of the strongly perturbed labeled image calculate their maximum values in the category dimension respectively, and name them confidence scores and as shown in the following formula.

[0101]

[0102] Similarly, the confidence scores of the prediction result of the weakly perturbed unlabeled image and the prediction result of the strongly perturbed unlabeled image are also obtained, as shown in the following formula.

[0103]

[0104] For the four confidence scores calculated above, divide them into each block respectively, and calculate the average confidence score of each block, taking the confidence score of the unlabeled weakly perturbed image as an example, as shown in the following formula.

[0105]

[0106] Among them, j represents the serial number of the j-th block; m represents the serial number of the m-th pixel in each block; k×k represents dividing the entire image and the confidence score into k×k blocks; represents the average confidence score of the j-th block of the unlabeled weakly perturbed image, reflecting whether the prediction result of a block is reliable and accurate. Similarly, calculate the average confidence score of the unlabeled strongly perturbed image the average confidence score of the labeled weakly perturbed image the average confidence score of the labeled strongly perturbed image

[0107] For the above four average confidence scores, find the block serial numbers with the lowest confidence scores respectively, as shown in the formula.

[0108]

[0109] Among them, j represents the j-th block; represents the block serial number with the lowest average confidence score of the unlabeled weakly perturbed image. Similarly, calculate the block serial number with the lowest average confidence score of the unlabeled strongly perturbed image the block serial number with the lowest average confidence score of the labeled weakly perturbed image the block serial number with the lowest average confidence score of the labeled strongly perturbed image

[0110] S55 Subsequently, perform an image replacement operation. Find the and blocks of the labeled weakly perturbed image and crop them. Find the and blocks of the unlabeled weakly perturbed image, and use the and blocks of the labeled weakly perturbed image for replacement operation to obtain a new image Find the and blocks of the labeled strongly perturbed image and crop them. Find the and blocks of the unlabeled strongly perturbed image, and use the and blocks of the labeled strongly perturbed image for replacement operation to obtain a new image

[0111] Input the image and the image into the segmentation network to obtain the prediction results of the new image and where is the prediction result of the image , is the prediction result of the image . Take as the pseudo-label to find the true label y l At the and blocks, crop them, and find the pseudo-label At the and blocks, use the true label y l At the and blocks for replacement operation to obtain a new pseudo-label The new pseudo-label and calculate the cross-entropy loss as shown in the following formula.

[0112]

[0113] where also need to find the and blocks and set the values of both blocks to 1.

[0114] The final loss function is as shown in the following formula, where α, β, and δ are all hyperparameters.

[0115] L = αL sup + βL semi + δL dp

[0116] Use backpropagation loss to optimize the network to complete the training. During the training process of each training epoch, use the validation dataset to verify the segmentation accuracy of the current epoch. If the segmentation accuracy on the validation dataset in the current epoch is the best, save the parameters of the current model. After the training process is completely finished, obtain the parameters of the model by selecting the best result on the validation dataset and get the test results on the prediction dataset. The validation and test processes do not need to execute the above training process. Just directly input the validation dataset and the test dataset into the segmentation model to obtain the prediction probability, and after performing the argmax c operation on the class dimension, output the pixel-level segmentation mask as shown in the following formula, where x test represents the data on the test set, F(·) represents the prediction probability generated by the trained segmentation model, and argmax c(·) represents the final pixel-level segmentation mask.

[0117] y test = argmax c (F(x test ))

[0118] For step S6, the measurement module tests the results of the detection module and the segmentation module specifically including the following steps:

[0119] Step S61: Grayscale conversion processing; The grayscale image only contains luminance information and no color information, which is convenient for us to find various edges in the image. Specifically, for the grayscale conversion processing, it actually utilizes the conversion from the human eye's perception of color to the perception of luminance. The specific conversion formula is as follows:

[0120] Grey = 0.299 ★ R + 0.587 ★ G + 0.114 ★ B

[0121] Step S62: Dilation processing; The dilation operation can increase the target feature value, resulting in the overall magnification of the target image. This step thickens the originally relatively thin edges, enabling the model to better distinguish the contour edge map of the image. The specific image dilation formula is:

[0122]

[0123] where A is the image, B is the corresponding structuring element, is the dilation operator, (x, y) are the pixel coordinates on the image A, and (i, j) are the coordinates of the structuring element B.

[0124] Step S63: Erosion processing; The erosion operation can eliminate noise and at the same time eliminate some boundary values, resulting in the overall reduction of the target image. Erosion and dilation are used together to widen the gap between the contour edges, enabling the adjacent contours to be separated, and allowing the model to better distinguish the contours corresponding to each attribute. The specific image erosion formula is:

[0125]

[0126] The image erosion formula is similar to the image dilation formula. Among them, A is the image, B is the corresponding structuring element, is the erosion operator, (x, y) are the pixel coordinates on the image A, and (i, j) are the coordinates of the structuring element B.

[0127] Step S64: Edge detection; Use the Canny operator to calculate the separated contour edge map, obtain the point set composed of the edge maps of each contour, and separate and identify the various phenotypic features.

[0128] Step S65: Minimum bounding rectangle; Add the corresponding minimum bounding rectangles to each contour map. Approximately regard the length and width of the minimum bounding rectangle as the pixel values of the length and width of the obtained phenotypic features, and the height is the width of each phenotypic feature in the side view.

[0129] Step S66: Calculate the phenotypic size. Use the calibrated digital block diagram (the distance between the midpoints of each digit is constant) obtained in Step S44 to calculate the length in reality corresponding to each pixel value, and multiply it by the pixel values of the approximate length, width, and height of the phenotypic features obtained in S65, and immediately obtain the lengths in reality of each automatically recognized phenotypic feature.

[0130] For Step S7, use the images in the test set to test the overall network trained with the training set images:

[0131] Load the trained overall model parameters and input the preprocessed test set images into the images.

[0132] Evaluate the results using different evaluation criteria according to different tasks:

[0133] For the detection module, calculate the accuracy, precision, and recall.

[0134] Accuracy represents the proportion of the number of correctly recognized samples in the total samples. In the following formula, TN is the number of samples where the predicted result is a negative sample and the actual result is also a negative sample; TP is the number of samples where the predicted result is a positive sample and the actual result is also a positive sample; FP is the number of samples where the predicted result is a positive sample and the actual result is a negative sample; FN is the number of samples where the predicted result is a negative sample and the actual result is a positive sample.

[0135]

[0136] Precision represents the proportion of true positive samples among the samples recognized as positive samples.

[0137]

[0138] Recall represents the proportion of correctly recognized positive samples in the total positive samples.

[0139]

[0140] For the segmentation module, mIoU (mean Intersection over Union) is very common in semantic segmentation (such as image segmentation) tasks, and mIoU is often used as an indicator when evaluating the accuracy and robustness of a model. The level of the mIoU value can reflect the overlapping degree between the predicted segmentation result of the model and the true label. mIoU is obtained by calculating the IoU (Intersection over Union) for each category and then averaging the IoUs of all categories. IoU is defined by calculating the ratio of the intersection to the union of the predicted region and the true label region, and the specific calculation method is shown in the following formula.

[0141]

[0142] Among them, Intersection is the intersection region of the predicted region and the true label region; Union is the union region of the predicted region and the true label region; A is the predicted region; B is the true label region. After calculating the IoU for each category, calculating their average value is mIoU, and the specific calculation method is shown in the following formula.

[0143]

[0144] Among them, N is the number of categories; A i is the predicted region of category i; B i is the true label region of category i.

[0145] For the measurement module, the specific calculation process can refer to the explanation in step S7 and will not be elaborated here.

[0146] For step S8, input the image data into the trained overall network to obtain the corresponding output result.

[0147] The above is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered by the protection scope of the present invention.

Claims

1. A crayfish phenotype detection method based on key point detection and image segmentation, characterized in that The method includes: Obtain crayfish images, and perform annotation processing on the crayfish images to form a crayfish image dataset; Preprocess the crayfish image dataset; Construct a crayfish morphology recognition model, and train the crayfish morphology recognition model based on the preprocessed crayfish image dataset; Recognize the crayfish images through the trained crayfish morphology recognition model, and output the phenotypic data of the crayfish.

2. The crayfish phenotype detection method based on key point detection and image segmentation according to claim 1, wherein The preprocessing of the crayfish image dataset includes resampling and normalizing the crayfish images, and performing one or more of the operations of cropping, expanding, randomly rotating, and flipping to enhance the image data.

3. The crayfish phenotype detection method based on key point detection and image segmentation according to claim 1, characterized in that, The crayfish morphology recognition model includes a detection module, a segmentation module, and a measurement module; when training the crayfish morphology recognition model based on the preprocessed crayfish image dataset, the detection module, the segmentation module, and the measurement module are trained in sequence.

4. The crayfish phenotype detection method based on key point detection and image segmentation according to claim 3, wherein The training of the detection module is specifically as follows: Design the backbone structure of the detection module, where the backbone structure includes a Conv module for feature extraction, a C2f module for optimizing gradient flow and reducing redundant parameters, and an SPPF module for splicing feature maps of different scales to enhance the detection ability for targets of different sizes; Design the head structure of the detection module, where the head structure includes a detection head and a classification head. The detection head is used to generate detection results, and the classification head uses global average pooling to classify the feature map and output the class probability distribution. The final detection results including bounding boxes, classes, and confidences are obtained by converting the fused feature map through the Detect module; Calculate the loss function of the detection module, where the loss function includes cross-entropy loss for measuring the difference between the prediction and the true probability distribution, DFL loss for correcting the bounding box regression error to improve the detection accuracy of blurred or unfocused images, and CIOU loss for adding the center point distance and relative ratio on the basis of the IOU loss to enhance the convergence speed and detection effect; Through multiple iterative trainings, continuously adjust the model parameters until the loss function converges to a preset threshold or reaches a preset number of training epochs to complete the training of the detection module.

5. The crayfish phenotype detection method based on key point detection and image segmentation according to claim 3, characterized in that, The training of the segmentation module is specifically as follows: Adopt a semi-supervised semantic segmentation method, and perform perturbation operations on the labeled images and unlabeled images respectively. Among them, weak perturbations are performed on the labeled images, such as cropping, rotating, and flipping; strong perturbations are performed on the unlabeled images, such as changing color contrast, brightness, cropping and covering, and adding noise; Input the perturbed labeled images and unlabeled images into the segmentation module respectively to obtain corresponding prediction results, and calculate the cross-entropy loss between the prediction results and the true labels or pseudo-labels; Implement a one-way replacement strategy from labeled to unlabeled, calculate the confidence scores and block average confidence scores of the labeled images and unlabeled images, find the block with the lowest average confidence score in the unlabeled images, replace it with the image block at the corresponding position in the labeled images, and update the pseudo-labels of the unlabeled images; The loss is fed back into the segmentation module through the backpropagation algorithm to adjust the parameters of the segmentation module, continuously optimize the network, evaluate the segmentation accuracy on the validation dataset, and save the parameters of the segmentation module when the segmentation accuracy reaches the best.

6. The crayfish phenotype detection method based on key point detection and image segmentation according to claim 3, characterized in that, The specific training of the measurement module is as follows: Using the training results of the detection module and the segmentation module, the image is subjected to gray-scale conversion processing to convert the color image into a gray-scale image containing only luminance information; The gray-scale image is subjected to dilation processing, and a preset dilation kernel is used to thicken the fine edges; The dilated image is subjected to erosion processing, and a preset erosion kernel is used to eliminate noise and some boundary values; The Canny operator is used to detect the edges of the eroded image to obtain the edge information of the image; A minimum bounding rectangle is added to the contour map after edge detection, and the length and width of the minimum bounding rectangle are approximated as the pixel values of the phenotypic features; The actual length corresponding to each pixel is calculated, so as to obtain the actual lengths of various phenotypic features of the crayfish and complete the training of the measurement module.

7. The crayfish phenotype detection method based on key point detection and image segmentation according to claim 1, characterized in that, The recognition of the crayfish image by the trained crayfish morphology recognition model and the output of the phenotypic data of the crayfish are specifically as follows: The trained detection module is used to recognize the input crayfish image, determine the positions of the crayfish and the calibration object in the image, and perform fixed-size segmentation on the original image; The segmented image is input into the trained segmentation module to perform image segmentation on different parts of the crayfish to obtain the segmentation masks of different parts of the crayfish; The trained measurement module recognizes the length of the calibration object according to the segmentation mask output by the segmentation module, and calculates the proportional relationship between the pixel length and the actual length of the calibration object; Based on the above proportional relationship, the measurement module calculates the actual lengths of different parts of the crayfish and finally outputs the phenotypic data of the crayfish.

8. A crayfish phenotype detection system based on key point detection and image segmentation, characterized in that, Including: An image acquisition and annotation unit for acquiring crayfish images and performing annotation processing on the crayfish images to form a crayfish image dataset; A data preprocessing unit for preprocessing the crayfish image dataset, including resampling, normalization processing, and image data enhancement using one or more of operations such as cropping, expansion, random rotation, and flipping; A model construction and training unit for constructing a crayfish morphology recognition model including a detection module, a segmentation module, and a measurement module, and sequentially training the detection module, the segmentation module, and the measurement module based on the preprocessed crayfish image dataset; A recognition and output unit for recognizing the crayfish image by the trained crayfish morphology recognition model and outputting the phenotypic data of the crayfish.

Images