A method and system for crayfish phenotypic detection based on key point detection and image segmentation

Abstract

This invention provides a method and system for crayfish phenotypic detection based on keypoint detection and image segmentation. The method includes: acquiring crayfish images and annotating the images to form a crayfish image dataset; preprocessing the crayfish image dataset; constructing a crayfish morphology recognition model and training the model based on the preprocessed crayfish image dataset; and using the trained model to recognize crayfish images and output crayfish phenotypic data. This invention enables rapid and accurate analysis of crayfish phenotypic data.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for detecting crayfish phenotypic patterns based on key point detection and image segmentation. Background Technology

[0002] With the continuous development of computer information technology and image processing technology, machine vision technology has been widely used in the field of aquatic animal attribute data collection and recognition. It can quickly, economically, and non-destructively detect the size, shape, color, and other appearance characteristics of aquatic animals. These characteristics are of great significance for assessing the growth status of aquatic animals and are one of the key factors for the development of aquaculture.

[0003] Accurate phenotypic data is crucial for studying the relationship between genotype, environment, and phenotype in crayfish, with precise identification of phenotypic data such as sex characteristics being particularly critical. However, the current method of sorting commercial crayfish (Procambarus clarkii) by size primarily relies on manual selection. This method depends on workers' accumulated experience, judging the integrity, vitality, and size of the crayfish before categorizing them. However, this process has significant drawbacks: individual worker subjectivity and variability lead to large errors in the categorization results; moreover, manual categorization is time-consuming, during which crayfish vitality decreases, stress mortality increases, and significant economic losses occur for the industry. Traditional methods of manually obtaining crayfish phenotypic data are not only time-consuming and labor-intensive but also fail to meet the demands of modern aquaculture for high efficiency and precision. Therefore, there is an urgent need for an efficient and accurate image morphology recognition method and system to solve these problems. Summary of the Invention

[0004] This invention aims to provide a crayfish phenotypic detection method and system based on key point detection and image segmentation, in order to solve the problem of time-consuming and labor-intensive manual observation and measurement to obtain crayfish phenotypic data. By introducing computer vision technology, it enables rapid and accurate analysis of crayfish phenotypic data, thereby saving time and labor costs for size classification in actual production.

[0005] The technical solution of the present invention is as follows:

[0006] A method for detecting crayfish phenotypic characteristics based on keypoint detection and image segmentation, the method comprising:

[0007] Acquire crayfish images and annotate them to form a crayfish image dataset;

[0008] The crayfish image dataset is preprocessed;

[0009] A crayfish morphology recognition model was constructed, and the model was trained based on a preprocessed crayfish image dataset.

[0010] The trained crayfish morphology recognition model is used to identify crayfish images and output crayfish phenotypic data.

[0011] Furthermore, the preprocessing of the crayfish image dataset includes resampling and normalizing the crayfish images, as well as performing image data enhancement using one or more of the following operations: cropping, augmentation, random rotation, and flipping.

[0012] Furthermore, the crayfish morphology recognition model includes a detection module, a segmentation module, and a measurement module; when training the crayfish morphology recognition model based on the preprocessed crayfish image dataset, the detection module, segmentation module, and measurement module are trained sequentially.

[0013] Furthermore, the training of the detection module specifically involves:

[0014] The backbone structure of the detection module is designed, which includes a Conv module for feature extraction, a C2f module for optimizing gradient flow and reducing redundant parameters, and an SPPF module for stitching feature maps of different scales to enhance the detection capability for targets of different sizes.

[0015] The head structure of the detection module is designed, which includes a detection head and a classification head. The detection head is used to generate detection results, and the classification head uses global average pooling to classify the feature map and output the class probability distribution. The Detect module converts the fused feature map into a final detection result containing bounding boxes, classes, and confidence scores.

[0016] The loss function of the detection module is calculated. The loss function includes cross-entropy loss, which measures the difference between the prediction and the true probability distribution; DFL loss, which is used to correct the bounding box regression error to improve the detection accuracy of blurred or unfocused images; and CIOU loss, which adds center point distance and relative proportion to the IOU loss to enhance the convergence speed and detection effect.

[0017] Through multiple iterations of training, the model parameters are continuously adjusted until the loss function converges to a preset threshold or reaches a preset number of training rounds, thus completing the training of the detection module.

[0018] Furthermore, the training of the segmentation module specifically involves:

[0019] A semi-supervised semantic segmentation method is adopted, and perturbation operations are performed on labeled and unlabeled images respectively. Weak perturbation, such as cropping, rotating and flipping, is applied to labeled images; strong perturbation, such as changing color contrast and brightness, cropping and covering, and adding noise, is applied to unlabeled images.

[0020] The perturbed labeled and unlabeled images are input into the segmentation module to obtain the corresponding prediction results, and the cross-entropy loss between the prediction results and the real labels or pseudo labels is calculated.

[0021] Implement a one-way replacement strategy from labeled to unlabeled images. Calculate the confidence scores and average block confidence scores of labeled and unlabeled images. Find the block with the lowest average confidence score in the unlabeled image and replace it with the corresponding image block in the labeled image. Update the pseudo-label of the unlabeled image.

[0022] The loss is fed back to the segmentation module through the backpropagation algorithm, the parameters of the segmentation module are adjusted, the network is continuously optimized, the segmentation accuracy is evaluated on the validation dataset, and the parameters of the segmentation module when the segmentation accuracy is optimal are saved.

[0023] Furthermore, the training of the measurement module specifically involves:

[0024] Using the training results of the detection and segmentation modules, the image is converted to grayscale, transforming the color image into a grayscale image containing only brightness information;

[0025] The grayscale image is dilated, and a preset dilation kernel is used to thicken the fine edges;

[0026] The dilated image is subjected to erosion processing, and a preset erosion kernel is used to eliminate noise and some boundary values;

[0027] Edge detection is performed on the eroded image using the Canny operator to obtain the edge information of the image;

[0028] Add a minimum bounding rectangle to the contour map after edge detection, and approximate the length and width of the minimum bounding rectangle to the pixel values ​​of the phenotypic features;

[0029] The actual length corresponding to each pixel is calculated, thereby obtaining the actual length of each phenotypic feature of the crayfish and completing the training of the measurement module.

[0030] Furthermore, the step of recognizing crayfish images using a trained crayfish morphology recognition model and outputting crayfish phenotypic data specifically involves:

[0031] The trained detection module is used to identify the input crayfish image, determine the position of the crayfish and the marker in the image, and perform fixed-size segmentation on the original image;

[0032] The segmented image is input into the trained segmentation module to segment different parts of the crayfish and obtain the segmentation mask for different parts of the crayfish.

[0033] The trained measurement module identifies the length of the calibration object based on the segmentation mask output by the segmentation module, and calculates the ratio between the pixel length of the calibration object and the actual length.

[0034] Based on the above proportional relationships, the measurement module calculates the actual length of different parts of the crayfish and finally outputs the crayfish's phenotypic data.

[0035] This invention also provides a crayfish phenotypic detection system based on key point detection and image segmentation, comprising:

[0036] The image acquisition and annotation unit is used to acquire crayfish images and perform annotation processing on the crayfish images to form a crayfish image dataset;

[0037] The data preprocessing unit is used to preprocess the crayfish image dataset, including resampling, normalization, and image data enhancement using one or more of the following operations: cropping, augmentation, random rotation, and flipping.

[0038] The model building and training unit is used to build a crayfish morphology recognition model that includes a detection module, a segmentation module, and a measurement module, and to train the detection module, segmentation module, and measurement module in sequence based on the preprocessed crayfish image dataset.

[0039] The recognition and output unit is used to recognize crayfish images using a trained crayfish morphology recognition model and output the phenotypic data of the crayfish.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] This invention strengthens the original data samples through preprocessing methods such as resampling, normalization, and image data enhancement, effectively mitigating the overfitting problem of the system, improving the robustness of the overall system, and enabling more effective capture of image detail information, thereby enhancing the ability to deeply understand and process image content.

[0042] This invention trains on a small amount of labeled data and enhances the generalization of the model by adding weak and strong perturbations to the data. It uses a unidirectional replacement strategy of labeled images with unlabeled images to make the generated pseudo-label results more accurate, the network training process more stable, and finally obtain more accurate segmentation results. Attached Figure Description

[0043] The accompanying drawings illustrate various embodiments generally by way of example rather than limitation, and are used, together with the specification and claims, to explain embodiments of the invention. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0044] Figure 1 This is a schematic flowchart of the crayfish phenotypic detection method of the present invention;

[0045] Figure 2 This is an overall structural diagram of the detection module designed for this invention;

[0046] Figure 3 The diagram illustrates the training process of the semi-supervised segmentation algorithm with labeled data designed for this invention.

[0047] Figure 4 The diagram illustrates the unlabeled data training process of the semi-supervised segmentation algorithm designed for this invention.

[0048] Figure 5 This is a diagram illustrating the one-way replacement process from labeled data to unlabeled data designed for this invention. Detailed Implementation

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] like Figure 1 As shown, this invention provides a crayfish phenotypic detection method based on keypoint detection and image segmentation, the method comprising:

[0051] Step S1: Obtain the required image dataset according to the crayfish image recognition task requirements;

[0052] Step S2: Preprocess the dataset;

[0053] Step S3: Divide the dataset into a training set and a test set;

[0054] Step S4: Design the overall network according to the task and train the detection module using training set images;

[0055] Step S5: Train the segmentation module using the training set images;

[0056] Step S6: Train the measurement module using the results from the detection module and the segmentation module.

[0057] Step S7: Test the overall network trained on the training set images using images from the test set;

[0058] Step S8: Input the image data into the trained network to obtain the corresponding output results.

[0059] For step S1: According to the requirements of the crayfish image recognition task, obtain the required image dataset.

[0060] Specifically, the data acquisition involves taking images of crayfish from different angles, at different growth stages, and under different environments according to the task requirements, and labeling the crayfish's location, key parts, and gender characteristics in the images to form a new image dataset.

[0061] For step S2: Preprocess the dataset.

[0062] Specifically, preprocessing methods include resampling, normalization, and image data augmentation, where data augmentation includes cropping, augmentation, random rotation, and flipping operations.

[0063] Normalization: Normalize the image by scaling the intensity of pixels or voxels in the dataset to [0,1], and setting the intensity of pixels or voxels outside the range to 0 and 1 respectively.

[0064] Data augmentation aims to enhance the generalization ability of a model by generating more training samples through transformation of the original image.

[0065] For step S3: Divide the images into training set and test set.

[0066] Specifically, this step ensures that the model can be learned on a subset (the training set) and then evaluated on another independent subset (the test set), thereby validating the model's generalization ability. A common split is 80% training set and 20% test set.

[0067] For step S4, training the detection module using the training set images specifically includes the following steps:

[0068] Step S41: Design the backbone structure of the detection module; such as... Figure 2 As shown, its AP stage is the backbone structure. The backbone structure is mainly responsible for feature extraction and multi-scale feature fusion, so it uses a series of convolutional and deconvolutional layers, residual connections and bottleneck structures to reduce the network size and improve performance, and an SPPF module is used to stitch feature maps of different scales together to improve the detection capability of targets of different sizes.

[0069] The backbone structure uses the Conv module, which is responsible for extracting features from the input image. It generates feature maps by performing element-wise multiplication and summation on the input data using sliding small matrices (called kernels or filters). The Conv module is the foundational module of the entire model.

[0070] In addition, a lightweight C2f module was used; the C2f module achieves higher feature extraction efficiency and a lighter network structure by optimizing gradient flow and reducing redundant parameters. Therefore, compared with other structures, it has fewer parameters and better feature extraction capabilities, and reduces redundant parameters.

[0071] To achieve multi-scale feature fusion, the SPPF module was used; the SPPF module is used for pooling operations at different scales. It stitches feature maps of different scales together, improving the detection capability for targets of different sizes.

[0072] Step S42: Design the head structure of the detection module; such as... Figure 2 As shown, its PB stage is the head structure. The head structure is mainly responsible for the final object detection and classification tasks, including a detection head and a classification head. The detection head contains a series of convolutional and deconvolutional layers to generate detection results; the classification head uses global average pooling to classify each feature map and outputs the probability distribution of each class.

[0073] In addition to the basic Conv and C2f modules, the backbone structure also uses a Detect module. The Detect module is responsible for converting the fused feature map into the final detection result, including predicted bounding boxes, categories, and confidence scores, in order to achieve the object detection and classification tasks required by the head structure.

[0074] Step S43: Calculate the loss function; the detection module specifically uses three types of loss: cross-entropy loss, DFL loss, and CIOU loss.

[0075] Cross-entropy loss measures the difference between the predicted probability distribution and the true probability distribution. In image classification tasks, the cross-entropy loss function is often used in conjunction with the softmax activation function to transform the model's raw output into a probability distribution.

[0076]

[0077] N represents the number of samples; K represents the number of categories; y ic One-hot encoding representing the target value of the sample. If sample p i If the true category is equal to c, then set it to 1; otherwise, set it to 0. (h) θ (x i ) c Representative sample x i The probability of belonging to category c. The closer the model's predicted probability distribution is to the true label probability distribution, the smaller the value of the cross-entropy loss function. The cross-entropy loss function can be directly used in the gradient descent algorithm to update the model's weights.

[0078] DFL loss is primarily used in the bounding box regression process of object detection tasks. Its main function is to correct errors in the model's prediction of object bounding boxes, and the optimized result can improve object detection accuracy to some extent, especially for blurry or out-of-focus images.

[0079] DFL(y i ,y i+1 )=-(i+1-y)log(y i )-(yi)log(y i+1 )

[0080] Where y is the actual label, yi and yi+1 represent the two predicted labels closest to the actual label y, and the variables in the formula satisfy... The lower the DFL loss during training, the better the model's performance in predicting bounding boxes.

[0081] CIOU loss is developed based on IOU loss, which is mainly used in object detection tasks to measure the overlap between anchor boxes and target boxes. It effectively masks the interference of bounding box size in a proportional manner, allowing the model to effectively balance the learning of large and small objects when using 1-IoU as the BBR loss. CIOU loss adds center point distance and relative proportion to IOU loss, further enhancing convergence speed and detection performance.

[0082]

[0083] This formula contains:

[0084]

[0085] Where v measures the consistency of the relative proportions of the two rectangles, and α is the weighting coefficient; where b and b gt Let represent the center point of the two bounding boxes, ρ represent the Euclidean distance between the two bounding boxes, and c represent the distance between the diagonals of the closure regions of the two bounding boxes; A represents the predicted image, B represents the ground truth label, and w and h represent the width and height of the predicted image, respectively. gt and h gt This represents the width and height of the ground truth label. The optimization goal of CIoU loss is to directly reduce the Euclidean distance between the center points of the two rectangles. The role of 'c' is to prevent the value of the loss function from becoming too large, thereby improving the convergence speed. At the same time, a relative proportion is added to it to penalize results where the predicted shape is inconsistent with the ground truth.

[0086] Step S44: Complete the training of the detection module.

[0087] For step S5, training the segmentation module using the training set images specifically includes the following steps:

[0088] S51 acquires the required image dataset and divides it into training, validation, and test sets according to different proportions. In a semi-supervised semantic segmentation scenario, the training set consists of a set of labeled images. and a set of unlabeled images Composition, in which Represents the i-th labeled image. This represents the segmentation mask corresponding to the i-th labeled image. This represents the i-th unlabeled image. Unlabeled images are images without corresponding segmentation masks, and the number of unlabeled images is much greater than the number of labeled images.

[0089] S52 for labeled images First, examine the labeled images. Weak perturbation operations a(·) and strong perturbation operations A(·) are performed respectively. Weak perturbation operations include image cropping, rotation, and flipping; strong perturbation operations include changing image attributes such as color contrast and brightness, cropping / covering the image, and adding noise. The perturbated images are then categorized into labeled weakly perturbated images. and labeled images with strong perturbations They are then used to obtain the corresponding prediction results through the segmentation model F(·). and in The prediction results represent labeled, weakly perturbated images. The prediction result for a labeled, strongly perturbed image is shown in the following formula.

[0090]

[0091] The cross-entropy loss is calculated between the obtained prediction results and the true labels to measure the difference between the predicted probability distribution and the true probability distribution. The specific operation is shown in the following formula.

[0092]

[0093] Where CE represents cross-entropy loss.

[0094] For unlabeled images For unlabeled images Perform the same weak perturbation a(·) and strong perturbation A(·) operations as described above to generate unlabeled weak perturbation images. and unlabeled strongly perturbated images After passing them through the segmentation model F(·), the corresponding prediction results are obtained. and in The prediction results represent unlabeled, weakly perturbated images. The prediction result for an unlabeled, strongly perturbed image is shown in the following formula.

[0095]

[0096] in H and W represent the image size, and C represents the number of categories. Following the FixMatch method, ... pseudo-labels for unlabeled images For supervision, both are used to calculate the cross-entropy loss, as shown in the following formula.

[0097]

[0098] Where L semi Represents unlabeled loss; argmax c This represents the index of the maximum value in the category dimension, indicating the category to which each pixel has the highest probability; max c This represents the maximum value in the category dimension; τ is a set hyperparameter representing the confidence threshold; 1(·) If the maximum predicted probability is greater than the set confidence threshold, retain the cross-entropy loss value at that position; otherwise, set it to 0. This position is not considered when calculating the final average.

[0099] S53 For unlabeled images, based on the above operations, the focus of this invention is to provide a new additional operation for the utilization and training of unlabeled images. In order to reduce the distribution gap between labeled and unlabeled images and ensure the stability of network training, this invention provides a one-way replacement strategy from labeled to unlabeled images, the specific operation of which is as follows.

[0100] Prediction results for labeled weakly perturbated images Prediction results for labeled strongly perturbed images Calculate their maximum values ​​in the category dimension and name them as confidence scores. and As shown in the following formula.

[0101]

[0102] Similarly, the prediction results for unlabeled weakly perturbated images are also obtained. Prediction results of unlabeled strongly perturbated images The confidence score is shown in the following formula.

[0103]

[0104] S54 divides the four confidence scores calculated above into blocks, calculates the average confidence score for each block, and uses the confidence score of the unlabeled weakly perturbed image as the basis for the calculation. For example, as shown in the following formula.

[0105]

[0106] Where j represents the index of the j-th block; m represents the index of the m-th pixel in each block; k×k represents dividing the entire image and confidence score into k×k blocks; The average confidence score representing the j-th block of an unlabeled weakly perturbated image reflects the reliability and accuracy of the prediction for that block. Similarly, the average confidence score for an unlabeled strongly perturbated image is calculated. Average confidence score of labeled weakly perturbated images Average confidence score of labeled strongly perturbed images

[0107] For each of the four average confidence scores mentioned above, find the block number with the lowest confidence score, as shown in the formula.

[0108]

[0109] Where j represents the j-th block; This represents the block number with the lowest average confidence score in the unlabeled weakly perturbed image. Similarly, the block number with the lowest average confidence score in the unlabeled strongly perturbed image is obtained. The block number with the lowest average confidence score for labeled weakly perturbated images The block number with the lowest average confidence score in the labeled strongly perturbed image

[0110] S55 then performs an image replacement operation to find the first image with a marked weak perturbation. and Block and crop to find the unlabeled weak perturbation image. and The block, using the first labeled weakly perturbation image and The block is replaced to obtain a completely new image. Find the first labeled strongly perturbation image and Block and crop to find the first unlabeled strongly perturbed image. and The block, using the first strongly perturbed image with labels. and The block is replaced to obtain a completely new image.

[0111] Image and images Input the data into the segmentation network to obtain prediction results for the new image. and in It is an image The prediction results It is an image The prediction results. As a pseudo-label Find the real label y l No. and Block and cut to find false labels No. and Block, using real label y l No. and The block is replaced to obtain a completely new pseudo-tag. New pseudo-labels and The cross-entropy loss is calculated as shown in the following formula.

[0112]

[0113] in, We still need to find the first and Set the value of both blocks to 1.

[0114] The final loss function is shown in the following equation, where α, β, and δ are hyperparameters.

[0115] L=αL sup +βL semi +δL dp

[0116] The network is trained using backpropagation loss optimization. During each training epoch, the segmentation accuracy is validated on a validation dataset. If the segmentation accuracy on the validation dataset is the best, the parameters of the current model are saved. After training is complete, the model's parameters are selected based on the best results from the validation dataset, and then tested on the prediction dataset. Alternatively, the validation and testing processes do not require the same training steps. The validation and testing datasets are directly input into the segmentation model to obtain predicted probabilities, which are then processed using argmax on the class dimension. c The operation outputs a pixel-level segmentation mask, as shown in the following formula, where x test The data represents the test set, F(·) represents the predicted probability generated by the trained segmentation model, and argmax represents the predicted probability. c(·) represents the final pixel-level segmentation mask.

[0117] y test =argmax c (F(x test ))

[0118] For step S6, testing the measurement module based on the results from the detection module and the segmentation module specifically includes the following steps:

[0119] Step S61: Grayscale conversion processing; Grayscale images only contain brightness information and not color information, making it easier for us to find edges in the image. The specific grayscale conversion processing actually utilizes the conversion between the human eye's perception of color and its perception of brightness. The specific conversion formula is as follows:

[0120] Grey = 0.299 ★ R+0.587 ★ G+0.114 ★ B

[0121] Step S62: Dilation Process; Dilation increases the target feature values, resulting in overall image magnification. This step thickens previously fine edges, allowing the model to better distinguish the image's contour edges. The specific image dilation formula is:

[0122]

[0123] Where A is the image and B is the corresponding structural element. For dilation, (x, y) are the pixel coordinates on image A, and (i, j) are the coordinates of structuring element B.

[0124] Step S63: Erosion Processing; Erosion can eliminate noise and some boundary values, resulting in an overall reduction in the target image size. Using erosion and dilation together widens the gaps between contour edges, allowing adjacent contours to be separated, and enabling the model to better distinguish the contours corresponding to each attribute. The specific image erosion formula is as follows:

[0125]

[0126] The formula for image erosion is similar to that for image dilation, where A represents the image and B represents the corresponding structuring element. For erosion, (x, y) are the pixel coordinates on image A, and (i, j) are the coordinates of struct element B.

[0127] Step S64: Edge detection; use the Canny operator to calculate the separated contour edge map, obtain the point set formed by the edge map of each contour, and separate and identify each phenotypic feature.

[0128] Step S65: Minimum bounding rectangle; Add the corresponding minimum bounding rectangle to each contour map. The length and width of the minimum bounding rectangle are approximated as the pixel values ​​of the length and width of the obtained phenotypic features, and the height is the width of each phenotypic feature in the side view.

[0129] Step S66: Calculate the phenotypic size. Using the calibration digit diagram obtained in step S44 (with constant spacing between the midpoints of each digit), calculate the actual length corresponding to each pixel value. Multiply this length by the pixel values ​​of the approximate length, width, and height of the phenotypic feature obtained in step S65 to obtain the actual length of each automatically recognized phenotypic feature.

[0130] For step S7, the overall network trained on the training set images is tested using images from the test set:

[0131] Load the trained overall model parameters and input the preprocessed test set images into the image.

[0132] Different evaluation criteria are used to evaluate the results based on different tasks:

[0133] For the detection module, calculate accuracy, precision, and recall.

[0134] Accuracy represents the proportion of correctly identified samples out of the total sample. In the following formula, TN is the number of samples that were predicted as negative and were actually negative; TP is the number of samples that were predicted as positive and were actually positive; FP is the number of samples that were predicted as positive and were actually negative; and FN is the number of samples that were predicted as negative and were actually positive.

[0135]

[0136] Precision represents the proportion of true positive class samples that are identified as positive class samples.

[0137]

[0138] Recall rate represents the proportion of correctly identified positive samples out of the total number of positive samples.

[0139]

[0140] For segmentation modules, mIoU (mean Intersection over Union) is very common in semantic segmentation tasks (such as image segmentation), and it is often used as a metric to evaluate the accuracy and robustness of a model. The mIoU value reflects the degree of overlap between the model's predicted segmentation results and the ground truth labels. mIoU is obtained by calculating the IoU (Intersection over Union) for each category and then averaging the IoUs across all categories. IoU is defined as the ratio of the intersection to the union of the predicted and ground truth regions, as shown in the following formula.

[0141]

[0142] Where Intersection is the intersection of the predicted region and the ground truth label region; Union is the union of the predicted region and the ground truth label region; A is the predicted region; and B is the ground truth label region. The average of the IoU values ​​for each category is calculated to obtain mIoU, as shown in the following formula.

[0143]

[0144] Where N is the number of categories; A i B is the predicted region for category i; i It is the actual label region of category i.

[0145] For the measurement module, the specific calculation process can be found in the explanation of step S7, and will not be repeated here.

[0146] For step S8, the image data is input into the trained overall network to obtain the corresponding output result.

[0147] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting crayfish phenotypic characteristics based on keypoint detection and image segmentation, characterized in that, The method includes: Acquire crayfish images and annotate them to form a crayfish image dataset; The crayfish image dataset is preprocessed; A crayfish morphology recognition model was constructed, and the model was trained based on a preprocessed crayfish image dataset. The trained crayfish morphology recognition model is used to identify crayfish images and output crayfish phenotypic data. The crayfish morphology recognition model includes a detection module, a segmentation module, and a measurement module. When training the crayfish morphology recognition model based on the preprocessed crayfish image dataset, the detection module, segmentation module, and measurement module are trained sequentially. The training of the detection module specifically involves: The backbone structure of the detection module is designed, which includes a Conv module for feature extraction, a C2f module for optimizing gradient flow and reducing redundant parameters, and an SPPF module for stitching feature maps of different scales to enhance the detection capability for targets of different sizes. The head structure of the detection module is designed, which includes a detection head and a classification head. The detection head is used to generate detection results, and the classification head uses global average pooling to classify the feature map and output the class probability distribution. The Detect module converts the fused feature map into a final detection result containing bounding boxes, classes, and confidence scores. The loss function of the detection module is calculated. The loss function includes cross-entropy loss, which measures the difference between the prediction and the true probability distribution; DFL loss, which is used to correct the bounding box regression error to improve the detection accuracy of blurred or unfocused images; and CIOU loss, which adds center point distance and relative proportion to the IOU loss to enhance the convergence speed and detection effect. Through multiple iterations of training, the model parameters are continuously adjusted until the loss function converges to a preset threshold or reaches a preset number of training rounds, thus completing the training of the detection module. The specific steps for training the segmentation module are as follows: A semi-supervised semantic segmentation method is adopted, and perturbation operations are performed on labeled and unlabeled images respectively. The labeled images are subjected to weak perturbation, including cropping, rotation, and flipping; the unlabeled images are subjected to strong perturbation, including changing color contrast and brightness, cropping and covering, and adding noise. The perturbed labeled and unlabeled images are input into the segmentation module to obtain the corresponding prediction results, and the cross-entropy loss between the prediction results and the real labels or pseudo labels is calculated. Implement a one-way replacement strategy from labeled to unlabeled images. Calculate the confidence scores and average block confidence scores of labeled and unlabeled images. Find the block with the lowest average confidence score in the unlabeled image and replace it with the corresponding image block in the labeled image. Update the pseudo-label of the unlabeled image. The loss is fed back to the segmentation module through the backpropagation algorithm, the parameters of the segmentation module are adjusted, the network is continuously optimized, the segmentation accuracy is evaluated on the validation dataset, and the segmentation module parameters when the segmentation accuracy reaches the best are saved. The specific steps of training the measurement module are as follows: Using the training results of the detection and segmentation modules, the image is converted to grayscale, transforming the color image into a grayscale image containing only brightness information; The grayscale image is dilated, and a preset dilation kernel is used to thicken the fine edges; The dilated image is subjected to erosion processing, and a preset erosion kernel is used to eliminate noise and some boundary values; Edge detection is performed on the eroded image using the Canny operator to obtain the edge information of the image; Add a minimum bounding rectangle to the contour map after edge detection, and approximate the length and width of the minimum bounding rectangle to the pixel values ​​of the phenotypic features; The actual length corresponding to each pixel is calculated, thereby obtaining the actual length of each phenotypic feature of the crayfish and completing the training of the measurement module.

2. The crayfish phenotypic detection method based on key point detection and image segmentation according to claim 1, characterized in that, The preprocessing of the crayfish image dataset includes resampling and normalizing the crayfish images, as well as image data enhancement using one or more of the following operations: cropping, augmentation, random rotation, and flipping.

3. The crayfish phenotypic detection method based on key point detection and image segmentation according to claim 1, characterized in that, The process of recognizing crayfish images using a trained crayfish morphology recognition model and outputting crayfish phenotypic data specifically involves: The trained detection module is used to identify the input crayfish image, determine the position of the crayfish and the marker in the image, and perform fixed-size segmentation on the original image; The segmented image is input into the trained segmentation module to segment different parts of the crayfish and obtain the segmentation mask for different parts of the crayfish. The trained measurement module identifies the length of the calibration object based on the segmentation mask output by the segmentation module, and calculates the ratio between the pixel length of the calibration object and the actual length. Based on the above proportional relationship, the measurement module calculates the actual length of different parts of the crayfish and finally outputs the phenotypic data of the crayfish.

4. A crayfish phenotypic detection system based on keypoint detection and image segmentation, applied to the crayfish phenotypic detection method based on keypoint detection and image segmentation as described in claim 1, characterized in that, The system includes: The image acquisition and annotation unit is used to acquire crayfish images and perform annotation processing on the crayfish images to form a crayfish image dataset; The data preprocessing unit is used to preprocess the crayfish image dataset, including resampling, normalization, and image data enhancement using one or more of the following operations: cropping, augmentation, random rotation, and flipping. The model building and training unit is used to build a crayfish morphology recognition model that includes a detection module, a segmentation module, and a measurement module, and to train the detection module, segmentation module, and measurement module in sequence based on the preprocessed crayfish image dataset. The recognition and output unit is used to recognize crayfish images using a trained crayfish morphology recognition model and output the phenotypic data of the crayfish.

Images