Cow mastitis assessment method and system based on small sample learning

Abstract

The invention relates to the technical field of artificial intelligence and computer vision, in particular to a dairy cow mastitis assessment method and system based on small sample learning, and the method comprises the steps: firstly constructing a dairy cow nipple image data set containing target detection and health classification; and carrying out high-quality generation and expansion on scarce unhealthy nipple images by adopting an image enhancement algorithm based on a generative adversarial network. In the first stage, an improved YOLOv8 model fusing spatial pyramid pooling and lightweight backbone network MobileNetV3 is adopted to realize accurate real-time detection and positioning of dairy cow nipples in a complex breeding environment; and in the second stage, an improved GoogLeNet model of an attention mechanism is introduced, end-to-end feature extraction is performed on the cut nipple region image, and a health condition score of the dairy cow nipple is output according to the roughness and thickness of the keratinous ring at the tail end of the nipple. Through data expansion and model structure optimization, the defects of high subjectivity and low efficiency of traditional manual detection are effectively overcome.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and in particular to a method and system for assessing mastitis in dairy cows based on few-shot learning. Background Technology

[0002] Current manual assessment methods for bovine mastitis suffer from high subjectivity, low efficiency, and poor consistency. Existing technologies utilize radiomics features to construct intelligent diagnostic models for bovine mastitis. These models employ automated or semi-automated software to non-invasively extract high-throughput image features that are difficult to detect with the naked eye from medical images such as ultrasound, infrared thermography, or optical coherence tomography (OCT), transforming these features into quantifiable radiomics indicators. Combined with deep data mining methods, machine learning algorithms are used to construct highly sensitive and specific diagnostic, risk stratification, and prognostic prediction models, providing scientific evidence and technical support for early screening, accurate classification, individualized intervention, and intelligent farm management of bovine mastitis. While deep learning-based methods have shown potential, they face the following challenges in practical applications:

[0003] 1. Small sample size and class imbalance: In real-world pastures, healthy teats constitute the vast majority, while samples of diseased teats requiring focused attention are extremely rare. This extreme class imbalance causes deep learning models to be severely biased towards the majority class, resulting in a very low recognition rate for the minority class and failing to meet the practical needs of disease early warning. Traditional data augmentation methods cannot generate images with novel semantic features, and their effectiveness in addressing this fundamental data scarcity problem is limited.

[0004] 2. Accurate Perception in Complex Environments: Farm environments often feature uneven lighting and cluttered backgrounds. Furthermore, the relatively small size of the teat makes it susceptible to obstruction and interference from the cow's abdomen, legs, and fecal stains, leading to missed and false detections by the detection model. In addition, differences in teat health are primarily reflected in the texture, thickness, and roughness of the keratin ring in a small area at the teat end. These subtle features are weak in the overall image and are easily obscured by irrelevant information such as the teat itself and skin blemishes, making it difficult for the classification model to learn discriminative pathological features.

[0005] Therefore, there is an urgent need for a dedicated assessment method that can solve the problem of imbalance in small samples at the algorithmic level and accurately locate and identify subtle pathological features in complex environments. Summary of the Invention

[0006] This invention provides a method and system for assessing mastitis in dairy cows based on few-shot learning. Through innovative data generation strategies and a carefully designed model structure, it effectively solves the problems of small sample size, class imbalance, and complex environmental perception.

[0007] To achieve the objectives of this invention, the technical solution adopted is: a method for assessing mastitis in dairy cows based on few-sample learning, comprising the following steps:

[0008] S1. Dataset Construction and Data Augmentation Based on Self-Attention GAN: Collect and label images of cow teats to construct object detection datasets and classification health score datasets; to address the class imbalance problem in the classification health score dataset, a generative adversarial network with self-attention modules embedded in both the generator and discriminator is used to generate high-quality images and augment data for minority class samples.

[0009] S2. Construction and training of the object detection model: The improved YOLOv8 detection model uses YOLOv8 as the basic framework, replaces its backbone network with MobileNetV3, and introduces a spatial pyramid pooling module after the feature pyramid of its neck network to form a lightweight and robust detection model.

[0010] S3. Construction and training of the target classification model: GoogLeNet is used as the basic framework. Convolutional block attention modules are introduced into its network structure, and a multi-head self-attention layer is introduced before the final fully connected layer to form a classification model with dual attention enhancement.

[0011] S4. Model Inference and Integration: Integrate the trained object detection model and object classification model into a unified inference process, and sequentially perform nipple localization and cropping and health scoring on the input image.

[0012] As an optimization of the present invention, in step S1, the selection criteria for the optimal GAN ​​network include:

[0013] FID score: measures the difference in the distribution of generated and real images in the feature space;

[0014] IS score: assesses the diversity and sharpness of the generated images.

[0015] As an optimization of the present invention, in step S1, a self-attention layer is added between the generator and discriminator of the selected optimal GAN ​​network. By calculating the correlation between all positions in the feature map, the global dependency is obtained, enabling the generative adversarial network to model the long-range structural correlation and local texture details of the keratin ring at the nipple end. Filtering is performed using the confidence of the pre-trained discriminator. The specific implementation steps for the confidence filtering of the pre-trained discriminator are as follows:

[0016] 1) Feature extraction: For the generated image I fake The discriminator D is used to extract multi-level features to obtain the final feature map f4;

[0017] 2) Confidence calculation: The confidence score is obtained by applying global average pooling and a fully connected layer to the final feature map f4;

[0018] 3) Threshold filtering: Set a dynamic threshold τ conf , τ conf =μ real -k·σ real ;

[0019] Where: μ real σ is the mean confidence level of the real image. real Here, k is the standard deviation, and k is the adjustment coefficient.

[0020] 4) Post-processing optimization: The selected images are post-processed as needed, including histogram matching, Gaussian filtering, and sharpening enhancement.

[0021] As an optimization of the present invention, in step S2, the improved YOLOv8 detection model uses MobileNetV3 as the backbone network, and its core component is the inverse residual bottleneck block, which is mathematically expressed as:

[0022] y = SE(Conv) 1×1 (ReLU6(BN(DWConv 3×3 (ReLU6(BN(Conv 1×1 (x))))))))

[0023] Where: SE is the compressed activation attention module, Conv is the convolution module, BN is the batch normalization operation, ReLU6 is the modified linear unit activation function that restricts the output to between 0 and 6, and DWConv is the depthwise separable convolution.

[0024] As an optimized solution of the present invention, in step S2, an adaptive spatial pyramid pooling module is introduced into the neck network, which includes four parallel branches: 1×1 convolution y1, 3×3 dilated convolution y2, 3×3 dilated convolution y3, global average pooling, and upsampling y4, and finally, through y = Conv 1×1 (Concat(y1,y2,y3,y4)) implements multi-scale feature fusion, where Conv is a convolution module and Concat is a concatenation operation.

[0025] As an optimized solution of the present invention, in step S2, a decoupled detection head structure is adopted to handle classification and regression tasks separately, wherein:

[0026] Classification branch output P cls =Softmax(Conv 3×3 (Conv 3×3 (F neck ))),

[0027] Regression branch output B box =Sigmoid(Conv 3×3 (Conv 3×3 (F neck ))),

[0028] Where: Softmax and Sigmoid are activation functions; F neck The feature map output by the neck network;

[0029] The loss function employs a multi-task learning framework: in: For the total loss function, For the complete intersection and union ratio loss, λ box for The weighting coefficients, For focal loss, λ cls for Weighting coefficients; For the distribution focus loss, λ dfl for The weighting coefficients.

[0030] As an optimized solution of the present invention, in step S3, a convolutional block attention module is embedded after each Inception module of GoogLeNet, which sequentially includes a channel attention submodule and a spatial attention submodule, wherein:

[0031] Channel attention is expressed as: M c =Sigmoid(MLP(GAP(x))+MLP(GMP(x))), spatial attention is expressed as: M s =Sigmoid(Conv 7×7 (Concat(GAP(x),GMP(x)))),

[0032] Where: MLP is a multilayer perceptron, GAP is global average pooling, GMP is global max pooling, and x is the original feature map input to the CBAM attention module;

[0033] Finally passed To achieve feature recalibration, where x′ is the feature map after dual weighting calibration using CBAM channel attention and spatial attention;

[0034] A multi-head self-attention layer, MultiHead, is introduced at the end of the classification network. Its computation process is as follows:

[0035] MultiHead=Concat(head1,...,head h W O ,

[0036] Where: head i For the i-th attention head, W O This is a linear transformation matrix used to convert the concatenated matrix back to its original dimensions.

[0037] As an optimization of the present invention, in step S3, a weighted cross-entropy loss function is adopted. Solving the class imbalance problem can be expressed as:

[0038] Where: w i Here, y represents the class weights, C represents the number of classes, and y represents the class weights. i For real labels, use label smoothing to obtain...

[0039] The optimization process uses the AdamW optimizer, combined with cosine annealing learning rate scheduling.

[0040] As an optimized solution of the present invention, in step S4, the bounding box coordinates of the nipple region are first output by improving the YOLOv8 model: B box =x min ,y min ,x max ,y max Then, the region of interest I is cropped based on the coordinates. roi =Crop(I original B box Finally, I roi The improved GoogLeNet model is used to obtain the probability distribution of health scores.

[0041] Among them: B box Let x be the location of the rectangular region of the target in the image. min ,y min ,x max ,y max These are the x-coordinates of the top-left corner, the y-coordinates of the top-left corner, the x-coordinates of the bottom-right corner, and the y-coordinates of the bottom-right corner of the bounding box, respectively. original For the original image, I roi The image shows the cropped region of interest; Crop is the cropping operation.

[0042] To achieve the purpose of this invention, the technical solution adopted is: an evaluation system based on a few-shot learning method for assessing mastitis in dairy cows, characterized in that it includes a dataset construction and data augmentation module based on self-attention GAN, a target detection model construction and training module, a target classification model construction and training module, and a model inference and integration module.

[0043] Dataset Construction and Self-Attention GAN-Based Data Augmentation Module: Collect and label images of cow teats to construct object detection datasets and classification health score datasets; to address the class imbalance problem in the classification health score dataset, a generative adversarial network with self-attention modules embedded in both the generator and discriminator is used to generate high-quality images and augment data for minority class samples.

[0044] The object detection model construction and training module: The improved YOLOv8 detection model uses YOLOv8 as the basic framework, replaces its backbone network with MobileNetV3, and introduces a spatial pyramid pooling module after the feature pyramid of its neck network to form a lightweight and robust detection model.

[0045] The object classification model construction and training module uses GoogLeNet as the basic framework, introduces a convolutional block attention module into its network structure, and introduces a multi-head self-attention layer before the final fully connected layer to form a classification model with dual attention enhancement.

[0046] Model Inference and Integration Module: Integrates the trained object detection model and object classification model into a unified inference process, sequentially performing nipple localization and cropping and health scoring on the input image.

[0047] This invention has positive effects:

[0048] 1) This invention alleviates the problems caused by data imbalance: Through self-attention GAN, high-quality and highly diverse minority class samples can be generated, which effectively alleviates the overfitting and recognition bias problems caused by data imbalance and significantly improves the recall rate of the model for high-risk health levels.

[0049] 2) The invention has strong detection robustness: The combination of the lightweight backbone and the ASPP module enables the detection model to maintain speed while having strong adaptability to complex environments, multi-scale targets and partial occlusion, and high detection accuracy.

[0050] 3) This invention improves classification accuracy: The introduction of the dual attention mechanism enables the classification model to have refined feature perception capabilities from local to global, and can accurately capture subtle pathological changes at the tip of the teat, thereby achieving high-precision intelligent identification of mastitis in dairy cows. Attached Figure Description

[0051] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0052] Figure 1 This is a flowchart illustrating the method of the present invention;

[0053] Figure 2This is an example diagram of the dataset used in the embodiments of the present invention;

[0054] Figure 3 This is an example image of a cow's teat generated according to an embodiment of the present invention;

[0055] Figure 4 This is an example diagram illustrating the use of a YOLOv8 network to individually detect cow teats in an embodiment.

[0056] Figure 5 This is an example diagram showing the final automatic assessment results of the health status of the cow's teats in the implementation embodiment;

[0057] Figure 6 This is an example diagram showing the data exported from the automatic assessment of the health status of cow teats in an embodiment. Detailed Implementation

[0058] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 An embodiment of the present invention provides a method for assessing mastitis in dairy cows based on few-sample learning, comprising the following steps:

[0060] S1. Dataset Construction and Data Augmentation Based on Self-Attention GAN: Collect and label images of cow teats to construct object detection datasets and classification health score datasets; to address the class imbalance problem in the classification health score dataset, a generative adversarial network with self-attention modules embedded in both the generator and discriminator is used to generate high-quality images and augment data for minority class samples.

[0061] In step S1, such as Figure 2As shown, 200 images of cow udders were collected in a pasture environment. Figure (a) shows the original cow abdomen image used for the object detection task. All nipple targets were labeled using the Labelme bounding box annotation tool to construct the object detection dataset. Subsequently, individual nipple images were cropped and labeled by professionals according to a scoring standard (1 point: healthy; 2 points: sub-healthy; 3 points: diseased; 4 points: severe disease), forming a categorized health score dataset. Figure 2 In the image, (b) is an example image of a 1-point cow teat, (c) is an example image of a 2-point cow teat, (d) is an example image of a 3-point cow teat, and (e) is an example image of a 4-point cow teat. There are 359 1-point samples, 207 2-point samples, 182 3-point samples, and 31 4-point samples.

[0062] After data acquisition, a GAN network is used as the base network to augment the data. In step S1, a noise-based GAN network (ProGAN) and a style transfer-based GAN network (CycleGAN) are trained and optimized to generate several batches of images. The optimal GAN ​​network is selected through a multi-index comprehensive evaluation method. The selection criteria include:

[0063] Fréchet Inception Distance (FID) score: measures the difference in distribution between the generated image and the real image in the feature space, and its expression is as follows:

[0064] FID = |μ r -μ g | 2 +Tr(Σ r +Σ g -2(Σ r Σ g ) 1 / 2 )

[0065] Where: μ r and μ g Let Σ represent the mean of the real image and the generated image in the feature space of the Inception-v3 model, respectively. r and Σ g Let be the corresponding covariance matrix, and Tr represent the trace of the matrix. The smaller the FID value, the higher the quality of the generated image.

[0066] Inception Score (IS): Evaluates the diversity and sharpness of the generated image, and its expression is as follows:

[0067]

[0068] Where: p(y|x) is the probability distribution predicted by the Inception-v3 model for image x, p(y) is the edge distribution, and D KL This represents the KL divergence. p g This represents the probability distribution of the generated image by the generative model. Ex represents the expected value of the generated image sample x. The larger the IS value, the better the quality of the generated image.

[0069] Different image generation networks were evaluated using FID and IS scores, and the score comparison results are shown in Table 1.

[0070] Table 1. Comparison of FID and IS scores between the improved GAN network and other generative networks in the embodiment.

[0071] PGGAN network CycleGAN network U-GAT-IT Network DDPM network Improve the network FID 203.81 150.12 170.68 136.43 126.33 IS 2.256 2.817 2.568 3.279 3.568

[0072] As shown in Table 1, the improved GAN network outperforms several other common image generation networks in terms of image generation performance. Furthermore, classification tests can be performed on the initial GoogLeNet classification network using both the original data and various enhanced data, with the weighted average of precision and recall, F1-score, used as the evaluation metric. The comparison results are shown in Table 2.

[0073] Table 2 compares the F1-scores of the improved GAN network and other generative networks on the initial GoogLeNet.

[0074] Original network PGGAN CycleGAN U-GAT-IT DDPM Improve the network F1 score 0.846 0.849 0.851 0.839 0.857 0.862

[0075] Table 2

[0076] As can be seen, the improved GAN network achieved a 1.89% improvement in F1-score when the images were classified on the GoogLeNet classification network, representing the largest improvement compared to other networks. In conclusion, the improved GAN network can achieve better results in image augmentation.

[0077] In step S1, the generator G and discriminator D of the GAN network are jointly optimized through adversarial training, wherein the structure of the generator G is represented as follows:

[0078] G(z) = G5(G4(G3(G2(G1(z)))))

[0079] Among them G i This represents the i-th upsampling block, which contains transposed convolution, batch normalization, and ReLU activation function:

[0080] G i (x)=ReLU(BN(Conv2DTranspose(x)))

[0081] Where: BN represents batch normalization layer, and Conv2DTranspose represents transposed convolutional layer.

[0082] The structure of discriminator D is represented as follows:

[0083] D(x)=Sigmoid(D5(D4(D3(D2(D1(x))))))

[0084] Where: D i This represents the i-th downsampling block, which includes convolution, batch normalization, and the LeakyReLU activation function:

[0085] D i (x)=LeakyReLU(BN(Conv2D(x)))

[0086] The adversarial loss function for generator G and discriminator D is:

[0087]

[0088] Where: x is the real sample, p data Let z be the probability distribution of the real data, and p be the random noise vector input to the generator. z Let Ex be the prior distribution of the noise. data The expectation of the true data distribution, Ez ~ p z Expectations regarding the noise distribution.

[0089] In step S1, a self-attention layer is added between the generator and discriminator layers of the selected optimal GAN ​​network to continuously optimize the results. The forward propagation process of the self-attention layer is as follows:

[0090] f(x) = W f x, g(x) = W g x,h(x)=W h x

[0091] s ij =f(x) i ) T g(x j )

[0092]

[0093] y i =γo i +x i

[0094] Among them, W f W g W hLet f(x), g(x), and h(x) be the learnable weight matrix, and f(x), g(x), and h(x) be the query, key, and value vectors obtained by different linear transformations of the input feature x. ij β is the original correlation score between i and j. j,i This represents the attention weight for the i-th region when generating the j-th region, o j Let j be the attention-weighted output feature, v be the weight matrix for linearly transforming the weighted value vector, N be the total number of input positions, γ be the learnable scaling parameter, and x be the input feature. i and x j The spatial location features are used as input to the feature map.

[0095] In step S1, after training, the generator is fixed, and a random noise vector is input to generate batches of 256×256 pixel images of unhealthy nipples. High-quality generated images are retained through manual screening or filtering using the confidence level of the pre-trained discriminator, and the number of samples in categories 3 and 4 is expanded to a level similar to that of categories 1 and 2, respectively. The specific implementation steps of the confidence level filtering of the pre-trained discriminator are as follows:

[0096] (1) Feature extraction: For the generated image I fake Multi-level features are extracted using discriminator D:

[0097] f1=D1(I fake )

[0098] f2 = D2(f1)

[0099] f3 = D3(f2)

[0100] f4 = D4(f3)

[0101] Wherein: D1~D4 are the four stages (sub-network modules) of discriminator D, and f1~f4 are multi-level feature maps extracted from shallow to deep.

[0102] (2) Confidence calculation: The confidence score is obtained by applying global average pooling and a fully connected layer to the final feature map f4.

[0103]

[0104] p real =Sigmoid(W T f gap +b)

[0105] Where: W is the weight vector, b is the bias term, H and W are the spatial dimensions of the feature map, and f gap The feature vector after global average pooling. This represents the confidence level by which the discriminator determines that the input image is a real image.

[0106] (3) Threshold filtering: Set a dynamic threshold τ conf τ conf =μ real -k·σ real

[0107] Where μ real σ is the mean confidence level of the real image. real denoted as standard deviation, and k is the adjustment coefficient.

[0108] (4) Post-processing optimization: The selected images are post-processed as needed, such as histogram matching, Gaussian filtering, and sharpening enhancement.

[0109] Finally, the high-quality generated images that are retained are added to the original dataset as data augmentation, such as... Figure 3 As shown, (a), (b), and (c) are example images of healthy cow teats generated, while (d), (e), and (f) are example images of unhealthy cow teats generated.

[0110] S2. Construction and training of the object detection model: YOLOv8 is used as the basic framework, its backbone network is replaced with MobileNetV3, and a spatial pyramid pooling module is introduced after the feature pyramid of its neck network to form a lightweight and robust detection model.

[0111] In step S2, the improved YOLOv8 detection model is trained. The improved YOLOv8 detection model includes the following:

[0112] (1) Input preprocessing: Normalize the input image:

[0113]

[0114] Where: μ and σ are the mean and standard deviation of the dataset, I norm This is the normalized value.

[0115] (2) Replace the default YOLOv8 CSPDarknet with MobileNetV3-Large. The specific structure includes the inverse residual bottleneck block:

[0116] y=SE(Conv1×1(ReLU6(BN(DWConv3×3(ReLU6(BN(Conv 1×1 (x))))))))

[0117] And the SE attention module z:

[0118] z=Sigmoid(FC2(ReLU(FC1(GAP(x)))))

[0119] y = z·x

[0120] Where DWConv is a depthwise separable convolution, SE is a compressed activation attention module, GAP is a global average pooling, FC is a fully connected layer, FC1 is a fully connected layer 1, FC2 is a fully connected layer 2, x is the input feature map, and y is the output feature map after being reweighted by the SE module.

[0121] (3) Adaptive spatial pyramid pooling module, which contains four parallel branches:

[0122] Branch 1: 1×1 convolution:

[0123] y1 = Conv1 × 1(x)

[0124] Branch 2: 3×3 dilated convolution:

[0125] y2=Conv3×3 dilation＝6 (x)

[0126] Branch 3: 3×3 dilated convolution:

[0127] y3 = Conv3 × 3 dilation＝12 (x)

[0128] Branch 4: Global average pooling + upsampling:

[0129] y4 = Upsample(GAP(x))

[0130] Use feature fusion:

[0131] y = Conv1×1(Concat(y1,y2,y3,y4))(4) A decoupled header structure is used to handle classification and regression tasks separately:

[0132] Classification branching is represented by P cls =Softmax(Conv3×3(Conv3×3(F neck ))),

[0133] The regression branch is expressed as B box =Sigmoid(Conv3×3(Conv3×3(F neck ))),

[0134] Wherein: F neck The feature map output by the neck network;

[0135] Bounding box encoding uses CIoU loss:

[0136]

[0137] Where: IoU is the intersection-union ratio, which is the area of ​​the intersection of the predicted bounding box and the ground truth bounding box divided by the area of ​​their union; ρ is the Euclidean distance; and b and b gt α and β are the center points of the predicted bounding box and the ground truth bounding box, respectively; c is the diagonal length of the minimum bounding rectangle; α is the weighting coefficient; and v is the aspect ratio consistency measure.

[0138] (5) The loss function is expressed as:

[0139]

[0140] in: Let λ be the total loss function. box , λ cls , λ dfl These are the weighting coefficients. To achieve a complete intersection and comparison loss, For the focus of loss, This represents the distribution focus loss.

[0141] Focal Loss Expressed as:

[0142]

[0143] Where: p t α is the probability predicted by the model. t γ and γ are the hyperparameters for adjustment.

[0144] Distribution Focal Loss is expressed as:

[0145]

[0146] Where: y is the real label, y i and y i+1 For adjacent boundaries, S i and S i+1 The bounding box position is in y i and y i+1 The probability value at that location.

[0147] In step S2, the inference process of the improved YOLOv8 detection model is as follows:

[0148] (1) Feature extraction: Input image I∈R 640×640×3 Multi-scale features are extracted using the MobileNetV3 backbone network: F1∈R 80×80×256 F2∈R 40×40×512 F3∈R 20×20×1024

[0149] (2) Feature fusion: Feature enhancement is performed through feature pyramids and the ASPP module (hollow spatial pyramid pooling module).

[0150] F fusion =ASPP(FPN(F1,F2,F3)) The parallel branches of the ASPP module are as follows:

[0151]

[0152] Where: F1, F2, and F3 are feature maps of different levels of the backbone network; the FPN feature pyramid network fuses multi-scale features; x is the feature map output by the FPN, which serves as the input to ASPP; dilation is the dilation rate of the dilated convolution, controlling the size of the receptive field; Upsample bilinear Bilinear interpolation upsampling restores spatial resolution. Concat stitches multiple feature maps along the channel dimension. y1 to y4 are the outputs of the four parallel branches of ASPP, where y is F. fusion This is the final enhanced feature map after fusion.

[0153] (3) Bounding box prediction: The detector head outputs prediction results at three scales: P1∈R 80×80×4 P2∈R 40×40×4 P3∈R 20×20×4 ;

[0154] (4) Post-processing: Non-maximum suppression is used to select the final detection result S. final :

[0155] Among them B i For bounding box, s i The confidence score is represented by NMS, a non-maximum suppression algorithm. N represents the total number of candidate detection boxes input to NMS.

[0156] To verify the effectiveness of the method in the embodiments, three evaluation metrics were used: detection precision, recall, and model intersection-union ratio (MAP50-95) at thresholds ranging from 0.5 to 0.95, with each threshold being 0.05. Experiments were conducted on the initial YOLOv8 network and the improved YOLOv8 network in the embodiments, and the results are shown in Table 3.

[0157] Table 3. Comparison of metrics between the initial and improved YOLOv8 networks.

[0158] Precision Recall mAP50-95 Initial YOLOv8 network 0.996 0.991 0.843 Improve YOLOv8 network 0.997 0.991 0.853

[0159] As shown in Table 3, the improved YOLOv8 network achieves a certain improvement in the mAP50-95 metric, reaching 1.1%, and the precision and recall values ​​remain at a high level, while the number of parameters is reduced, which proves the effectiveness of the improved method in the embodiments.

[0160] S3. Construction and training of the target classification model: GoogLeNet is used as the basic framework. Convolutional block attention modules are introduced into its network structure, and a multi-head self-attention layer is introduced before the final fully connected layer to form a classification model with dual attention enhancement.

[0161] In step S3, the improved GoogLeNet classification model is trained. The improved GoogLeNet classification model includes the following:

[0162] (1) Input preprocessing of the cropped nipple area:

[0163] I input =RandomCrop(RandomHorizontalFlip(I roi ))

[0164] Among them: I input This is the preprocessed image of the nipple region. RandomCrop is a random cropping operation, and RandomHorizontalFlip is a random horizontal flipping operation. roi This is an image of the cropped nipple region.

[0165] (2) A dual attention mechanism is introduced based on the original Inception module. The original basic Inception module is expressed as:

[0166] Inception(x)=Concat(Conv1×1(x),Conv3×3(Conv1×1(x)),Conv5×5(Conv1

[0167] ×1(x)),MaxPool3×3(x))

[0168] The specific formula is expressed as follows:

[0169]

[0170] output=Concat(branch1,branch2,branch3,branch4)

[0171] Add a convolutional block attention module after each Inception module:

[0172] Channel attention is expressed as: M c =Sigmoid(MLP(GAP(x))+MLP(GMP(x)))

[0173] Where: x is the input feature map, GAP is global average pooling, GMP is global max pooling, MLP is multilayer perceptron, Inception(x) is the enhanced feature map after multi-scale fusion, and MaxPool... 3×3 It is a 3×3 max-pooling layer.

[0174] Spatial attention is expressed as: M s =Sigmoid(Conv 7×7 (Concat(GAP(x),GMP(x))))

[0175] Use feature recalibration:

[0176] (3) Introduce a multi-head self-attention mechanism before the final classification layer:

[0177] Establish a query, key, and value mapping: Q = W Q x, K = W K x, V = W V x;

[0178] Establish scaling dot product attention:

[0179] Where Q is the query matrix, K is the key matrix, V is the value matrix, and W is the value matrix. Q To query the weight matrix, W K W is the key weight matrix. V Let d be the value weight matrix. k Let be the dimension of the key vector. This is a scaling factor to prevent the gradient from vanishing due to an excessively large dot product.

[0180] Ultimately, a multi-head attention mechanism is established, expressed as:

[0181] MultiHead=Concat(head1,...,head h W O ;

[0182] in: W O For multi-head output projection matrix, Let Q be the Q matrix of the i-th head. Let K be the matrix of the i-th head. Let V be the V matrix of the i-th head.

[0183] (4) In the classification head design, feature aggregation is expressed as:

[0184] f global =GAP(MultiHead(Inception) output ))

[0185] A fully connected layer is expressed as:

[0186] P = Softmax(Wf) global +b)

[0187] MultiHead is a multi-head self-attention mechanism, W is the weight matrix, b is the bias term, P is the probability distribution of the output, and Inception... Output The feature map output by the Inception backbone network. This is the final image-level global feature vector.

[0188] (5) The loss function uses weighted cross-entropy loss, expressed as:

[0189]

[0190] in, The class weights are C=4, where C is the number of classes and N is the number of classes. total The total number of samples, Let be the number of samples in the i-th class.

[0191] Use label smoothing:

[0192] Where ε is the smoothing parameter, y i This is a real label.

[0193] (6) Using the AdamW optimizer as the optimizer for the model, expressed as:

[0194]

[0195] in, and The first and second moments are estimated after bias correction, and the specific calculations are as follows:

[0196]

[0197] Where, m t and v t These are the first-order moment estimate and the second-order moment estimate, respectively, where β1 and β2 are the decay rates, η is the learning rate, and θ is the second-order moment estimate. t This is an updated version of Adam.

[0198] Using cosine annealing as the learning rate η t The scheduling method is expressed as:

[0199]

[0200] Where, η min To minimize the learning rate, η max For the maximum learning rate, Tcur T represents the current training steps. max This represents the total number of training steps.

[0201] In step S3, the inference process of the improved GoogLeNet classification model is as follows:

[0202] (1) Feature extraction: Input image I∈R 640×640×3 Features were extracted using several improved Inception modules: F1 = Inception3a(I), F2 = Inception3b(F1), F3 = Inception4a(F2), F4 = Inception4b(F3);

[0203] (2) Attention Enhancement: Apply CBAM attention after each Inception module output:

[0204] F i ′=CBAM(F i for i = 1, 2, 3, 4

[0205] (3) Capturing long-range dependencies through a multi-head self-attention layer:

[0206] F attention =MultiHead(Concat(F1′,F2′,F3′,F4′))

[0207] (4) The final output is the probability distribution of the four health levels:

[0208] [p1,p2,p3,p4]=Softmax(FC(LayerNorm(F attention )))

[0209] Here, p1 to p4 correspond to the probabilities of four levels: healthy, sub-healthy, diseased, and severely diseased, respectively, and LayerNorm is layer normalization.

[0210] To verify the effectiveness of the method in the embodiments, three evaluation metrics were used: precision, recall, and the weighted average of precision and recall, F1-score. Experiments were conducted on the initial GoogLeNet network and the improved GoogLeNet network in the embodiments, and the results are shown in Table 4.

[0211] Table 4. Comparison of metrics between the initial GoogLeNet network and the improved GoogLeNet network.

[0212] Precision Recall F1 score Initial GoogLeNet network 0.832 0.881 0.862 Improve the GoogLeNet network 0.876 0.903 0.889

[0213] As shown in Table 4, the improved GoogLeNet network exhibits significant improvements in precision, recall, and F1-score. Specifically, precision is improved by 5.29%, recall by 2.50%, and F1-score by 3.13%, demonstrating the effectiveness of the improved method in the embodiments.

[0214] S4. Model Inference and Integration: Integrate the trained two-stage model into a unified inference process, and sequentially perform nipple localization and cropping and health scoring on the input image.

[0215] In step S4, a comprehensive reasoning script is written, and the process is as follows:

[0216] like Figure 2 As shown, input an image of a cow's udder, such as... Figure 4 As shown, the improved YOLOv8 model is used for inference, outputting the bounding box coordinates of all nipples. Based on each bounding box, the corresponding nipple region image is cropped from the original image. Each cropped nipple image is resized to 224×224 and subjected to the same normalization process as during training. The processed images are then sequentially input into the improved GoogLeNet model to obtain the probability distribution of each nipple belonging to a score of 1-4. The argmax is taken as the final health score, and the final output image includes the detection boxes and corresponding health scores. The test results of the entire inference script on a computer are shown below. Figure 5 As shown, the detection results are then exported to local storage for management. The exported results are as follows: Figure 6 As shown.

[0217] This invention first constructs a dataset of cow teat images, including object detection and health classification. Addressing the severe imbalance between healthy and unhealthy samples in the dataset, a generative adversarial network-based image enhancement algorithm is employed to generate and augment the scarce unhealthy teat images with high quality. Subsequently, a two-stage evaluation model is designed: the first stage uses an improved YOLOv8 model that integrates spatial pyramid pooling and a lightweight backbone network, MobileNetV3, to achieve accurate real-time detection and localization of cow teats in complex farming environments; the second stage uses an improved GoogLeNet model incorporating an attention mechanism to perform end-to-end feature extraction on the cropped teat region images, and finally outputs a health status score for the cow teat based on the roughness and thickness of the keratin ring at the teat tip. This invention, through innovative data augmentation and model structure optimization, effectively overcomes the shortcomings of traditional manual detection, such as high subjectivity and low efficiency.

[0218] This invention also provides a computer device, which may be a server. The computer device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor in this computer design provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store the data corresponding to this embodiment. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0219] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0220] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0221] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assessing mastitis in dairy cows based on few-sample learning, characterized in that, Includes the following steps: S1. Dataset Construction and Data Augmentation Based on Self-Attention GAN: Collect and label images of cow teats to construct object detection datasets and classification health score datasets; to address the class imbalance problem in the classification health score dataset, a generative adversarial network with self-attention modules embedded in both the generator and discriminator is used to generate high-quality images and augment data for minority class samples. S2. Construction and training of the object detection model: The improved YOLOv8 detection model uses YOLOv8 as the basic framework, replaces its backbone network with MobileNetV3, and introduces a spatial pyramid pooling module after the feature pyramid of its neck network to form a lightweight and robust detection model. S3. Construction and training of the target classification model: GoogLeNet is used as the basic framework. Convolutional block attention modules are introduced into its network structure, and a multi-head self-attention layer is introduced before the final fully connected layer to form a classification model with dual attention enhancement. S4. Model Inference and Integration: Integrate the trained object detection model and object classification model into a unified inference process, and sequentially perform nipple localization and cropping and health scoring on the input image.

2. The method for assessing mastitis in dairy cows based on few-sample learning according to claim 1, characterized in that: In step S1, the selection criteria for the optimal GAN ​​network include: FID score: measures the difference in the distribution of generated and real images in the feature space; IS score: assesses the diversity and sharpness of the generated images.

3. The method for assessing mastitis in dairy cows based on few-sample learning according to claim 2, characterized in that: In step S1, a self-attention layer is added between the generator and discriminator layers of the selected optimal GAN ​​network. By calculating the correlation between all locations in the feature map, global dependencies are obtained, enabling the generative adversarial network to model the long-range structural correlations and local texture details of the keratin ring at the nipple tip. The confidence score of the pre-trained discriminator is used for filtering. The specific steps for filtering using the confidence score of the pre-trained discriminator are as follows: 1) Feature extraction: For the generated image I fake The discriminator D is used to extract multi-level features to obtain the final feature map f4; 2) Confidence calculation: The confidence score is obtained by applying global average pooling and a fully connected layer to the final feature map f4; 3) Threshold filtering: Set a dynamic threshold τ conf , τ conf =μ real -k·σ real ; Where: μ real σ is the mean confidence level of the real image. real Here, k is the standard deviation, and k is the adjustment coefficient. 4) Post-processing optimization: The selected images are post-processed as needed, including histogram matching, Gaussian filtering, and sharpening enhancement.

4. The method for assessing mastitis in dairy cows based on few-sample learning according to claim 3, characterized in that: In step S2, the improved YOLOv8 detection model uses MobileNetV3 as its backbone network, and its core component is the inverse residual bottleneck block, which is mathematically expressed as: and=SE(Conv 1×1 (ReLU6(BN(DWConv 3×3 (ReLU6(BN(Conv 1×1 (x)))))))) Where: SE is the compressed activation attention module, Conv is the convolution module, BN is the batch normalization operation, ReLU6 is the modified linear unit activation function that restricts the output to between 0 and 6, and DWConv is the depthwise separable convolution.

5. The method for assessing mastitis in dairy cows based on few-sample learning according to claim 4, characterized in that: In step S2, an adaptive spatial pyramid pooling module is introduced into the neck network, which includes four parallel branches: 1×1 convolution y1, 3×3 dilated convolution y2, 3×3 dilated convolution y3, global average pooling, and upsampling y4. Finally, through y = Conv 1×1 (Concat(y1,y2,y3,y4)) implements multi-scale feature fusion, where Conv is a convolution module and Concat is a concatenation operation.

6. The method for assessing mastitis in dairy cows based on few-sample learning according to claim 5, characterized in that: In step S2, a decoupled detection head structure is used to process the classification and regression tasks separately, wherein: the classification branch outputs P cls =Softmax(Conv 3×3 (Conv 3×3 (F neck ))), Regression branch output B box =Sigmoid(Conv 3×3 (Conv 3×3 (F neck ))), Where: Softmax and Sigmoid are activation functions; F neck The feature map output by the neck network; The loss function employs a multi-task learning framework: in: For the total loss function, For the complete intersection and union ratio loss, λ box for The weighting coefficients, For focal loss, λ cls for Weighting coefficients; For the distribution focus loss, λ dfl for The weighting coefficients.

7. The method for assessing mastitis in dairy cows based on few-sample learning according to claim 6, characterized in that: In step S3, a convolutional block attention module is embedded after each Inception module of GoogLeNet, which sequentially includes a channel attention submodule and a spatial attention submodule, wherein: Channel attention is expressed as: M c =Sigmoid(MLP(GAP(x))+MLP(GMP(x))), Spatial attention is expressed as: M s =Sigmoid(Conv 7×7 (Concat(GAP(x),GMP(x)))), Where: MLP is a multilayer perceptron, GAP is global average pooling, GMP is global max pooling, and x is the original feature map input to the CBAM attention module; Finally passed To achieve feature recalibration, where x′ is the feature map after dual weighting calibration using CBAM channel attention and spatial attention; A multi-head self-attention layer, MultiHead, is introduced at the end of the classification network. Its computation process is as follows: MultiHead=Concat(head1,...,head h )W O , Where: head i For the i-th attention head, W O This is a linear transformation matrix used to convert the concatenated matrix back to its original dimensions.

8. The method for assessing mastitis in dairy cows based on few-shot learning according to claim 7, characterized in that: In step S3, a weighted cross-entropy loss function is used. Solving the class imbalance problem can be expressed as: Where: w i Here, y represents the class weights, C represents the number of classes, and y represents the class weights. i For real labels, use label smoothing to obtain... The optimization process uses the AdamW optimizer, combined with cosine annealing learning rate scheduling.

9. The method for assessing mastitis in dairy cows based on few-sample learning according to claim 8, characterized in that: In step S4, the bounding box coordinates of the nipple region are first output by improving the YOLOv8 model: B box =x min ,y min ,x max ,y max Then, the region of interest I is cropped based on the coordinates. roi =Crop(I original B box Finally, I roi The improved GoogLeNet model is used to obtain the probability distribution of health scores. Among them: B box Let x be the location of the rectangular region of the target in the image. min ,y min ,x max ,y max These are the x-coordinates of the top-left corner, the y-coordinates of the top-left corner, the x-coordinates of the bottom-right corner, and the y-coordinates of the bottom-right corner of the bounding box, respectively. original For the original image, I roi The image shows the cropped region of interest; Crop is the cropping operation.

10. An assessment system employing the small-sample learning-based method for assessing mastitis in dairy cows as described in claim 9, characterized in that: It includes modules for dataset construction and data augmentation based on self-attention GAN, construction and training of object detection models, construction and training of object classification models, and model inference and integration. Dataset Construction and Self-Attention GAN-Based Data Augmentation Module: Collect and label images of cow teats to construct object detection datasets and classification health score datasets; to address the class imbalance problem in the classification health score dataset, a generative adversarial network with self-attention modules embedded in both the generator and discriminator is used to generate high-quality images and augment data for minority class samples. The object detection model construction and training module: The improved YOLOv8 detection model uses YOLOv8 as the basic framework, replaces its backbone network with MobileNetV3, and introduces a spatial pyramid pooling module after the feature pyramid of its neck network to form a lightweight and robust detection model. The object classification model construction and training module uses GoogLeNet as the basic framework, introduces a convolutional block attention module into its network structure, and introduces a multi-head self-attention layer before the final fully connected layer to form a classification model with dual attention enhancement. Model Inference and Integration Module: Integrates the trained object detection model and object classification model into a unified inference process, sequentially performing nipple localization and cropping and health scoring on the input image.

Images