Improved Mask2Former-based intelligent identification method and device for livestock affiliation of herdsman

Abstract

The invention provides an improved Mask2Former-based intelligent identification method and device for the livestock affiliation of a herdsman. The method comprises the following steps: acquiring an image when livestock is stocked; inputting the photos and the videos into a trained improved Mask2Former image segmentation model, and outputting color classification corresponding to the backs of the livestock; and according to the color classification, determining the corresponding herdsman belonging to the color classification. In the scheme provided by the invention, on the basis of a Mask2Former network, Haar wavelet transform is introduced to improve the local feature extraction capability, an attention module is utilized to pay attention to key features, multi-scale feature fusion is optimized through a feature pyramid network, an improved Mask2Former image segmentation model is constructed, and the backs of the livestock are classified into different colors; the method is advantaged in that livestock back color segmentation precision and pasture affiliation identification accuracy are improved, the method is especially suitable for scenes with complex backgrounds and dense livestock, an innovative solution is provided for sheep flock management and pasture affiliation identification in Qinghai-Tibet Plateau areas, and a new research path is developed for image segmentation tasks based on color features.

Description

Technical Field

[0001] The present application relates to the technical field of livestock ownership identification, and in particular to a method and device for intelligently identifying livestock ownership of herders based on an improved Mask2Former. Background Art

[0002] The Qinghai-Tibet Plateau, known as the "Roof of the World", "the Third Pole of the Earth" and "Water Tower of Asia", is one of the most unique and fragile ecosystems on Earth.

[0003] However, due to its unique climate and geography, the Qinghai-Tibet Plateau's ecosystem is extremely sensitive to external disturbances. In recent years, with the intensification of human activities, particularly the expansion of grazing, the grassland ecosystem in Qinghai Province has shown a trend of gradual degradation. Traditional production methods, primarily based on sheep herding, have placed increasing pressure on grasslands as the number of herders increases and grazing areas expand, becoming one of the main drivers of grassland degradation.

[0004] Overgrazing in the plateau is primarily caused by livestock such as yaks and sheep, with sheep considered a key factor in grassland degradation. This problem is particularly pronounced in fragile ecosystems such as alpine steppes and meadows. Scientific grazing management helps achieve a balance between grass and livestock, improves grassland productivity, and promotes sustainable economic development in pastoral areas. Therefore, managing livestock and avoiding overgrazing in the plateau region has become a key component of ecological protection and regional sustainable development.

[0005] Currently, research in livestock monitoring and management primarily focuses on the application of object detection and image segmentation technologies. Traditional object detection methods such as YOLO (You Only Look Once) and Faster R-CNN are widely used for animal recognition tasks in agriculture and animal husbandry. These methods achieve livestock identification and location by delineating and classifying target regions. Although YOLO offers rapid detection speed, it is prone to false detections and missed detections in scenes with densely populated livestock, significant overlap, or complex backgrounds. Faster R-CNN improves detection accuracy with the help of a region proposal network (RPN), but similarly struggles with complex scenes characterized by high occlusion and densely populated objects. With the advancement of deep learning technology, image segmentation methods have gradually become a key tool for livestock monitoring. U-Net (proposed in 2015), a classic medical image segmentation model, has demonstrated excellent detail preservation and high segmentation accuracy in animal monitoring, but its robustness against occluded objects or similar backgrounds remains limited. Subsequently, the DeepLabV3+ model was proposed. This model enhances the extraction of multi-scale information by introducing dilated convolutions, improving performance in complex backgrounds. However, it still suffers from blurred boundaries in dense scenes and poor adaptability to illumination variations and environmental noise. Subsequently, the Vision Transformer (ViT) was proposed in 2020. By dividing images into fixed-size patches and applying the Transformer architecture, it achieved significant results in image classification tasks and has gradually been applied to object detection and image segmentation. Compared to traditional convolutional neural networks (CNNs), ViT has stronger global modeling capabilities, but its high dependence on large-scale data and high computational overhead limit its application in real-time surveillance. In the 2020s, new segmentation models such as SegFormer and Mask2Former emerged. SegFormer, based on the Transformer architecture, overcomes the limitations of traditional CNNs, more effectively extracting global information from images and showing good adaptability in complex environments. However, its high computational requirements may lead to poor performance on resource-constrained devices. Mask2Former combines the advantages of Transformer and convolutional neural networks, showing stronger accuracy and robustness in processing natural scenes with dense targets and complex backgrounds. It is particularly suitable for high-precision livestock segmentation and ownership recognition tasks in natural grazing environments.

[0006] Although previous research has attempted to apply image segmentation and object detection technologies to livestock monitoring, practical applications still face numerous challenges. The complex and ever-changing natural environment of the Qinghai-Tibet Plateau, characterized by varying lighting conditions, unstable weather, and high levels of overlap between livestock, severely impacts the stability and accuracy of existing models in dynamic scenarios. Traditional methods often fail to accurately identify livestock boundaries in densely populated areas or complex backgrounds, leading to frequent false or missed detections. Furthermore, accurately distinguishing livestock belonging to different herders within the same image remains a technical challenge, especially in the absence of clear markings. In areas like the Qinghai-Tibet Plateau, livestock from different herders often graze together. Traditional methods, which rely on manual identification of sheep back markings, are inefficient and inaccurate, and are no longer able to meet the needs of modern, large-scale pasture management. These challenges severely hinder the widespread adoption and application of these technologies in large-scale pastoral areas.

[0007] Therefore, how to accurately identify the herders of different livestock under complex environmental conditions, especially in scenarios where multiple herders are mixed, has become a technical problem to be solved. Summary of the Invention

[0008] Multiple aspects of the present application provide a method and device for intelligently identifying the ownership of livestock by herders based on an improved Mask2Former, which utilizes advanced image segmentation technology to achieve accurate recognition of livestock back color and herder ownership analysis, thereby improving the accuracy and efficiency of livestock identification and management.

[0009] To achieve the above technical effects, one aspect of the present application provides an intelligent identification method for livestock ownership based on an improved Mask2Former, comprising:

[0010] Acquire images of livestock while grazing;

[0011] Input the photos and videos into the trained improved Mask2Former image segmentation model to output the color classification corresponding to the back of the livestock;

[0012] The corresponding herder is determined according to the color classification.

[0013] According to a preferred embodiment of the present invention, the step of acquiring images of livestock grazing further comprises:

[0014] Use drones or cameras to film livestock on the ground while they are grazing, collecting original photos and video images under different time, lighting, weather and background conditions;

[0015] OpenCV is used to extract the original video frame by frame, remove blurry and highly repetitive images, and obtain an image set containing the backs of livestock.

[0016] According to a preferred embodiment of the present invention, inputting the photos and videos into a trained improved Mask2Former image segmentation model and outputting the color classification corresponding to the back of the livestock further includes:

[0017] The improved Mask2Former image segmentation model was obtained by training the neural network model using the color classification corresponding to historical livestock as training samples;

[0018] Use image segmentation evaluation metrics to evaluate and adjust the performance of the improved Mask2Former image segmentation model;

[0019] The livestock image to be identified is input into the trained improved Mask2Former image segmentation model, and the color classification corresponding to the livestock's back is output.

[0020] According to a preferred embodiment of the present invention, the improved Mask2Former image segmentation model obtained by training the neural network model using the color classification corresponding to historical livestock as training samples further includes:

[0021] Obtain historical livestock grazing images and corresponding color classifications;

[0022] Use the X-AnyLabeling tool to label the livestock back area in the image with the corresponding color;

[0023] Using the convolutional neural network ResNet-50 as the backbone network, the Haar wavelet transform module and the attention module are added to construct an improved Mask2Former image segmentation model;

[0024] The color marked on the backs of livestock was used as the evaluation indicator, and historical images of livestock grazing were input into the model for training, and finally a trained improved Mask2Former image segmentation model was obtained.

[0025] According to a preferred embodiment of the present invention, the improved Mask2Former image segmentation model constructed by using the convolutional neural network ResNet-50 as the backbone network and adding the Haar wavelet transform module and the attention module further includes:

[0026] The convolutional neural network ResNet-50 is used as the backbone network, and the Haar wavelet transform module is used to replace the maximum pooling layer in the network;

[0027] The attention module is embedded in each bottleneck block in the network.

[0028] According to a preferred embodiment of the present invention, the improved Mask2Former image segmentation model further includes:

[0029] Feature Pyramid Network, a connection layer placed between the backbone network and the pixel decoder.

[0030] According to a preferred embodiment of the present invention, the using of image segmentation evaluation indicators to evaluate and adjust the performance of the improved Mask2Former image segmentation model further includes:

[0031] Set image segmentation evaluation indicators: mean intersection over union (mIoU), mean precision (mPrecision), mean recall (mRecall), and F1 score (mF1-score);

[0032] The image segmentation evaluation index is used to evaluate the index score of the improved Mask2Former image segmentation model, and the model parameters are adjusted according to the index score.

[0033] Another aspect of the present application provides an intelligent livestock identification device for herders based on an improved Mask2Former, comprising:

[0034] Image acquisition module: used to obtain images of livestock grazing;

[0035] Color classification module: used to input the photos and videos into the trained improved Mask2Former image segmentation model and output the color classification corresponding to the livestock's back;

[0036] The attribution determination module is used to determine the corresponding herder according to the color classification.

[0037] Another aspect of the present application provides a herder’s livestock ownership intelligent identification device based on an improved Mask2Former, the device comprising:

[0038] at least one processor; and

[0039] a memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0041] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the above method.

[0042] In the solution provided in the embodiment of the present application, by introducing the Haar wavelet transform on the basis of the Mask2Former network to improve the local feature extraction capability, using the attention module to focus on key features, and optimizing the multi-scale feature fusion through the feature pyramid network, an improved Mask2Former image segmentation model is constructed and the backs of livestock are classified into different colors. It has stronger recognition ability in scenes with similar target colors and blurred boundaries, improves the color segmentation accuracy of livestock backs and the accuracy of herder identification, and is particularly suitable for scenes with complex backgrounds and dense livestock. It provides an innovative solution for sheep herd management and herder identification in the Qinghai-Tibet Plateau, and provides key technical support for the construction of an intelligent and sustainable modern pastoral management system in the Qinghai-Tibet Plateau, and also opens up a new research path for image segmentation tasks based on color features. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0045] Figure 1 A flowchart of an intelligent livestock identification method based on an improved Mask2Former provided in one embodiment of the present application;

[0046] Figure 2 A comparison of the original image and the color-annotated image provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of the Mask2Former network architecture provided in one embodiment of the present application;

[0048] Figure 4 Schematic diagram of the improved ResNet-50 network architecture provided in one embodiment of the present application

[0049] Figure 5 A schematic diagram of the overall structure and feature fusion process of the Haar wavelet transform module in a network provided in one embodiment of the present application;

[0050] Figure 6 A schematic diagram of the network structure of the attention module provided in one embodiment of the present application;

[0051] Figure 7A schematic diagram of a feature pyramid network structure provided in one embodiment of the present application;

[0052] Figure 8 The loss value, F1-score, and mIoU curves during the model training process provided in one embodiment of the present application;

[0053] Figure 9 This is a rendering of an intelligent livestock identification method for herders based on an improved Mask2Former according to an embodiment of the present application;

[0054] Figure 10 A schematic diagram of the structure of an intelligent livestock identification device for herders based on an improved Mask2Former according to one embodiment of the present application;

[0055] Figure 11 The present invention is a schematic structural diagram of a device suitable for implementing the solution in the embodiment of the present application.

[0056] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0057] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0058] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0059] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0060] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. The information can be computer program instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc-read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0061] In actual scenarios, the execution subject of this method can be a user device, or a device formed by integrating a user device and a network device through a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablets, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer collections, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computers.

[0062] This invention aims to combine the practical needs of plateau grassland ecological protection with refined pastoral management, leveraging advanced image segmentation technology to accurately identify sheep back color and analyze herder affiliation. Addressing the inaccuracy of traditional methods in the complex natural environment and high-density grazing environment of the Qinghai-Tibet Plateau, this method utilizes advanced semantic segmentation models to precisely extract livestock back color regions. This method, combined with the herder affiliation relationships represented by different colors, enables automated identification of herders in mixed herder scenarios.

[0063] For ease of understanding, the embodiments of the present invention are illustrated using a flock of sheep as an example. This method has strong adaptability and generalization performance in different scenarios. It is not only suitable for sheep color recognition, but can also be extended to other livestock and even related tasks such as ecological monitoring and natural resource management.

[0064] Figure 1 A flowchart of an intelligent identification method for livestock ownership by herders based on an improved Mask2Former is provided in one embodiment of the present application. Figure 1 As shown, the method includes:

[0065] S101. Acquire images of livestock grazing.

[0066] Specifically, the team used drones and cameras to capture sheep grazing on the Qinghai-Tibet Plateau, collecting original photos and video footage, focusing primarily on the color of the sheep's backs as a distinguishing feature. To ensure data diversity and complexity, the collection process encompassed varying lighting conditions, weather conditions, and diverse terrain, capturing the true distribution of sheep in their natural state.

[0067] OpenCV was used to extract the original video frame by frame, removing blurred and highly repetitive images, and obtaining an image set containing the backs of livestock. The image set contained photos of the backs of tens of thousands of sheep.

[0068] S102: Input the photos and videos into a trained improved Mask2Former image segmentation model, and output the color classification corresponding to the back of the livestock.

[0069] Specifically, the model was trained using historical images of livestock grazing and their corresponding color classifications as training samples. When selecting the dataset, the X-AnyLabeling tool was first used to label the corresponding colors of the livestock back regions in the sample images. These regions are crucial for segmenting the flock and identifying herders.

[0070] The dataset is divided into multiple categories based on the number of herders. This example uses four categories as an example: red (red sheep back color), blue (blue sheep back color), yellow (yellow sheep back color), and colorless (colorless sheep back). Different colors not only represent the color characteristics of the sheep's back but also correspond to the sheep's affiliation with different herders. Red, blue, and yellow respectively mark flocks from different herders, while colorless flocks do not belong to any specific herder. Table 1 shows the correspondence between sheep back color and herder affiliation, clarifying the actual meaning of each label.

[0071] Table 1 Correspondence between sheep back color and sheep ownership

[0072]

[0073] Figure 2 The original image and the color-annotated comparison image provided in one embodiment of the present application are as follows: Figure 2As shown in the figure, for easier visualization, the sheep with different back colors are labeled with their corresponding colors. This color labeling allows for a more intuitive representation of the sheep's back color segmentation results. The dataset ultimately contained 89,179 labels, of which the red label appeared 17,062 times, the blue label 34,524 times, the yellow label 13,303 times, and the colorless label 24,290 times.

[0074] The sheep back color segmentation dataset used in this example contains a total of 5130 images. After color annotation, to ensure the effectiveness of training and evaluation, the dataset was divided into a training set and a validation set in a 4:1 ratio, with the training set containing 4104 images and the validation set containing 1026 images. This dataset covers different sheep flocks, lighting conditions, background environments, and other factors, providing good diversity and representativeness to ensure that the model can cope with various complex situations. The table below shows the detailed partitioning information of the dataset, including the distribution of the number of labels per category.

[0075] Table 2 Label category distribution of dataset division

[0076] Tag Category training set Validation set sum Red 13413 3649 17062 Blue 27553 6971 34524 Yellow 10599 2704 13303 Colourless 19288 5002 24290 Alllabels 70853 18326 89179

[0077] As shown in Table 2, the distribution of label categories in the dataset remains consistent across both the training and validation sets to ensure the model's generalization capabilities. Overall, the training and validation sets generally meet the 4:1 split criteria for the number of samples per category. Even with slight differences in sample size for certain categories (such as red), the overall ratio remains relatively balanced, ensuring a reasonable data partitioning.

[0078] The color marked on the sheep's back is used as the evaluation indicator, and the images of sheep grazing in the training set are input into the model for training. The model uses the convolutional neural network ResNet-50 as the backbone network, and adds the Haar wavelet transform module and the attention module to construct an improved Mask2Former image segmentation model.

[0079] In recent years, the Transformer architecture has driven breakthroughs in computer vision, particularly in image segmentation. Compared to the local receptive field limitations of traditional convolutional neural networks (CNNs) for feature extraction, the Visual Transformer (ViT) and its derivative architectures successfully model spatial relationships across images through a global self-attention mechanism and sequence modeling. This innovation not only significantly improves the performance of fundamental tasks such as image classification and object detection, but also opens up a new paradigm for pixel-level prediction tasks such as semantic segmentation, instance segmentation, and panoptic segmentation.

[0080] Mask2Former, an innovative Transformer-based image segmentation model, redefines the technical framework for pixel-level prediction tasks through a collaborative modeling mechanism combining global perception and local decoupling. Unlike traditional CNN models, which are limited by local receptive fields and static weight sharing, Mask2Former incorporates a global self-attention mechanism to capture local details while comprehensively mining the image's global context. This feature makes it particularly effective when dealing with complex backgrounds and long-range dependencies, overcoming the limitations of traditional CNNs in modeling cross-regional correlations.

[0081] The core advantage of mask2Former lies in the Mask Prediction strategy it introduces, which fully leverages the advantages of Transformer in sequence modeling. Through a multi-layer mask generation mechanism, it can not only accurately generate segmentation masks, but also improve the coherence and detail accuracy of the segmentation results. This enables Mask2Former to demonstrate excellent pixel-level segmentation capabilities when processing difficult image scenes (such as complex backgrounds or occlusions). In addition, an efficient feature encoding and decoding architecture is constructed, combined with the hierarchical feature fusion and self-attention mechanism of Transformer, to further enhance the ability to capture fine-grained local features. By performing fine segmentation on feature maps of different scales, Mask2Former not only performs outstandingly in semantic segmentation tasks, but also provides strong support for tasks such as instance segmentation and panoramic segmentation.

[0082] Figure 3 A schematic diagram of the Mask2Former network architecture provided in an embodiment of the present application is shown in FIG. Figure 3 As shown in the figure, the network architecture integrates a Feature Pyramid Network (FPN) to fuse the multi-scale features extracted by the backbone network. The white blocks in the figure represent convolutional layers (Conv2D), which are used to unify the feature maps at four scales into 256 channels. Although not explicitly shown in the figure, the backbone network incorporates a Haar wavelet transform module and a CBAM, both of which play a key role in enhancing feature representation and focusing on key image areas.

[0083] The improved Mask2Former network in the embodiment of the present invention consists of three parts: Pixel Decoder, Transformer Decoder and Segmentation Module, among which: Pixel Decoder upsamples the image layer by layer after the backbone network extracts features, and passes the multi-scale feature map to Transformer Decoder; Transformer Decoder uses the mask attention mechanism to process the feature map passed in by Pixel Decoder and performs fine-grained modeling on the Query features; Segmentation Module classifies the Query output by Transformer Decoder, generates a mask embedding, and performs a dot multiplication with the output of Pixel Decoder to obtain a binary mask, and finally outputs the segmentation result.

[0084] The decoder of Mask2Former adopts a Transformer Decoder-based architecture and is the core component of the entire model. By introducing learnable query embeddings, the decoder interacts with the encoder's multi-scale feature maps, extracting target-related information using cross-attention and modeling relationships between targets through self-attention. In the multi-layer decoder, each layer generates an intermediate mask result, which is gradually refined through layer-by-layer optimization to improve segmentation accuracy and consistency. In particular, the decoder adopts a design that decouples mask generation from target category prediction, allowing query embeddings to be used for both pixel-level mask generation and target classification, thereby improving the model's expressiveness and inference efficiency.

[0085] Mask2Former provides a variety of optional backbone networks (Backbone) to adapt to different task requirements and computing resource constraints. Common versions include the ResNet series based on traditional convolutional neural networks (CNN) (such as ResNet-50, ResNet-101, etc.) and the Swin Transformer series based on Vision Transformer (such as Swin-B and Swin-L). Among them, the ResNet series (such as ResNet-50, ResNet-101) as a classic CNN architecture has good deep feature extraction capabilities and model stability, and performs well in various visual tasks. Compared with the more complex ViT architecture, ResNet has lower computational overhead while maintaining high performance, and is especially suitable for resource-constrained edge computing scenarios.

[0086] ResNet (Residual Network) significantly alleviates the performance degradation problem that occurs during the training of deep networks by introducing the residual block mechanism. As the number of network layers increases, gradients are prone to vanishing or exploding when training deep networks, resulting in decreased model performance or even degradation. To solve this problem, ResNet introduced a "residual connection" in each module, that is, the output of a certain layer is not only passed to the next layer, but also directly added to the output of the subsequent layer through the residual connection. This design enables the network to explicitly learn the "residual" between input and output, effectively alleviating the difficulties in gradient propagation. The residual block mainly consists of two parts: one or more convolutional layers and a shortcut connection (direct connection, Shortcut Connection).

[0087] In the standard residual block, the input x is convolved to obtain the output F(x). At the same time, the input x is directly passed through the shortcut connection and added to (x) to finally obtain the output y. The formula is as follows:

[0088] y=F(x,W i )+x (1)

[0089] Among them, W i is the weight parameter of the convolution layer in the residual block. i ) represents the result of transformations such as convolution and activation on the input x, while x is the input signal passed directly through the shortcut connection.

[0090] When the input and output dimensions are inconsistent (such as when downsampling), the input dimension needs to be transformed to maintain the consistency of the residual connection. To make the dimensions of the residual connection match, ResNet introduced the downsampling residual block. In the downsampling residual block, a 1x1 convolution layer is added to the direct connection edge to adjust the input dimension to match the output dimension. The formula for the downsampling residual block is:

[0091] y=F(x,W i )+W s (x) (2)

[0092] Among them, W s Represents the weight parameter of the 1x1 convolution layer, which is responsible for adjusting the dimension of the input x.

[0093] The embodiment of the present invention uses ResNet-50 as the backbone network. ResNet-50 consists of 50 layers and has strong feature extraction capabilities. It is widely used in tasks such as image classification, target detection, and semantic segmentation. ResNet-50 is a deep residual network that effectively solves the degradation problem in deep network training by introducing residual connections, allowing the network to be stably trained and extract effective deep features. The network starts with the input image, first passing through a 7x7 convolutional layer (64 filters), and then downsampling through a 3x3 maximum pooling layer. Next, the network gradually extracts features through four stages. Each stage consists of multiple residual blocks, using a different number of convolution filters. Specifically, the first stage (Stage 1) consists of three residual blocks, each consisting of a convolutional layer with 64 filters; the second stage (Stage 2) includes four residual blocks, each containing 128 filters; the third stage (Stage 3) consists of six residual blocks, each containing 256 filters; and the fourth stage (Stage 4) consists of three residual blocks, each with 512 filters. After the convolution operation within each residual block, the input is passed directly to the next layer through a "shortcut connection", thereby helping to learn the "residual" between the input and output.

[0094] The network structure of ResNet-50 also includes two basic Bottleneck Blocks: Conv Block and IDBlock. The first residual block in each stage is the Conv Block, which is mainly used for downsampling and adjusting the number of channels. The subsequent residual block is the ID Block, which keeps the dimensions of the input and output consistent and transmits information through residual connections. At the end of the network, each channel is averaged through global average pooling to generate the final feature representation. Global average pooling effectively reduces the number of parameters of the model and improves the generalization ability of the model by calculating the average value of each feature map. Next, the features pass through the fully connected layer to map the extracted features to the output category for classification or regression tasks.

[0095] To further enhance the model's feature extraction capabilities and adapt to the complexity of diverse environments, this embodiment of the present invention introduces the Haar Wavelet Transform and the Convolutional Block Attention Module (CBAM) module based on ResNet-50. The Haar Wavelet Transform helps capture the multi-scale frequency information of the image, improving the model's robustness under varying lighting and background conditions. The CBAM module, through spatial and channel-wise attention mechanisms, automatically focuses on key color areas on the sheep's backs, further improving image segmentation accuracy. Figure 4 This is a schematic diagram of the improved ResNet-50 network architecture provided in one embodiment of the present application. The figure details the various components including the Haar wavelet transform and CBAM modules, and presents the overall design of the architecture. Figure 4 As shown in the figure, Haar wavelet transform is used to replace the traditional maximum pooling operation to enhance the multi-scale frequency information modeling; the CBAM module is embedded in each Bottleneck Block to enhance the model's attention response to important areas.

[0096] In an embodiment of the present invention, key improvements are made to the traditional ResNet-50 architecture to improve its performance in the tasks of sheep back color segmentation and herder identification. First, the original maximum pooling layer (Max Pooling) is replaced with Haar Wavelet Transform (Haar Wavelet Transform). By capturing multi-scale frequency information, the model's perception of details and texture changes is enhanced. Secondly, a CBAM module (convolutional block attention module) is introduced and embedded between the Conv Block and the ID Block of each Bottleneck Block, after the convolution operation and before the residual connection. By strengthening the attention mechanism in space and channels, CBAM can help the model focus on the color area on the back of the sheep, thereby improving the segmentation accuracy and adaptability of the model. These improvements effectively enhance the performance of ResNet-50 in complex environments, enabling the model to provide more accurate sheep segmentation and herder identification results in changing scenarios.

[0097] In computer vision tasks, convolutional neural networks (CNNs) typically rely on convolution operations combined with maximum pooling to extract spatial features of images and achieve dimensionality reduction of feature maps. However, while maximum pooling compresses the size of feature maps and enhances translation invariance, it also inevitably causes information loss, especially in terms of fine-grained textures and edge structures. This information loss is particularly evident in tasks that require fine segmentation. To alleviate this problem, an embodiment of the present invention introduces the Haar Wavelet Transform as an alternative downsampling mechanism to encode feature maps from the perspective of the frequency domain to improve feature retention capabilities and enhance the model's perception of multi-scale details.

[0098] The Haar wavelet transform is a discrete wavelet transform method with a simple structure and efficient computation. Its core idea is to decompose an image into subbands of different frequencies, thereby analyzing image features at different scales. Specifically, the Haar wavelet decomposes an image into a low-frequency component (LL) and three high-frequency components (LH, HL, and HH) through a linear transformation, corresponding to the overall image contour and detailed information in different directions, respectively. Unlike traditional maximum pooling operations that only retain local extreme values, the Haar wavelet not only has the ability to downsample, but also retains both low-frequency and high-frequency information, helping to enhance the model's perception of local textures, edge structures, and detailed features.

[0099] In this embodiment of the present invention, the Haar wavelet transform is applied to the sheep back color segmentation task, significantly improving the network's ability to respond to local salient areas, especially in images with complex color variations and susceptible to background interference. By introducing the Haar wavelet, the model can more accurately capture subtle texture and edge changes while maintaining spatial structure stability, providing a more stable and discriminative feature foundation for subsequent attention mechanisms and mask segmentation.

[0100] Assume that the original signal is a one-dimensional vector x=[x1,x2,…,x N ], a Haar wavelet transform decomposes the signal into low-frequency and high-frequency information. Specifically, assuming the length of the input signal is 2n, the output after the Haar wavelet transform includes two sequences:

[0101] Approximation coefficients:

[0102]

[0103] Among them, W0(x) represents the low-frequency part, that is, the result after averaging the input signal.

[0104] Detail coefficients:

[0105]

[0106] Here, W1(x) represents the high-frequency part, that is, the difference of the signal, which is usually used to represent the details in the image, such as edges or textures.

[0107] For two-dimensional images The Haar wavelet transform operation is divided into two steps: first, a one-dimensional Haar wavelet transform is performed on each row, and then the same transform is performed on each column. Finally, the image is decomposed into four sub-bands:

[0108] 1. Low-frequency component (Approximation)

[0109]

[0110] 2. Horizontal Detail

[0111]

[0112] 3. Vertical Detail:

[0113]

[0114] 4. Diagonal Detail:

[0115]

[0116] Among them, I 00 ,I 01 ,I 10 ,I 11 The four pixel values ​​of a 2×2 block in the image are decomposed into a low-frequency part (LL) and three high-frequency parts (LH, HL, HH), which represent the global information and detail information of the image respectively.

[0117] In the network structure, the embodiment of the present invention adopts the DwTForward module in the PyTorch Wavelets toolkit, which implements the above operations based on the orthogonal Haar wavelet. Perform a wavelet decomposition and output a low-frequency component and a high-frequency component group In the high-frequency component, the three channels correspond to the HL, LH, and HH sub-bands respectively.

[0118] In order to facilitate feature fusion and subsequent convolution processing, the embodiment of the present invention converts the low-frequency component y LThe high-frequency components are concatenated along the channel dimension to form a 4C×H / 2×W / 2 feature map. This is then compressed and fused through a 1x1 convolution, combined with BatchNormalization and ReLU activation functions to enhance feature expression. The above operation process can be expressed as:

[0119] x′=ReLU(BN(Conv 1×1 (Concat(LL,HL,LH,HH)))) (9)

[0120] Figure 5 A schematic diagram of the overall structure and feature fusion process of the Haar wavelet transform module in the network provided in one embodiment of the present application is shown in FIG. Figure 5 As shown in Figure 1, it covers steps such as wavelet decomposition, feature fusion, and convolution processing. The HWD module mainly consists of two submodules: (1) a lossless feature encoding block, which is responsible for frequency domain transformation and downsampling; (2) a feature representation learning block, which consists of standard convolutional layers, BatchNorm, and ReLU, and is used to extract discriminative information from the transformed features.

[0121] This structure not only retains multi-scale frequency information, but also avoids the information loss problem caused by maximum pooling, effectively enhancing the model's ability to perceive details and edges, and providing richer and more stable semantic features for subsequent mask segmentation tasks.

[0122] The Haar wavelet transform enhances the perception of image details and edge features through multi-scale feature extraction and frequency domain analysis. In an embodiment of the present invention, the introduction of the Haar wavelet transform enables the network to not only extract features at multiple scales but also retain both low-frequency and high-frequency information, thereby improving the ability to capture fine-grained texture and edge features in images. By combining the Haar wavelet transform with traditional convolution operations, this embodiment of the present invention further optimizes the model's performance in complex backgrounds, achieving superior performance in the sheep back color segmentation task.

[0123] The attention module is embedded in each bottleneck block in the network.

[0124] In computer vision tasks, especially image segmentation tasks, the model needs to effectively identify key features in the image and suppress irrelevant background information. Although the standard convolutional neural network (CNN) can extract rich local features, its attention is often scattered when dealing with complex backgrounds and diverse details, making it difficult to focus on the key information areas in the image. In particular, in the sheep back color segmentation task in the embodiment of the present invention, due to the subtle color differences and the presence of natural black areas, missegmentation is easily caused. To this end, the embodiment of the present invention introduces a convolutional block attention module (CBAM). This module combines spatial attention and channel attention mechanisms, which can dynamically guide the network to focus on important feature areas in the image, especially in the segmentation of sheep back color, significantly reducing the occurrence of misjudgment. Through these two attention mechanisms, CBAM effectively enhances the expression of convolutional features, enabling the model to focus on the target area in a complex background, improving the perception of details, and thus improving the accuracy of the segmentation task.

[0125] Specifically, the CBAM module adopts a two-stage attention mechanism: spatial attention mechanism and channel attention mechanism. Figure 6 This is a schematic diagram of the attention module network structure provided in one embodiment of the present application, as shown in FIG. Figure 6 As shown in Figure 2, the spatial attention mechanism learns the spatial features of the image to determine which locations are most important for the current task. The channel attention mechanism further strengthens the network's perception of specific features by weighting each channel.

[0126] Channel Attention Mechanism: The channel attention mechanism compresses the spatial information of feature maps to generate channel attention maps. During this process, average pooling and max pooling are used to aggregate the spatial dimensions of the feature maps. Average pooling feeds back all pixels in the feature map, while max pooling only feeds back the gradients in the feature map with the largest response. After these operations, a fully connected layer is used to generate the channel attention map.

[0127] The formula is as follows:

[0128] A channel =σ(FC(MaxPool(X))+FC(AvgPool(X))) (10)

[0129] Among them, A channel Represents the channel attention map, FC represents the fully connected layer, MaxPool and AvgPool are maximum pooling and average pooling respectively.

[0130] Spatial Attention Mechanism: The spatial attention mechanism compresses the channel information of the feature map to generate a spatial attention map. Using average pooling and max pooling, it extracts the highest number, width submaximum, and average value at each location, respectively. The extracted feature maps are then combined to generate a final spatial attention map. After processing by the spatial attention mechanism, the spatial dimensions of the feature map are weighted, allowing the network to focus more on important spatial regions.

[0131] The formula is as follows:

[0132] A spatial =σ(Conv(MaxPool(X),AvgPool(X))) (11)

[0133] Among them, A spatial is the spatial attention map, and Conv represents the convolution operation.

[0134] Through channel attention and spatial attention, the CBAM module provides the network with a dual attention mechanism, enabling it to capture key information areas in the image in both the spatial and channel dimensions while suppressing interference from irrelevant areas, thereby significantly improving image segmentation accuracy. CBAM's advantages are reflected in several aspects: first, it can adaptively adjust the focus of attention based on input features; second, by enhancing the network's perception of key areas, especially in complex backgrounds, CBAM can significantly improve the model's segmentation accuracy in diverse environments; finally, while maintaining a lightweight structure, it incurs minimal computational overhead, making it suitable for real-time segmentation and resource-constrained applications. Therefore, this embodiment of the present invention embeds the CBAM module into each bottleneck block of ResNet-50, specifically between the convolutional portion and the residual connection. This design significantly improves the model's attention to the color region on the sheep's backs, thereby achieving superior performance in fine-grained segmentation tasks.

[0135] Preferably, the improved Mask2Former image segmentation model further includes a feature pyramid network (FPN), which is placed in the connection layer between the backbone network and the pixel decoder.

[0136] One of the main challenges in image segmentation, especially when dealing with sheepback color segmentation, is how to effectively handle multi-scale features. Since sheepbacks can appear at different sizes and colors in an image, with significant multi-scale variations, this places higher demands on the model in terms of capturing both detailed information and global semantics. Therefore, developing a mechanism that can integrate features across scales is crucial for improving segmentation performance.

[0137] To address this issue, the present invention introduces a Feature Pyramid Network (FPN). FPN is a classic multi-scale feature fusion architecture, initially widely used in object detection tasks. By combining a bottom-up feature extraction process with a top-down feature enhancement process, it achieves efficient fusion of multi-level semantic features. This architecture significantly enhances the model's ability to perceive objects of varying sizes, making it ideally suited for the scale-varying sheepback color region segmentation task in the present invention.

[0138] The core concept of FPN is to fuse features from multiple layers of the backbone network, leveraging both low-level detail information and high-level semantic information. In this embodiment of the present invention, FPN helps address the scale variation of the sheepback region and effectively improves the overall segmentation performance of the model by enhancing segmentation capabilities at different resolutions.

[0139] Its structure primarily consists of a bottom-up convolutional network (such as ResNet) and a top-down upsampling module. The former extracts features layer by layer, gradually decreasing the resolution of the feature maps while enhancing their semantics. The latter, through progressive upsampling, fuses high-level features (high semantics but low resolution) with finer features from lower layers. This fusion strategy effectively preserves detailed information while enhancing semantic expression.

[0140] Figure 7 A schematic diagram of a feature pyramid network structure provided in an embodiment of the present application is shown in FIG. Figure 7 The specific calculation process is as follows:

[0141]

[0142] Among them, P l Represents the feature map extracted from the lth layer, Conv(P l ) represents the feature map P of the current layer l Perform convolution operation, Upsample(P l+1 ) means the feature map P of the previous layer l+1 Through this process, FPN can effectively fuse low-level detail information with high-level semantic information, enhancing the performance of the model in multi-scale tasks.

[0143] In an embodiment of the present invention, a feature pyramid network (FPN) is integrated into the Mask2Former architecture and placed in the connection layer between the backbone network and the pixel decoder. This design allows the model to make full use of multi-level features during the decoding process, thereby improving the recognition accuracy of sheep back areas of different scales. At the same time, combined with the ResNet-50 backbone network and the CBAM module, FPN further improves the network's ability to capture fine-grained features and improves the recognition ability of complex backgrounds and small targets. Through this integrated solution, FPN not only optimizes the segmentation results, but also improves the performance of the entire image segmentation task, enabling this method to be better applied to grassland monitoring and sheep management tasks.

[0144] After the improved Mask2Former image segmentation model is constructed, the model performance needs to be evaluated. The embodiment of the present invention uses image segmentation evaluation indicators to evaluate and adjust the performance of the improved Mask2Former image segmentation model.

[0145] Set image segmentation evaluation indicators: mean intersection over union (mIoU), mean precision (mPrecision), mean recall (mRecall), and F1 score (mF1-score). Use the image segmentation evaluation indicators to evaluate the indicator scores of the improved Mask2Former image segmentation model, and adjust the model parameters according to the indicator scores.

[0146] Let C be the total number of categories, TP i is the true positive of the i-th class, that is, the number of pixels correctly predicted to be the i-th class; FP i is the false positive of the i-th class, that is, the number of pixels that are incorrectly predicted to be the i-th class; FN i is the false negative of the i-th class, that is, the number of pixels that are incorrectly predicted as non-i-th class. The definition and calculation formula of each indicator are as follows:

[0147] Mean Intersection over Union (mIoU):

[0148]

[0149] mIoU measures the overlap between the predicted and true regions. It comprehensively evaluates the model's performance across categories and is the most critical and commonly used evaluation metric in semantic segmentation tasks. Higher mIoU values ​​indicate better segmentation accuracy across different categories.

[0150] Mean Precision (mPrecision):

[0151]

[0152] Precision focuses on the proportion of samples predicted as positive by the model that are actually positive, reflecting the model's prediction accuracy. mPrecision averages the Precision of each class to reflect the model's overall accuracy across all classes. A higher mPrecision indicates a lower false positive rate in the model's predictions.

[0153] Mean Recall (mRecall):

[0154]

[0155] Recall focuses on how many samples are actually positive that the model successfully predicts as positive, reflecting the model's detection capability. mRecall averages the recall values ​​for each class and reflects how many real objects the model can identify. A higher mRecall value indicates that the model is better able to identify positive regions and avoid missed detections.

[0156] Mean F1 Score (mF1 Score):

[0157]

[0158] The F1 Score is a harmonic mean of both Precision and Recall, achieving a balance between the two. A higher F1 Score generally indicates that the model maintains high accuracy while also possessing better detection capabilities, resulting in more stable and reliable overall performance.

[0159] These indicators can reflect the accuracy, completeness and stability of the model's prediction effect in the sheep back color segmentation task from different dimensions. In order to quantitatively evaluate, the embodiment of the present invention conducted a performance test on the test set, and the results are shown in Table 4:

[0160] index Score mIoU 82.98％ mPrecision 89.38％ mRecall 91.50％ mF1-score 90.41％

[0161] Table 4 Core performance index evaluation results

[0162] Table 4 shows that this method achieves excellent results across all core metrics. The mIoU (mInfluence over Union) score reaches 82.98%, demonstrating good consistency in the model's positioning and delineation of the target region. mPrecision and mean Recall reach 89.38% and 91.50%, respectively, demonstrating that the model not only ensures segmentation accuracy but also effectively covers the true target region, striking a balance between detection rate and false positive rate. The mF1-score score reaches 90.41%, comprehensively reflecting the model's good balance between classification accuracy and detection capability.

[0163] Overall, the model not only accurately segments the color regions on the sheep's back, but also demonstrates good adaptability and robustness across different categories and scenarios. The simultaneous improvement in both precision and recall demonstrates the method's strong anti-interference capabilities in challenging scenarios such as blurred colors and complex backgrounds, providing a solid foundation for practical applications.

[0164] In order to deeply analyze the segmentation performance of the model on different sheepback color categories, the embodiment of the present invention counted the IoU and F1-score indicators of the four categories of labels, Red, Blue, Yellow, and Colorless, on the test set. The results are shown in Table 5:

[0165] category IoU (%) F1-score (%) Red 87.21％ 93.17 Blue 80.32％ 89.08 Yellow 79.04％ 88.30 Colourless 69.83％ 82.24

[0166] Table 5 Segmentation performance index results of each category

[0167] As can be seen from Table 5, the model achieved high segmentation results across all categories. The Red category achieved the highest IoU and F1-score, reaching 87.21% and 93.17%, respectively, demonstrating the model's strong segmentation capabilities in categories with distinct colors and clear regional boundaries. The Blue and Yellow categories also performed well, with IoUs of 80.32% and 79.04%, respectively, and F1-scores exceeding 88%, indicating that the model was able to distinguish between different color features. In contrast, the Colorless category achieved relatively low IoU and F1-scores of 69.83% and 82.24%, respectively. This is primarily due to the fact that the colorless regions share a certain degree of texture similarity with the background color, which leads to ambiguity and errors in the model's boundary judgment, resulting in slightly lower segmentation accuracy.

[0168] In summary, the proposed method shows strong generalization ability on all kinds of color labels, especially high performance on the red category, which verifies the effectiveness of this method in color feature extraction and semantic region separation, and has good practical application potential.

[0169] In order to comprehensively evaluate the training effect and performance stability of the model, the curves of training loss (Loss), F1-score and mIoU changing with the number of training steps are fitted experimentally. Figure 8 The loss value, F1-score and mIoU curves during the model training process provided in one embodiment of the present application are as follows: Figure 8 As shown in the figure, the training loss curve shows that the loss value decreases rapidly in the early stages of training, indicating that the model can quickly capture key features in the data and complete initial parameter optimization. As training progresses, the rate of loss decreases gradually and eventually stabilizes, indicating that the model focuses more on refining and optimizing features in the later stages of learning, thereby further improving prediction accuracy. There are no significant fluctuations or rebounds throughout the training process, indicating that the model optimization process is stable and converges well.

[0170] The F1-score curve demonstrates the model's learning ability in classification tasks. It rapidly increases in the early stages of training, indicating that the model is able to quickly learn effective features for classification tasks. As training progresses, the F1-score gradually stabilizes, ultimately reaching around 90%. This demonstrates the model's strong learning ability and stability in classification tasks.

[0171] The mIoU curve reflects the model's performance on the segmentation task. It shows that mIoU improves significantly at the beginning of training, then continues to grow at a relatively steady rate, ultimately converging around 82.9%. Throughout training, the mIoU curve remains stable with no significant fluctuations, showing no signs of overfitting, further validating the model's generalization capabilities.

[0172] In summary, Figure 8 The training curves reveal several key characteristics: the loss value continues to decrease and stabilize, indicating good model convergence; the F1-score and mIoU both steadily increase, ultimately stabilizing at approximately 90% and above 82%, respectively, with no significant overfitting. This demonstrates that the model achieves stable and efficient training results in both classification and segmentation tasks, effectively extracting semantic features from images, and exhibits good generalization and practicality, laying a solid foundation for subsequent experiments and deployment.

[0173] The embodiment of the present invention also analyzes the impact of each module (Mask2Former, FPN, Haar, CBAM) on the model performance.

[0174] The model efficiency is mainly evaluated from four dimensions: parameters, floating point operations (FLOPs), inference time, and inference throughput (FPS). The results are as follows:

[0175] As shown in Table 6.

[0176]

[0177]

[0178] The addition of the FPN, Haar, and CBAM modules increases the overall model parameter count, as shown in Table 6. The original Mask2Former configuration has 44 million parameters, but after the module enhancements, the number of parameters increases to 46.63 million, an increase of approximately 6%. The introduction of these modules significantly enhances the model's ability to handle complex tasks, but due to its rational design, the parameter increase remains manageable.

[0179] In terms of computational complexity, FLOPs is a key metric for measuring the model's computational workload. After adding FPN, FLOPs increased from 21.469 GFLOPs to 36.32 GFLOPs, an increase of approximately 70%. Despite the increased computational complexity, the addition of this module effectively improves image feature extraction capabilities, enabling the model to demonstrate greater expressiveness when handling more complex tasks. However, the increase in FLOPs may affect inference performance, particularly with a decrease in inference throughput, but this change remains within an acceptable range.

[0180] From the perspective of inference performance, the original Mask2Former configuration has an inference time of 14.091 milliseconds and an FPS of 76.35. After adding FPN, the inference time increases slightly to 15.95 milliseconds, and the FPS drops to 69.3, a decrease of about 9%. Despite the decrease in inference performance, the model still maintains a high level of inference capability when handling more complex tasks. The inference time of the model after the fusion module increases by about 2.4 milliseconds, and the FPS decreases by less than 10 FPS. For example, with the full configuration, the inference time is 15.387 milliseconds and the FPS is 70.05, which still meets the real-time requirements. It is worth noting that the introduction of the Haar module has minimal impact on the number of parameters, FLOPs, inference time, and FPS, showing good lightweight characteristics.

[0181] Overall, while the addition of FPN, Haar, and CBAM increases the model's computational overhead, particularly in FLOPs and inference time, the model's performance is significantly improved. The introduction of these modules strikes a balance between improved task performance and a decrease in inference performance. Therefore, the optimized Mask2Former configuration is suitable for challenging real-time segmentation tasks and offers excellent computational efficiency.

[0182] The embodiments of the present invention also verify the effectiveness of each improved module, that is, its contribution to model performance. On the one hand, it evaluates the effect of different combinations of FPN, Haar, and CBAM modules on model performance improvement. On the other hand, it explores the impact of different insertion positions of the CBAM module in the Bottleneck structure on the results.

[0183] First, building on the Mask2Former, we introduced the Haar wavelet transform, FPN multi-scale feature fusion, and CBAM attention modules to evaluate their impact on segmentation performance. The effectiveness of each module was quantified using four metrics: mIoU, mPrecision, mRecall, and mF1-score. The results are shown in Table 7.

[0184] Table 7 Evaluation results of the impact of different modules on model segmentation performance

[0185] Model mIoU (%) mPrecision(%) mRecall (%) mF1-score (%) Mask2Former 81.09 88.12 90.31 89.19 Haar 81.44 88.15 90.75 89.41 FPN 81.28 88.7 90.04 89.32 CBAM 81.38 88.41 90.34 89.35 Haar+FPN 82.54 89.14 91.20 90.13 Haar+CBAM 82.63 89.24 91.19 90.18 CBAM+FPN 82.31 89.21 91.23 90.23 Final 82.98 89.38 91.50 90.41

[0186] The baseline model Mask2Former achieved 81.09% in terms of mIoU, while mPrecision and mRecall were 88.12% and 90.31% respectively. Although the baseline model showed good segmentation accuracy, there is still room for improvement in terms of precision and recall.

[0187] After introducing Haar and FPN, the model's mIoU reached 81.44% and 81.28%, respectively, while mPrecision and mRecall significantly improved to 88.15% and 90.75%, and 88.70% and 90.04%, respectively. This demonstrates that Haar effectively enhances the extraction of details and edge features, while FPN optimizes the processing of objects of varying sizes through multi-scale feature fusion. Although the improvement in mIoU when using these modules alone is modest, the significant improvement in mRecall demonstrates their significant contribution to recall.

[0188] Furthermore, the combination of Haar and CBAM improves mIoU to 82.63%, while mPrecision and mRecall reach 89.24% and 91.19%. This demonstrates that the CBAM module, through its attention-guided feature selection mechanism, significantly enhances the capture of color regions and key information, further improving segmentation accuracy. CBAM is particularly effective in the task of segmenting the color details of sheep backs.

[0189] Ultimately, by combining Haar wavelet transform, FPN, and CBAM, the model achieved mIoU of 82.98%, mPrecision of 89.38%, mRecall of 91.50%, and an mF1-score of 90.41%. This result fully demonstrates the synergy between these modules. The combination of the three not only effectively improves segmentation accuracy and recall, but also optimizes the extraction of detailed features, performing particularly well when processing tasks with complex color regions.

[0190] Overall, the synergistic effect of the Haar wavelet transform, FPN multi-scale feature fusion, and CBAM attention mechanism significantly improves the segmentation performance of the model. Specifically, the Haar module enhances the extraction of details and edge features, FPN provides multi-scale information fusion, further improving the recognition ability of targets of different sizes, and CBAM optimizes the extraction of key feature areas through the attention mechanism. The organic combination of these modules not only effectively improves the precision and recall rate of the model in the sheep back color segmentation task, but also enhances the robustness of the model in complex scenes. The final model performs well in key indicators such as mIoU, mPrecision, mRecall, and mF1-score, demonstrating its superiority in fine-grained segmentation tasks. It has strong application potential and is particularly suitable for practical scenarios requiring high-precision segmentation.

[0191] The embodiment of the present invention further determines the optimal insertion position of the attention mechanism CBAM in the network structure.

[0192] Ablation experiments were conducted after embedding CBAM into different convolutional layers (Conv1, Conv2, and Conv3) in the Bottleneck Block. The performance of different configurations was compared based on metrics such as mIoU, mPrecision, mRecall, and mF1-score. The experimental results are shown in Table 8.

[0193] Model mIoU (%) mPrecision(%) mRecall (%) mF1-score (%) Mask2Former 81.09 88.12 90.31 89.19 CBAMin Conv1 81.68 88.65 90.74 89.58 CBAMin Conv2 82.03 88.92 91.01 89.83 CBAMin Conv3 82.98 89.38 91.50 90.41

[0194] Table 8 Performance comparison of CBAM inserted in different convolution positions

[0195] As shown in Table 8, compared to the basic Mask2Former model, inserting CBAM at different positions effectively improves segmentation performance. When CBAM is embedded in the Conv1 position, the model's mIoU increases from 81.09% to 81.68%, and mPrecision and mF1-score also increase by 0.53% and 0.39%, respectively. This improvement is mainly attributed to the enhancement of local texture information by low-level features, but since the feature expression at this stage is still relatively rudimentary, the overall improvement is limited. When CBAM is inserted into the Conv2 position, the model performance is further improved, with mIoU reaching 82.03% and F1-score also increasing to 89.83%. At this stage, the network features incorporate more mid-level semantic information, and the attention mechanism can more effectively filter feature areas related to the target, resulting in better performance than Conv1. The best results were achieved when CBAM was inserted into Conv3, with mIoU significantly improving to 82.98%, mPrecision reaching 89.38%, mRecall reaching 91.50%, and F1-score rising to 90.41%. This phenomenon demonstrates that deep features contain richer and more abstract semantic information. Introducing the attention mechanism on this basis can maximize the highlighting of key feature areas and significantly improve overall segmentation accuracy. In particular, in the application scenario of fine-grained segmentation of sheep back color features, high-level attention guidance can effectively improve the model's ability to identify boundary regions and model regional consistency.

[0196] To fully verify the performance advantages of the improved Mask2Former image segmentation model in semantic segmentation tasks, this paper selected current mainstream image segmentation methods, including U-Net, DeepLabv3+, SegFormer, K-Net, ViT, and Mask2Former, as comparison models and used the same training strategy and dataset for experimental evaluation. Evaluation metrics include mean intersection over union (mIoU), mean precision (mPrecision), mean recall (mRecall), and F1-score.

[0197] The performance of each model on the sheep back color segmentation task is shown in Table 9.

[0198] Model mIoU (%) mPrecision(%) mRecall (%) mF1-score (%) U-Net 71.60 83.86 81.81 82.48 DeepLabv3+ 76.15 85.04 86.76 85.82 SegFormer 75.64 84.90 86.27 85.44 ViT 70.45 82.50 81.20 81.41 K-Net 80.94 88.89 89.42 89.09 Mask2Former 81.09 88.12 90.31 89.19 Ours 82.98 89.38 91.50 90.41

[0199] Table 9 Performance comparison of each model in the sheep back color segmentation task

[0200] As shown in Table 9, traditional encoder-decoder architectures such as U-Net and DeepLabv3+ perform poorly in this fine-grained task, achieving mIoU scores of 71.60% and 76.15%, respectively. Despite their high accuracy, these architectures are sensitive to changes in object boundaries and color, affecting overall segmentation consistency. SegFormer and ViT, which introduce Transformer architectures and possess some global modeling capabilities, still struggle with insufficient local feature awareness in fine-grained segmentation scenarios, achieving mIoU scores of 75.64% and 70.45%, respectively.

[0201] In comparison, K-Net and Mask2Former, more advanced semantic segmentation frameworks, achieve superior performance through dynamic convolution, query mechanisms, and multi-scale feature fusion, with mean Intersection Over Union (MIoU) exceeding 80%. Mask2Former, the improved infrastructure of the present invention, achieves an mIoU of 81.09% and an F1-score of 89.19%, demonstrating strong segmentation capabilities.

[0202] On this basis, the improved model proposed in the embodiment of the present invention (the Mask2Former model integrating FPN, Haar, and CBAM) comprehensively introduces multi-scale enhancement (FPN), efficient texture features (Haar), and the attention mechanism (CBAM), achieving significant improvements in multiple indicators. The final model proposed in the embodiment of the present invention (Mask2Former+FPN+Haar+CBAM) achieved excellent performance in all evaluation indicators, reaching 82.98%, 89.38%, 91.50%, and 90.41% in mIoU, precision, recall, and F1-score, respectively. Compared with the basic model Mask2Former, it has improved by 1.89% and 1.22%, and is comprehensively superior to all compared models.

[0203] Traditional U-Net and DeepLabv3+ algorithms have limited ability to identify color boundaries on sheep backs, resulting in color regions clumping and blurred boundaries. SegFormer and ViT offer improvements in their overall structure, capturing differences in some areas, but still suffer from misjudgments when dealing with similar colors or detailed boundaries. K-Net and Mask2Former offer stronger feature modeling capabilities, resulting in predictions closer to the true labels, but still exhibit slight inaccuracies in some detailed areas.

[0204] In contrast, the improved model proposed in this embodiment of the present invention not only achieves sharper segmentation boundaries but also exhibits greater coherence and accuracy within regions of color consistency, effectively avoiding misclassification and omission. This advantage demonstrates the comprehensive effectiveness of this model in integrating multi-scale structure (FPN), local edge information (Haar), and spatial attention mechanism (CBAM), further demonstrating its adaptability and practical application value in color-driven semantic segmentation tasks.

[0205] The results clearly demonstrate that the proposed method demonstrates enhanced feature representation and regional consistency modeling capabilities in fine-grained color perception and structure preservation, particularly in semantic segmentation tasks where color features are the primary distinguishing factor. Specifically, in the sheep back color segmentation scenario of interest in this study, the model more accurately extracts color boundaries and regional consistency, enabling precise demarcation of the entire sheep area, providing a reliable basis for subsequent determination of the herder group to which the sheep belong.

[0206] In order to intuitively demonstrate the segmentation effect of the model, the embodiment of the present invention provides several sets of visualization diagrams of experimental results. Figure 9 This is a rendering of an intelligent identification method for livestock ownership based on an improved Mask2Former provided in an embodiment of the present application, as shown in FIG. Figure 9 As shown in the figure, each set of images includes the input image, the ground truth, and the model's predicted segmentation results. These comparisons clearly demonstrate the model's performance in the sheep back color segmentation task. The model utilizes the sheep's back color as a distinct feature for segmentation, and by accurately identifying the back color region, it gradually expands to the entire sheep body, achieving complete object detection and segmentation.

[0207] Figure 9The improved Mask2Former image segmentation model in

[15] achieves high segmentation accuracy in areas with clear color boundaries, such as red and blue. The high overlap between the true labels and the predicted results in these areas demonstrates the model's ability to locate and identify images in scenes with strong color contrast. For the yellow area, despite a certain degree of color similarity between the yellow area and the background, the model is still able to effectively extract the semantic features of the area and achieve segmentation, demonstrating its robustness to complex backgrounds and color transition areas. In the case of blurred boundaries or color gradients, the model can still maintain reasonable region recognition capabilities, avoiding over-segmentation or mis-segmentation problems caused by color confusion. However, the segmentation accuracy of colorless areas is slightly lower. This may be due to the high similarity between the color and the background texture, which makes the model face greater challenges in boundary judgment. However, even in such low-contrast areas, the model can still complete segmentation to a certain extent, demonstrating its adaptability in complex scenes.

[0208] Overall, the proposed method shows good visual segmentation performance in different color categories, verifying its usability and robustness in practical applications.

[0209] After verifying the performance of the improved Mask2Former image segmentation model, the livestock image to be identified is input into the trained improved Mask2Former image segmentation model, and the color classification corresponding to the livestock's back is output.

[0210] S103: Determine the corresponding herder according to the color classification.

[0211] Specifically, after improving the Mask2Former image segmentation model to output the color classification corresponding to the livestock back, the sheep back color segmentation results are combined with the herder affiliation rules to determine the herders corresponding to different colors.

[0212] In the method provided by the embodiment of the present invention, by introducing the Haar wavelet transform on the basis of the Mask2Former network to improve the local feature extraction capability, using the attention module to focus on key features, and optimizing the multi-scale feature fusion through the feature pyramid network, an improved Mask2Former image segmentation model is constructed and the backs of livestock are classified into different colors. It has stronger recognition ability in scenes with similar target colors and blurred boundaries, improves the color segmentation accuracy of livestock backs and the accuracy of herder affiliation identification, and is particularly suitable for scenes with complex backgrounds and dense livestock. It provides an innovative solution for sheep herd management and herder affiliation identification in the Qinghai-Tibet Plateau, and provides key technical support for the construction of an intelligent and sustainable modern pastoral management system in the Qinghai-Tibet Plateau, and also opens up a new research path for image segmentation tasks based on color features.

[0213] Figure 10A schematic diagram of the structure of an intelligent identification device for livestock ownership based on an improved Mask2Former provided in one embodiment of the present application is shown in FIG. Figure 10 As shown, the device includes:

[0214] Image acquisition module 11: used to obtain images of livestock during grazing;

[0215] Color classification module 22: used to input the photos and videos into the trained improved Mask2Former image segmentation model and output the color classification corresponding to the back of the livestock;

[0216] The attribution determination module 33 is used to determine the corresponding herder household according to the color classification.

[0217] The above device can execute the intelligent identification method of livestock ownership of herders based on the improved Mask2Former in the above embodiment, wherein the image acquisition module 11 executes step S101; the color classification module 22 executes step S102; and the ownership determination module 33 executes step S103.

[0218] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application. The method corresponding to the electronic device may be the method for intelligently identifying the ownership of livestock by herders based on the improved Mask2Former in the aforementioned embodiment, and the principle of solving the problem is similar to that of the method. The electronic device provided in the embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0219] The electronic device may be a user device, or a device formed by integrating a user device and a network device via a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablets, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer collections, and can be used to implement some of the processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computers.

[0220] Figure 11The structure of a device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown. The device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 into the random access memory (RAM) 1203. Various programs and data required for system operation are also stored in the RAM 1203. The CPU 1201, ROM 1202 and RAM 1203 are connected to each other through a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0221] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, and the like; an output section 1207 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, and a speaker; a storage section 1208 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, and a semiconductor memory; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 1209 performs communication processing via a network such as the Internet.

[0222] In particular, the methods and / or embodiments of the present application can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the method of the present application are performed.

[0223] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application.

[0224] Specifically, the present embodiment can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0225] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0226] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0227] The computer program code for performing the operations of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).

[0228] The flow chart or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the equipment, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code include one or more executable instructions for realizing the logical function of the specification. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs the function or operation of the specification, or can be implemented with a combination of dedicated hardware and computer instructions.

[0229] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0230] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0231] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0232] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0233] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0234] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

Claims

1. An intelligent identification method for livestock ownership based on improved Mask2Former, characterized in that: include: Acquire images of livestock while grazing; Input the photos and videos into the trained improved Mask2Former image segmentation model to output the color classification corresponding to the back of the livestock; The corresponding herder is determined according to the color classification.

2. The intelligent identification method for livestock ownership of herders based on improved Mask2Former according to claim 1 is characterized in that: The obtaining of images of livestock grazing further comprises: Use drones or cameras to film livestock on the ground while they are grazing, collecting original photos and video images under different time, lighting, weather and background conditions; OpenCV is used to extract the original video frame by frame, remove blurry and highly repetitive images, and obtain an image set containing the backs of livestock.

3. The intelligent identification method for livestock ownership of herders based on improved Mask2Former according to claim 1 is characterized in that: Inputting the photos and videos into the trained improved Mask2Former image segmentation model and outputting the color classification corresponding to the back of the livestock further includes: The improved Mask2Former image segmentation model was obtained by training the neural network model using the color classification corresponding to historical livestock as training samples; Use image segmentation evaluation metrics to evaluate and adjust the performance of the improved Mask2Former image segmentation model; The livestock image to be identified is input into the trained improved Mask2Former image segmentation model, and the color classification corresponding to the livestock's back is output.

4. The intelligent identification method for livestock ownership of herders based on improved Mask2Former according to claim 3 is characterized in that: The improved Mask2Former image segmentation model obtained by training the neural network model using the color classification corresponding to the historical livestock as training samples further includes: Obtain historical livestock grazing images and corresponding color classifications; Use the X-AnyLabeling tool to label the livestock back area in the image with the corresponding color; Using the convolutional neural network ResNet-50 as the backbone network, the Haar wavelet transform module and the attention module are added to construct an improved Mask2Former image segmentation model; The color marked on the backs of livestock was used as the evaluation indicator, and historical images of livestock grazing were input into the model for training, and finally a trained improved Mask2Former image segmentation model was obtained.

5. The intelligent identification method for livestock ownership of herders based on improved Mask2Former according to claim 4 is characterized in that: The improved Mask2Former image segmentation model constructed by using the convolutional neural network ResNet-50 as the backbone network and adding the Haar wavelet transform module and the attention module further includes: The convolutional neural network ResNet-50 is used as the backbone network, and the Haar wavelet transform module is used to replace the maximum pooling layer in the network; The attention module is embedded in each bottleneck block in the network.

6. The intelligent identification method for livestock ownership of herders based on improved Mask2Former according to claim 4 is characterized in that: The improved Mask2Former image segmentation model also includes: Feature Pyramid Network, a connection layer placed between the backbone network and the pixel decoder.

7. The intelligent identification method for livestock ownership of herders based on improved Mask2Former according to claim 3 is characterized in that: The using of image segmentation evaluation indicators to evaluate and adjust the performance of the improved Mask2Former image segmentation model further includes: Set image segmentation evaluation indicators: mean intersection over union (mIoU), mean precision (mPrecision), mean recall (mRecall), and F1 score (mF1-score); The image segmentation evaluation index is used to evaluate the index score of the improved Mask2Former image segmentation model, and the model parameters are adjusted according to the index score.

8. An intelligent identification device for livestock ownership based on improved Mask2Former, characterized in that: include: Image acquisition module: used to obtain images of livestock grazing; Color classification module: used to input the photos and videos into the trained improved Mask2Former image segmentation model and output the color classification corresponding to the livestock's back; The attribution determination module is used to determine the corresponding herder according to the color classification.

9. An intelligent livestock identification device for herders based on an improved Mask2Former, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable medium having computer program instructions stored thereon, characterized in that: The computer program instructions are executable by a processor to implement the method according to any one of claims 1 to 7.

Images