A beef cattle target detection method based on efficient selection and fusion of high and low frequency characteristics

Abstract

The application provides a beef target detection method based on efficient selection and fusion of high and low frequency characteristics, comprising the following steps: S1, constructing a beef target detection data set; S2, constructing a beef target detection model; S201, constructing a feature extraction network MPCPKI-ResNet; S202, constructing a hybrid encoder; S203, constructing an IoU perception query selection mechanism; S204, constructing a decoder with an auxiliary prediction head; S3, training the beef target detection model; S4, beef target detection; the application can efficiently and accurately detect beef in different scenes and complex environments, significantly improving the performance of real-time beef target detection; the application is specially designed for real-time beef target detection; the method of the application uses an efficient P4 feature layer selection module of a multi-path cooperative gating mechanism, effectively integrates various low-level features extracted by a shallow network, improves the efficiency of feature fusion, and enhances the image detail information capturing capability of the model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of video image understanding, and relates to beef cattle target detection, in particular to a beef cattle target detection method based on efficient selection and fusion of high-frequency and low-frequency features. BACKGROUND

[0002] The beef cattle target detection task is the basis for monitoring the beef cattle breeding process in a scientific manner, and the identification and positioning of beef cattle targets are beneficial to the realization of fine management under the large-scale breeding of beef cattle and have important significance for the intelligent perception of individual behavior analysis and health status of beef cattle. In the past few decades, foreign scholars have made various attempts on the application of target detection technology in the field of livestock breeding, such as traditional image analysis-based target detection algorithm, statistical theory-based target detection algorithm and deep learning-based target detection algorithm.

[0003] The target detection algorithm based on traditional image processing theory has threshold segmentation method, frame difference method and the like. However, the traditional detection algorithm has various limitations and cannot meet the application requirements in complex real scenes. Unlike the algorithm based on traditional image processing theory, the statistical theory-based target detection algorithm uses mathematical statistical methods and probability theory to establish a dynamic background model, and adaptively adjusts the model parameters according to the current environment, so as to continuously follow the environmental changes. Representative algorithms include feature matching method, background difference and Gaussian mixture model. However, the traditional statistical theory algorithm usually depends on manually selected features such as color, texture and shape. The expression ability of these features is limited and may not be able to fully capture the complex patterns in the image. With the development of computer vision technology, the deep learning-based target detection algorithm provides a new solution for the recognition and positioning of breeding animals in complex scenes. Compared with traditional detection models, these algorithms have significant improvement in accuracy and real-time performance, and have become the mainstream in the field of target detection. Although the deep learning-based target detection algorithm has obvious improvement in accuracy and real-time performance compared with the traditional algorithm, it mainly solves the detection problem of individuals and sparse group distribution. In the actual detection environment, there are still challenges, such as various obstacles in the real farm environment, for example, dense, occlusion and complex background, which may affect the accuracy and robustness of the detection system. How to make the target detection algorithm still achieve high-precision real-time detection of multiple categories and multiple targets in the case of variable environment, complex background and dense occlusion is an extremely important research problem. SUMMARY

[0004] In view of the deficiencies in the prior art, the purpose of the present application is to provide a beef cattle target detection method based on efficient selection and fusion of high-frequency and low-frequency features, so as to solve the technical problem that the accuracy of the target detection method in the prior art needs to be further improved in a complex environment.

[0005] To solve the above technical problems, the application adopts the following technical solutions to achieve:

[0006] A beef cattle target detection method based on high and low frequency feature efficient selection fusion, which comprises the following steps:

[0007] Step S1, constructing a beef cattle target detection data set:

[0008] Collect beef cattle videos of different breeding scenes under real monitoring angles, label the beef cattle videos, and construct a beef cattle target detection data set, which is named Beef-Cattle36.

[0009] Step S2, constructing a beef cattle target detection model:

[0010] Step S201, constructing a feature extraction network MPCPKI-ResNet:

[0011] The feature extraction network MPCPKI-ResNet comprises a basic residual module, an efficient P4 feature layer selection module based on a multi-path cooperative gating mechanism, and a CSP-PKINet P5 feature layer selection module.

[0012] The input image data is sent to the feature extraction network MPCPKI-ResNet, and the feature extraction network MPCPKI-ResNet generates three levels of feature maps {P3, P4, P5}, wherein: the P3 feature map is output by the basic residual module; the P4 feature map is output by the efficient P4 feature layer selection module based on the multi-path cooperative gating mechanism; and the P5 feature map is output by the CSP-PKINet P5 feature layer selection module.

[0013] Step S202, constructing a hybrid encoder:

[0014] The hybrid encoder comprises a scale intra-interaction module based on high and low frequency self-attention and a context feature selection fusion pyramid network.

[0015] The P5 feature map obtained in step S201 is input into the scale intra-interaction module based on high and low frequency self-attention for internal interaction of high and low frequency features, and an F5 feature map is output.

[0016] The P3 feature map, the P4 feature map and the F5 feature map obtained in step S201 are input into the context feature selection fusion pyramid network, and the P3 feature map, the P4 feature map and the F5 feature map are cross-scale high and low frequency feature fusion spliced, and a Memory feature sequence is output.

[0017] Step S203, constructing an IoU-aware query selection mechanism:

[0018] The regression and classification loss of the predicted bounding box corresponding to each feature in the Memory feature sequence obtained in the calculation step S202 and the real bounding box is calculated, and the IoU is combined in the classification loss to realize the consistency of the position confidence and the category confidence of the network output, and Top-K features are selected according to the loss.

[0019] Step S204, a decoder with an auxiliary prediction head is constructed:

[0020] The Decoder receives the Top-K initial target queries screened by the IoU-aware query selection mechanism constructed in step S203 and the Memory feature sequence output by the hybrid encoder constructed in step S202, iteratively optimizes the target queries through multiple layers of self-attention and cross-attention, and finally outputs the category and coordinates of the beef cattle detection frame.

[0021] Step S3, training a beef cattle target detection model:

[0022] Based on the beef cattle target detection data set Beef-Cattle36 obtained in step S1, when training the beef cattle target detection model, the loss function L used includes a target bounding box regression loss L box and a classification loss L cls The loss function L is back-propagated, and the iteration is repeated until the iteration reaches the preset initial value, and the training of the beef cattle target detection model is completed.

[0023] Step S4, beef cattle target detection:

[0024] Given the beef cattle image data, input into the beef cattle target detection model trained in step S3, and output the category and coordinates of the beef cattle detection frame.

[0025] The present application also has the following technical features:

[0026] In step S1, the different breeding scenes are five different breeding scenes.

[0027] In step S1, the beef cattle video is labeled using DarkLabel.

[0028] In step S3, the target bounding box regression loss L box is defined as the weighted sum of SIoU loss and L1 loss.

[0029] In step S3, the classification loss L cls is a variable focus loss.

[0030] Compared with the prior art, the present application has the following technical effects:

[0031] (I) The application can efficiently and accurately detect beef cattle in different scenes and complex environments (such as occlusion, shadow and appearance similarity), significantly improving the performance of real-time beef cattle target detection.

[0032] (II) The application is specially designed for real-time beef cattle target detection. The method of the application effectively integrates various low-level features extracted by the shallow network through the efficient P4 feature layer selection module of the multi-path cooperative gating mechanism, improves the efficiency of feature fusion, and enhances the model's ability to capture image detail information.

[0033] (III) The application considers the global information and detail features of the image through the scale intra-interaction module based on high and low frequency self-attention, so that the model can more accurately process the target detection task in complex scenes. Furthermore, considering the size, calculation and performance requirements of the model in the target detection task, the parameter quantity and calculation quantity of the model are reduced through the CSPPKINetP5 feature layer selection module, the context feature selection fusion pyramid network is introduced, and the SCYLLA-IoU loss function is introduced. The model parameter quantity and calculation load are effectively reduced, the deployability of the model is improved, and the performance in the target detection task is maintained. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a structure diagram of the beef cattle target detection method based on efficient selection and fusion of high and low frequency features.

[0035] Figure 2 It is a structure diagram of the efficient P4 feature layer selection module based on the multi-path cooperative gating mechanism of the application.

[0036] Figure 3 It is a structure diagram of the CSP-PKINetP5 feature layer selection module of the application.

[0037] Figure 4 It is a structure diagram of the scale intra-interaction module based on high and low frequency self-attention of the application.

[0038] Figure 5 It is a structure diagram of the feature selection context anchor attention of the application.

[0039] Figure 6 It is a structure diagram of the context feature selection fusion pyramid network of the application.

[0040] Figure 7 It is a detection error type distribution diagram of each module gradually added to the comparative example in the embodiment.

[0041] Figure 8 It is a result comparison diagram of the method and the comparative example in the embodiment in different breeding scenes.

[0042] Figure 9 is a comparison chart of the results of the method in the examples and the thermal map of the comparative examples.

[0043] Figure 10 is a comparison chart of the robustness of the method in the examples and the comparative examples under different occlusions.

[0044] The specific content of the present application is further explained in detail below in combination with the examples. DETAILED DESCRIPTION

[0045] It should be noted that all algorithms, modules and networks in the present application, unless otherwise specified, all use known algorithms, modules and networks in the prior art.

[0046] In order to meet the demand of real breeding link for beef cattle target detection, aiming at the problems of variable breeding environment, complex background, dense occlusion and the like, a real beef cattle target detection data set named "Beef-Cattle36" is collected first, the data set is captured in different scenes, and contains beef cattle detection under various interference challenges, such as occlusion, shadow, similar appearance and the like. This is more in line with the real cattle field environment, in which the cattle herd is collectively raised, and the cattle are often not captured by the camera in the center of the image. Based on the real beef cattle target detection data set, the present application proposes a real-time DETR (HLFFESF-RTDETR) based on high and low frequency feature efficient selection and fusion, which is specially designed for real-time beef cattle target detection. First, in order to more effectively integrate various low-level features extracted by the shallow network, the present application proposes an efficient P4 feature layer selection module (Efficient P4 based on multi-path cooperative gating mechanism, EP4MCGM) based on multi-path cooperative gating mechanism. Secondly, the scale intra-interaction module (Intrascale Feature Interaction Module Based on High-Low Frequency, HiLo-IFIM) based on high and low frequency self-attention is proposed, so that it can process the global information of the image while considering the bottom-level and detailed features. Finally, in order to improve the deployability of the model, a model lightweight scheme is proposed in the scheme, in which a P5 feature layer selection module based on the Cross-Stage Partial Poly Kernel Inception Network (CSPPKINetP5) is proposed to reduce the parameter quantity and calculation quantity of the model, a context feature selection fusion pyramid network (CFSF-FPN) is proposed and the SCYLLA-IoU loss function is introduced, which is designed to work with the CSPPKINetP5 feature layer selection module, to reduce the model size and calculation quantity while ensuring the performance of the model in the target detection task.

[0047] Further, the beef target detection method based on efficient selection and fusion of high and low frequency features of the present application can also be called a beef target detection method based on Real-Time-DETR and efficient selection and fusion of high and low frequency features. The method of the present application is composed of a plurality of key components, including a feature extraction network, a hybrid encoder, an IoU-aware query selection mechanism, and a decoder with an auxiliary prediction head. The feature extraction network MPCPKI-ResNet is composed of a basic residual module, an efficient P4 feature layer selection module based on a multi-path cooperative gating mechanism, and a CSPPKINetP5 feature layer selection module. The hybrid encoder includes a scale intra-interaction module based on high and low frequency self-attention and a context feature selection fusion pyramid network. The structure is as shown in Figure 1 As shown in the figure. First, the input beef image data is sent into the MPCPKI-ResNet to generate three levels of feature maps {P3, P4, P5}. Among them, the P3 feature map is output by the basic residual module, and the P4 and P5 feature maps are output by the EP4MCGM feature layer selection module and the CSP-PKINetP5 feature layer selection module, respectively. First, the P5 feature map is input into the HiLo-IFIM branch for internal interaction of high and low frequency features, and the F5 feature map is output. Then, the P3, P4, and F5 feature maps are input into the CFSF-FPN branch, where the P3, P4, and F5 feature maps are cross-scale high and low frequency feature fusion and splicing, and the Memory is output. Using the self-attention mechanism and pyramid network structure, the high and low frequency features within and between scales are effectively combined, which is conducive to improving the performance and accuracy of target detection. Then Memory is input into IoU-aware Query Selection and Decoder&Head at the same time. In IoU-aware Query Selection, the regression and classification loss of the predicted bounding box and the real bounding box corresponding to each feature in Memory is calculated, and the IOU is considered in the classification loss to ensure the consistency of the position confidence and the class confidence output by the network. According to the loss, the Top-K features are selected as the initial input of Decoder&Head, which ensures that the selected object query has good quality, helps to accelerate the convergence of the decoder, and accurately identifies the target in the image. In Decoder&Head, Decoder receives the initial target query selected by IoU-aware Query Selection and the Memory feature sequence output by the hybrid encoder, iteratively optimizes the target query through multiple layers of self-attention and cross-attention, and finally outputs the class and coordinates of the detection box. Head maps the query features output by Decoder to the detection box. Through this end-to-end design, the present application realizes efficient and accurate multi-target detection, and provides an effective solution for real-time beef target detection tasks.

[0048] In the present application:

[0049] The English full name of HLFFESF-RTDETR is Real-Time DETR Based on Efficient Selection and Fusion of High and Low-Frequency Features, and its Chinese meaning is based on high and low frequency feature efficient selection and fusion Real-time DETR.

[0050] The English full name of Real-Time-DETR is Real-Time DEtection TRansformer, and its Chinese meaning is Real-time detection transformer.

[0051] The English full name of EP4MCGM is Efficient P4 based on multi-path cooperative gating mechanism, and its Chinese meaning is Efficient P4 based on multi-path cooperative gating mechanism.

[0052] The English full name of CSPPKINetP5 is P5 feature layer selection module based on the Cross-Stage Partial Poly Kernel Inception Network, and its Chinese meaning is P5 feature layer selection module based on the Cross-Stage Partial Poly Kernel Inception Network.

[0053] The English full name of MPCPKI-ResNet is ResNet Feature Extraction Network Based on Multi-path Cooperative and Poly Kernel Inception, and its Chinese meaning is ResNet Feature Extraction Network Based on Multi-path Cooperative and Poly Kernel Inception.

[0054] The English full name of HiLo-IFIM is Intrascale Feature Interaction Module Based on

[0055] High-Low Frequency, Chinese meaning: Scale interaction module based on high and low frequency self-attention.

[0056] The full name of CFSF-FPN in English is context feature selection fusion pyramid network, and its Chinese meaning is context feature selection fusion pyramid network.

[0057] IoU-aware Query Selection refers to an IoU-aware query selection mechanism.

[0058] Decoder&Head refers to a decoder with an auxiliary prediction head.

[0059] DarkLabel refers to a video and image labeling tool.

[0060] Memory refers to the feature map of an image.

[0061] Top-K refers to a selection strategy that selects only the top K features with the smallest loss at each generation step.

[0062] Decoder refers to a decoder.

[0063] Head refers to an auxiliary prediction head.

[0064] IoU refers to Intersection over Union in English, and its Chinese meaning is intersection over union.

[0065] SIoU, which is SCYLLA-IoU, refers to Scalable Intersection over Union in English, and its Chinese meaning is scalable intersection over union.

[0066] RepConv refers to Reparameterized Convolution.

[0067] PWConv refers to Point-Wise Convolution.

[0068] According to the above technical solutions, the following specific embodiments of the present application are given. It should be noted that the present application is not limited to the following specific embodiments, and any equivalent transformation based on the technical solutions of the present application falls within the scope of protection of the present application.

[0069] Embodiment:

[0070] The present embodiment provides a beef target detection method based on efficient selection and fusion of high and low frequency features, which comprises the following steps:

[0071] Step S1, constructing a beef target detection data set:

[0072] Collect beef cattle videos in five different breeding scenes under real monitoring view, use DarkLabel to label the beef cattle videos, and build a beef cattle target detection dataset, which is named Beef-Cattle36.

[0073] In this step, specifically, videos in five different breeding scenes under real monitoring view are collected, the average duration of these videos is 35 minutes, the MP4 format is adopted, the resolution is 1920 pixels (width) x 1080 pixels (height), and the frame rate is 24 frames / second. In particular, in different scenes of beef cattle breeding, there are various interference detection performance challenges, including the scale change of beef cattle in the video due to the camera hanging on the top corner of the shed, the change of light intensity due to the large video sampling time span, the mutual occlusion between beef cattle, the occlusion caused by the ceiling, fence and other objects, the dust interference caused by the behavior of beef cattle running, and the background interference caused by the fact that the appearance of beef cattle is not as obvious as the appearance of pedestrians. Then, the beef cattle videos are labeled by using DarkLabel. Finally, the application constructs a beef cattle target detection dataset, which is named Beef-Cattle36, including 11936 images as a training set, 1492 images as a test set, and 1492 images as a test set, the ratio of the training set, the validation set and the test set is 8:1:1.

[0074] Step S2, constructing a beef cattle target detection model:

[0075] Step S201, constructing a feature extraction network MPCPKI-ResNet:

[0076] The feature extraction network MPCPKI-ResNet includes a basic residual module, an efficient P4 feature layer selection module based on a multi-path cooperative gating mechanism, and a CSP-PKINetP5 feature layer selection module.

[0077] The input image data is sent into the feature extraction network MPCPKI-ResNet, and the feature extraction network MPCPKI-ResNet generates three levels of feature maps {P3, P4, P5}, wherein: the P3 feature map is output by the basic residual module; the P4 feature map is output by the efficient P4 feature layer selection module based on the multi-path cooperative gating mechanism; and the P5 feature map is output by the CSP-PKINetP5 feature layer selection module.

[0078] In this step, the structure of the efficient P4 feature layer selection module based on the multi-path cooperative gating mechanism is as shown in Figure 2 The input of this module is X P3 , wherein denotes the set of real numbers, C denotes the number of channels, H and W denote 1 / 8 of the height and width of the input image, respectively. It adopts a multi-branch architecture, and the information flow y is denoted as y = m(P3) + r(P3) + x(P3), which represents the three branches of the building block, where m(P3) is the main branch, r(P3) is the residual branch, and x(P3) is the identity mapping branch.

[0079] In the main branch, the present application divides the features obtained by 3x3 convolution along the channel direction, applies RepConv to the 1 / 4 input channels for feature extraction, and keeps the remaining channels unchanged. This method reduces redundant computation and memory access, thereby achieving efficiency. Then the features are input into two PWConv and a multi-path collaborative gating mechanism (MCGM) to more effectively extract feature information. The MCGM includes a spatial (height and width) based gating mechanism and a global context based gating mechanism. The spatial gating mechanism can capture long-range dependencies on the top and bottom and left and right sides of the image, while preserving the precise position information in the vertical and horizontal directions. This helps to capture low-frequency information in the image while preserving high-frequency information. The global gating mechanism considers the overall information of the input feature map and can capture global and extensive low-frequency information. The specific operation is as follows:

[0080] First, the height H and width W of the input feature X are obtained. Then, the input feature X is encoded in the vertical and horizontal directions using dimensions of (1, W) and (H, 1) for each channel of the input feature X. At the same time, the input feature X is globally encoded. The encoding process is as follows:

[0081]

[0082] In the formula:

[0083] h and w represent the height and width of the input X, respectively;

[0084] e n (h, t) is the pixel value of the nth channel at the (h, t) position;

[0085] denotes the encoded output of the nth channel with a height of h;

[0086] e n (k, w) is the pixel value of the nth channel at the (k, w) position;

[0087] is the encoded output of the nth channel with a width of w;

[0088] e n (i, j) is the pixel value of the nth channel at the (i, j) position;

[0089] G(e) is the encoding output of the nth channel with height h and width w;

[0090] and can be regarded as the spatial information of this channel;

[0091] G(e) can be regarded as the global information of this channel.

[0092] Next, the encoding along the channel dimension in the vertical and horizontal directions is spliced, sent to the 3x1 convolution fusion and spatial feature extraction, 1x1 convolution is applied to the extracted features, and the output is converted to a weight between 0 and 1 through the sigmoid function, to obtain:

[0093]

[0094] In the formula:

[0095] cat[·,·] represents the splicing operation along the spatial dimension;

[0096] Conv represents the convolution operation;

[0097] Cat represents the splicing operation;

[0098] ∑ represents the sigmoid activation function;

[0099] gate represents the gating weight in the height and width directions.

[0100] The gating weights gate h / w = split h / ww[gate] and the features in the height and width directions

[0101] In the formula:

[0102] h and w represent the split dimensions;

[0103] gate h represents the gating weight in the height direction;

[0104] gate w represents the gating weight in the width direction;

[0105] V h corresponds to the feature in the height direction;

[0106] V w corresponds to the feature in the width direction.

[0107] The spatial gating mechanism sgm is realized by weighting the features in the height and width directions using the attention weights in the height and width directions respectively. h / w=∑(V h / w ·gate h / w )。

[0108] In the formula:

[0109] denotes element-wise multiplication operation, also known as Hadamard product.

[0110] This means that the elements of the two tensors involved in the Hadamard product at the corresponding positions will be multiplied.

[0111] Perform Mean operation on the gate weights in the height and width directions to obtain the global gate weight, and weight the global encoding G of the input feature X to realize the global gating mechanism ggm=∑(Mean(gate)·G). Finally, the output of the multi-path cooperative gating mechanism of the present application can be written as MCGM=X·sgm h ·sgm ww ·ggm.

[0112] The residual branch contains a 1x1 convolutional layer to learn the conversion of the input feature so as to add it to the output of the main branch to obtain a better feature representation. The identity mapping branch keeps the integrity of the input feature information, enabling the module to selectively learn more complex feature conversion without losing the information of the input feature.

[0113] In this step, the CSP-PKINetP5 feature layer selection module structure is as shown in Figure 3 The overall structure integrates the Cross Stage Partial (CSP) structure, as well as the Poly Kernel Inception (PKI) and Context Anchor Attention (CAA) modules. This design helps to extract target features at various scales and capture local and long-range contexts while reducing computational requirements. The input feature Here H' and W' represent 1 / 16 of the height and width of the input image, respectively. After 3x3 convolution, it is split into two parts S p5′ , S p5″ =split channel [Conv 3×3 (X)], and sent to two paths. Among them, channel is Conv 3×3In (X), the number of channels is 1 / 2, and H″ and W″ represent 1 / 32 of the input image height and width, respectively. One path is a feed-forward network (FFN), and the other is a PKI block. The outputs of the two paths are concatenated and then subjected to a 1x1 convolution operation to obtain the final output. This design enables the model to process information in parallel and combine the results, enhancing its ability to extract and learn complex features.

[0114] o=Conv 1×1 (Cat[FFN(S P5′ ), PKIBlock M (S P5′′ )])

[0115] Where:

[0116] PKIBlock M Indicates that the path contains M PKI blocks. This means that the path will process the input through M PKI blocks in sequence.

[0117] Step S202: construct a hybrid encoder:

[0118] The hybrid encoder consists of an intra-scale interaction module based on high- and low-frequency self-attention and a fusion pyramid network for contextual feature selection.

[0119] The P5 feature map obtained in step S201 is input into the intra-scale interaction module based on high- and low-frequency self-attention for internal interaction of high- and low-frequency features, and the F5 feature map is output.

[0120] The P3 feature map, P4 feature map, and F5 feature map obtained in step S201 are input into the context feature selection fusion pyramid network, and the high- and low-frequency features of the P3 feature map, P4 feature map, and F5 feature map are fused and spliced ​​across scales to output a memory feature sequence.

[0121] Specifically in this step, the scale interaction module based on high and low frequency self-attention is as follows Figure 4 As shown. This module uses a single-layer Transformer encoder and only processes the P5 features generated by step S201. First, the P5 feature map is divided into small blocks of fixed size, each of which is a vector of fixed dimension. Since the Transformer structure cannot capture the sequence order, it is necessary to add position encoding to each block vector to retain the spatial position information. These two parts serve as the input of the entire module, and the HiLo attention mechanism is used for intra-scale feature interaction. Subsequently, the features are further extracted and nonlinear transformations are performed through the feedforward neural network. After self-attention and feedforward neural networks, normalization and residual connection operations are performed to stabilize the training of the model and prevent the gradient from disappearing. The process is as follows:

[0122] F5 = LN[FFN(LN[Linear(HiLo(Q, K, V) + P5)])

[0123] + LN[Linear(HiLo(Q, K, V) + P5)]]

[0124] Q = K = V = Flatten(P5)

[0125] Q = Q + Pos, K = K + Pos

[0126] In the formula:

[0127] Q, K and V represent query, key and value in attention mechanism, respectively;

[0128] Flatten is to convert two-dimensional feature P5 into a vector;

[0129] Pos is a position embedding;

[0130] LN is a normalization layer;

[0131] Linear is a linear output layer;

[0132] FFN is composed of several linear layers and nonlinear activation functions.

[0133] In this step, the context feature selection fusion pyramid network is as shown in Figure 5 It includes four main processes: feature selection, attention weight generation, feature fusion and output splicing.

[0134] Firstly, the present application inputs the feature maps generated in steps S201 and S202 into a feature selection context anchor attention (FSCAA) module for feature selection and weighting. The FSCAA module uses global information and local semantic information of features of different scales to adaptively learn and select features with important semantic information. The structure of the FSCAA module is as shown in Figure 6As shown. For a certain scale of input feature map, the application can use X-gf to select features. In the formula, X e {P3, P4, F5} represents the input features of a certain scale, and gf is the gating factor. After selecting F5 features, the obtained high-level features are input into the CAA module again to convert them into corresponding attention weights. These attention weights are used to filter the necessary semantic information contained in the low-level features, thereby improving the quality and effectiveness of the features. It should be noted that the CAA used here is the same as the CAA in the step S202 CSP-PKINet P5 feature layer selection module, which returns a gating factor. In this process, in order to unify the dimensions of high-level features and low-level features, transpose convolution is used to sample high-level features. The filtered low-level features are added element-wise with the high-level semantic features to obtain the fused feature representation. These features are further processed by the CSPRep module to enhance their representation ability and semantic information. The structure of the CSPRep module is as shown. Figure 7 Finally, the processed feature maps are spliced along the channel direction to output.

[0135] gf = å (Conv 1×1 (DWConv 11×1 (DWConv 1×11 (Conv 1×1 (Avg(I))))))

[0136] In the formula:

[0137] Avg is the average pooling operation;

[0138] DWConv 1×11 and DWConv 11×1 are horizontal and vertical convolution kernel DWConv operations with (1, 11) and (11, 1), respectively.

[0139] Step S203, build an IoU-aware query selection mechanism:

[0140] Calculate the regression and classification loss of each feature in the Memory feature sequence obtained in step S202 corresponding to the predicted bounding box and the real bounding box, and combine IoU in the classification loss to realize the consistency of the position confidence and the class confidence of the network output, and select Top-K features according to the loss.

[0141] Step S204, build a decoder with an auxiliary prediction head:

[0142] The Decoder receives the Top-K initial target queries filtered by the IoU-aware query selection mechanism constructed in step S203 and the Memory feature sequence output by the hybrid encoder constructed in step S202, iteratively optimizes the target queries through multi-layer self-attention and cross-attention, and finally outputs the class and coordinates of the beef cattle detection box.

[0143] In step S204, the Head maps the query features output by the Decoder to the detection box.

[0144] In step S3, the beef cattle target detection model is trained.

[0145] Based on the beef cattle target detection dataset Beef-Cattle36 obtained in step S1, the loss function L used in training the beef cattle target detection model includes two parts: target bounding box regression loss L box and classification loss L cls The loss function L is back-propagated, and the iteration is repeated until the iteration reaches the preset initial value, and the training of the beef cattle target detection model is completed.

[0146] The target bounding box regression loss L box is defined as the weighted sum of SIoU (SCYLLA-IoU) loss and L1 loss:

[0147] The classification loss L cls uses a variable focal loss (VFL, varifocal loss):

[0148] L = L box + L cls

[0149]

[0150] In the formula:

[0151] represents the predicted bounding box;

[0152] b represents the real bounding box;

[0153] L SIoU represents the SIoU loss;

[0154] λ represents a weight parameter for adjusting the proportion of L SIoU loss in L box loss;

[0155] represents a weight parameter for adjusting the proportion of L1 loss in L box loss.

[0156] VFL represents variable focus loss;

[0157] p represents a predicted instance-aware classification score, which is the classification score of each instance predicted by the model;

[0158] q represents a target score;

[0159] Both a and k represent hyperparameters;

[0160] When the target score q > 0, VFL has no hyperparameters for positive samples, that is, there is no any decay;

[0161] When the target score q = 0, VFL has a hyperparameter term for negative samples, where k reduces the contribution of negative samples, and a is to prevent over-suppression.

[0162] Step S4, beef cattle target detection:

[0163] Given the beef cattle image data, input into the beef cattle target detection model trained in step S3, output the class and coordinates of the beef cattle detection box.

[0164] In the present application, the evaluation indicators for measuring detection accuracy are precision, recall, mAP0.5 and mAP0.5:0.95. In addition to measuring detection accuracy, the model's parameter quantity (Params), computational quantity (Gflops), inference time (Inference Time) and FPS (Frame Per Second) indicators are also very important for measuring the overall performance of the model. In addition, the present application uses the TIDE framework to analyze the specific advantages and disadvantages of the modules proposed in the present application from a more fine-grained perspective. The framework divides detection errors into main error types and other error types, where the main error types include six types, which are: Cls, Loc, Both, Dupe, Bkg and Miss. Cls is a classification error, indicating that the detected object is misclassified but correctly positioned. Loc is a positioning error, indicating that the detected object is correctly classified but not accurately positioned. Both is a classification and positioning error, indicating that the detected object is both misclassified and not accurately positioned. Dupe is a duplicate detection error, indicating that the correct object is detected multiple times. Bkg is a background error, indicating that the background is mistaken for a foreground object. Miss is a missed detection error, indicating that the real object is not detected (i.e. false negative). The other error types are two types, FP and FN.

[0165] Comparative Example 1:

[0166] The present comparative example compares the proposed method with 26 known classical advanced target detection methods, including Faster R-CNN (NeurIPS'2015), RetinaNet (ICCV'2017), Cascade R-CNN (CVPR'2018), Libra R-CNN (CVPR'2019), ATSS (CVPR'2020), Double-Head R-CNN (CVPR'2020), GFL (Advances in Neural Information Processing Systems'2020), EfficientDet (CVPR'2020), Dynamic R-CNN (ECCV'2020), DDOD (ACM MM'2021), VarifocalNet (CVPR'2021), TOOD (ICCV'2021), Deformable DETR (ICLR'2021), DAB-DETR (ICLR'2022), DDQ (CVPR'2023), DINO (ICLR'2023), H-DINO (CVF'2023), Align-DETR (arXiv'2023), DiffusionDet (ArXiv'2023), CO-DETR (ICCV'2023), YOLOF, YOLOV3, YOLOX-X, YOLOv5-M, YOLOV6-M, YOLOV8-M. There are 5 evaluation indexes, respectively mAP50, mAP50-95, Params, Gflops and FPS.

[0167] The results of the present comparative example are shown in Table 1. The mAP0.5 of the embodiment of the present application is 90.3%, the mAP0.5-0.95 is 59.6%, the parameter amount is 26.4M, the Gflops is 50.8, and the FPS is 100.3. Among all the other object detection methods listed in Table 1, it performs best in mAP0.5 and mAP0.5-0.95, and has a highly competitive advantage in parameter amount, Gflops and FPS. Specifically, the embodiment of the present application exceeds the highly competitive YOLOv5 and YOLOv8 methods by 0.4%, 0.4% and 0.8%, 0.4% in mAP0.5 and mAP0.5-0.95, respectively. In terms of parameter amount and Gflops, it reduces these indicators by 45.0%, 20.6%, 46.8% and 35.5% compared with YOLOv5 and YOLOv8, respectively. These results show that the embodiment of the present application improves detection performance and significantly reduces model complexity and computational requirements. This combination of high precision, reduced parameter amount and computational efficiency increases the possibility of future deployment and application in resource-constrained environments, laying a solid foundation for robust and reliable performance in real-world scenarios.

[0168] Table 1 compares the embodiment of the present application with other state-of-the-art object detection methods

[0169]

[0170]

[0171] Comparative Example 2

[0172] The present comparative example gives an object detection model, which gradually adds the EP4MCGM module in step S201, the HiLo-IFIM in step S202, and the model lightening scheme, wherein the model lightening scheme includes the CSPPKINetP5 module in step S201, the CFSF-FPN in step S202, and the optimization of the loss function in step S3, to finally form the embodiment. There are 8 evaluation indexes, which are Precision, Recall, mAP50, mAP50-95, Params(M), Gflops, IT(Inference Time, ms / bs=16) and FPS(bs=1), respectively.

[0173] Table 2 shows the results of beef cattle object detection after gradually adding modules

[0174]

[0175]

[0176] The results of the present comparative example are shown in Table 2. The performance of the method of the present example is improved in Precision, Recall, mAP, Params, Gflops compared with Comparative Example 2. The Precision, Recall, mAP50, mAP50-95 are improved by 3.4%, 0.9%, 2.0%, 1.3% respectively. The Params and Gflops are reduced by 31.6% and 10.7% respectively. However, the performance of Inference Time and FPS is only slightly decreased. From the perspective of adding modules, it can be found that after adding EP4MCGM to Comparative Example 2, the Precision, Recall, mAP50, mAP50-95 can be improved by 1.7%, 0.7%, 1.3%, 0.7% respectively. However, the model does not increase in calculation amount but decreases, which indicates that the addition of EP4MCGM module enables the model to more effectively play the role of P4 feature layer and improve the accuracy of target detection. Then, by continuously introducing HiLo-IFIM, the Precision, Recall, mAP50, mAP50-95 of the model are further improved by 1.2%, 0.3%, 0.6%, 0.6% respectively. The calculation amount of the model is slightly increased by 0.1 Gflops compared with the model only adding EP4MCGM module. However, compared with the baseline model, the calculation amount is still reduced, which indicates that HiLo-IFIM can effectively promote the interaction of different frequency features within the scale and improve the performance of the model. In terms of model lightweight scheme, the addition of CSPPKINetP5, CFSF-FPN and SIoU can greatly reduce the parameter amount and calculation amount of the model while improving the Precision and mAP. The Recall, Inference Time and FPS performance only have slight decrease, which indicates that the lightweight scheme can effectively reduce the calculation amount and parameter amount of the model while maintaining the performance of the model. Here, the Inference Time and FPS of the model after +CFSF-FPN and +SioU hardly change, because the loss function is mainly used in the training stage. When the model has been trained and inference is performed, the loss function is no longer in effect. These results show that the proposed modules have a positive impact on the performance of the model to varying degrees, providing an effective technical means for achieving efficient and accurate beef cattle target detection.

[0177] Comparative Example 3

[0178] The present comparative example gives a target detection model, which is the same as the model in Comparative Example 2. The difference between the model and Comparative Example 2 is that the evaluation indicators are quantified by eight indicators of Cls, Loc, Both, Dupe, Bkg, Miss, FP and FN.

[0179] Table 3. The distribution of the target detection errors of the beef cattle after adding the modules step by step

[0180]

[0181] The results of the comparative example are shown in Table 3 and Figure 7 The main error types of Comparative Example 3 and the method of the present example are positioning errors, duplicate detection errors, background errors, and missed detection errors. After adding the modules in the method of the present example, these errors are reduced to varying degrees compared to the baseline, indicating that the added modules significantly improve the detection performance of the model in various aspects and reduce various types of detection errors. From the analysis of adding each module, it can be found that after adding the EP4MCGM module, the Loc, Dupe, Bkg, Miss, FP, and FN values of the model are all reduced, indicating that this module can enhance the positioning, distinguish different targets, distinguish foreground and background, and capture the actual target of the baseline model. After introducing the HiLo-IFIM, the model focuses more on distinguishing foreground and background and capturing the actual target compared to the baseline and the model with only the EP4MCGM module. Continuing to add the CSPPKINetP5 module, the model's ability to reduce various types of detection errors has declined, because as one of the components of the model lightweight scheme, the main role of this module is to reduce the parameter quantity and computational quantity of the model. As shown in Table 2, the addition of the CSPPKINetP5 module reduces the parameter quantity and computational quantity of the model by 24.8% and 7.2%, respectively. It successfully reduces the parameter quantity and computational quantity of the model, making the model more lightweight and more suitable for running on resource-constrained devices. At the same time, the module still maintains a low missed detection error, indicating that it still has strong ability in effectively capturing the actual target. The subsequent CFSF-FPN and SIoU, as the remaining components of the model lightweight scheme, aim to improve the performance of the model while maintaining the parameter quantity and computational quantity of the model after adding the CSPPKINetP5 module. Finally, the method of the present example has a low duplicate detection error rate, background error rate, and false positive rate, indicating that the method of the present example has a significant advantage in reducing duplicate detection errors, background errors, and false positive rates. These improvements enable the model to maintain efficient and accurate detection performance in various complex scenarios, thereby greatly improving the practicality and reliability of the model.

[0182] Comparative Example 4:

[0183] The present comparative example gives a target detection model which is the same as the model in Comparative Example 3. The robustness of the present example method and the comparative example method is compared in different breeding scenarios, heat map situations and in different occlusion situations. The results of the comparative example and the present example method are compared at the same confidence level. The green box represents the correctly detected target, the red box represents the missed target, and the blue box represents the false detection. The area where occlusion occurs is marked with a yellow dashed box.

[0184] The comparative results of the present comparative example in different breeding scenarios are shown in Figure 8 Through these visualizations, the missed and false detection characteristics of Comparative Example 4 are analyzed. The present invention finds that when the target is too small, occluded or in a dark, dark condition, missed detection often occurs. False detection is usually due to positioning, repeated detection and background error, which is caused by the difficulty in distinguishing the appearance of the cattle. The present example method shows significant improvement and its robustness and accuracy can be observed in various scenarios and environments. For example, in an indoor scenario, the present example method can still accurately detect the outline and position of a small target cattle and correctly distinguish the background similar to the color of the cattle. In the case of a target covered by a shadow, the present example method also shows strong anti-interference ability and correctly identifies the target in the shadow.

[0185] The comparative results of the present comparative example in heat map situations are shown in Figure 9 Compared with Comparative Example 4, the heat map of the present example method shows more accurate and concentrated attention distribution. Overall, the present example method extracts more feature representations. This result is consistent with the results in Tables 2 and 3, proving the effectiveness of the present example method in beef cattle detection.

[0186] The comparative results of the present comparative example in different occlusion situations are shown in Figure 10 The present example method demonstrates robustness and practicality in beef cattle detection under occlusion conditions that may occur in real farm scenarios. Through analysis of the results of Comparative Example 4, the present invention also finds that certain types of occlusion, such as the simultaneous occurrence of scene 2 mutual occlusion and ceiling diagonal rod occlusion or scene 5 mutual occlusion and indistinguishable appearance, have a greater impact on performance. The present example method will also fail in these cases, which will guide the present invention to further improve the model in future work to better cope with these challenging situations.

Claims

1. A beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features, characterized in that: The method comprises the following steps: Step S1: Construct a beef cattle target detection dataset: We collected beef cattle videos from different farming scenarios under real monitoring perspectives, annotated the beef cattle videos, and constructed a beef cattle target detection dataset, which we named Beef-Cattle36. Step S2: Build a beef cattle target detection model: Step S201: Construct a feature extraction network MPCPKI-ResNet: The feature extraction network MPCPKI-ResNet includes a basic residual module, an efficient P4 feature layer selection module based on a multi-path collaborative gating mechanism, and a CSPPKINetP5 feature layer selection module; The input image data is fed into the feature extraction network MPCPKI-ResNet, which generates feature maps at three levels: {P3, P4, and P5}. The P3 feature map is output by the basic residual module; the P4 feature map is output by the efficient P4 feature layer selection module based on the multi-path collaborative gating mechanism; and the P5 feature map is output by the CSP-PKINet P5 feature layer selection module. Step S202: construct a hybrid encoder: The hybrid encoder includes an intra-scale interaction module based on high- and low-frequency self-attention and a fusion pyramid network for contextual feature selection; The P5 feature map obtained in step S201 is input into the intra-scale interaction module based on high- and low-frequency self-attention to perform internal interaction of high- and low-frequency features, and output the F5 feature map; The P3 feature map, P4 feature map, and F5 feature map obtained in step S201 are input into the context feature selection fusion pyramid network, and cross-scale high- and low-frequency features of the P3 feature map, P4 feature map, and F5 feature map are fused and spliced ​​to output a memory feature sequence; Step S203: Construct an IoU-aware query selection mechanism: Calculate the regression and classification losses of the predicted bounding box and the true bounding box corresponding to each feature in the memory feature sequence obtained in step S202, and combine the IoU in the classification loss to achieve consistency between the position confidence and category confidence of the network output, and select the top-K features based on the loss; Step S204: Construct a decoder with an auxiliary prediction head: The decoder receives the IoU-aware query selection mechanism constructed in step S203, filters out the top-K initial target queries and the memory feature sequence output by the hybrid encoder constructed in step S202, it iteratively optimizes the target query through multi-layer self-attention and cross-attention, and finally outputs the category and coordinates of the beef cattle detection box; Step S3: training the beef cattle target detection model: Based on the beef cattle target detection dataset Beef-Cattle36 obtained in step S1, when training the beef cattle target detection model, the loss function L used includes the target bounding box regression loss L box and classification loss L cls There are two parts: back-propagation of the loss function L, and repeated iterations until the number of iterations reaches the preset initial value to complete the training of the beef cattle target detection model; Step S4, beef cattle target detection: Given beef cattle image data, input it into the beef cattle target detection model trained in step S3, and output the category and coordinates of the beef cattle detection frame.

2. The beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features according to claim 1 is characterized in that: In step S1, the different breeding scenarios are 5 different breeding scenarios.

3. The beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features according to claim 1 is characterized in that: In step S1, the beef cattle video is labeled using DarkLabel.

4. The beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features according to claim 1, characterized in that: In step S3, the target bounding box regression loss L box It is defined as the weighted sum of SIoU loss and L1 loss.

5. The beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features according to claim 4 is characterized in that: In step S3, the classification loss L cls A variable focus loss is used.

6. The beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features according to claim 5, characterized in that: In step S3: L=L box +L cls Where: represents the predicted bounding box; b represents the ground-truth bounding box; L SIoU represents SIoU loss; λ represents the weight parameter, which is used to adjust L SIoU Loss in L box proportion of losses; Represents the weight parameter, which is used to adjust the L1 loss in L box proportion of losses; VFL stands for variable focus loss; p represents the predicted instance-aware classification score, which is the classification score for each instance predicted by the model; q represents the target score; Both α and κ represent hyperparameters.

Images