Beef cattle target detection method based on high and low frequency feature efficient selection fusion

By adopting the high-low-frequency feature selection fusion method in beef cattle target detection, combining the multi-path collaborative gating mechanism and the high-low-frequency self-attention module, the problem of insufficient target detection accuracy in beef cattle in complex environments is solved, and efficient and accurate beef cattle detection is achieved , reduces model complexity and computing requirements.

CN118968411AActive Publication Date: 2024-11-15NORTHWEST A & F UNIV

Patent Information

Application Number
CN202411029906.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-15
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The accuracy of beef cattle target detection in complex environments needs to be improved, especially in the case of occlusion, shadow and appearance, it is difficult to achieve high-precision, multi-category, and multi-objective real-time detection.

Method used

Using the beef cattle target detection method based on high and low frequency features, the feature extraction network MPCPKI-ResNet, hybrid encoder and IoU perceived query selection mechanism is constructed, combining multi-path collaborative gating mechanism, high and low frequency self-attention module and context features Select a fusion pyramid network to achieve efficient and accurate beef cattle target detection.

Benefits of technology

The performance of beef cattle target detection is significantly improved in complex environments, and can detect beef cattle efficiently and accurately, reducing the number of model parameters and calculation load, and improving the deployability of the model and the performance of detection tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968411A_ABST
    Figure CN118968411A_ABST
Patent Text Reader

Abstract

The invention provides a beef cattle target detection method based on high and low frequency feature efficient selection fusion. The method comprises the following steps: S1, constructing a beef cattle target detection data set; step S2, constructing a beef cattle target detection model; step S201, constructing a feature extraction network MPCPKI-ResNet; step S202, constructing a hybrid encoder; step S203, constructing an IoU perception query selection mechanism; step S204, constructing a decoder with an auxiliary prediction head; s3, training a beef cattle target detection model; and S4, beef cattle target detection. According to the method, beef cattle can be efficiently and accurately detected in different scenes and complex environments, and the performance of real-time beef cattle target detection is remarkably improved. The method is specially designed for real-time beef cattle target detection. According to the method, various low-level features extracted by a shallow network are effectively integrated through an efficient P4 feature layer selection module of a multi-path collaborative gating mechanism, the efficiency of feature fusion is improved, and the capturing capability of the model on image detail information is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of video image understanding, relates to beef cattle target detection, and specifically relates to a beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features. Background Art

[0002] The beef cattle target detection task is the basis for monitoring the beef cattle breeding process in a scientific way. The identification and positioning of beef cattle targets is conducive to the fine management of large-scale beef cattle breeding, and is of great significance for the intelligent perception of individual behavior analysis and health status of beef cattle. Over the past few decades, foreign scholars have made various attempts to apply target detection technology in the field of animal husbandry, such as target detection algorithms based on traditional image analysis, target detection algorithms based on statistical theory, and target detection algorithms based on deep learning.

[0003] The target detection algorithms based on traditional image processing theory include threshold segmentation method, frame difference method, etc. However, traditional detection algorithms have various limitations and cannot meet the application requirements in complex real-world scenarios. Different from the algorithms based on traditional image processing theory, the target detection algorithms based on statistical theory use mathematical statistical methods and probability theory to establish dynamic background models, and adaptively adjust the model parameters according to the current environment, so as to continuously follow the changes in the environment. Representative algorithms include feature matching methods, background difference, and Gaussian mixture models. However, traditional statistical theory algorithms usually rely on manually selected features such as color, texture, and shape. These features have limited expressive power and may not be able to fully capture the complex patterns in the image. With the development of computer vision technology, target detection algorithms based on deep learning provide new solutions for the identification and positioning of farmed animals in complex scenarios. These algorithms have significantly improved compared with traditional detection models in terms of accuracy and real-time performance, and have become the mainstream in the field of target detection. Although target detection algorithms based on deep learning have obvious improvements in accuracy and real-time performance compared with traditional algorithms, they mostly solve detection problems under individual and sparse group distribution. There are still challenges in the actual detection environment. There are many obstacles in the real farm environment, such as dense, occluded and complex backgrounds, which may affect the accuracy and robustness of the detection system. How to make the target detection algorithm still achieve high-precision real-time detection of multiple categories and multiple targets in the case of changing environment, complex background and dense occlusion is an extremely important research issue. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features, so as to solve the technical problem that the detection accuracy of the target detection method in the prior art in complex environments needs to be further improved.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:

[0006] A method for detecting beef cattle targets based on efficient selection and fusion of high-frequency and low-frequency features, the method comprising the following steps:

[0007] Step S1, constructing a beef cattle target detection dataset:

[0008] Beef cattle videos of different breeding scenes under real monitoring perspectives are collected, the beef cattle videos are annotated, and a beef cattle target detection dataset is constructed. The beef cattle target detection dataset is named Beef-Cattle36.

[0009] Step S2, constructing a beef cattle target detection model:

[0010] Step S201, constructing a feature extraction network MPCPKI-ResNet:

[0011] The feature extraction network MPCPKI-ResNet includes a basic residual module, an efficient P4 feature layer selection module based on a multi-path collaborative gating mechanism, and a CSPPKINetP5 feature layer selection module.

[0012] The input image data is sent to the feature extraction network MPCPKI-ResNet, and the feature extraction network MPCPKI-ResNet generates three levels of feature maps {P3, P4, P5}, among which: the P3 feature map is output by the basic residual module; the P4 feature map is output by the efficient P4 feature layer selection module based on the multi-path collaborative gating mechanism; the P5 feature map is output by the CSP-PKINet P5 feature layer selection module.

[0013] Step S202, constructing a hybrid encoder:

[0014] The hybrid encoder includes an intra-scale interaction module based on high- and low-frequency self-attention and a contextual feature selection fusion pyramid network.

[0015] The P5 feature map obtained in step S201 is input into the intra-scale interaction module based on high- and low-frequency self-attention for internal interaction of high- and low-frequency features, and the F5 feature map is output.

[0016] The P3 feature map, P4 feature map and F5 feature map obtained in step S201 are input into the context feature selection fusion pyramid network, and the high and low frequency features of the P3 feature map, P4 feature map and F5 feature map are fused and spliced ​​across scales to output a Memory feature sequence.

[0017] Step S203: construct an IoU-aware query selection mechanism:

[0018] The regression and classification losses of the predicted bounding box and the true bounding box corresponding to each feature in the memory feature sequence obtained in step S202 are calculated, and the IoU is combined in the classification loss to achieve the consistency of the position confidence and category confidence output by the network, and the Top-K features are selected according to the loss.

[0019] Step S204, construct a decoder with an auxiliary prediction head:

[0020] The Decoder receives the IoU-aware query selection mechanism constructed in step S203, selects the Top-K initial target queries and the memory feature sequence output by the hybrid encoder constructed in step S202, iteratively optimizes the target query through multi-layer self-attention and cross-attention, and finally outputs the category and coordinates of the beef cattle detection box.

[0021] Step S3, training the beef cattle target detection model:

[0022] Based on the beef cattle target detection dataset Beef-Cattle36 obtained in step S1, when training the beef cattle target detection model, the loss function L used includes the target bounding box regression loss L box and classification loss L cls There are two parts: back propagating the loss function L and repeating the iteration until the number of iterations reaches the preset initial value to complete the training of the beef cattle target detection model.

[0023] Step S4, beef cattle target detection:

[0024] Given beef cattle image data, the data is input into the beef cattle target detection model trained in step S3, and the category and coordinates of the beef cattle detection frame are output.

[0025] The present invention also has the following technical features:

[0026] In step S1, the different breeding scenarios are 5 different breeding scenarios.

[0027] In step S1, the beef cattle video is labeled using DarkLabel.

[0028] In step S3, the target bounding box regression loss L box It is defined as the weighted sum of SIoU loss and L1 loss.

[0029] In step S3, the classification loss L cls A variable focus loss is used.

[0030] Compared with the prior art, the present invention has the following technical effects:

[0031] (I) The present invention can efficiently and accurately detect beef cattle in different scenarios and complex environments (such as occlusion, shadows and similar appearance), significantly improving the performance of real-time beef cattle target detection.

[0032] (II) The present invention is specially designed for real-time beef cattle target detection. The method of the present invention effectively integrates various low-level features extracted by shallow networks through an efficient P4 feature layer selection module with a multi-path collaborative gating mechanism, improves the efficiency of feature fusion, and enhances the model's ability to capture image detail information.

[0033] (III) The present invention takes into account both the global information and detail features of the image through an intra-scale interaction module based on high and low frequency self-attention, so that the model can more accurately handle target detection tasks in complex scenes; furthermore, considering the size of the model, calculation and performance requirements in target detection tasks, the CSP-PKINetP5 feature layer selection module is used to reduce the number of model parameters and the amount of calculation, and the pyramid network is fused through context feature selection and the SCYLLA-IoU loss function is introduced to work together with the CSP-PKINetP5 feature layer selection module, which effectively reduces the number of model parameters and the computational load, improves the deployability of the model, and maintains the performance in target detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a structural diagram of a beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features.

[0035] Figure 2 It is a structural diagram of the efficient P4 feature layer selection module based on the multi-path collaborative gating mechanism of the present invention.

[0036] Figure 3 It is a structural diagram of the feature layer selection module based on CSP-PKINetP5 of the present invention.

[0037] Figure 4 It is a structural diagram of the intra-scale interaction module based on high and low frequency self-attention of the present invention.

[0038] Figure 5 It is a structural diagram of the feature selection context anchor attention of the present invention.

[0039] Figure 6 It is a structural diagram of the context feature selection fusion pyramid network of the present invention.

[0040] Figure 7 It is a distribution diagram of error types detected by each module in the embodiment gradually added to the comparative example.

[0041] Figure 8 It is a comparison chart of the results of the method in the embodiment and the comparative example in different breeding scenarios.

[0042] Fig. 9 It is a comparison chart of the thermodynamic results of the method in the embodiment and the comparative example.

[0043] Fig.10 It is a comparison diagram of the robustness of the method in the embodiment and the comparative example under different occlusion conditions.

[0044] The specific contents of the present invention are further explained in detail below in conjunction with embodiments. DETAILED DESCRIPTION

[0045] It should be noted that, unless otherwise specified, all algorithms, modules and networks in the present invention adopt algorithms, modules and networks known in the prior art.

[0046] In order to meet the needs of beef cattle target detection in the real breeding process, a real beef cattle target detection dataset named "Beef-Cattle36" was first collected to address the problems of changeable farm environment, complex background, dense occlusion, etc. The dataset was captured in different scenes and includes beef cattle detection under various interference challenges, such as occlusion, shadow, similar appearance, etc. This is more in line with the real cattle farm environment, where cattle are raised collectively and cattle are often not captured in the center of the image by the camera. Based on the real beef cattle target detection dataset, this paper proposes a real-time DETR based on efficient selection and fusion of high and low frequency features (HLFFESF-RTDETR), which is designed for real-time beef cattle target detection. First, in order to more effectively integrate various low-level features extracted by shallow networks, this paper proposes an efficient P4 feature layer selection module (Efficient P4 based on multi-path cooperative gating mechanism, EP4MCGM) based on multi-path cooperative gating mechanism. Secondly, an intrascale feature interaction module based on high-low frequency self-attention (HiLo-IFIM) is proposed to enable it to process the global information of the image while taking into account the underlying and detailed features. Finally, in order to improve the deployability of the model, the present invention proposes a model lightweight solution, in which a feature layer selection module based on CSPPKINetP5 (P5 feature layer selection module based on the Cross-Stage Partial Poly Kernel InceptionNetwork) is proposed to reduce the number of parameters and the amount of calculation of the model, and a context feature selection fusion pyramid network (CFSF-FPN) is proposed and the SCYLLA-IoU loss function is introduced, which aims to work with the CSPPKINetP5 feature layer selection module to reduce the model size and the amount of calculation while ensuring the performance of the model in the target detection task.

[0047] Furthermore, the beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features of the present invention can also be called a beef cattle target detection method based on Real-Time-DETR and efficient selection and fusion of high- and low-frequency features. The method of the present invention is composed of multiple key components, including a feature extraction network, a hybrid encoder, an IoU-aware query selection mechanism, and a decoder with an auxiliary prediction head. The feature extraction network MPCPKI-ResNet is composed of a basic residual module, an efficient P4 feature layer selection module based on a multi-path collaborative gating mechanism, and a feature layer selection module based on CSPPKINetP5. The hybrid encoder includes an intra-scale interaction module based on high- and low-frequency self-attention and a contextual feature selection fusion pyramid network. The structure is as follows Figure 1 As shown. First, the input beef cattle image data is sent to MPCPKI-ResNet to generate feature maps at three levels {P3, P4, P5}. Among them, the P3 feature map is output by the basic residual module, and the P4 and P5 feature maps are output by the EP4MCGM feature layer selection module and the CSP-PKINetP5 feature layer selection module respectively. First, the P5 feature map is input to the HiLo-IFIM branch for internal interaction of high and low frequency features, and the F5 feature map is output. Then the P3, P4, and F5 feature maps are input to the CFSF-FPN branch. In the CFSF-FPN branch, the P3, P4, and F5 feature maps are fused and spliced ​​across scales of high and low frequency features, and the Memory is output. The self-attention mechanism and pyramid network structure are used to effectively combine the high and low frequency features within and between scales, which is conducive to improving the performance and accuracy of target detection. After that, the memory is simultaneously input into the IoU-aware Query Selection and the Decoder&Head. In the IoU-aware Query Selection, the regression and classification losses of the predicted bounding box and the true bounding box corresponding to each feature in the memory are calculated, and the IOU is considered in the classification loss to ensure the consistency of the position confidence and category confidence of the network output. According to the loss, the Top-K features are selected as the initial input of the Decoder&Head to ensure that the selected object query has good quality, which helps to accelerate the convergence of the decoder and accurately identify the target in the image. In the Decoder&Head, the Decoder receives the initial target query and the memory feature sequence output by the hybrid encoder filtered by the IoU-aware Query Selection, and iteratively optimizes the target query through multi-layer self-attention and cross-attention, and finally outputs the category and coordinates of the detection box. The Head maps the query features output by the Decoder to the detection box. Through this end-to-end design, the present invention realizes efficient and accurate multi-target detection, and provides an effective solution for the real-time target detection task of beef cattle.

[0048] In the present invention:

[0049] The full name of HLFFESF-RTDETR is Real-Time DETR Based on Efficient Selection and Fusion of High and Low-Frequency Features, which means real-time DETR based on efficient selection and fusion of high and low-frequency features.

[0050] The full name of Real-Time-DETR is Real-Time DEtection TRansformer, which means real-time detection converter in Chinese.

[0051] EP4MCGM stands for Efficient P4 based on multi-path cooperative gating mechanism.

[0052] The full English name of CSPPKINetP5 is P5 feature layer selection module based on the Cross-Stage Partial Poly Kernel Inception Network, which means P5 feature layer selection module based on the cross-stage partial polynomial core Inception network in Chinese.

[0053] The full English name of MPCPKI-ResNet is ResNet Feature Extraction Network Based on Multi-path Cooperative and Poly Kernel Inception, which means ResNet feature extraction network based on multi-path collaboration and polynomial core Inception in Chinese.

[0054] The full name of HiLo-IFIM is Intrascale Feature Interaction Module Based on

[0055] High-Low Frequency, in Chinese, means an in-scale interaction module based on high- and low-frequency self-attention.

[0056] The full name of CFSF-FPN in English is context feature selection fusion pyramid network, which means context feature selection fusion pyramid network in Chinese.

[0057] The IoU-aware query selection mechanism refers to IoU-aware Query Selection.

[0058] The decoder with auxiliary prediction head is referred to as Decoder&Head.

[0059] DarkLabel refers to a video and image annotation tool.

[0060] Memory refers to the feature map of the image.

[0061] Top-K refers to a selection strategy that selects only the top K features with the smallest loss at each generation step.

[0062] Decoder refers to the decoder.

[0063] Head refers to the auxiliary prediction head.

[0064] The full name of IoU is Intersection over Union, which means intersection over union in Chinese.

[0065] SIoU, also known as SCYLLA-IoU, is the full name of Scalable Intersection over Union in English, and its Chinese meaning is scalable intersection over union ratio.

[0066] RepConv refers to Reparameterized Convolution.

[0067] PWConv refers to Point-Wise Convolution.

[0068] In accordance with the above technical scheme, specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical scheme of this application fall within the protection scope of the present invention.

[0069] Example:

[0070] This embodiment provides a method for detecting beef cattle targets based on efficient selection and fusion of high-frequency and low-frequency features, and the method comprises the following steps:

[0071] Step S1, constructing a beef cattle target detection dataset:

[0072] Beef cattle videos of five different breeding scenarios under real monitoring perspectives were collected, and the beef cattle videos were annotated using DarkLabel to construct a beef cattle target detection dataset, which was named Beef-Cattle36.

[0073] Specifically in this step, videos of 5 different breeding scenes under the real monitoring perspective are collected. The average duration of these videos is 35 minutes, using MP4 format, with a resolution of 1920 pixels (width) × 1080 pixels (height) and a frame rate of 24 frames per second. In particular, in different scenes of beef cattle breeding, there are various challenges in interference detection performance, including the scale change of beef cattle in the video due to the camera hanging at the top corner of the shed, the change of light intensity caused by the large time span of video sampling, the mutual occlusion between beef cattle, the occlusion caused by external objects such as ceilings and fences, the dust interference caused by the running of beef cattle, and the background interference caused by the color of the mud ground similar to the body color of beef cattle due to the appearance of beef cattle not being as obvious as pedestrians. Afterwards, the present invention uses DarkLabel to annotate the beef cattle video. Finally, the present invention constructed a beef cattle target detection dataset, named Beef-Cattle36, which includes 11936 images as training sets, 1492 images as test sets, and 1492 images as test sets. The ratio of training set, validation set and test set is 8:1:1.

[0074] Step S2, constructing a beef cattle target detection model:

[0075] Step S201, constructing a feature extraction network MPCPKI-ResNet:

[0076] The feature extraction network MPCPKI-ResNet includes a basic residual module, an efficient P4 feature layer selection module based on a multi-path collaborative gating mechanism, and a P5 feature layer selection module based on CSPPKINet.

[0077] The input image data is sent to the feature extraction network MPCPKI-ResNet, and the feature extraction network MPCPKI-ResNet generates three levels of feature maps {P3, P4, P5}, among which: the P3 feature map is output by the basic residual module; the P4 feature map is output by the efficient P4 feature layer selection module based on the multi-path collaborative gating mechanism; the P5 feature map is output by the CSP-PKINetP5 feature layer selection module.

[0078] Specifically in this step, the structure of the efficient P4 feature layer selection module based on the multi-path collaborative gating mechanism is as follows: Figure 2 The input of this module is X P3 ,in Represents a set of real numbers, C represents the number of channels, H and W represent 1 / 8 of the height and width of the input image, respectively. It adopts a multi-branch architecture, and the information flow y is expressed as y=m(P3)+r(P3)+x(P3), which represent the three branches of the building block, where m(P3) is the main branch, r(P3) is the residual branch, and x(P3) is the identity mapping branch.

[0079] In the main branch, the present invention divides the features obtained by 3×3 convolution along the channel direction, applies RepConv to 1 / 4 of the input channels for feature extraction, and keeps the remaining channels unchanged. This method reduces redundant calculations and memory accesses, thereby achieving efficiency. The features are then input into two PWConvs and a multi-path collaborative gate mechanism (MCGM) to more efficiently extract feature information. MCGM includes a gating mechanism based on space (height and width) and a gating mechanism based on global context. The spatial gating mechanism can capture long-range dependencies between the upper and lower and left and right sides of the image while retaining precise position information in the vertical and horizontal directions. This helps capture low-frequency information in the image while retaining high-frequency information. The global gating mechanism takes into account the overall information of the input feature map and can capture global and wide-range low-frequency information. The specific operations are as follows:

[0080] First, obtain the height H and width W of the input feature X. Then, use the size (1, W) and (H, 1) to encode each channel of the input feature X vertically and horizontally. At the same time, the input feature X is globally encoded. The encoding process is:

[0081]

[0082] Where:

[0083] h and w represent the height and width of input X respectively;

[0084] e n (h, t) is the pixel value of the nth channel at position (h, t);

[0085] represents the encoded output of the nth channel with height h;

[0086] e n (k, w) is the pixel value of the nth channel at position (k, w);

[0087] is the encoded output of the nth channel with width w;

[0088] e n (i, j) is the pixel value of the nth channel at position (i, j);

[0089] G(e) is the encoded output of the nth channel with height h and width w;

[0090] and It can be regarded as the spatial information of this channel;

[0091] G(e) can be regarded as the global information of this channel.

[0092] Next, the vertical and horizontal codes are concatenated along the channel dimension and sent to the 3x1 convolution for fusion and extraction of spatial features. A 1x1 convolution is applied to the extracted features, and the output is converted to a weight between 0 and 1 through the sigmoid function to obtain:

[0093]

[0094] Where:

[0095] cat[·,·] represents the concatenation operation along the spatial dimension;

[0096] Conv stands for convolution operation;

[0097] Cat represents the splicing operation;

[0098] ∑ represents the sigmoid activation function;

[0099] gate represents the gating weight in the height and width directions.

[0100] Use the split operation to obtain the gate weights gate separately h / w=split h / ww[gate] and the characteristics of height and width

[0101] Where:

[0102] h and w represent the dimensions of segmentation;

[0103] gate h represents the gating weight in the height direction;

[0104] gate w represents the gating weight in the width direction;

[0105] V h Corresponding to the features in the height direction;

[0106] V w Corresponds to the features in the width direction.

[0107] The attention weights in the height and width directions are used to weight the features in the height and width directions respectively, thereby realizing the spatial gating mechanism SGM. h / w=∑(V h / w ·gate h / w ).

[0108] Where:

[0109] Represents the element-wise multiplication operation, also known as the Hadamard product.

[0110] This means that elements of the two tensors involved in the Hadamard product at corresponding positions will be multiplied.

[0111] The gating weights in the height and width directions are operated with Mean to obtain the global gating weight, and the global encoding G of the input feature X is weighted to realize the global gating mechanism ggm=∑(Mean(gate)·G). Finally, the output of the multi-path collaborative gating mechanism of the present invention can be written as MCGM=X·sgm h ·sgm ww ·ggm.

[0112] The residual branch contains a 1×1 convolutional layer to learn the transformation of the input features so that they can be added to the output of the main branch for better feature representation. The identity mapping branch maintains the integrity of the input feature information, enabling the module to selectively learn more complex feature transformations without losing information about the input features.

[0113] Specifically in this step, the CSP-PKINetP5 feature layer selection module structure is as follows Figure 3 As shown in the figure. The overall structure combines the Cross Stage Partial (CSP) structure, as well as the Poly Kernel Inception (PKI) and Context Anchor Attention (CAA) modules. This design helps to extract target features at various scales and capture local and long-range contexts while reducing computational requirements. Input features Here H' and W' represent 1 / 16 of the height and width of the input image, respectively. After 3x3 convolution, it is split into two parts along the channel dimension S p5′ , S p5″ =split channel [Conv 3×3 (X)], And is sent into two paths. Among them, channel is Conv 3×3In (X), H″ and W″ represent 1 / 32 of the input image height and width, respectively. One path is the Feed-Forward Network (FFN), and the other is the PKI block. The outputs of the two paths are concatenated and then a 1x1 convolution operation is performed to obtain the final output. This design enables the model to process information in parallel and combine the results, enhancing its ability to extract and learn complex features.

[0114] o=Conv 1×1 (Cat[FFN(S P5′ ), PKIBlock M (S P5′′ )])

[0115] Where:

[0116] PKIBlock M Indicates that the path contains M PKI blocks. This means that the path will process the input through M PKI blocks in sequence.

[0117] Step S202, constructing a hybrid encoder:

[0118] The hybrid encoder contains an intra-scale interaction module based on high- and low-frequency self-attention and a fusion pyramid network for contextual feature selection.

[0119] The P5 feature map obtained in step S201 is input into the intra-scale interaction module based on high- and low-frequency self-attention for internal interaction of high- and low-frequency features, and the F5 feature map is output.

[0120] The P3 feature map, P4 feature map and F5 feature map obtained in step S201 are input into the context feature selection fusion pyramid network, and the high and low frequency features of the P3 feature map, P4 feature map and F5 feature map are fused and spliced ​​across scales to output a Memory feature sequence.

[0121] Specifically, in this step, the scale interaction module based on high- and low-frequency self-attention is as follows Figure 4 As shown. This module uses a single-layer Transformer encoder and only processes the P5 features generated by step S201. First, the P5 feature map is divided into small blocks of fixed size, each of which is a vector of fixed dimension. Since the Transformer structure cannot capture the sequence order, it is necessary to add position encoding to each block vector to retain the spatial position information. These two parts serve as the input of the entire module, and the HiLo attention mechanism is used for intra-scale feature interaction. Subsequently, features are further extracted and nonlinearly transformed through a feedforward neural network. After self-attention and feedforward neural networks, normalization and residual connection operations are performed to stabilize the training of the model and prevent gradient disappearance. The process is as follows:

[0122] F5=LN[FFN(LN[Linear(HiLo(Q,K,V)+P5)])

[0123] +LN[Linear(HiLo(Q,K,V)+P5)]]

[0124] Q=K=V=Flatten(P5)

[0125] Q=Q+Pos,K=K+Pos

[0126] Where:

[0127] Q, K, and V represent the query, key, and value in the attention mechanism, respectively;

[0128] Flatten converts the two-dimensional feature P5 into a vector;

[0129] Pos is the position embedding;

[0130] LN is the normalization layer;

[0131] Linear is the linear output layer;

[0132] FFN is composed of several linear layers and non-linear activation functions.

[0133] Specifically in this step, the context feature selection fusion pyramid network is as follows Figure 5 As shown in Figure 2, it includes four main processes: feature selection, attention weight generation, feature fusion and output concatenation.

[0134] First, the present invention inputs the feature graphs generated in steps S201 and S202 into the Feature Selection Contextual Anchor Attention (FSCAA) module for feature selection and weighting. The FSCAA module uses the global information and local semantic information of feature graphs of different scales to adaptively learn and select features with important semantic information. The structure of the FSCAA module is as follows: Figure 6As shown. For an input feature map of a certain scale, the present invention can use X·gf for feature selection. In the formula, X∈{P3, P4, F5} represents an input feature of a certain scale, and gf is a gating factor. After selecting the F5 feature, the obtained high-level features are input into the CAA module again to convert them into corresponding attention weights. These attention weights are used to filter the necessary semantic information contained in the low-level features, thereby improving the quality and effectiveness of the features. It should be noted that the CAA used here is the same as the CAA in the CSP-PKINetP5 feature layer selection module in step S202, which returns a gating factor. In this process, in order to unify the dimensions of high-level features and low-level features, transposed convolution is used to sample high-level features. The filtered low-level features are element-wise added to the high-level semantic features to obtain a fused feature representation. These features are further processed by the CSPRep module to enhance their representation capabilities and semantic information. The structure of the CSPRep module is shown in Figure 7 Finally, the processed feature maps are concatenated along the channel direction for output.

[0135] gf=∑(Conv 1×1 (DWConv 11×1 (DWConv 1×11 (Conv 1×1 (Avg(I))))))

[0136] Where:

[0137] Avg is the average pooling operation;

[0138] DWConv 1×11 and DWConv 11×1 The horizontal and vertical convolution kernels DWConv operations are (1, 11) and (11, 1) respectively.

[0139] Step S203: construct an IoU-aware query selection mechanism:

[0140] The regression and classification losses of the predicted bounding box and the true bounding box corresponding to each feature in the memory feature sequence obtained in step S202 are calculated, and the IoU is combined in the classification loss to achieve the consistency of the position confidence and category confidence output by the network, and the Top-K features are selected according to the loss.

[0141] Step S204, construct a decoder with an auxiliary prediction head:

[0142] The Decoder receives the IoU-aware query selection mechanism constructed in step S203, selects the Top-K initial target queries and the memory feature sequence output by the hybrid encoder constructed in step S202, iteratively optimizes the target query through multi-layer self-attention and cross-attention, and finally outputs the category and coordinates of the beef cattle detection box.

[0143] In step S204, the Head maps the query features output by the Decoder to the detection box.

[0144] Step S3, training the beef cattle target detection model:

[0145] Based on the beef cattle target detection dataset Beef-Cattle36 obtained in step S1, when training the beef cattle target detection model, the loss function L used includes the target bounding box regression loss L box (Bounding Box Regeression Loss) and classification loss L cls The loss function L is back-propagated into two parts, and the training of the beef cattle target detection model is completed when the number of iterations reaches the preset initial value.

[0146] Object bounding box regression loss L box It is defined as the weighted sum of SIoU (SCYLLA-IoU) loss and L1 loss:

[0147] Classification loss L cls The variable focus loss (VFL) is used:

[0148] L=L box +L cls

[0149]

[0150] Where:

[0151] represents the predicted bounding box;

[0152] b represents the ground-truth bounding box;

[0153] L SIoU represents SIoU loss;

[0154] λ represents the weight parameter, which is used to adjust L SIoU Loss in L box proportion of losses;

[0155] Represents the weight parameter, which is used to adjust the L1 loss in L box proportion of losses;

[0156] VFL stands for variable focal loss;

[0157] p represents the predicted instance-aware classification score, which is the classification score for each instance predicted by the model;

[0158] q represents the target score;

[0159] Both α and κ represent hyperparameters;

[0160] When the target score q>0, VFL has no hyperparameters for positive samples, that is, there is no attenuation;

[0161] When the target score q = 0, VFL has hyperparameters for negative samples, where k reduces the contribution of negative samples and α is to prevent over-suppression.

[0162] Step S4, beef cattle target detection:

[0163] Given beef cattle image data, the data is input into the beef cattle target detection model trained in step S3, and the category and coordinates of the beef cattle detection frame are output.

[0164] In the present invention, the evaluation indicators for measuring detection accuracy are accuracy (Precision), recall rate (Recall), mAP0.5 and mAP0.5:0.95. In addition to measuring detection accuracy, the model's parameter quantity (Params), computational quantity (Gflops), inference time (Inference Time) and FPS (Frame Per Second) indicators are also very important for measuring the overall performance of the model. In addition, the present invention uses the TIDE framework to analyze the specific advantages and disadvantages of the module proposed in the present invention from a more fine-grained perspective. The framework divides detection errors into main error types and other error types, among which the main error types include six types, namely: Cls, Loc, Both, Dupe, Bkg and Miss. Cls is a classification error, indicating that the detected object is misclassified but correctly positioned. Loc is a positioning error, indicating that the detected object is correctly classified but inaccurately positioned. Both is a classification and positioning error, indicating that the detected object is both misclassified and inaccurately positioned. Dupe is a repeated detection error, indicating that the correct object is detected multiple times. Bkg is a background error, indicating that the background is mistaken for a foreground object. Miss is a missed detection error, which means that the real object cannot be detected (i.e., false negative). There are two other error types, FP and FN.

[0165] Comparative Example 1:

[0166] This comparison compares the proposed method with 26 known classic advanced object detection methods. The specific comparison methods are Faster R-CNN(NeurIPS'2015), RetinaNet(ICCV'2017), Cascade R-CNN(CVPR'2018), Libra R-CNN(CVPR'2019), ATSS(CVPR'2020), Double-Head R-CNN(CVPR'2020), GFL(Advances in Neural Information Processing Systems'2020), EfficientDet(CVPR'2020), Dynamic R-CNN(ECCV'2020), DDOD(ACM MM'2021), VarifocalNet(CVPR'2021), TOOD(ICCV'2021), Deformable DETR (ICLR'2021), DAB-DETR (ICLR'2022), DDQ (CVPR'2023), DINO (ICLR'2023), H-DINO (CVF'2023), Align-DETR (arXiv'2023), DiffusionDet (ArXiv'2023), CO-DETR (ICCV'2023), YOLOF, YOLOV3, YOLOX-X, YOLOv5-M, YOLOV6-M, YOLOV8-M. There are 5 evaluation indicators, namely mAP50, mAP50-95, Params, Gflops and FPS.

[0167] The results of this comparative example are shown in Table 1. The embodiment of the present invention achieves 90.3% mAP0.5, 59.6% mAP0.5-0.95, 26.4M parameters, 50.8 Gflops, and 100.3 FPS. Among all other target detection methods listed in Table 1, it performs best in mAP0.5 and mAP0.5-0.95, and has highly competitive advantages in parameter volume, Gflops, and FPS. Specifically, the embodiment of the present invention exceeds the highly competitive YOLOv5 and YOLOv8 methods by 0.4%, 0.4% and 0.8%, 0.4% in mAP0.5 and mAP0.5-0.95, respectively. In terms of parameter volume and Gflops, it reduces these indicators by 45.0%, 20.6%, 46.8%, and 35.5%, respectively, compared with YOLOv5 and YOLOv8. These results show that the embodiment of the present invention improves detection performance and significantly reduces model complexity and computational requirements. This combination of high accuracy, reduced number of parameters, and computational efficiency increases the possibility of future deployment and application in resource-constrained environments, laying a solid foundation for robust and reliable performance in real-world scenarios.

[0168] Table 1 compares the embodiments of the present invention with other state-of-the-art object detection methods.

[0169]

[0170]

[0171] Comparative Example 2:

[0172] This comparative example provides a target detection model, which is gradually added with the EP4MCGM module in step S201, the HiLo-IFIM in step S202, and the model lightweight solution, wherein the model lightweight solution includes the CSPPKINetP5 module based on step S201, the CFSF-FPN in step S202, and the optimized loss function in step S3, and finally constitutes an embodiment. There are 8 evaluation indicators, namely Precision, Recall, mAP50, mAP50-95, Params (M), Gflops, IT (Inference Time, ms / bs = 16) and FPS (bs = 1).

[0173] Table 2. Results of beef cattle target detection after gradually adding modules

[0174]

[0175]

[0176] The results of this comparative example are shown in Table 2. Compared with comparative example 2, the performance of the method of this embodiment in terms of Precision, Recall, mAP, Params, and Gflops is improved. The Precision, Recall, mAP50, and mAP50-95 indicators are improved by 3.4%, 0.9%, 2.0%, and 1.3%, respectively. Params and Gflops can be reduced by 31.6% and 10.7%, respectively, but the performance of the Inference Time and FPS indicators is only slightly reduced. From the perspective of adding modules, it can be found that after adding EP4MCGM to the base comparative example 2, its Precision, Recall, mAP50, and mAP50-95 indicators can be improved by 1.7%, 0.7%, 1.3%, and 0.7%, respectively, but the model has no increase in computational complexity but has decreased. This shows that the addition of the EP4MCGM module enables the model to more effectively play the role of the P4 feature layer and improve the accuracy of target detection. Next, HiLo-IFIM was introduced, and the model's Precision, Recall, mAP50, and mAP50-95 were further improved by 1.2%, 0.3%, 0.6%, and 0.6%, respectively. The model's computational complexity increased slightly compared to the model with only the EP4MCGM module added, increasing by 0.1 Gflops, but compared to the baseline model, its computational complexity was still reduced, indicating that HiLo-IFIM can effectively promote the interaction of different frequency features within the scale and improve the performance of the model. In terms of the model lightweight solution, after adding CSPPKINetP5, CFSF-FPN, and SIoU, the model's parameters and computational complexity can be greatly reduced while improving the Precision and mAP indicators, while the Recall, inference time, and FPS performance only slightly decreased, indicating that the lightweight solution can effectively reduce the model's computational complexity and parameter complexity while maintaining model performance. Here, the InferenceTime and FPS of the model after +CFSF-FPN and +SioU are almost unchanged, because the loss function is mainly used in the training stage. When the model has been trained and inference is performed, the loss function no longer works. These results show that the proposed modules have a positive impact on the model performance to varying degrees, providing an effective technical means for achieving efficient and accurate beef cattle target detection.

[0177] Comparative Example 3:

[0178] This comparative example provides a target detection model, which is the same as the model in comparative example 2. The difference between this model and comparative example 2 is that the evaluation index is quantified by eight indicators: Cls, Loc, Both, Dupe, Bkg, Miss, FP and FN.

[0179] Table 3 Distribution of errors in beef cattle target detection after gradually adding modules

[0180]

[0181] The results of this comparative example are shown in Table 3 and Figure 7 As shown, the main error types of Comparative Example 3 and the method of the present embodiment are positioning errors, repeated detection errors, background errors and missed detection errors. After adding the modules in the embodiment method, these errors are reduced to varying degrees compared to the baseline, indicating that the added modules significantly improve the detection performance of the model in various aspects and reduce various types of detection errors. From the analysis of adding each module, it can be found that after adding the EP4MCGM module, the Loc, Dupe, Bkg, Miss, FP and FN values ​​of the model are all reduced, indicating that the module can enhance the positioning of the baseline model, distinguish different targets, distinguish foreground and background, and capture the actual target. After the introduction of HiLo-IFIM, the model pays more attention to distinguishing the foreground and background and the ability to capture the actual target compared to the baseline and the model with only the EP4MCGM module added. Continue to add the CSPPKINetP5 module, the model's ability to reduce various types of detection errors has declined. This is because the module, as one of the components of the model lightweight solution, plays a major role in reducing the number of parameters and the amount of calculation of the model. As shown in Table 2, the addition of the CSPPKINetP5 module reduces the number of parameters and the amount of calculation of the model by 24.8% and 7.2%, respectively. It successfully reduces the number of parameters and the amount of calculation of the model, making the model more lightweight and more suitable for running on resource-constrained devices. At the same time, the module still maintains a low level of missed detection errors, indicating that it still has a strong ability to effectively capture actual targets. The subsequent CFSF-FPN and SIoU as the remaining components of the model lightweight solution are intended to maintain the number of parameters and the amount of calculation of the model after adding the CSPPKINetP5 module, and improve the performance of the model. Finally, the method of this embodiment has a low repeated detection error rate, background error rate and false alarm rate, indicating that the method of this embodiment has significant advantages in reducing repeated detection errors, background errors and false alarm rates. These improvements enable the model to maintain efficient and accurate detection performance in various complex scenarios, thereby greatly improving the practicality and reliability of the model.

[0182] Comparative Example 4:

[0183] This comparative example provides a target detection model, which is the same as the model in comparative example 3. The robustness of the method of this embodiment and the comparative example method in different farming scenes, heat map situations and different occlusion situations is compared. At the same confidence level, the results of the comparative example and the method of this embodiment are compared. The green box represents the correctly detected target, the red box represents the missed target, and the blue box represents the false detection. The area where occlusion occurs is marked with a yellow dotted box.

[0184] The comparison results of this comparison in different breeding scenarios are as follows Figure 8 As shown, through these visualizations, the missed detection and false detection characteristics of comparative example 4 are analyzed. The present invention finds that missed detection often occurs when the target is too small, obscured, or in shaded or dark conditions. False detections are usually due to positioning, repeated detections, and background errors, which are caused by the difficult-to-distinguish appearance of the cattle. The method of this embodiment shows significant improvements, and its robustness and accuracy can be observed in various scenarios and environments. For example, in indoor scenes, the method of this embodiment can still accurately detect the outline and position of small target cattle, and correctly distinguish the background with a similar color to the cattle. In the case where the target is covered by shadows, the method of this embodiment also demonstrates strong anti-interference ability and correctly identifies the target in the shadow.

[0185] The comparison results of the heat map of this comparative example are as follows Fig. 9 As shown, the heat map of the method of this embodiment shows a more accurate and concentrated attention distribution compared with Comparative Example 4. In general, the method of this embodiment extracts more feature representations. This result is consistent with the results in Tables 2 and 3, proving the effectiveness of the method of this embodiment in beef cattle detection.

[0186] The robustness comparison results of this comparison under different occlusion conditions are as follows Fig.10 As shown, the method of this embodiment demonstrates the robustness and practicality of beef cattle detection under occlusion conditions that may occur in real farm scenes. By analyzing the results of comparative example 4, the present invention also found that certain types of occlusion, such as the mutual occlusion of scene 2 and the occlusion of the ceiling diagonal rod, or the mutual occlusion of scene 5 and the appearance is difficult to distinguish, have a greater impact on performance. The method of this embodiment will also fail in these cases, which will guide the present invention to further improve the model in future work to better cope with these challenging situations.

Claims

1. A beef cattle target detection method based on efficient selection and fusion of high- and low-frequency features, characterized in that: The method comprises the following steps: Step S1, constructing a beef cattle target detection dataset: We collected beef cattle videos of different breeding scenes from real monitoring perspectives, annotated the beef cattle videos, and constructed a beef cattle target detection dataset, which we named Beef-Cattle36. Step S2, constructing a beef cattle target detection model: Step S201, constructing a feature extraction network MPCPKI-ResNet: The feature extraction network MPCPKI-ResNet includes a basic residual module, an efficient P4 feature layer selection module based on a multi-path collaborative gating mechanism, and a CSPPKINetP5 feature layer selection module; The input image data is sent to the feature extraction network MPCPKI-ResNet, which generates feature maps at three levels {P3, P4, P5}, where: the P3 feature map is output by the basic residual module; the P4 feature map is output by the efficient P4 feature layer selection module based on the multi-path collaborative gating mechanism; the P5 feature map is output by the CSP-PKINet P5 feature layer selection module; Step S202, constructing a hybrid encoder: The hybrid encoder includes an intra-scale interaction module based on high- and low-frequency self-attention and a contextual feature selection fusion pyramid network; The P5 feature map obtained in step S201 is input into the intra-scale interaction module based on high- and low-frequency self-attention to perform internal interaction of high- and low-frequency features, and output the F5 feature map; The P3 feature map, the P4 feature map, and the F5 feature map obtained in step S201 are input into the context feature selection fusion pyramid network, and the P3 feature map, the P4 feature map, and the F5 feature map are fused and spliced ​​across scales of high and low frequency features to output a Memory feature sequence; Step S203: construct an IoU-aware query selection mechanism: Calculate the regression and classification losses of the predicted bounding box and the true bounding box corresponding to each feature in the memory feature sequence obtained in step S202, and combine the IoU in the classification loss to achieve the consistency of the position confidence and category confidence of the network output, and select the Top-K features according to the loss; Step S204, construct a decoder with an auxiliary prediction head: The Decoder receives the IoU-aware query selection mechanism constructed in step S203, selects the Top-K initial target queries and the memory feature sequence output by the hybrid encoder constructed in step S202, iteratively optimizes the target query through multi-layer self-attention and cross-attention, and finally outputs the category and coordinates of the beef cattle detection box; Step S3, training the beef cattle target detection model: Based on the beef cattle target detection dataset Beef-Cattle36 obtained in step S1, when training the beef cattle target detection model, the loss function L used includes the target bounding box regression loss L box and classification loss L cls Two parts: back propagation loss function L, repeated iteration until the number of iterations reaches the preset initial value to complete the training of the beef cattle target detection model; Step S4, beef cattle target detection: Given beef cattle image data, the data is input into the beef cattle target detection model trained in step S3, and the category and coordinates of the beef cattle detection frame are output.

2. The beef cattle target detection method based on efficient selection and fusion of high and low frequency features as claimed in claim 1, characterized in that: In step S1, the different breeding scenarios are 5 different breeding scenarios.

3. The beef cattle target detection method based on efficient selection and fusion of high and low frequency features as claimed in claim 1, characterized in that: In step S1, the beef cattle video is labeled using DarkLabel.

4. The beef cattle target detection method based on efficient selection and fusion of high and low frequency features as claimed in claim 1, characterized in that: In step S3, the target bounding box regression loss L box It is defined as the weighted sum of SIoU loss and L1 loss.

5. The beef cattle target detection method based on efficient selection and fusion of high and low frequency features as claimed in claim 4 is characterized in that: In step S3, the classification loss L cls A variable focus loss is used.

6. The method for detecting beef cattle targets based on efficient selection and fusion of high- and low-frequency features as claimed in claim 5, characterized in that: In step S3: L=L box +L cls Where: represents the predicted bounding box; b represents the ground-truth bounding box; L SIoU represents SIoU loss; λ represents the weight parameter, which is used to adjust L SIoU Loss in L box proportion of losses; Represents the weight parameter, which is used to adjust the L1 loss in L box proportion of losses; VFL stands for variable focal loss; p represents the predicted instance-aware classification score, which is the classification score predicted by the model for each instance; q represents the target score; Both α and κ represent hyperparameters.

Citation Information

Patent Citations

  • Underwater target detection method based on high and low frequency characteristic difference pyramid neural network

    CN115761467A

  • Target detection method based on transformer architecture

    CN117315223A

  • Dark light scene target detection method based on self-supervised feature learning

    CN117809024A

  • Cow detection method based on ResMO-Dense-YOLO

    CN117912062A

Cited By

  • Pork grading evaluation method and device based on image recognition

    CN119229435A

  • Remote sensing image target dynamic detection method based on multi-kernel

    CN119580116A