Cross-species target detection method based on cross-layer feature fusion and linear attention optimization

Abstract

This invention discloses a cross-species target detection method based on cross-layer feature fusion and linear attention optimization. It utilizes a hybrid self-attention encoder module to acquire global contextual information, establish long-range dependencies on the target, and extract features at various levels of the cross-species target. A cross-attention decoder module fully integrates features from each level, achieving joint encoding and alignment of low-level and high-level information, adjusting the contribution of each feature to the output features, highlighting effective information, and suppressing redundant information. Finally, a keypoint heatmap detection module predicts the target's location based on the target features modeled by the encoder-decoder. This invention can fully integrate features from various levels, improving the accuracy of cross-species target detection; and by decomposing and optimizing the attention process, it reduces the performance overhead of cross-species target detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a cross-species target detection method based on cross-layer feature fusion and linear attention optimization. Background Technology

[0002] With the widespread use of smartphones, computers, tablets, and other electronic products, images have become deeply integrated into people's daily lives and are rapidly becoming the most important channel for information transmission. As image data continues to increase, human understanding and recognition of this information alone is insufficient to meet the demands. Therefore, computer technology is needed to achieve information processing of images. How to effectively and efficiently extract and analyze information from images has become particularly important, and object detection technology can address this problem. The object detection task involves using computers to digitally analyze input images, locating and analyzing targets appearing in the image, and determining the target's position information and category. Compared to other complex visual tasks, object detection, while simpler, can provide effective technical support for more complex tasks such as semantic segmentation, pose estimation, and object tracking.

[0003] Cross-species target detection is now widely used in production and daily life. For example, during vehicle driving, cross-species target detection can rely on cameras to replace the human visual system for observation, accurately locating vehicles, pedestrians, and traffic lights captured by the cameras, thereby avoiding traffic risks in advance. In agricultural production, cross-species target detection can effectively improve production efficiency by providing data support and technical assistance, such as detecting and counting fruits and vegetables to effectively assess yield per acre; and detecting pests and diseases to effectively guide prevention and control efforts.

[0004] In the field of smart healthcare, cross-species target detection can be applied to analyze massive amounts of medical data. By identifying valuable information, it can help doctors make more accurate diagnoses, improve diagnostic efficiency, detect diseases early, and take effective intervention measures.

[0005] However, current cross-species object detection networks mostly rely on simple feature concatenation or addition operations to fuse multi-level object features without suppressing redundant information. This limited utilization of features significantly restricts the detection performance of cross-species object detection networks. Furthermore, when applying attention to model features, the computational complexity of the attention module in cross-species object detection networks is too high, consuming excessive hardware memory and computing resources, making it difficult to use in practice. Summary of the Invention

[0006] The purpose of this invention is to address the technical deficiencies in existing technologies by providing a cross-species target detection method based on cross-layer feature fusion and linear attention optimization. This method aims to solve the following problems in current neural network-based cross-species target detection: firstly, it fails to fully utilize the features at each level within the neural network, resulting in unsatisfactory detection performance in practical applications due to the influence of complex scenarios such as small-sized targets, overlapping targets, and targets with unclear foreground-background contrast; secondly, its modeling process for each level of features is too complex, consuming excessive hardware memory and computing resources, making it difficult to apply in practice.

[0007] The technical solution adopted to achieve the purpose of this invention is:

[0008] A cross-species target detection method based on cross-layer feature fusion and linear attention optimization is proposed. This method utilizes a cross-species target detection network that processes the input image to be detected and predicts the target location. The network includes an attention encoding / decoding module and a keypoint heatmap detection module. The attention encoding / decoding module comprises at least two levels of attention encoders / decoders. The output of the previous level's attention encoder / decoder serves as the input to the next level's attention encoder / decoder. Each level's attention encoder / decoder includes at least two levels of hybrid self-attention encoders and at least two levels of cross-attention decoders. The method includes the following steps:

[0009] The hybrid self-attention encoder module, based on a window partitioning strategy and an attention linear optimization strategy, segments the input image into multiple segmentation feature maps, restricts attention calculation to one window, linearly maps the segmentation feature maps to the attention calculation feature space through a linear mapping layer, and recombines multiple features obtained through attention calculation to extract features of cross-species targets at various levels from the image to be detected.

[0010] The cross-attention decoder module, through cross-attention calculation, fuses the features of each level proposed by the hybrid self-attention encoder module to achieve joint encoding and alignment of low-level and high-level information, and outputs the feature map to be detected.

[0011] The key point heatmap detection and processing module processes the target feature map to be detected using Gaussian heatmap encoding based on the target features modeled by the encoder-decoder. It predicts the target's location by predicting the target's center point position, the center point offset during downsampling, and the length and width of the bounding box, forming corresponding predicted values. The predicted values ​​are compared with the true values ​​to obtain the loss. The loss value is backpropagated to update the network parameters to continuously correct the prediction until the predicted value equals the true value, and then the prediction result is output.

[0012] In this method, Gaussian heatmap encoding is used to fit the distribution of the target center point in two-dimensional space. The center value of the Gaussian heatmap corresponds to the target center point, which is close to 1 and gradually decreases from the center to the edge, approaching 0. Through the learning behavior of the network, the predicted key point heatmap conforms to the target distribution, thereby obtaining an accurate prediction of the target location.

[0013] The key point heatmap detection and processing module includes three detection modules:

[0014] The key point heatmap detection module is used to obtain the predicted values ​​of the key point heatmap. The true value Y of the obtained key point heatmap xyc The comparison was performed, and the loss value L was calculated using pixel-level logistic regression (Focal Loss). k ;

[0015]

[0016] Where α and β are hyperparameters, and N is the number of targets to be detected;

[0017] The positioning error detection module predicts the offset of the center point during downsampling for each center point. The loss between the actual value and the predicted value is calculated using the L1 norm, as shown in the following formula:

[0018]

[0019] In the formula, The predicted value representing the center point offset, O k The true value representing the offset, where k is the kth target to be detected;

[0020] The target size detection module predicts the length and width of the bounding box corresponding to each point. The loss between the actual value and the predicted value is calculated using the L1 norm, as shown in the following formula:

[0021]

[0022] In the formula, The predicted value representing the target size, s k Represents the actual value of the size.

[0023] The system comprises at least two stages of hybrid attention encoders connected in sequence, with each subsequent hybrid self-attention encoder extracting features from the output of the preceding hybrid self-attention encoder. Each hybrid self-attention encoder includes a spatial self-attention encoder and a channel self-attention encoder. The output of the spatial self-attention encoder is the input of the channel self-attention encoder.

[0024] The encoding process of the hybrid self-attention encoder involves segmenting the input image X into windows. Segmentation feature map X slice The segmentation feature map X slice A linear mapping layer is used to linearly map to a spatial attention computation feature space, and the spatial segmentation feature map X is obtained through spatial attention computation. space The spatial segmentation feature map X space A linear mapping layer is used to linearly map to a channel attention computation feature space, resulting in the channel segmentation feature map X. chanl By using window reorganization operations, the feature maps X of multiple channels are segmented. chanl Recombined into output feature map F selfattn h and w are the dimensions of the input image, and M is the dimensions of the window.

[0025] The system comprises at least two levels of cross-attention decoders connected sequentially. Each cross-attention decoder includes a cross-spatial attention decoder and a cross-channel attention decoder. The cross-spatial attention decoder is used to jointly encode and align the high-level features captured by the spatial self-attention encoder and the low-level features before capture in the spatial dimension. The cross-channel attention decoder is used to jointly encode and align the high-level features captured by the channel self-attention encoder and the low-level features before capture in the channel dimension. The cross-attention mechanism controls the transmission ratio of high-level and low-level information to achieve effective information utilization.

[0026] The decoding process of the cross-attention decoder includes the following steps:

[0027] First, the cross-spatial attention decoder receives the high-level feature map F captured by the hybrid self-attention encoder. selfattn and the low-level feature map F before capture repl Obtained through window partitioning Each corresponding windowed high-level feature map F split and low-level feature map R split ;

[0028] For each windowed high-level feature map F split and low-level feature map R split Mapped to two feature spaces K via a linear mapping layer sinfo V sinfo and Q repl , high-level feature map F spli and low-level feature map R split Encoded as information components K of the same size sinfo V sinfo and Q repl ;

[0029] K sinfo V sinfo =Linear(F split )

[0030] Q repl =Linear(R) split )

[0031] Using K sinfo V sinfo and Q repl Calculate cross-space attention, with M 2 The window size is C, where C is the number of channels, and the output feature I within the control window is... sattn Gaining K's trust sinfo Or Q repl The calculation formula is shown below;

[0032]

[0033] Where λ1 is a user-defined constant;

[0034] Output feature I sattn After full connection layer and regularization processing, its calculation is shown in the following formula;

[0035] I sattn =Normliazation(A1I sattn +B1), where A1 and B1 are the parameters to be learned;

[0036] Will I sattn After being mapped to the feature space Q through a linear layer sattn K sattn V sattn The self-attention weights are calculated as shown in the following formula:

[0037] Q sattn K sattn V sattn =Linear(I sattn )

[0038] F space =SpaceAttn(Q sattn ,K sattn V sattn )

[0039] Second, the cross-channel attention decoder receives the high-level feature map F output by the cross-spatial attention decoder. space and the low-level feature map F before capture repl Low-level feature map F repl Obtained through window partitioning. R segmentation feature maps split ;

[0040] For each high-level feature map F space and low-level feature map R split Mapped to two feature spaces K through a linear mapping layer chinfo V chinfo and Q repl , to high-level features F space and low-level features R split Encoded as feature components K of the same size chinfo V chinfo and Q repl The linear mapping is shown in the following equation:

[0041] K chinfo V chinfo =Linear(F space )

[0042] Q repl =Linear(R) split )

[0043] Using K chinfo V chinfo and Q repl Calculate cross-channel attention, with M 2 Using a window of size C as the unit, control the information characteristics I within the window. chattn Gaining K's trust chinfo Or Q repl The calculation is shown in the following formula:

[0044]

[0045] Where λ2 is a user-defined constant;

[0046] Will I chattn After full connection layer and regularization, the calculation is as shown in the following formula;

[0047] I chattn =Normliazation(A2I xhattn +B2), where A2 and B2 are the parameters to be learned;

[0048] Will I chattn After being mapped to the feature space Q through a linear layer chattn K chattn V chattn The self-attention weights are calculated as shown in the following formula:

[0049] Q chattn K chattn V chattn =Linear(I chattn )

[0050] F chanl =ChanlAttn(Q chattn K chattn V chattn )

[0051] Third, the output is calculated through cross-channel attention calculation using window reorganization operations. F segmentation feature map chanl Recombined into feature map F croxattn This serves as the input for the key point heatmap detection and processing module.

[0052] This invention is based on cross-species target detection technology using cross-layer feature fusion and linear attention optimization. It constructs a cross-layer fusion cross-species target detection network through self-attention-cross-attention design to fully integrate features from each layer and improve the accuracy of cross-species target detection. Furthermore, it reduces the performance overhead of cross-species target detection by decomposing and optimizing the attention process. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall structure of the cross-species target detection network based on cross-layer feature fusion and linear attention optimization of the present invention.

[0054] Figure 2 This is a schematic diagram of the network structure of the hybrid self-attention encoder module according to an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the network structure of the cross-attention decoder module according to an embodiment of the present invention. Detailed Implementation

[0056] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0057] This invention utilizes a hybrid self-attention encoder module to acquire global contextual information and establish long-range dependencies on the target, thereby more effectively extracting valuable features and extracting features at various levels for cross-species targets. A cross-attention decoder module is used to fully integrate features at various levels, achieving joint encoding and alignment of low-level and high-level information, adjusting the contribution of each feature to the output features, highlighting effective information, and suppressing redundant information. Finally, a keypoint heatmap detection module is used to predict the target's location based on the target features modeled by the encoder-decoder.

[0058] In the process of extracting features of cross-species targets at various levels in the hybrid self-attention encoder module, a window partitioning strategy and an attention linear optimization strategy are adopted to reduce overhead.

[0059] like Figure 1 As shown, the cross-species target detection method based on cross-layer feature fusion and linear attention optimization according to an embodiment of the present invention includes the following steps:

[0060] Step 1: Processing of the Hybrid Self-Attention Encoder Module

[0061] Current cross-species object detection networks encode target images as vector sequences. The self-attention function iteratively learns feature representations by linking any pair of vectors in the sequence. This results in the network's computational complexity being quadratically related to the number of input vectors, or in other words, quadratically related to the image resolution. This quadratic computational complexity prevents cross-species object detection networks from modeling high-resolution images, and the high computational cost makes them unsuitable for real-world scenarios.

[0062] To address this issue, this invention employs window partitioning and linear attention to solve the problem of excessive computational complexity in the self-attention calculation process. Linear attention internally approximates exponential operations through first-order Taylor expansion and reduces computation by reusing intermediate results. The self-attention encoder module constructs a more efficient modeling process, achieving modeling results with better global approximation and generalization performance.

[0063] The hybrid self-attention encoder module mainly includes the following four steps:

[0064] First, window partitioning reduces the computational complexity of attention calculations: This operation restricts attention calculations to a single window, introducing locality of reference into the convolution operation and saving computational resources. For example, the cross-species target image X∈R can be partitioned into a window. hwC The self-attention encoder module is trained using input, and the results are obtained through window partitioning operations. Each segmentation feature map Where h and w are the width and height of the input image, C is the number of color channels of the input image, and M is the width and height of the window.

[0065] Second, the segmentation feature map output for each window partitioning operation. Mapped to the feature space through a linear mapping layer. (Attention calculation query matrix) (Attention calculation key matrix) (Attention calculation matrix)

[0066] The linear mapping process is shown in formula (1).

[0067] Q space K space V space =Linear(X) slice (1)

[0068] Using Q space K space V space The computational space self-attention is calculated as described in formula (2).

[0069]

[0070] The j-th column calculated by formula (2) is shown in formula (3), and the intermediate results generated in the formula calculation are as follows. and By reusing intermediate results, the computation process is simplified, ultimately bringing the computational complexity to a linear scale.

[0071]

[0072] in Q space K space V space The column vectors in the matrix represent the j-th column of the query matrix, the ith-th column of the key matrix, and the ith-th column of the value matrix, respectively.

[0073] Third, for each spatial self-attention, the intermediate segmentation feature map is calculated as output. All are mapped to the feature space through a linear mapping layer. (Attention calculation query matrix) (Attention calculation key matrix) (Attention calculation matrix). The linear mapping process is shown in formula (4).

[0074] Q chanl K chanl V chanl =Linear(X) space (4)

[0075] Using Q chanl K chanl V chanl The computational space self-attention is calculated as described in formula (5).

[0076]

[0077] The i-th row calculated by formula (5) is shown in formula (6), and the intermediate results generated during the calculation are as follows. and It can be reused, simplifying the calculation process by reusing intermediate results, and ultimately bringing the computational complexity to a linear scale. O() is a function representing the complexity of an algorithm. If the input problem size is N, the computational resources consumed are N. Here, the input image is of size hwC, and the computational resources consumed are hwC. 2 .

[0078]

[0079] Where q i ∈R C k j ∈R C v j ∈R C Q chanl T K chanl T V chanl T The column vectors in the matrix represent the i-th column of the query matrix, the j-th column of the key matrix, and the j-th column of the value matrix, respectively.

[0080] Fourth, through window reorganization operations... The intermediate segmentation feature map is calculated from the output of the self-attention algorithm for each channel. Recombined into output feature map F selfattn ∈R hwC That is, high-level feature map.

[0081] In summary, it can be seen that the self-attention encoder module models target information from both spatial and channel perspectives. After passing through several self-attention encoders, it sequentially extracts feature representations from low-level to high-level. Multiple self-attention encoders are connected sequentially, and each subsequent self-attention encoder extracts features from the output of the previous one. This makes the entire feature construction process more efficient than other mainstream cross-species target detection networks. The feature representations generated first are low-level features, and the feature representations generated later are high-level features.

[0082] Step 2: Processing of the cross-attention decoder module

[0083] The cross-attention decoder module includes a cross-spatial attention decoder and a cross-channel attention decoder. The extracted multi-level target features contain high-level semantic information (clear objects in high-level feature maps) and low-level positional detail information (low-level information such as corners, textures, and contours in low-level feature maps). This information is crucial for improving detection performance. However, existing cross-species target detection methods still have many problems in utilizing multi-level features. Most methods use simple feature concatenation or addition operations to fuse high-level and low-level features, failing to suppress redundant information and greatly limiting the network's performance in complex scenes.

[0084] Therefore, in this embodiment of the invention, a cross-spatial attention decoder module is used to jointly encode and align the high-level features captured by the self-attention encoder module and the low-level features before capture in the spatial dimension. A cross-channel attention decoder module is used to jointly encode and align the high-level features captured by the self-attention encoder module and the low-level features before capture in the channel dimension. By controlling the transmission weight of high-level and low-level information through cross-attention, the utilization of effective information is promoted. Figure 3 As shown.

[0085] The cross-attention decoder module specifically includes three steps:

[0086] First, receive the high-level feature map F captured by the hybrid self-attention encoder module. selfattn ∈R hwC and the low-level feature map F before capture repl ∈R hwC Obtained through window partitioning Each corresponding windowed high-level feature map and low-level feature maps

[0087] For each windowed high-level feature map F split and low-level feature map R split Both are mapped to two feature spaces through a linear mapping layer. (Attention calculation key matrix) (Attention calculation matrix) and (Attention calculation query matrix), this process will use the high-level feature map F split and low-level feature map R split Encoded as information components K of the same size sinfo V sinfo and Q repl The linear mapping process is shown in equations (7) and (8).

[0088] K sinfo V sinfo =Linear(F split (7)

[0089] Q repl =Linear(R) spl (8)

[0090] Using K sinfo V sinfo and Q repl Calculate the cross-space attention, the process of which is based on M 2 Using a window of size C as the unit, control the output feature I within the window. sattnGaining trust from high-level information K sinfo Or gaining trust from lower-level information Q repl The entire calculation process is shown in formula (9), where λ1 is a user-defined constant and hyperparameter.

[0091]

[0092] Output feature I sattn After full connection layer and regularization, its calculation process is shown in formula (10), where A1 and B1 are the parameters to be learned.

[0093] I sattn =Normliazation(A1I sattn +B1) (10)

[0094] Will I sattn Mapped to the feature space through a linear layer (Attention calculation query matrix) (Attention calculation key matrix) (Attention Calculation Matrix) Calculate the self-attention weights. The entire calculation process is shown in formulas (11) and (12).

[0095] Q sattn K sattn V sattn =Linear(I sattn (11)

[0096] F space =SpaceAttn(Q sattn ,K sattn V sattn (12)

[0097] Second, the high-level feature map output by receiving cross-spatial attention. and low-level feature map F repl ∈R hwC Low-level feature map F repl Obtained through window partitioning. Each segmentation feature map

[0098] For each high-level feature map F space and low-level feature map R split Both are mapped to two feature spaces through a linear mapping layer. (Attention calculation key matrix) (Attention calculation matrix) and (Attention calculation query matrix), this process will use high-level features F space and low-level features R splitEncoded as feature components K of the same size chinfo V chinfo and Q repl The linear mapping process is shown in equations (13) and (14).

[0099] K chinfo V chinfo =Linear(F space (13)

[0100] Q repl =Linear(R) split (14)

[0101] Using K chinfo V chinfo and Q repl Calculate cross-channel attention, the process of which is M 2 Using a window of size C as the unit, control the information characteristics I within the window. chattn Gaining trust from high-level information K chinfo Or gaining trust from lower-level information Q repl The entire calculation process is shown in formula (15), where λ2 is a user-defined constant and hyperparameter.

[0102]

[0103] Will I chattn After full connection layer and regularization, its calculation process is shown in formula (16), where A2 and B2 are the parameters to be learned.

[0104] I chattn =Normliazation(A2I chattn +B2) (16)

[0105] Will I chattn Mapped to the feature space through a linear layer (Attention calculation query matrix) (Attention calculation key matrix) (Attention Calculation Matrix) Calculate the self-attention weights. The calculation process is shown in formula (17) and formula (18).

[0106] Q chattn K chattn V chattn =Linear(I chattn (17)

[0107] F chanl =ChanlAttn(Q chattn K chattn V chattn (18)

[0108] Third, through window reorganization operation The segmentation feature map output by the cross-channel attention calculation Recombined into feature map F croxattn ∈R hwC .

[0109] Step 3: Detection by the key point heatmap detection and processing module

[0110] The distribution of the target center point in two-dimensional space is fitted using Gaussian heatmap encoding. The center value of the Gaussian heatmap corresponds to the target center point, which is close to 1, and gradually decreases from the center to the edge, approaching 0. Through the learning behavior of the network, the predicted keypoint heatmap conforms to the target distribution, thereby obtaining an accurate prediction of the target location.

[0111] The key point heatmap detection and processing module detects the target through the following two steps:

[0112] First, receive the output feature map F from the cross-attention decoder module. croxattn ∈R hwC The goal of the key point heatmap detection and processing module is to obtain the true value Y of the key point heatmap. xyc ∈[0,1] h,w,C .

[0113] For any point p on the key point heatmap · ∈R [x,y] Each has a key point p∈R derived from the real label. [px,py] Correspondingly, by using a Gaussian kernel Establish keypoint p from the real label and surrounding points p on the keypoint heatmap. · The correspondence between them, where δ p This is a size-adaptive standard deviation, where x and y represent the horizontal and vertical coordinates of the keypoint p on the true label, and px and py represent the keypoint heatmap p. · The horizontal and vertical coordinates of the coordinates.

[0114] If two Gaussian points of the same category overlap, place one of the points on the reserved empty category page Y. xy(c+1) Meanwhile, two overlapping Gaussian points are retained, and the calculation process is shown in formula (19).

[0115]

[0116] Second, output feature map F croxattn ∈R hwCThe data is fed into three detection modules, which respectively predict the center point position, the center point offset during downsampling, and the length and width of the bounding box, forming corresponding predicted values. The predicted values ​​are then compared with the true values ​​to obtain the loss. The loss value is backpropagated to update the network parameters, continuously correcting the data until the predicted value equals the true value.

[0117] The first detection module is the keypoint heatmap detection module, which obtains the predicted values ​​of the keypoint heatmap. Then, the actual value Y of the obtained key point heatmap was... xyc ∈[0,1] h,w,C The comparison is performed, and the loss value is calculated using pixel-level logistic regression Focal Loss, as shown in formula (20). Here, α and β are hyperparameters, and N is the number of targets to be detected.

[0118]

[0119] The second detection module is the positioning error detection module, which predicts the center point offset during downsampling for each center point. The loss between the true value and the predicted value is calculated using the L1 norm, and the calculation process is shown in formula (21).

[0120]

[0121] In the formula, The predicted value representing the center point offset, o k The true value of the offset is represented by N, where N is the number of targets to be detected and k is the kth target to be detected.

[0122] The third detection module is the target size detection module, which predicts the length and width of the bounding box for each point. The loss between the true value and the predicted value is calculated using the L1 norm, and the calculation process is shown in formula (22).

[0123]

[0124] In the formula, The predicted value representing the target size, s k The actual value of the size is represented by N, which is the number of targets to be detected, and k is the kth target to be detected.

[0125] Experimental verification:

[0126] The results of the cross-species target detection technique based on cross-layer feature fusion and linear attention optimization in this embodiment of the invention were compared with those of other target detection methods on MS COCO 2017.

[0127] The MS COCO (Microsoft COCO: Common Objects in Context) dataset is considered one of the most challenging datasets available, and it has become an important tool for benchmarking object detection tasks. The latest release, MS COCO 2017, includes 80 different object categories, 118,287 training images, 860,001 training instances, 5,000 test images, and 36,781 test instances.

[0128] mAP was used. S mAP M mAP L As the primary evaluation metric, mAP refers to the mean accuracy across all 80 categories in the MS COCO 2017 dataset. S mAP M mAP L These are the average accuracy rates for the small, medium, and large target classes, in that order. During the training process of this invention's method, the images are randomly cropped and scaled to 511x511 pixels. Common data augmentation methods, such as color jitter and brightness jitter, are also applied. During the testing process of this invention's method, the original input image resolution is kept constant, and the input image is padded with zeros to a fixed size before being fed into the network for prediction.

[0129] Model Training Input Validate input mAP <![CDATA[mAP S ]]> <![CDATA[mAP M ]]> <![CDATA[mAP L ]]> °DeNet 512×512 512×512 33.8 12.3 36.1 50.8 Faster RCNN 1000×600 1000×600 34.9 15.6 38.7 50.9 Cascade RCNN 1000×600 1000×600 42.8 26.6 23.7 45.5 GFLV2 2000×1200 2000×1200 53.3 35.7 56.1 65.8 LSNet 1400×840 1400×840 53.5 35.7 56.4 65.8 RetainNet 800×800 800×800 39.1 21.8 42.7 50.2 RefineDet 512×512 512×512 41.8 25.6 45.1 54.1 CornerNet 511×511 .ori 42.1 20.8 44.8 56.7 ExtremeNet 511×511 .ori 43.2 24.1 46.0 57.1 CenterNet 511×511 .ori 45.1 26.6 47.1 57.7 CentripetalNet 511×511 .ori 48.0 28.9 49.9 58.9 Ours 511×511 .ori 54.3 37.3 56.4 66.9

[0130] Table 1

[0131] As shown in Table 1, the present invention was compared with networks such as DeNet, Faster R-CNN, Cascade R-CNN, GFLV2, LSNet, RetinaNet, RefineDet, CornerNet, ExtremeNet, CenterNet (Objects as Points), and CentripetalNet. It can be concluded that the detection network of the present invention can obtain more accurate detection results.

[0132] As can be seen, the cross-species target detection technology based on cross-layer feature fusion and linear attention optimization in this invention can fully utilize high-level semantic information and low-level positional details in the detection network to locate cross-species targets. Its superior globality and generalization performance in modeling can effectively address challenges in complex scenarios (such as small-sized targets, structurally complex targets, and targets with low contrast to the background). Furthermore, due to its low computational cost, this invention can be deployed on resource-constrained hardware devices, making it possible to implement cross-species target detection in practical engineering applications.

[0133] With the rapid development of artificial intelligence and deep learning, more and more intelligent algorithms have entered our lives, such as in smart wearables and smart homes. In the future, artificial intelligence will advance into more intelligent sensing scenarios. In this process, the complex internal structure of targets and the ever-changing external scenes pose challenges to target detection algorithms. This invention can construct a cross-layer fusion cross-species target detection network. This network can not only detect various small-sized targets, overlapping targets, and targets with unclear foreground-background contrast, but also provides the possibility for cross-species target detection to be applied to all aspects of production and life. For example, applying this invention to vehicle driving can identify complex road conditions and accurately locate vehicles, pedestrians, and traffic lights, thereby avoiding traffic risks in advance.

[0134] Furthermore, when applied to agricultural production, this invention can be used to detect pests and diseases under different lighting conditions, thereby effectively guiding prevention and control efforts.

[0135] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention.

[0136] Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of the equivalents of the claims be included within the invention.

[0137] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A cross-species target detection method based on cross-layer feature fusion and linear attention optimization, characterized in that, A cross-species target detection network is implemented through cross-layer feature fusion and linear attention optimization. It processes the input image to be detected and predicts the target location. The cross-species target detection network includes an attention encoding and decoding module and a key point heatmap detection and processing module. The attention encoding and decoding module includes at least two levels of attention encoders and decoders. The output of the previous level of attention encoder and decoder is the input of the next level of attention encoder and decoder. Each level of attention encoder and decoder includes at least two levels of hybrid self-attention encoders and at least two levels of cross-attention decoders. include step: The hybrid self-attention encoder, based on a window partitioning strategy and an attention linear optimization strategy, segments the input image into multiple segmentation feature maps, restricts attention calculation to one window, linearly maps the segmentation feature maps to the attention calculation feature space through a linear mapping layer, and recombines multiple features obtained through attention calculation to extract features of cross-species targets at various levels from the image to be detected. The cross-attention decoder, through cross-attention calculation, fuses the features of each level proposed by the hybrid self-attention encoder to achieve joint encoding and alignment of low-level and high-level information, and outputs the feature map to be detected. The key point heatmap detection and processing module processes the target feature map to be detected using Gaussian heatmap encoding based on the target features modeled by the encoder-decoder. It predicts the target's location by predicting the target's center point position, the center point offset during downsampling, and the length and width of the bounding box, forming corresponding predicted values. The predicted values ​​are compared with the true values ​​to obtain the loss. The loss value is backpropagated to update the network parameters to continuously correct the prediction until the predicted value equals the true value, and then the prediction result is output.

2. The cross-species target detection method based on cross-layer feature fusion and linear attention optimization according to claim 1, characterized in that, The distribution of the target center point in two-dimensional space is fitted using Gaussian heatmap encoding. The center value of the Gaussian heatmap corresponds to the target center point, which is close to 1 and gradually decreases from the center to the edge, approaching 0. Through the learning behavior of the network, the predicted key point heatmap conforms to the target distribution, thereby obtaining an accurate prediction of the target location.

3. The cross-species target detection method based on cross-layer feature fusion and linear attention optimization according to claim 2, characterized in that, The key point heatmap detection and processing module includes three detection modules: The key point heatmap detection module is used to obtain the predicted values ​​of the key point heatmap. The actual values ​​of the key point heatmap obtained. Comparison, loss values ​​are calculated using pixel-level logistic regression (Focal Loss). L k ; ; in, and This is a hyperparameter, where N is the number of targets to be detected; The positioning error detection module predicts the offset of the center point during downsampling for each center point. The loss between the true value and the predicted value is calculated using the L1 norm, and the calculation process is as follows: ; In the formula, The predicted value representing the center point offset. The true value representing the offset. k It is the kth target to be detected; The target size detection module predicts the length and width of the bounding box corresponding to each point. The loss between the true value and the predicted value is calculated using the L1 norm, and the calculation process is shown in the following formula: ; In the formula, The predicted value representing the target size. Represents the actual value of the size.

4. The cross-species target detection method based on cross-layer feature fusion and linear attention optimization according to claim 1, characterized in that, At least two hybrid attention encoders are connected in sequence, with each subsequent hybrid self-attention encoder extracting features from the output of the preceding hybrid self-attention encoder; each hybrid self-attention encoder includes a spatial self-attention encoder and a channel self-attention encoder; the output of the spatial self-attention encoder is the input of the channel self-attention encoder.

5. The cross-species target detection method based on cross-layer feature fusion and linear attention optimization according to claim 4, characterized in that, The encoding process of the hybrid self-attention encoder is to divide the input image into windows. Divided into Each segmentation feature map Segmentation feature map A linear mapping layer is used to linearly map to a spatial attention computation feature space, and the spatial segmentation feature map is obtained through spatial attention computation. Spatial segmentation feature map A linear mapping layer is used to linearly map to a channel attention feature space, resulting in a channel segmentation feature map. By using window reassembly operations, feature maps from multiple channels are segmented. Reorganized into output feature map h and w are the dimensions of the input image, and M is the dimensions of the window.

6. The cross-species target detection method based on cross-layer feature fusion and linear attention optimization according to claim 5, characterized in that, At least two levels of cross-attention decoders are connected sequentially; each cross-attention decoder includes a cross-spatial attention decoder and a cross-channel attention decoder. A cross-spatial attention decoder is used to jointly encode and align the high-level features captured by the spatial self-attention encoder and the low-level features before capture in the spatial dimension; a cross-channel attention decoder is used to jointly encode and align the high-level features captured by the channel self-attention encoder and the low-level features before capture in the channel dimension; and the transmission ratio of high-level information and low-level information is controlled by cross-attention to achieve effective information utilization.

7. The cross-species target detection method based on cross-layer feature fusion and linear attention optimization according to claim 6, characterized in that, The decoding process of the cross-attention decoder includes the following steps: First, the cross-spatial attention decoder receives high-level feature maps captured by the hybrid self-attention encoder. and low-level feature maps before capture Obtained through window partitioning Each corresponding windowed high-level feature map and low-level feature maps ; For each windowed high-level feature map and low-level feature maps Mapped to two feature spaces via a linear mapping layer , and High-level feature map and low-level feature maps Encoded as information components of the same size and ; ; ; use and Calculate cross-space attention, in order to The size of the window is the unit. The number of channels controls the output features within the control window. gain trust or The calculation formula is shown below; ; in For user-defined constants; Output features After full connection layer and regularization processing, its calculation is shown in the following formula; ,in and These are the parameters to be learned; Will Mapped to the feature space through a linear layer , , The self-attention weights are calculated as shown in the following formula: ; ; Second, the cross-channel attention decoder receives the high-level feature map output by the cross-spatial attention decoder. and low-level feature maps before capture Low-level feature maps Obtained through window partitioning. Each segmentation feature map ; For each high-level feature map and low-level feature maps Mapped to two feature spaces through a linear mapping layer. , and High-level characteristics and low-level features Encoded as feature components of the same size and The linear mapping is shown in the following equation: ; ; use and Calculate cross-channel attention, in order to The size of the window is used as a unit to control the information characteristics within the window. gain trust or The calculation is shown in the following formula: ; in For user-defined constants; Will After full connection layer and regularization, the calculation is as shown in the following formula; ,in and These are the parameters to be learned; Will Mapped to the feature space through a linear layer , The self-attention weights are calculated as shown in the following formula: ; ; Third, the output is calculated through cross-channel attention calculation using window reorganization operations. Each segmentation feature map Reorganized into feature map This serves as the input for the key point heatmap detection and processing module.

Images