Target re-identification method and system based on target-aware high-frequency enhanced Transformer network

By introducing a Transformer-based target-perception high-frequency enhancement network into animal recognition technology, the challenge of animal recognition in different species and environments is solved, and higher recognition accuracy and robustness are achieved.

CN118587741BActive Publication Date: 2024-11-29WUHAN UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410757224.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-13
Publication Date
2024-11-29
Estimated Expiration
2044-06-13

AI Technical Summary

Technical Problem

The application of existing animal re-identification technologies among different species is limited, and it is difficult to accurately identify the identity of the same individual under different time frames, environments or perspectives.

Method used

A target-aware high-frequency enhancement network based on Transformer is used to extract high-frequency image features through Fourier transform and Gaussian high-pass filter, and combine the block generation module and the depth Transformer module to re-identify animal targets.

Benefits of technology

It significantly improves the accuracy of animal re-identification, can better capture the contours and texture features of animals, and enhances the robustness and scalability of recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118587741B_ABST
    Figure CN118587741B_ABST
Patent Text Reader

Abstract

The present invention discloses a target re-identification method and system based on a target-aware high-frequency enhanced Transformer network. First, the image is Fourier transformed, and then the frequency domain image is multiplied point by point with a high-pass filter, and the transformed image is inversely Fourier transformed; then, the original image and the high-frequency image are divided into several small blocks in a partially overlapping manner through a block generation module; after the small blocks are flattened into a sequence, a random initialization small block is added to the original image block and the high-frequency image block respectively, and then the position information is embedded, and all the small blocks are input into the Transformer layer; then in each generation, the original image block is first learned to obtain the original features, the network layer is frozen, and then the high-frequency block is based on the original image attention-guided learning to obtain the high-frequency features; finally, the network update is restored, and the triplet loss and identity loss are respectively used for the high-frequency features and the original features. The present invention helps Transformer to better learn the contour and texture features of animals and improve the accuracy of animal re-identification retrieval.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision target retrieval, and relates to a target re-identification method and system, and specifically to a target re-identification method and system based on a target perception high-frequency enhanced Transformer network, which can be applied to animal recognition. Background Art

[0002] Object re-identification (Re-ID) is a task to retrieve specific objects from non-overlapping cameras (References 1-3). The application of object re-identification (Re-ID) technology has made significant progress in pedestrian and vehicle retrieval. Nowadays, with the increasing demand for wildlife protection, animal re-identification has gradually become an important research field (References 4-5). This technology has great potential value for biodiversity conservation, ecosystem monitoring, and wildlife research. However, compared with datasets related to pedestrians and vehicles, most animal datasets lack annotations of cropping boxes, cameras, and viewpoints, and the interference of the wild environment is more significant. The challenges of existing animal re-identification are mainly: 1) significant differences in identifiable features between different species. Compared with humans who have significant features such as clothes and backpacks, animals do not have such distinguishing features. 2) The huge variation in body posture complicates the task. Animals present a variety of body postures, such as reclining, standing, and swimming.

[0003] The existing field of animal re-identification is still in its early stages and has many limitations. Many existing methods exploit the unique features of specific species to develop their methods, and some specifically focus on curating datasets for certain body parts of animals. These methods have proven challenging in the wide application of Re-ID between different species and have shown limited scalability. In addition, many animal-related studies are not fine-grained tasks, and some of them are coarse individual classifications. They usually only focus on distinguishing different individuals, and rarely pay attention to whether these individuals can be reliably re-identified in different environments. Existing animal Re-ID methods adopt different experimental settings. Some of these works are experimentally conducted in closed-world environments, mainly for identifying objects in known and limited categories. This invention proposes for the first time a new task of accurately identifying the identity of the same individual in different time frames, environments or perspectives, organizes most of the existing animal datasets into re-identification tasks, and gives methods for this task that are effective in multiple animal datasets.

[0004] At present, animal re-identification methods are mainly divided into three categories: global image-based methods, local feature-based methods, and additional annotation information-assisted methods. Many existing global image-based studies refer to traditional methods for pedestrian re-identification. Wang et al. (Reference 6) designed a multi-stream structure to learn local and global features of giant pandas, and then performed feature fusion to extract the most discriminative features based on local features during the dataset collection stage. For fine-grained elephant ear (Reference 7) and whale tail (Reference 8) images, Weideman et al. (Reference 9) extracted boundary information through color and texture conversion and intensity changes to identify the contours of key areas. The additional annotation information-assisted method inputs additional annotation information into the network to assist learning. Zhang et al. (Reference 10) used a simplified definition to use the left or right head of the yak as an auxiliary supervision signal to enhance feature learning. Li et al. (Reference 11) used the results of pose key point estimation to model tiger images and decompose them into seven parts, including the trunk, front legs, hind legs, etc., to learn local features. At present, these methods are mainly proposed for specific species, and their widespread application in Re-ID between different species has great limitations.

[0005] A large number of animal re-identification methods are based on convolutional neural networks (References 6, 10, 11, 12). However, compared with pedestrians, the pose and perspective of animal images vary greatly, which will lead to irregular distribution of important features in space. Convolutional neural networks may be limited in handling such spatial variations. Due to its ability to process global information, Vision Transformer (Reference 13) is more robust to such changes in perspective. In addition, in animal ReID, for some animals, the global contour information of the animal also plays a large role in the entire recognition process. The convolution of convolutional neural networks operates between local adjacent pixels (Reference 14), which will focus on local features (such as spots, stripes or other marks), but may ignore the importance of global contour information, thereby affecting the accuracy of overall recognition. In contrast, Vision Transformer, due to its global receptive field, can capture both local details and overall contours, enabling it to understand the image content more comprehensively, thereby improving the accuracy and robustness of overall recognition in animal ReID tasks. These properties motivate us to study animal re-identification methods based on the ViT framework.

[0006] Zhang et al. demonstrated that high-frequency information effectively improves fine-grained recognition in pedestrian re-identification because it emphasizes the edge and texture information of pedestrians (such as backpack patterns, clothing patterns, etc.), improving the accuracy of recognition (Reference 16). Inspired by them, considering the great differences in the characteristics of different animals, the present invention deeply studies the potential of applying high-frequency information to animal re-identification scenarios. Our research found that high-frequency information plays a vital role in animal recognition, especially in the emphasis on animal contours and texture details, thereby significantly improving recognition accuracy. For animals that lack significant features, such as dolphins and sharks, high-frequency information can more accurately outline their contours; for animals with obvious textures, such as tigers and cheetahs, high-frequency information can highlight their texture details and strengthen the recognition of these details, thereby promoting fine-grained learning. The research of this application reveals the importance of applying high-frequency information in animal recognition, which is not only a major breakthrough in the existing technology, but also provides a new perspective for the field of animal re-identification. The application of high-frequency information can significantly improve recognition accuracy, especially when distinguishing animals that lack distinctive features (such as whale tails, sharks) and animals with complex patterns and textures (such as zebra or leopard stripes). However, we also noticed that in the wild, the background of animals is usually complex and changeable, which may lead to interference from environmental noise in high-frequency images, such as leaves and grass textures, which may be overemphasized. Therefore, directly using high-frequency images as a means of data enhancement may cause the model to pay too much attention to background information and ignore the key features of the target animal. When applying high-frequency information for animal recognition, further research is needed on how to effectively reduce the interference of background noise to ensure that the model can accurately focus on the key features of the target animal.

[0007] In summary, it is crucial to design a Transformer-based target-aware high-frequency enhanced animal re-ID model for animal Re-ID to solve the above problems.

[0008] [Reference 1] Ying-Cong Chen, Xiatian Zhu, Wei-Shi Zheng, and Jian-HuangLai. 2017. Person re-identification by camera correlation aware feature augmentation. IEEE TPAMI 40, 2 (2017), 392–408.

[0009] [Reference 2] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven C. H. Hoi. 2021. Deep learning for person re-identification: A survey and outlook. IEEE TPAMI (2021), 1–1.

[0010] [Reference 3] Liang Zheng, Yi Yang, and Alexander G Hauptmann. 2016. Person reidentification: Past present and future. arXiv preprint arXiv:1610.02984 (2016).

[0011] [Reference 4] L. I. Kuncheva, F. Williams, S. L. Hennessey and J. J. Rodríguez, "A Benchmark Database for Animal Re-Identification and Tracking." IEEE, 2022, pp. 1-6.

[0012] [Reference 5] S. Schneider, G. W. Taylor, S. Linquist, and S. C. Kremer, “Past, present and future approaches using computer vision for animal re-identification from camera trap data,” Methods in Ecology and Evolution, vol. 10, no. 4, pp. 461–470, 2019.

[0013] [Reference 6] L. Wang, R. Ding, Y. Zhai, Q. Zhang, W. Tang, N. Zheng, and G. Hua, “Giant panda identification,” IEEE TIP, vol. 30, pp. 2837– 2849, 2021.

[0014] [Literature 7] H. J. Weideman, Z. M. Jablons, J. Holmberg, K. Flynn, J. Calambokidis, R. B. Tyson, J. B. Allen, R. S. Wells, K. Hupman, K. Urian et al., “Integral curvature representation and matching algorithms for identification of dolphins and whales,” in ICCV Workshop, 2017, pp. 2831–2839.

[0015] [Literature 8] T. Cheeseman, K. Southerland, J. Park, M. Olio, K. Flynn, J. Calambokidis, L. Jones, C. Garrigue, A. Frisch Jordan, A. Howard et al., “Advanced image recognition: a fully automated, high-accuracy photo-identification matching system for humpback whales,” Mammalian Biology, vol. 102, no. 3, pp. 915–929, 2022.

[0016] [Literature 9] H. Weideman, C. Stewart, J. Parham, J. Holmberg, K. Flynn, J. Calambokidis, D. B. Paul, A. Bedetti, M. Henley, F. Pope et al., “Extracting identifying contours for african elephants and humpback whales using a learned appearance model,” in WACV, 2020, pp. 1276–1285.

[0017] [Chapter 10]T. Zhang, Q. Zhao, C. Da, L. Zhou, L. Li, and S. Jiancuo,“Yakreid- 103: Benchmark for yak re-identification.” IEEE , 2021 , pp. 107 - 100 . 1–8

[0018] [Chapter 11]S. Li, J. Li, H. Tang, R. Qian, and W. Lin, “Atrw: benchmark for amur tiger re-identification in the wild,” arXiv preprintarXiv:1906.05586,

[0019] [Chapter 12]L. Bergamini, A. Porrello, AC Dondona, E. Del Negro, M. Mattioli, N. D'alterio, and S. Calderara. IEEE, 2018, p. 184–191.

[0020] [Figure 13] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, DirkWeissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, MatthiasMinderer, Georg

[0021] Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words:Transformers

[0022] for image recognition to scale. arXiv preprint arXiv:2010.11929(2020).

[0023] [Document 14] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In CVPR. 7794–7803.

[0024] [Document 15] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In CVPR. 7794–7803.

[0025] [Document 16] G. Zhang, Y. Zhang, T. Zhang, B. Li and S. Pu, "PHA: Patch-Wise High-Frequency Augmentation for Transformer-Based Person Re-Identification," 2023 IEEE / CVF Conference on Computer Vision and PatternRecognition (CVPR), Vancouver, BC, Canada, 2023, pp. 14133-14142, doi:10.1109 / CVPR52729.2023.01358. Summary of the invention

[0026] In order to solve the above technical problems, the present invention provides a target re-identification method and system based on a target-perceived high-frequency enhanced Transformer network, which improves the accuracy of animal target re-identification.

[0027] The technical solution adopted by the method of the present invention is: a target re-identification method based on a target-aware high-frequency enhancement Transformer network, which uses a target-aware high-frequency enhancement Transformer network to perform target re-identification; the target-aware high-frequency enhancement Transformer network includes a high-frequency map generation module, a block generation module and a depth of h Transformer module, where h is the default value;

[0028] The high-frequency image generation module includes a Fourier transform layer, a Gaussian high-pass filter and an inverse Fourier transform layer; the Fourier transform is used to apply Fourier transform to the input original image and convert it into the frequency domain; the Gaussian high-pass filter is used to multiply the frequency domain image and the high-pass filter point by point; the inverse Fourier transform layer is used to perform inverse Fourier transform on the frequency domain image to which the high-pass filter is applied, convert it back to the spatial domain, and output a high-frequency image;

[0029] The block generation module includes a convolution layer, a flattening layer, a global feature block adding layer and a position information embedding layer; the convolution layer uses an overlapping segmentation method to segment the original image and the high-frequency image with N*N pixels as blocks, wherein N is a preset value; the flattening layer is used to flatten the small blocks into a one-dimensional sequence; the global feature block adding layer is used to add a randomly initialized global feature block to the serialized small blocks of the original image and the high-frequency image respectively; the position information embedding layer is used to record the relative position of each small block in the original image;

[0030] The Transformer module includes a multi-head self-attention layer MSA and a two-layer fully connected network MLP with a GELU activation function. A layer normalization layer LayerNorm is set before the MSA and the MLP, and a residual connection is added.

[0031] Preferably, the method comprises the following steps:

[0032] Step 1: Through the high-frequency image generation module, the original image is Fourier transformed to convert it from the spatial domain to the frequency domain, and then the frequency domain image is multiplied point by point by the Gaussian high-pass filter. Finally, the frequency domain image with the high-pass filter is inversely Fourier transformed to convert it back to the spatial domain and output a high-frequency image;

[0033] Step 2: Through the block generation module, the original image and the high-frequency image are divided into several small blocks with fixed step size and fixed size convolution blocks in a partially overlapping manner;

[0034] Step 3: The small blocks are flattened into a one-dimensional sequence through the flattening layer, and a randomly initialized global feature block is added to the serialized small blocks of the original image and the high-frequency image through the global feature block adding layer; the position information embedding layer introduces position information embedding for each small block, and the processed small blocks are then sent to the Transformer module with a depth of h;

[0035] Step 4: Classify and identify the processed images through MLP and LayerNorm respectively.

[0036] Preferably, the high frequency image generation module inputs the original image Apply a Fourier transform to convert to the frequency domain:

[0037] ;

[0038] in, is the original image The spatial domain coordinates of is the frequency domain coordinate, represents the Fourier transform, j is an imaginary unit;

[0039] Apply a centering transform to move low-frequency components to the center of the spectrum:

[0040] ;

[0041] in, is the frequency domain representation after centralization;

[0042] Apply a Gaussian high pass filter Get high frequency components :

[0043] ;

[0044] ;

[0045] in, is the center coordinate in the frequency domain, D is the cutoff frequency of the Gaussian high-pass filter;

[0046] Use inverse Fourier transform to reconstruct the image and obtain a high-frequency image :

[0047] .

[0048] Preferably, the convolution layer of the block generation module uses a convolution kernel size of 16×16 and a step size of 12; an overlapping segmentation method is adopted to segment the original image and the high-frequency image respectively with 16×16 pixels as blocks.

[0049] Preferably, the position information embedding layer of the block generation module is combined with the feature vector of the small block through an addition operation, thereby giving the network spatial perception ability without destroying the original feature representation.

[0050] Preferably, the Transformer module classifies and identifies the processed images through classifiers using MLP and LayerNorm respectively.

[0051] Preferably, the target-aware high-frequency enhancement Transformer network is a trained network;

[0052] In each round of training, the network first focuses on learning the original image blocks to extract basic visual features; then, by freezing the Transformer module, the high-frequency image blocks are used to accurately learn the high-frequency features under the guidance of the original image features, ensuring the consistency and complementarity of the extracted high-frequency features with the original image features; finally, the network update is restored, and the high-frequency features processed by the Transformer module and the original features are optimized using triplet loss and identity loss respectively.

[0053] Preferably, the high frequency feature is to convert the high frequency image , where H is the image height, W is the image width, and C is the number of channels, split and projected into an N-dimensional vector ; Add a global classification feature , the vector input to the network is ,in It is the embedding of position information; it is based on the original image attention guidance to learn high-frequency features,

[0054] ;

[0055] ;

[0056] in, is the original image input sequence, is a learnable weight matrix, is the attention weight of high-frequency features, is a high-frequency input sequence, Q is the query, K is the key, and V is the value. is the dimension of key K, is the weighted representation of the input high-frequency sequence.

[0057] As a preference, the loss function used in the training process is:

[0058] ;

[0059] ;

[0060] Among them, the global classification feature of the original graph is expressed as , the global classification feature of the high-frequency graph is expressed as ; is the original image feature loss function, is the high-frequency feature loss function, To calculate the triple classification loss for the global classification features of the original image, To calculate the identity loss of the global classification features of the original image, To calculate the triple classification loss for the global classification features of high-frequency images, To calculate the identity loss for global classification features of high-frequency images;

[0061] The total loss function is:

[0062] ;

[0063] in, and Represent the proportion of original features and high-frequency features respectively.

[0064] The technical solution adopted by the system of the present invention is: a target re-identification system based on target-aware high-frequency enhanced Transformer network, comprising:

[0065] one or more processors;

[0066] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the target re-identification method based on the target-aware high-frequency enhanced Transformer network.

[0067] The present invention has the following advantages:

[0068] (1) This paper applies Transformer to animal re-identification and unifies multiple animal datasets as the experimental setting for the re-identification task. By leveraging the Transformer's ability to process global information, we propose a general network framework for re-identification of multiple animals.

[0069] (2) This paper introduces a new high-frequency information enhancement learning strategy, which helps Transformer better learn the contour and texture features of animals.

[0070] (3) Compared with other existing models, the present invention significantly improves the retrieval accuracy on multiple animal re-identification datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.

[0072] Figure 1 This is a diagram of the target-aware high-frequency enhancement Transformer network structure of an embodiment of the present invention;

[0073] Figure 2 is a flow chart of a method according to an embodiment of the present invention;

[0074] Figure 3 This is a schematic diagram of the target perception high-frequency enhancement Transformer network training process of an embodiment of the present invention;

[0075] Figure 4 It is a schematic diagram of the enhancement effect of the high-frequency image processing technology on the contour and texture features of animals according to an embodiment of the present invention, as well as a general schematic diagram of network training and testing. DETAILED DESCRIPTION

[0076] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0077] The core idea of ​​this invention is to design a new image high-frequency information enhancement learning strategy based on ViT (Vision Transformer) to enhance the effectiveness of learning multiple animal features, and to learn high-frequency information features under the guidance of the original image attention to avoid the model paying too much attention to the background, thereby focusing on the learning of the details and contours of the target object.

[0078] This embodiment provides a target re-identification method based on a target-aware high-frequency enhanced Transformer network, which uses a target-aware high-frequency enhanced Transformer network to perform animal target re-identification;

[0079] Please see Figure 1 The target-aware high-frequency enhancement Transformer network described in this embodiment includes a high-frequency graph generation module, a block generation module and a depth of h Transformer module, where h is the default value;

[0080] The high-frequency image generation module includes a Fourier transform layer, a Gaussian high-pass filter and an inverse Fourier transform layer; the Fourier transform is used to apply Fourier transform to the input original image and convert it into the frequency domain; the Gaussian high-pass filter is used to multiply the frequency domain image and the high-pass filter point by point; the inverse Fourier transform layer is used to perform inverse Fourier transform on the frequency domain image to which the high-pass filter is applied, convert it back to the spatial domain, and output a high-frequency image;

[0081] The block generation module includes a convolution layer, a flattening layer, a global feature block adding layer and a position information embedding layer; the convolution layer uses an overlapping segmentation method to segment the original image and the high-frequency image into blocks of N×N pixels, wherein N is a preset value; the flattening layer is used to flatten the small blocks into a one-dimensional sequence; the global feature block adding layer is used to add a randomly initialized global feature block to the serialized small blocks of the original image and the high-frequency image respectively; the position information embedding layer is used to record the relative position of each small block in the original image;

[0082] The Transformer module includes a multi-head self-attention layer MSA and a two-layer fully connected network MLP with a GELU activation function. A layer normalization layer LayerNorm is set before the MSA and the MLP, and a residual connection is added.

[0083] In one embodiment, the high frequency image generation module inputs the original image Apply a Fourier transform to convert to the frequency domain:

[0084] ;

[0085] in, is the original image The spatial domain coordinates of is the frequency domain coordinate, represents the Fourier transform, j is an imaginary unit;

[0086] Apply a centering transform to move low-frequency components to the center of the spectrum:

[0087] ;

[0088] in, is the frequency domain representation after centralization;

[0089] Apply a Gaussian high pass filter Get high frequency components :

[0090] ;

[0091] ;

[0092] in, is the center coordinate in the frequency domain, D is the cutoff frequency of the Gaussian high-pass filter;

[0093] Use inverse Fourier transform to reconstruct the image and obtain a high-frequency image :

[0094] .

[0095] In one implementation, the convolution layer of the block generation module uses a convolution kernel size of 16×16 and a step size of 12; an overlapping segmentation method is used to segment the original image and the high-frequency image into blocks of 16×16 pixels.

[0096] In one embodiment, the position information embedding layer of the block generation module is combined with the feature vector of the small block through an addition operation, thereby giving the network spatial perception ability without destroying the original feature representation.

[0097] In one implementation, the Transformer module classifies and identifies the processed images through classifiers using MLP and LayerNorm respectively.

[0098] Please see Figure 2 , this embodiment provides a target re-identification method based on a target-aware high-frequency enhanced Transformer network, comprising the following steps:

[0099] Step 1: Through the high-frequency image generation module, the original image is Fourier transformed to convert it from the spatial domain to the frequency domain, and then the frequency domain image is multiplied point by point by the Gaussian high-pass filter, thereby retaining the high-frequency components and removing the low-frequency components. Finally, the frequency domain image with the high-pass filter is inversely Fourier transformed to convert it back to the spatial domain and output a high-frequency image;

[0100] Step 2: Through the block generation module, the original image and the high-frequency image are divided into several small blocks with fixed step size and fixed size convolution blocks in a partially overlapping manner;

[0101] Step 3: The small blocks are flattened into a one-dimensional sequence through the flattening layer to adapt to the input requirements of the Transformer module. In order to enhance the network's ability to understand the global context, this embodiment adds a randomly initialized global feature block to the serialized small blocks of the original image and the high-frequency image through the global feature block adding layer; the position information embedding layer introduces position information embedding for each small block to ensure that the network can understand the relative position of each small block in the original image, which is crucial for maintaining the spatial relationship within the image. The position embedding is combined with the feature vector of the small block through an addition operation, thereby giving the network spatial perception without destroying the original feature representation. All small blocks that have undergone this series of processing are then sent to a Transformer module with a depth of h;

[0102] Step 4: Classify and identify the processed images through MLP and LayerNorm respectively.

[0103] In one embodiment, see Figure 3 , the target-aware high-frequency enhanced Transformer network is a trained network;

[0104] In each round of training, the network first focuses on learning the original image blocks to extract basic visual features; then, by freezing the Transformer module, the high-frequency image blocks are used to accurately learn the high-frequency features under the guidance of the original image features, ensuring the consistency and complementarity of the extracted high-frequency features with the original image features; finally, the network update is restored, and the high-frequency features processed by the Transformer module and the original features are optimized using triplet loss and identity loss respectively.

[0105] In one embodiment, the high frequency feature is to convert the high frequency image , where H is the image height, W is the image width, and C is the number of channels, split and projected into an N-dimensional vector ; Add a global classification feature , the vector input to the network is ,in It is the location information embedding;

[0106] In each generation, we first learn the original image blocks to obtain the original features, then freeze the network layer, prohibit gradient descent, and then learn the high-frequency blocks based on the original image attention guidance to obtain high-frequency features;

[0107] Based on the original image attention guidance, learn high-frequency features.

[0108] ;

[0109] ;

[0110] in, is the original image input sequence, is a learnable weight matrix, is the attention weight of high-frequency features, is a high-frequency input sequence, Q is the query, K is the key, and V is the value. is the dimension of key K, is the weighted representation of the input high-frequency sequence.

[0111] In one embodiment, the loss function used in the training process is:

[0112] ;

[0113] ;

[0114] Among them, the global classification feature of the original graph is expressed as , the global classification feature of the high-frequency graph is expressed as ; is the original image feature loss function, is the high-frequency feature loss function, To calculate the triple classification loss for the global classification features of the original image, To calculate the identity loss of the global classification features of the original image, To calculate the triple classification loss for the global classification features of high-frequency images, To calculate the identity loss for global classification features of high-frequency images;

[0115] The total loss function is:

[0116] ;

[0117] in, and Represent the proportion of original features and high-frequency features respectively.

[0118] This embodiment also provides a target re-identification system based on a target-aware high-frequency enhanced Transformer network, including:

[0119] one or more processors;

[0120] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the target re-identification method based on the target-aware high-frequency enhanced Transformer network.

[0121] The principle of this embodiment is further explained below in conjunction with specific experiments.

[0122] The deep learning framework used in this implementation example is Pytorch. The hardware environment of the experiment is NVIDIA

[0123] GeForceRTX3090×8 graphics card, processor is Intel(R)Xeon(R)Gold 6240. The experimental process is as follows:

[0124] Step 1: Building a target-aware high-frequency enhanced Transformer network;

[0125] In the experiment, the Vision Transformer (ViT) network is used as a feature extractor, Fourier transform and Gaussian filter are used to extract high-frequency information of the image, the original image attention is used to guide the learning of high-frequency features, and finally the constraints between the original features and high-frequency features are established. Identity classification loss, triple loss, and cross entropy loss are used to combine the end-to-end feature extractor and the high-frequency information enhancement branch recognition network.

[0126] Step 2: Network training;

[0127] This experiment selected datasets containing different species for evaluation. Since these datasets only provide original images and corresponding identity annotations, this experiment uniformly divides them into training sets and test sets for re-identification tasks. Specifically, 70% of the identities in each dataset are used as training data, and the remaining 30% are used as test data. Ensure that the identities in the test set have never appeared in the training set. Generate high-frequency images from the training set images first, and then send the two images to the high-frequency information enhanced branch recognition network for training. Use forward propagation and backpropagation to optimize and update the network parameters.

[0128] Step 3: Network testing;

[0129] Please see Figure 4 In the test phase, each image in the test set is used as a query, and all images in the test set except the query image constitute the query set. The best model in the training process is used for inference to obtain the final retrieval result on the test set. The evaluation indicators are Rank-1, mAP, and mINP matching accuracy, which reflects the retrieval probability of the correct re-identified image.

[0130] This experiment was tested on three animal datasets: iPanda-50, ATRW, and ELPephants. Since the dataset only provides the original images and the corresponding identity annotations, there is no unified division for re-identification, making it difficult to compare indicators. This experiment evenly divides it into training and test sets for the re-identification task. This experiment uses 70% of the identities in each dataset as training data and the remaining 30% as test data. Make sure that the identities in the test set do not appear in the training set. In the test phase, this experiment treats each image in the test set as a query, and all images in the test set except the query image constitute the query;

[0131] iPanda-50 is a panda re-identification dataset collected through live panda videos, containing 6874 images of 50 giant pandas of different ages, including cubs, subadults and adults. ATRW is a dataset containing 3649 images of 92 Siberian tigers collected from multiple large wildlife parks, with bounding boxes, pose keypoints and identity annotations. Elpephants is a dataset containing 2078 images of 276 elephants in the Dzanga bai clearing in the Dzanga-Ndoki National Park in the Central African Republic, with one identity label for each image.

[0132] In this experiment, the images are uniformly resized to 256×256. In addition, random rotation of 15 degrees, 50% probability of random adjustment of image brightness, 50% probability of random adjustment of image contrast, and 10 pixels of padding are used in the training data. The network parameters are initialized with the parameters pre-trained by ImageNet-1K. In the overlapping block embedding stage, the patch size is set to 16 and the stride size is set to 12. In the high-frequency map generation module, the cutoff frequency of the filter is set to 50. For the original features extracted from the backbone and the rotated high-frequency features, the triplet loss without margin is used, and the cross entropy loss is used after the features pass through the batch generalization layer. The weight of the original feature λ is 0.5, and the weight of the high-frequency feature 1-λ is 0.5. During training, the stochastic gradient descent (SGD) optimizer is used. The initial learning rate is 0.001, and the cosine learning rate decay is used. The number of training times is 150. The batch size is set to 32, including 8 identities, each with 4 images. In the test phase, only the original features are used to calculate the distance matrix. The implementation of the entire experiment is based on PyTorch.

[0133] In order to verify the effectiveness of the present invention, this experiment compares the retrieval results of the present invention with the existing target re-identification methods. The existing target re-identification methods mainly include:

[0134] (1)MGN: S. Wang, Y. Cui and Y. Kang, "Learning Multiple GranularityFeatures for Unsupervised Person Re-Identification," 2022 IEEE International Conference on Multimedia and Expo (ICME), 2022, pp.

[0135] (2) AGW: Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. 2021. Deep learning for person re-identification: A survey and outlook. IEEE TPAMI (2021), 1–1.

[0136] (3)BoT: H. Luo, W. Jiang, Y. Gu, F. Liu, X. Liao, S. Lai, and J. Gu, “A strong baseline and batch normalization neck for deep personreidentification,” IEEE TMM, vol. 22, no. 10, pp. 2597–2609, 2019.

[0137] (4) TransReID: Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. 2021. Transreid: Transformer-based object re-identification. InICCV. 15013–15022.

[0138] Tested on iPanda-50, ATRW, ELPephants datasets, the results are shown in Table 1

[0139] Table 1

[0140]

[0141] It can be seen from Table 1 that compared with the Re-ID methods in recent years, the method proposed in the present invention has achieved an improvement in the retrieval rate in multiple animal re-identification tasks, among which elephants lack significant pattern features and focus on regional contour line recognition, and tigers belong to fine-grained pattern recognition. On the iPanda-50 dataset, the performance of the proposed method is significantly better than all the methods in the table. The proposed method is 4.6% and 4.3% better than the current optimal method TransRe-ID in Rank1 and mAP respectively. On the ELPephants dataset, mAP is 8.1% better than the current optimal ReID method, and mINP is 4.3% better than the current optimal ReID method. On the ATRW dataset, the method proposed in the present invention achieved a mINP accuracy of 34.3% and a mAP of 34.4% without using any auxiliary information during testing, surpassing all other methods. The experimental results on three animal collection datasets demonstrate the effectiveness and generalization of the method proposed in the present invention.

[0142] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0143] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A target re-identification method based on target-aware high-frequency enhanced Transformer network, characterized in that: The following steps are involved: Step 1: Through the high-frequency image generation module, the original image is Fourier transformed to convert it from the spatial domain to the frequency domain, and then the frequency domain image is multiplied point by point by the Gaussian high-pass filter. Finally, the frequency domain image with the high-pass filter is inversely Fourier transformed to convert it back to the spatial domain and output a high-frequency image; The high frequency image generation module inputs the original image Apply a Fourier transform to convert to the frequency domain: ; in, is the original image The spatial domain coordinates of is the frequency domain coordinate, represents the Fourier transform, j is an imaginary unit; Apply a centering transform to move low-frequency components to the center of the spectrum: ; in, It is the frequency domain representation after centralization; Apply a Gaussian high pass filter Get high frequency components : ; ; in, is the center coordinate in the frequency domain, D is the cutoff frequency of the Gaussian high-pass filter; Use inverse Fourier transform to reconstruct the image and obtain a high-frequency image : ; Step 2: Through the block generation module, the original image and the high-frequency image are divided into several small blocks with fixed step size and fixed size convolution blocks in a partially overlapping manner; Step 3: The small blocks are flattened into a one-dimensional sequence through the flattening layer, and a randomly initialized global feature block is added to the serialized small blocks of the original image and the high-frequency image through the global feature block adding layer; the position information embedding layer introduces the position information embedding for each small block, and the processed small blocks are then sent to the depth h In the Transformer module; Step 4: Classify and identify the processed images through MLP and LayerNorm respectively.

2. The target re-identification method based on target-aware high-frequency enhanced Transformer network according to claim 1, characterized in that: The convolution layer of the block generation module uses a convolution kernel size of 16×16 and a step size of 12; an overlapping segmentation method is used to segment the original image and the high-frequency image into blocks of 16×16 pixels.

3. The target re-identification method based on target-aware high-frequency enhanced Transformer network according to claim 1, characterized in that: The position information embedding layer of the block generation module is combined with the feature vector of the small block through an addition operation, thereby giving the network spatial perception ability without destroying the original feature representation.

4. The target re-identification method based on target-aware high-frequency enhanced Transformer network according to claim 1, characterized in that: The Transformer module classifies and identifies the processed images through classifiers using MLP and LayerNorm.

5. The target re-identification method based on target-aware high-frequency enhanced Transformer network according to any one of claims 1 to 4, characterized in that: The target-aware high-frequency enhanced Transformer network is a trained network; In each round of training, the network first focuses on learning the original image blocks to extract basic visual features; then, by freezing the Transformer module, the high-frequency image blocks are used to accurately learn the high-frequency features under the guidance of the original image features, ensuring the consistency and complementarity of the extracted high-frequency features with the original image features; finally, the network update is restored, and the high-frequency features processed by the Transformer module and the original features are optimized using triplet loss and identity loss respectively.

6. The target re-identification method based on target-aware high-frequency enhanced Transformer network according to claim 5, characterized in that: The high-frequency feature is to convert the high-frequency image , where H is the image height, W is the image width, and C is the number of channels, split and projected into an N-dimensional vector ; Add another global classification feature , the vector input to the network is ,in It is the embedding of position information; it is based on the original image attention guidance to learn high-frequency features, ; ; in, is the original image input sequence, is a learnable weight matrix, is the attention weight of high-frequency features, is a high-frequency input sequence, Q is the query, K is the key, and V is the value. is the dimension of key K, is the weighted representation of the input high-frequency sequence.

7. The target re-identification method based on target-aware high-frequency enhanced Transformer network according to claim 5, characterized in that: The loss function used in the training process is: ; ; Among them, the global classification feature of the original graph is expressed as , the global classification feature of the high-frequency graph is expressed as ; is the original image feature loss function, is the high-frequency feature loss function, To calculate the triple classification loss for the global classification features of the original image, To calculate the identity loss of the global classification features of the original image, To calculate the triple classification loss for the global classification features of high-frequency images, To calculate the identity loss for global classification features of high-frequency images; The total loss function is: ; in, and Represent the proportion of original features and high-frequency features respectively.

8. The target re-identification method based on target-aware high-frequency enhanced Transformer network according to claim 1, characterized in that: The target-aware high-frequency enhancement Transformer network includes a high-frequency graph generation module, a block generation module and a depth of h Transformer module, where h is the default value; The high-frequency image generation module includes a Fourier transform layer, a Gaussian high-pass filter and an inverse Fourier transform layer; the Fourier transform is used to apply Fourier transform to the input original image and convert it into the frequency domain; the Gaussian high-pass filter is used to multiply the frequency domain image and the high-pass filter point by point; the inverse Fourier transform layer is used to perform inverse Fourier transform on the frequency domain image to which the high-pass filter is applied, convert it back to the spatial domain, and output a high-frequency image; The block generation module includes a convolution layer, a flattening layer, a global feature block adding layer and a position information embedding layer; the convolution layer uses an overlapping segmentation method to segment the original image and the high-frequency image into blocks of N×N pixels, wherein N is a preset value; the flattening layer is used to flatten the small blocks into a one-dimensional sequence; the global feature block adding layer is used to add a randomly initialized global feature block to the serialized small blocks of the original image and the high-frequency image respectively; the position information embedding layer is used to record the relative position of each small block in the original image; The Transformer module includes a multi-head self-attention layer MSA and a two-layer fully connected network MLP with a GELU activation function. A layer normalization layer LayerNorm is set before the MSA and the MLP, and a residual connection is added.

9. A target re-identification system based on target-aware high-frequency enhanced Transformer network, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the target re-identification method based on the target-aware high-frequency enhanced Transformer network as described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Pedestrian cross-mirror re-identification method and system based on background and attitude normalization

    CN114120363A

  • Unmanned aerial vehicle target re-identification method, system and equipment with rotation invariance

    CN115661688A

  • Unsupervised target re-identification method and system based on perceptual assisted learning Transform model

    CN116403015A

  • Weather variability text pedestrian re-recognition algorithm based on high-frequency information guidance

    CN117095326A

  • Person re-identification method, apparatus, and device based on heterogeneous network feature interaction

    WO2024021283A1