A target object re-identification method
By combining global and high-frequency feature extraction methods, the problem of ignoring structural and high-frequency information in existing technologies is solved, achieving more efficient target object recognition and behavior recognition, and improving recognition accuracy and anti-interference ability.
Patent Information
- Application Number
- CN202411995621.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2044-12-31
AI Technical Summary
Existing transformer-based target object re-identification methods ignore structural and high-frequency information, resulting in low identification efficiency, especially in monitoring scenarios where it is difficult to accurately identify objects and their behaviors.
A method combining global feature extraction and high-frequency feature extraction is adopted. The global and high-frequency features of the target object are obtained through the first transformer network and the second discrete wavelet transform. The edge perception module is used to perform feature fusion to identify the target object.
It improves the accuracy and generalization ability of target object recognition, and enhances the network's anti-interference ability in different scenarios, especially the ability to recognize small target objects.
Smart Images
Figure CN119919965B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of computer vision, in particular to the field of target object recognition. BACKGROUND
[0002] Object re-identification aims to associate pedestrian images captured by different cameras, and realize the identification and retrieval of target objects such as pedestrians or vehicles across cameras and scenes, and is widely used in the field of intelligent monitoring. In recent years, pedestrian re-identification methods based on deep convolutional networks have achieved excellent results. Most object re-identification methods based on transformers usually divide the entire image into a picture sequence and then use global image tokens as object features.
[0003] Most methods use the method of directly dividing the picture, which not only ignores part of the structure information, but also makes the transformer network converge slowly. In addition, the high-frequency information is not fully considered when extracting information, and the outline and details are ignored, which hinders the identification of objects and their behaviors in the monitoring pictures. SUMMARY
[0004] In order to overcome the above technical defects, the present application provides a target object re-identification method.
[0005] To achieve the above purpose, the present application is realized according to the following technical scheme:
[0006] A target object re-identification method, comprising:
[0007] obtaining a plurality of monitoring pictures corresponding to a target object;
[0008] performing global feature extraction processing on the plurality of monitoring pictures to obtain global features containing the target object;
[0009] performing high-frequency feature extraction processing on the plurality of monitoring pictures to obtain high-frequency features containing the target object;
[0010] fusing the global features containing the target object and the high-frequency features containing the target object to identify the target object.
[0011] Optionally, the global feature extraction processing on the plurality of monitoring pictures to obtain global features containing the target object comprises:
[0012] The plurality of monitoring pictures are token segmented to obtain a plurality of first tokens, and the plurality of first tokens are input into a first transformer network and constrained using an ID loss and a triplet loss, and global features containing the target object are output.
[0013] Optionally, the plurality of monitoring pictures are high-frequency feature extraction processed to obtain high-frequency features containing the target object; the method comprises:
[0014] The plurality of monitoring pictures are subjected to secondary discrete wavelet transform extraction processing to obtain a plurality of high-frequency components containing the target object.
[0015] Based on the plurality of high-frequency components containing the target object, high-frequency features containing the target object are obtained.
[0016] Optionally, the plurality of monitoring pictures are subjected to secondary discrete wavelet transform extraction processing to obtain a plurality of high-frequency components containing the target object, which comprises:
[0017] The plurality of monitoring pictures are subjected to secondary discrete wavelet transform extraction processing to remove a plurality of low-frequency components in the first wavelet transform and the second wavelet transform, thereby obtaining a plurality of high-frequency components containing the target object.
[0018] Optionally, the plurality of high-frequency components containing the target object are used to obtain high-frequency features containing the target object, which comprises:
[0019] The plurality of high-frequency components containing the target object are subjected to wavelet inverse transform reconstruction to obtain a plurality of reconstructed pictures containing the target object; wherein the reconstructed pictures containing the target object are the same size as the corresponding monitoring pictures.
[0020] The plurality of reconstructed pictures containing the target object are input into an edge perception module, the edge perception module uses a pre-trained unet segmentation model to segment the plurality of reconstructed pictures containing the target object to obtain a plurality of segmented binary images containing the target object, the plurality of segmented binary images containing the target object are token segmented to obtain a plurality of second tokens containing the target object, and the plurality of second tokens are comprehensively scored, and the second tokens with a score higher than a first threshold value are output as high-frequency features containing the target object.
[0021] Optionally, the plurality of second tokens are comprehensively scored, which comprises:
[0022] The plurality of second tokens containing the target object and the class token correlation are used to comprehensively score the plurality of second tokens.
[0023] Optionally, the global feature containing the target object and the high-frequency feature containing the target object are fused to identify the target object.
[0024] The global feature containing the target object and the high-frequency feature containing the target object are input into a second transformer network for fusion processing to identify the target object, wherein the first transformer network and the second transformer network share weights.
[0025] Optionally, the fusion processing is at least one of the following fusion manners: attention mechanism fusion, feature splicing fusion, and multi-layer perception fusion.
[0026] The application has the following beneficial effects:
[0027] The target object re-identification method provided by the application provides the overall structure by the global feature and presents the details by the high-frequency feature, so that the combination of the two makes the feature more complete and rich. The identification accuracy can be improved, the complementary information can be used to distinguish the subtle differences, the generalization ability and the anti-interference ability can be improved, the contour feature of the target object is obtained by using the secondary wavelet transform, the ability of the network to identify small target objects is enhanced, and the identification of the target object in different scenes is adapted.
[0028] In addition to the purposes, features and advantages described above, the application has other purposes, features and advantages. The application will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS
[0029] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application, and are incorporated herein for explanation by reference. In the drawings:
[0030] Figure 1 is a flowchart of a target object re-identification process provided by an embodiment of the application.
[0031] Figure 2 is a schematic diagram of a global feature extraction process for a monitoring picture containing a target object provided by an embodiment of the application.
[0032] Figure 3 is a schematic diagram of a wavelet transform process for a monitoring picture containing a target object provided by an embodiment of the application.
[0033] Figure 4 is a schematic diagram of a processing process of an edge perception module for a reconstructed picture containing a target object provided by an embodiment of the application. DETAILED DESCRIPTION
[0034] The embodiments of the present application are described in detail below with reference to the accompanying drawings, but the present application can be implemented in various different ways as defined and covered by the claims.
[0035] Object re-identification is used for target object identification and retrieval across camera scenes, and is widely used in intelligent monitoring. Pedestrian re-identification based on deep convolutional network has excellent results, but most transformer-based methods have problems, direct segmentation of pictures ignores structural information and converges slowly, and also ignores high-frequency information, hindering monitoring object and behavior recognition.
[0036] To solve the above problems, as shown in Figure 1 The present application proposes a target object re-identification method, the specific steps are as follows:
[0037] Step S101: obtaining a plurality of monitoring pictures corresponding to a target object;
[0038] The target object mentioned in the present application is generally an object that can be obtained by a monitoring camera, for example, a pedestrian, a vehicle, and an animal. The plurality of monitoring pictures mentioned in the present application can be a plurality of monitoring pictures of a time line in which the target object appears, or a plurality of monitoring pictures of a space in which the target object appears.
[0039] Step S102: performing global feature extraction processing on the plurality of monitoring pictures to obtain global features containing the target object;
[0040] Global feature extraction is to obtain feature information of the target object from the whole, reflecting its overall properties and structure. As shown in Figure 2 The plurality of monitoring pictures (original pictures) are cut into a plurality of first tokens by global image tokenization, thereby obtaining a plurality of first tokens, each token having a part of the monitoring picture. The plurality of first tokens are input into a first transformer network for network feature extraction D and constrained by ID loss and triplet loss, and global features containing the target object are output.
[0041] The first transformer network is a transformer network, and the transformer is a deep learning architecture mainly composed of an encoder and a decoder, which utilizes self-attention mechanism. It performs well in natural language processing and other fields, and can effectively capture image features, process long-distance dependencies, and improve the performance and efficiency of image recognition, classification and other tasks.
[0042] If a single monitoring picture is R H×W×C If it is cut into m tokens, the sequence dimension β obtained is R (m +1)×DD represents the latent space dimension. The position token dimension is the same as the sequence dimension, i.e. E pos The dimension is R (m+1)×D The input finally entering the first transformer network is i: β + E pos Where a single transformer block can be formulated as:
[0043] i ' = MSA(LN(i l-1 ))+i l-1 (l = 1...L); i1=MLP(LN(i ' ))+i ' (l = 1...L) (1)
[0044] In the formula, MSA is a multi-attention mechanism, and MLP is a multi-layer perception.
[0045]
[0046] In the formula, N is the number of target object categories, q is the supervised label, and p is the predicted label; IDloss is used in this application to enhance the feature discrimination of the target object and stabilize the model (first transformer) training, solve the intra-class difference and inter-class similarity problem, that is: the appearance, posture, etc. of different target objects may have large differences, and the same target object may also have certain changes in different scenes. ID loss can help the model better handle these intra-class differences and inter-class similarities, and through fine adjustment of the features, the model can accurately identify different target objects, even in the case of large intra-class differences or high inter-class similarity, the performance can be maintained.
[0047] Triplet loss extracts K instances for each identity by randomly sampling P identities, forming a minibatch of size P*K; each picture x o is selected as an anchor, the positive sample picture x p with the farthest distance and the negative sample picture x n with the closest distance in the batch are selected to form a triplet to train the first transformer network and enhance the generalization ability of the first transformer network.
[0048] Triplet loss:
[0049]
[0050] After the first transformer network processes the monitoring picture, the global features of the target object can be obtained.
[0051] Step S103: performing high-frequency feature extraction processing on the plurality of monitoring pictures to obtain high-frequency features of the target object;
[0052] The plurality of monitoring pictures containing the target object are processed by secondary discrete wavelet transform to obtain a plurality of high-frequency components containing the target object. The specific process is as follows:
[0053] The input original picture is R H×W×C , the image size is HxW, and after one wavelet transform to extract high-frequency components, LL, LH, HL and HH are obtained, and the sizes of them are all After secondary wavelet transform, LL2, LH2, HL2 and HH2 are obtained, and the sizes of them are all Then, LL and LL2 in the one wavelet transform and the secondary wavelet transform are taken out, that is, the low-frequency components in the one wavelet transform and the secondary wavelet transform are removed, so as to obtain a plurality of high-frequency components containing the target object. The low-frequency component, that is, removing the target background and redundancy, makes the features of the target object more prominent and reduces the interference of the background. The above wavelet transform process is as shown in Figure 3 .
[0054] After obtaining the plurality of high-frequency components containing the target object, the bilinear interpolation method is used to perform wavelet inverse transform reconstruction on the plurality of high-frequency components to obtain a plurality of reconstructed pictures containing the target object. The plurality of reconstructed pictures containing the target object are the same size as the corresponding monitoring pictures.
[0055] As shown in Figure 4 , then the reconstructed picture R H×W×C containing the target object is input to the edge perception module, and the edge perception module uses the pre-trained unet segmentation model to segment the plurality of reconstructed pictures containing the target object to obtain a plurality of segmented binary pictures R H×W×1 containing the target object. The segmented binary picture is used to guide feature extraction to improve the network's ability to extract high-frequency features.
[0056] The plurality of segmented binary pictures containing the target object are further tokenized to obtain a plurality of second tokens containing the target object. Then, the correlation between the plurality of second tokens containing the target object and the class token is used to comprehensively score the plurality of second tokens, so that the second tokens with a score higher than a first threshold value are output as high-frequency features containing the target object. The specific process of scoring is as follows:
[0057] Suppose the input segmented binary picture is divided into n tokens, and the set of tokens is tokens: Next, the linear projection is performed to obtain In addition, the class token is denoted as The input is Assuming that vit has L layers, the score of this part is Wherein Denotes the key, Denotes the query. K represents the number of heads, and sigma represents the sigmoid function.
[0058] In addition, the segmented binary graph containing the target object is denoted as A, and A i Represents the value at the i-th pixel, B i Represents the value at the i-th pixel, and the two are constrained by loss, wherein the loss is Wherein ∈ is a smoothing term, μ is a coefficient and 0<μ<1.
[0059] The final score is:
[0060]
[0061] The higher the score on a certain category, the higher the probability that the detected target belongs to this category.
[0062] The first threshold value can be set according to the specific category of the target object, or can be set according to the past historical recognition experience, and this paper does not make specific restrictions.
[0063] Step S104: The global feature containing the target object and the high-frequency feature containing the target object are fused and processed to identify the target object.
[0064] After obtaining the global feature containing the target object and the high-frequency feature containing the target object, the global feature containing the target object and the high-frequency feature containing the target object are input into the second transformer network for fusion processing to identify the target object. If the target object is a pedestrian, the behavior of the target object can also be identified.
[0065] Wherein, the first transformer network and the second transformer network share weights. The meaning of sharing weights is that when the network processes the same type of data, the knowledge learned by one network is directly transferred to another network through weight sharing, improving the generalization ability of the network. In addition, in the above fusion processing process, balanced loss is used to balance the high-frequency feature and the global feature, so as to avoid losing the original visual information.
[0066] Assuming that T g Represents the tokens of the global feature, T h Represents the tokens of the high-frequency feature. The balanced loss formula is as follows:
[0067]
[0068] In the formula, C represents the first input monitoring picture, and D represents the first token in the input.
[0069] The second transformer network is also a transformer network, which is substantially the same as the first transformer network. When performing fusion processing, at least one of the following fusion modes is selected: attention mechanism fusion, feature splicing fusion, and multi-layer perception fusion. When selecting the fusion mode, the corresponding fusion mode can also be selected according to different target objects.
[0070] The above method used in the present application uses a two-level wavelet transform to obtain the contour features of the target object, enhances the ability of the network to identify small target objects, and uses a segmentation map to guide feature extraction, thereby improving the ability of the network to extract high-frequency features. A comprehensive scoring mechanism is used to score the tokens from two aspects of contour accuracy and category sensitivity, thereby finally improving the recognition accuracy of the target object. The global features provide the overall structure, and the high-frequency features present the details, and the combination of the two makes the features more complete and rich. The recognition accuracy can be improved, the complementary information can be used to distinguish subtle differences, the generalization ability and the anti-interference ability can be improved, and the target object recognition in different scenes can be adapted.
[0071] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A target object re-identification method, characterized in that, The method comprises the following steps: obtaining a plurality of monitoring pictures corresponding to a target object; performing global feature extraction processing on the plurality of monitoring pictures to obtain global features of the target object; performing high-frequency feature extraction processing on the plurality of monitoring pictures to obtain high-frequency features of the target object; fusing the global features of the target object and the high-frequency features of the target object to identify the target object; wherein the high-frequency feature extraction processing on the plurality of monitoring pictures to obtain the high-frequency features of the target object comprises: performing secondary discrete wavelet transform extraction processing on the plurality of monitoring pictures to obtain a plurality of high-frequency components of the target object; performing inverse wavelet transform reconstruction on the plurality of high-frequency components of the target object to obtain a plurality of reconstructed pictures of the target object; wherein the plurality of reconstructed pictures of the target object are the same size as the corresponding monitoring pictures; inputting the plurality of reconstructed pictures of the target object into an edge perception module, using a pre-trained unet segmentation model to segment the plurality of reconstructed pictures of the target object to obtain a plurality of segmented binary maps of the target object, performing token segmentation on the plurality of segmented binary maps of the target object to obtain a plurality of second tokens of the target object, and performing comprehensive scoring on the plurality of second tokens to output the second tokens with a score higher than a first threshold as the high-frequency features of the target object.
2. The method of claim 1, wherein, The global feature extraction processing on the plurality of monitoring pictures to obtain the global features of the target object comprises: performing token segmentation on the plurality of monitoring pictures to obtain a plurality of first tokens, inputting the plurality of first tokens into a first transformer network, and using ID loss and triplet loss for constraint to output the global features of the target object.
3. The method of claim 1, wherein, The secondary discrete wavelet transform extraction processing on the plurality of monitoring pictures to obtain the plurality of high-frequency components of the target object comprises: performing secondary discrete wavelet transform extraction processing on the plurality of monitoring pictures to remove a plurality of low-frequency components in the first wavelet transform and the second wavelet transform to obtain the plurality of high-frequency components of the target object.
4. The method of claim 1, wherein, The comprehensive scoring on the plurality of second tokens comprises: comprehensive scoring on the plurality of second tokens by using the correlation between the plurality of second tokens of the target object and a class token.
5. The method of claim 2, wherein, The fusion processing of the global features of the target object and the high-frequency features of the target object to identify the target object comprises: inputting the global features of the target object and the high-frequency features of the target object into a second transformer network for fusion processing to identify the target object; wherein the first transformer network and the second transformer network share weights.
6. The method of claim 5, wherein, The fusion processing mode is at least one of the following fusion modes: attention mechanism fusion, feature splicing fusion, and multi-layer perception fusion.