Target object re-identification method

Abstract

The invention provides a target object re-identification method. The target object re-identification method comprises the following steps: acquiring a plurality of monitoring pictures corresponding to a target object; performing global feature extraction processing on the plurality of monitoring pictures to obtain a global feature containing a target object; performing high-frequency feature extraction processing on the plurality of monitoring pictures to obtain high-frequency features including a target object; and performing fusion processing on the global feature containing the target object and the high-frequency feature containing the target object, and identifying the target object. The global features provide an overall structure, and the high-frequency features present details, so that the features are richer and more complete. According to the method, the identification accuracy can be improved, subtle differences are distinguished by using complementary information, the generalization capability and the anti-interference capability can be improved, the contour features of the target object are obtained by using secondary wavelet transform, the capability of the network for identifying the small target object is enhanced, and the method is suitable for identification of the target object in different scenes.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to the field of target object recognition. Background Art

[0002] Object re-identification aims to associate pedestrian images captured by different cameras, and realize the recognition and retrieval of target objects such as pedestrians or vehicles across cameras and scenes. It is widely used in the field of intelligent monitoring. In recent years, pedestrian re-identification methods based on deep convolutional networks have achieved excellent results. Most transformer-based object re-identification methods usually segment the entire image as an image sequence and feed it into the transformer network, and then use the global image tag (token) as the object feature.

[0003] Most methods use direct image segmentation, which not only ignores some structural information, but also slows down the convergence of the transformer network. In addition, high-frequency information is not fully considered when extracting information, and contours and details are ignored. This situation causes certain obstacles in the process of identifying objects and their behaviors in surveillance images. Summary of the invention

[0004] In order to overcome the above technical defects, the present application provides a target object re-identification method.

[0005] To achieve the above objectives, this application is implemented according to the following technical solutions:

[0006] A target object re-identification method, comprising:

[0007] Obtain multiple surveillance images corresponding to the target object;

[0008] Performing global feature extraction processing on the multiple monitoring images to obtain global features containing the target object;

[0009] Performing high-frequency feature extraction processing on the multiple monitoring images to obtain high-frequency features containing the target object;

[0010] The global features including the target object and the high-frequency features including the target object are fused to identify the target object.

[0011] Optionally, performing global feature extraction processing on the multiple surveillance images to obtain global features containing the target object includes:

[0012] The multiple monitoring images are tokenized to obtain multiple first tokens, and the multiple first tokens are input into a first transformer network, and are constrained by using ID loss and triplet loss to output global features containing the target object.

[0013] Optionally, the step of performing high-frequency feature extraction processing on the plurality of surveillance images to obtain high-frequency features containing the target object comprises:

[0014] Performing secondary discrete wavelet transform extraction processing on the multiple monitoring images to obtain multiple high-frequency components containing the target object;

[0015] Based on the multiple high-frequency components containing the target object, a high-frequency feature containing the target object is obtained.

[0016] Optionally, the performing of secondary discrete wavelet transform extraction processing on the multiple surveillance images to obtain multiple high-frequency components containing the target object includes:

[0017] The plurality of monitoring images are subjected to secondary discrete wavelet transform extraction processing, and a plurality of low-frequency components in the primary wavelet transform and the secondary wavelet transform are removed to obtain a plurality of high-frequency components containing the target object.

[0018] Optionally, obtaining a high-frequency feature containing the target object based on the multiple high-frequency components containing the target object includes:

[0019] The multiple high-frequency components containing the target object are reconstructed by inverse wavelet transform to obtain multiple reconstructed pictures containing the target object; wherein the reconstructed pictures containing the target object have the same size as their corresponding monitoring pictures.

[0020] The multiple reconstructed images containing the target objects are input into an edge perception module, and the edge perception module uses a pre-trained UNET segmentation model to segment the multiple reconstructed images containing the target objects to obtain multiple segmentation binary images containing the target objects, tokenize the multiple segmentation binary images containing the target objects to obtain multiple second tokens containing the target objects, comprehensively score the multiple second tokens, and output the second tokens with scores higher than a first threshold as high-frequency features containing the target objects.

[0021] Optionally, the comprehensively scoring the plurality of second tokens includes:

[0022] The plurality of second tokens containing the target objects are comprehensively scored by utilizing the correlation between the plurality of second tokens and the class token.

[0023] Optionally, the fusing of the global features containing the target object and the high-frequency features containing the target object to identify the target object comprises:

[0024] The global features containing the target object and the high-frequency features containing the target object are input into a second transformer network for fusion processing to identify the target object; wherein the first transformer network and the second transformer network share weights.

[0025] Optionally, the fusion processing method is at least one of the following fusion methods: attention mechanism fusion, feature splicing fusion, and multi-layer perceptron fusion.

[0026] This application has the following beneficial effects:

[0027] The target object re-identification method provided by this application uses global features to provide the overall structure and high-frequency features to present details. The combination of the two makes the features richer and more complete. It can improve recognition accuracy, use complementary information to distinguish subtle differences, and improve generalization and anti-interference capabilities. It also uses secondary wavelet transform to obtain the contour features of the target object, enhance the network's ability to identify small target objects, and adapt to the recognition of target objects in different scenarios.

[0028] In addition to the above-described purposes, features and advantages, the present application has other purposes, features and advantages. The present application will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0030] Figure 1 It is a flowchart of a target object re-identification provided in an embodiment of the present application.

[0031] Figure 2 It is a schematic diagram of a global feature extraction process for a monitoring image containing a target object provided in an embodiment of the present application.

[0032] Figure 3 It is a schematic diagram of a wavelet transform process for a monitoring image containing a target object provided in an embodiment of the present application.

[0033] Figure 4 It is a schematic diagram of the processing process of the edge perception module provided in an embodiment of the present application on a reconstructed image containing a target object. DETAILED DESCRIPTION

[0034] The embodiments of the present application are described in detail below with reference to the accompanying drawings; however, the present application can be implemented in many different ways as defined and covered by the claims.

[0035] Object re-identification is used to identify and retrieve target objects in cross-camera scenarios and is widely used in intelligent surveillance. Pedestrian re-identification based on deep convolutional networks has achieved excellent results, but most transformer-based methods have problems. Direct image segmentation ignores structural information and converges slowly. It also ignores high-frequency information, which hinders the recognition of monitored objects and behaviors.

[0036] In order to solve the above problems, Figure 1 As shown, this application proposes a target object re-identification method, the specific steps are as follows:

[0037] Step S101: Acquire multiple surveillance images corresponding to the target object;

[0038] The target object mentioned in this application is generally an object that can be captured by a surveillance camera, such as a pedestrian, a vehicle, or an animal. The multiple surveillance images mentioned in this application can be multiple surveillance images of the timeline in which the target object appears, or multiple surveillance images of the space in which the target object appears.

[0039] Step S102: performing global feature extraction processing on the multiple monitoring images to obtain global features containing the target object;

[0040] Global feature extraction is to obtain the feature information of the target object as a whole, reflecting its overall attributes and structure. Figure 2 As shown, multiple monitoring pictures (original pictures) are segmented by global image tags (tokens), thereby obtaining multiple first tokens, wherein each of the multiple first tokens has a part of the monitoring picture. The multiple first tokens are input into the first transformer network, and network feature extraction D is performed and constrained using ID loss and triplet loss, and the global features containing the target object are output.

[0041] The first transformer network is the transformer network, which is a deep learning architecture mainly composed of an encoder and a decoder, and uses a self-attention mechanism. It performs well in fields such as natural language processing. In image processing, it can also effectively capture image features, handle long-distance dependencies, and improve the performance and efficiency of tasks such as image recognition and classification.

[0042] If a single monitoring image is R H×W×C , if it is divided into m tokens, the resulting sequence dimension β is R (m +1)×D,D represents the latent space dimension. The position token dimension is the same as the sequence dimension, that is, E pos The dimension is R (m+1)×D , the final input to the first converter network is i:β+E pos . A single transformer block can be formulated as:

[0043] i′=MSA(LN(i l-1 ))+i l-1 (l=1....L);i 1 =MLP(LN(i′))+i′(l=1...L) (1)

[0044] In the formula, MSA is the multi-attention mechanism and MLP is the multi-layer perceptron.

[0045]

[0046] Where N is the number of target object categories, q is the supervision label, and p is the predicted label; ID loss is used here in this application to enhance the feature discrimination of the target object and stabilize the model (first transformer) training to solve the intra-class difference and inter-class similarity problems, that is, the appearance and posture of different target objects may be quite different, and the same target object may also have certain changes in different scenes. ID loss can help the model better handle these intra-class differences and inter-class similarities. By fine-tuning the features, the model can accurately identify different target objects, and maintain good performance even when the intra-class differences are large or the inter-class similarities are high.

[0047] Triplet loss randomly samples P identities, extracts K instances from each identity, and forms a minibatch of size P*K; selects each image x in the batch in turn. o As an anchor point, select the positive sample image x with the longest distance in the batch p and the nearest negative sample image x n The triplets are formed to train the first transformer network, thereby enhancing the generalization ability of the first transformer network.

[0048]

[0049] After the monitoring image is processed by the first transformer network, the global features of the target object can be obtained.

[0050] Step S103: performing high-frequency feature extraction processing on the plurality of monitoring images to obtain high-frequency features containing the target object;

[0051] Multiple surveillance images containing target objects are processed by secondary discrete wavelet transform to obtain multiple high-frequency components containing target objects. The specific process is as follows:

[0052] Input the original image as R H×W×C , the image size is H×W, after a wavelet transform to extract the high-frequency components, we get LL, LH, HL and HH, their sizes are After the second wavelet transform, we get LL2, LH2, HL2, and HH2, and their sizes are Then, LL and LL2 in the first wavelet transform and the second wavelet transform are taken out, that is, the low-frequency components in the first wavelet transform and the second wavelet transform are removed, so as to obtain multiple high-frequency components containing the target object. The low-frequency components, that is, the target background and redundancy are removed, so that the characteristics of the target object are more prominent and the background interference is reduced. The above wavelet transform process, such as Figure 3 shown.

[0053] After obtaining multiple high-frequency components containing the target object, the multiple high-frequency components are reconstructed by wavelet inverse transform using a bilinear interpolation method to obtain multiple reconstructed images containing the target object. These multiple reconstructed images containing the target object are the same size as their corresponding monitoring images.

[0054] like Figure 4 As shown, the reconstructed image R containing the target object is then H×W×C , which is input to the edge perception module. The edge perception module uses the pre-trained unet segmentation model to segment multiple reconstructed images containing target objects and obtain multiple segmentation binary images R containing target objects. H×W×1 . Use segmentation binary images to guide feature extraction and improve the network's ability to extract high-frequency features.

[0055] Then tokenize multiple segmented binary images containing target objects to obtain multiple second tokens containing target objects. Then, use the correlation between multiple second tokens containing target objects and class token to comprehensively score multiple second tokens, so that the second tokens with scores higher than the first threshold are output as high-frequency features containing target objects. The specific scoring process is as follows:

[0056] Assume that the input segmentation binary image is divided into n tokens, the set Next will Perform linear projection to obtain In addition, class token is recorded as The input is Assuming that vit has L layers, the score for this part is in Indicates key represents query. K represents the number of heads, and σ represents the sigmoid function.

[0057] In addition, the segmentation binary image containing the target object is recorded as A, A i represents the value at the i-th pixel, B i Represents the value at the i-th pixel, and the two are constrained by loss. In this case, the loss is Where ∈ is a smoothing term, μ is a coefficient and 0<μ<1.

[0058] Final rating:

[0059]

[0060] The higher the score for a certain category, the higher the probability that the detected target belongs to this category.

[0061] The first threshold value can be set by yourself according to the specific category of the target object, or it can be set according to previous historical recognition experience. This article does not make any specific restrictions on this.

[0062] Step S104: fusing the global features containing the target object and the high-frequency features containing the target object to identify the target object.

[0063] After obtaining the global features and high-frequency features of the target object, the global features and high-frequency features are input into the second transformer network for fusion processing to identify the target object. If the target object is a pedestrian, the behavior of the target object can also be identified.

[0064] The first transformer network and the second transformer network share weights. The significance of sharing weights is that when the networks process the same type of data, the knowledge learned by one network is directly transferred to another network through weight sharing, thereby improving the generalization ability of the network. In addition, in the above fusion process, balanced loss is used to balance high-frequency features and global features to avoid losing original visual information.

[0065] Assume T g Tokens representing global features, T h Represents high-frequency feature tokens. The balanced loss formula is as follows:

[0066]

[0067] In the formula, C represents the number of the input monitoring image, and D represents the number of tokens in the input.

[0068] The second transformer network is also a transformer network, which is substantially the same as the first transformer network. When performing the fusion process, at least one of the following fusion methods is selected: attention mechanism fusion, feature splicing fusion, and multi-layer perceptron fusion. When selecting a fusion method, the corresponding fusion method can also be selected according to different target objects.

[0069] The above method used in this application uses a secondary wavelet transform to obtain the contour features of the target object, enhance the network's ability to identify small target objects, and use a segmentation map to guide feature extraction and enhance the network's ability to extract high-frequency features. Using a comprehensive scoring mechanism, tokens are scored from two aspects: contour accuracy and category sensitivity, ultimately improving the recognition accuracy of the target object. Global features provide the overall structure, and high-frequency features present details. The combination of the two makes the features richer and more complete. It can improve recognition accuracy, use complementary information to distinguish subtle differences, and improve generalization and anti-interference capabilities to adapt to the recognition of target objects in different scenarios.

[0070] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A target object re-identification method, characterized in that: include: Obtain multiple surveillance images corresponding to the target object; Performing global feature extraction processing on the multiple monitoring images to obtain global features containing the target object; Performing high-frequency feature extraction processing on the multiple monitoring images to obtain high-frequency features containing the target object; The global features including the target object and the high-frequency features including the target object are fused to identify the target object.

2. The method according to claim 1, characterized in that: The performing global feature extraction processing on the multiple monitoring images to obtain global features containing the target object includes: The multiple monitoring images are tokenized to obtain multiple first tokens, and the multiple first tokens are input into a first transformer network, and are constrained by using ID loss and triplet loss to output global features containing the target object.

3. The method according to claim 2, characterized in that The high-frequency feature extraction process is performed on the plurality of monitoring images to obtain high-frequency features containing the target object; comprising: Performing secondary discrete wavelet transform extraction processing on the multiple monitoring images to obtain multiple high-frequency components containing the target object; Based on the multiple high-frequency components containing the target object, a high-frequency feature containing the target object is obtained.

4. The method according to claim 3, characterized in that The performing of secondary discrete wavelet transform extraction processing on the plurality of monitoring images to obtain a plurality of high-frequency components containing the target object includes: The plurality of monitoring images are subjected to secondary discrete wavelet transform extraction processing, and a plurality of low-frequency components in the primary wavelet transform and the secondary wavelet transform are removed to obtain a plurality of high-frequency components containing the target object.

5. The method according to claim 3 or 4, characterized in that: The obtaining of high-frequency features containing the target object based on the multiple high-frequency components containing the target object comprises: The multiple high-frequency components containing the target object are reconstructed by inverse wavelet transform to obtain multiple reconstructed pictures containing the target object; wherein the reconstructed pictures containing the target object have the same size as their corresponding monitoring pictures. The multiple reconstructed images containing the target objects are input into an edge perception module, and the edge perception module uses a pre-trained UNET segmentation model to segment the multiple reconstructed images containing the target objects to obtain multiple segmentation binary images containing the target objects, tokenize the multiple segmentation binary images containing the target objects to obtain multiple second tokens containing the target objects, comprehensively score the multiple second tokens, and output the second tokens with scores higher than a first threshold as high-frequency features containing the target objects.

6. The method according to claim 4, characterized in that The comprehensively scoring the plurality of second tokens includes: The plurality of second tokens containing the target objects are comprehensively scored by utilizing the correlation between the plurality of second tokens and the class token.

7. The method according to claim 5, characterized in that The fusing process of the global features containing the target object and the high-frequency features containing the target object to identify the target object comprises: The global features containing the target object and the high-frequency features containing the target object are input into a second transformer network for fusion processing to identify the target object; wherein the first transformer network and the second transformer network share weights.

8. The method according to claim 6, characterized in that: The fusion processing method is at least one of the following fusion methods: attention mechanism fusion, feature splicing fusion, and multi-layer perceptron fusion.

Images