Pedestrian target association method fusing local attention and visual similarity
Patent Information
- Application Number
- CN202511280019.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-09
- Publication Date
- 2026-10-16
- Estimated Expiration
- 2045-09-09
AI Technical Summary
[0002]现有的行人目标关联方法存在相似区域干扰及视觉冲突问题，导致复杂场景下行人目标关联的准确性较低、鲁棒性较差
[0034] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a pedestrian target association method that integrates local attention and visual similarity, which can effectively alleviate the problems of interference in similar areas and visual conflict, and significantly improve the accuracy and robustness of pedestrian target association in complex scenes.
Smart Images
Figure CN121191191B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and more specifically to a method for pedestrian target association that integrates local attention and visual similarity. Background Technology
[0002] Existing methods for pedestrian target association suffer from interference from similar regions and visual conflicts, resulting in low accuracy and poor robustness in pedestrian target association in complex scenarios.
[0003] Therefore, how to provide a pedestrian target association method that can effectively alleviate interference and visual conflict in similar areas and significantly improve the accuracy and robustness of pedestrian target association in complex scenes is an urgent problem to be solved in this field. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide a pedestrian target association method that integrates local attention and visual similarity.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] A pedestrian target association method that integrates local attention and visual similarity includes the following steps:
[0007] The pedestrian image to be associated is input into a trained human feature extraction network model that integrates local attention and visual similarity to obtain the global features and N local features of the pedestrian image to be associated; where N≥2;
[0008] The global features and N local features of the pedestrian image to be associated are concatenated to obtain the overall features of the pedestrian image to be associated.
[0009] The overall feature with the highest similarity to the overall feature of the pedestrian image to be associated is selected from the preset overall feature library, and the pedestrian image corresponding to the overall feature with the highest similarity is used as the associated image of the pedestrian image to be associated.
[0010] The acquisition method of each overall feature in the preset overall feature library is consistent with the acquisition method of the overall features of the pedestrian image to be associated.
[0011] Preferably, the basic architecture of the human feature extraction network model that integrates local attention and visual similarity adopts the Transformer deep learning architecture for visual tasks;
[0012] The human feature extraction network model that integrates local attention and visual similarity adds an overall token and N local tokens to the image block token sequence;
[0013] The human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the local token and the image block at a preset position in the input image when calculating the attention of each local token. The attention score between the local token and the remaining image blocks in the input image is set to negative infinity. The image block at the preset position in the input image is the image block in the input image that the local token is interested in.
[0014] Preferably, the trained human feature extraction network model that integrates local attention and visual similarity is obtained based on the following steps:
[0015] S1: Initialize the local central dynamic library;
[0016] S2: Input the training sample images into the human keypoint detection network model to obtain the keypoint heatmap of the training sample images;
[0017] The key point heatmap of the training sample image is divided to obtain N local key point heatmaps; wherein, the human body regions of interest in the N local key point heatmaps correspond one-to-one with the human body regions of interest in the N local tokens.
[0018] The training sample image and the N local key point heatmaps are subjected to the same random transformation and then input into the human feature extraction network model that integrates local attention and visual similarity to obtain the N local features of the training sample image, the attention map of the N local features, and the predicted identity information.
[0019] The N local key point heatmaps are sequentially binarized and normalized at the image block level to obtain N normalized local key point heatmaps.
[0020] To minimize the total loss function, the network parameters of the human feature extraction network model that integrates local attention and visual similarity are updated using the backpropagation method; wherein, the total loss function is obtained based on N local features of the training sample image, the attention map and predicted identity information of the N local features, and the N normalized local keypoint heatmaps;
[0021] The local center dynamic library is updated using all local features obtained from the current batch of training sample images.
[0022] S3: Repeat S2 continuously to obtain the trained human feature extraction network model that integrates local attention and visual similarity.
[0023] Preferably, initializing the local central dynamic library specifically includes the following steps:
[0024] All training sample images are input into a human feature extraction network model that integrates local attention and visual similarity to obtain local features of each training sample image; each training sample image obtains N local features, and each training sample image is labeled with the pedestrian's orientation information and identity information;
[0025] Local features of the same identity information, same orientation information, and same human body region are classified into the same type of features to obtain an initialized local center dynamic library; wherein, each type of feature is set with a feature center. If there is only one local feature of the same identity information, same orientation information, and same human body region, then the feature center is the local feature; if there are multiple local features of the same identity information, same orientation information, and same human body region, then the feature center is the average value of the multiple local features.
[0026] Preferably, the total loss function is obtained by weighted summation of the local attention loss, visual similarity loss, identity information classification loss, and global feature triplet loss.
[0027] Preferably, the local attention loss is obtained by weighted summation of the regularization loss and the alignment loss; wherein, the regularization loss is the sum of the regularization losses of N local features; the alignment loss is the sum of N KL divergences; and the KL divergence of the attention map of a local feature and the KL divergence of the normalized local keypoint heatmap corresponding to the same human body region constitutes one of the KL divergences.
[0028] Preferably, the visual similarity loss is obtained by weighted summation of the local clustering loss and the triplet loss of local features;
[0029] The local clustering loss is the sum of the losses of N centers; the center loss of a local feature and the feature centers of the same type of feature constitutes one of the center losses.
[0030] The triplet loss of the local feature is the sum of the triplet losses of N local features; if the distance difference between two local features with different identity information, the same orientation information, and the same human body region and the feature center of their respective types is less than a preset threshold, then the triplet loss of the local feature is zero.
[0031] Preferably, updating the local center dynamic library using all local features obtained from the current batch of training sample images specifically includes the following steps:
[0032] All local features obtained from the current batch of training sample images are input into the local center dynamic library, and the feature centers of various features are updated using momentum update.
[0033] Preferably, the attention map is the average value of the attention maps output by all Transformer encoder layers in ViT.
[0034] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a pedestrian target association method that integrates local attention and visual similarity, which can effectively alleviate the problems of interference in similar areas and visual conflict, and significantly improve the accuracy and robustness of pedestrian target association in complex scenes. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0036] Figure 1 This is a flowchart of a pedestrian target association method that integrates local attention and visual similarity, provided in an embodiment of the present invention. Detailed Implementation
[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0038] like Figure 1 As shown, a pedestrian target association method that integrates local attention and visual similarity includes the following steps:
[0039] The pedestrian image to be associated is input into a trained human feature extraction network model that integrates local attention and visual similarity to obtain the global features and N local features of the pedestrian image to be associated; where N≥2;
[0040] It is understandable that local features can be head features, trunk features, leg features, foot features, upper features (i.e., head features plus trunk features), middle features (i.e., trunk features plus leg features, lower limb features (i.e., leg features plus foot features), lower head features (i.e., trunk features plus leg features plus foot features), and upper foot features (i.e., head features plus trunk features plus leg features).
[0041] In one or more embodiments, the architecture of the human feature extraction network model that integrates local attention and visual similarity adopts a Transformer deep learning architecture for visual tasks.
[0042] It is understood that the basic architecture of the human feature extraction network model that integrates local attention and visual similarity can adopt various Transformer deep learning architectures for vision tasks, such as ViT.
[0043] The human feature extraction network model that integrates local attention and visual similarity adds an overall token and N local tokens to the image block token sequence;
[0044] The human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the local token and the image block at a preset position in the input image when calculating the attention of each local token. The attention score between the local token and the remaining image blocks in the input image is set to negative infinity. The image block at the preset position in the input image is the image block in the input image that the local token is interested in.
[0045] It should be noted that the image patches in the input image that each local token focuses on do not overlap.
[0046] In one embodiment, the N local features are head features, trunk features, and lower limb features;
[0047] The human feature extraction network model that integrates local attention and visual similarity adds an overall token and three local tokens to the image block token sequence; wherein, the three local tokens include a head token, a torso token, and a lower limb token;
[0048] When calculating the attention of the head token, the human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the head token and the upper 1 / 3 of the input image, and sets the attention score between the head token and the remaining 2 / 3 of the image to negative infinity.
[0049] When calculating the attention of the torso token, the human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the torso token and the middle 1 / 3 of the input image, and sets the attention score between the torso token and the remaining 2 / 3 of the image to negative infinity.
[0050] When calculating the attention of the lower limb token, the human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the lower limb token and the lower 1 / 3 of the input image, and sets the attention score between the lower limb token and the remaining 2 / 3 of the image to negative infinity.
[0051] In one embodiment, the N local features are head features and lower limb features;
[0052] The human feature extraction network model that integrates local attention and visual similarity adds an overall token and two local tokens to the image block token sequence; wherein, the two local tokens include a head token and a lower limb token;
[0053] When calculating the attention of the head token, the human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the head token and the upper 1 / 3 of the input image, and sets the attention score between the head token and the remaining 2 / 3 of the image to negative infinity.
[0054] When calculating the attention of the lower limb token, the human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the lower limb token and the lower 1 / 3 of the input image, and sets the attention score between the lower limb token and the remaining 2 / 3 of the image to negative infinity.
[0055] The processing methods for selecting other different local features are similar and will not be elaborated here.
[0056] The global features and N local features of the pedestrian image to be associated are concatenated to obtain the overall features of the pedestrian image to be associated.
[0057] In one or more embodiments, N local features and global features of the pedestrian image to be associated are concatenated according to preset weights and then normalized to obtain the overall features of the pedestrian image to be associated.
[0058] The overall feature with the highest similarity to the overall feature of the pedestrian image to be associated is selected from the preset overall feature library, and the pedestrian image corresponding to the overall feature with the highest similarity is used as the associated image of the pedestrian image to be associated.
[0059] In one or more embodiments, the similarity is cosine similarity.
[0060] The acquisition method of each overall feature in the preset overall feature library is consistent with the acquisition method of the overall features of the pedestrian image to be associated.
[0061] In one or more embodiments, the trained human feature extraction network model that integrates local attention and visual similarity is obtained based on the following steps:
[0062] S1: Initialize the local central dynamic library;
[0063] In one or more embodiments, initializing the local central dynamic library specifically includes the following steps:
[0064] All training sample images are input into a human feature extraction network model that integrates local attention and visual similarity to obtain local features of each training sample image; each training sample image obtains N local features, and each training sample image is labeled with the pedestrian's orientation information and identity information;
[0065] Local features of the same identity information, same orientation information, and same human body region are classified into the same type of features to obtain an initialized local center dynamic library; wherein, each type of feature is set with a feature center. If there is only one local feature of the same identity information, same orientation information, and same human body region, then the feature center is the local feature; if there are multiple local features of the same identity information, same orientation information, and same human body region, then the feature center is the average value of the multiple local features.
[0066] It is understandable to take N local features as examples, namely head features, trunk features, and lower limb features:
[0067] Initializing the local central dynamic library specifically includes:
[0068] Head features with the same identity information and the same orientation information are classified into the same type of features;
[0069] If there is only one head feature for the same identity information and the same orientation information, then the feature center of this type of feature is that head feature; if there are N1 head features for the same identity information and the same orientation information, then the feature center of this type of feature is the average of the N1 head features; N1 is a positive integer greater than 1.
[0070] Classify torso features with the same identity information and the same orientation information into the same type of features;
[0071] If there is only one torso feature with the same identity information and orientation information, then the feature center of that type of feature is that torso feature; if there are N2 torso features with the same identity information and orientation information, then the feature center of that type of feature is the average of the N2 torso features; N2 is a positive integer greater than 1.
[0072] Lower limb features with the same identity information and the same orientation information are classified into the same type of features;
[0073] If there is only one lower limb feature with the same identity information and the same orientation information, then the feature center of this type of feature is that lower limb feature; if there are N3 lower limb features with the same identity information and the same orientation information, then the feature center of this type of feature is the average of the N3 lower limb features; N3 is a positive integer greater than 1.
[0074] S2: Input the training sample images into the human keypoint detection network model to obtain the keypoint heatmap of the training sample images;
[0075] In one or more embodiments, the human keypoint detection network model adopts any open-source human keypoint detection network model, such as HRNet or OpenPose.
[0076] The key point heatmap of the training sample image is divided to obtain N local key point heatmaps; wherein, the human body regions of interest in the N local key point heatmaps correspond one-to-one with the human body regions of interest in the N local tokens.
[0077] The training sample image and the N local key point heatmaps are subjected to the same random transformation and then input into the human feature extraction network model that integrates local attention and visual similarity to obtain the N local features of the training sample image, the attention map of the N local features, and the predicted identity information.
[0078] The N local key point heatmaps are sequentially binarized and normalized at the image block level to obtain N normalized local key point heatmaps.
[0079] To minimize the total loss function, the network parameters of the human feature extraction network model that integrates local attention and visual similarity are updated using the backpropagation method; wherein, the total loss function is obtained based on N local features of the training sample image, the attention map and predicted identity information of the N local features, and the N normalized local keypoint heatmaps;
[0080] It is understandable to take N local features as examples, namely head features, trunk features, and lower limb features:
[0081] Specifically:
[0082] The key point heatmaps of the training sample images are divided into head key point heatmaps, trunk key point heatmaps, and lower limb key point heatmaps.
[0083] Specifically, based on the key point locations of the key point heatmaps of the training sample images, they are divided into head key point heatmaps, trunk key point heatmaps, and lower limb key point heatmaps.
[0084] The training sample image, the head key point heatmap, the torso key point heatmap, and the lower limb key point heatmap are subjected to the same random transformation and then input into the human feature extraction network model that integrates local attention and visual similarity to obtain the head features, torso features, lower limb features, attention maps of head features, attention maps of torso features, attention maps of lower limb features, and predicted identity information of the training sample image.
[0085] It is understandable that the random number generator state is saved before the random transformation and restored after the random transformation.
[0086] In one or more embodiments, the attention map is the average of the attention maps output by all Transformer encoder layers in ViT.
[0087] The head key point heatmap, the torso key point heatmap, and the lower limb key point heatmap are subjected to image block-level binarization and normalization to obtain normalized head key point heatmap, normalized torso key point heatmap, and normalized lower limb key point heatmap.
[0088] Specifically:
[0089] The head key point heatmap is divided into several image blocks (the size of the image blocks is the same as the size of the image blocks divided by ViT). The block value of the image block that includes key points is set to 1, and the block value of the image block that does not include key points is set to 0. The image blocks with a block value of 1 are divided by M1 to obtain the normalized head key point heatmap, where M1 represents the number of image blocks with a block value of 1 in the head key point heatmap.
[0090] The torso key point heatmap is divided into several image blocks (the size of the image blocks is the same as the size of the image blocks divided by ViT). The block value of the image block that includes the key points is set to 1, and the block value of the image block that does not include the key points is set to 0. The image blocks with a block value of 1 are divided by M2 to obtain the normalized torso key point heatmap, where M2 represents the number of image blocks with a block value of 1 in the torso key point heatmap.
[0091] The lower limb key point heatmap is divided into several image blocks (the size of the image blocks is the same as the size of the image blocks divided by ViT). The block value of the image block that includes the key point is set to 1, and the block value of the image block that does not include the key point is set to 0. The image blocks with a block value of 1 are divided by M3 to obtain the normalized lower limb key point heatmap, where M3 represents the number of image blocks with a block value of 1 in the lower limb key point heatmap.
[0092] To minimize the total loss function, the network parameters of the human feature extraction network model that integrates local attention and visual similarity are updated using the backpropagation method. The total loss function is obtained based on the head features, trunk features, lower limb features, attention maps of the head features, trunk features, and lower limb features of the training sample images, the predicted identity information, the normalized head keypoint heatmap, the normalized trunk keypoint heatmap, and the normalized lower limb keypoint heatmap.
[0093] In one or more embodiments, the total loss function is obtained by weighted summation of local attention loss, visual similarity loss, identity information classification loss, and global feature triplet loss.
[0094] Specifically: the expression for the total loss function is:
[0095] L total =λ1L attention +λ2L visul +λ3L id +λ4L T-triplet ;
[0096] In the formula, L total The total loss function is represented by λ1, λ2, λ3, and λ4, which represent weighting coefficients; L attention Indicates the local attention loss, L visul L represents the visual similarity loss; id The loss represents the classification loss of the identity information, L. T-triplet The triplet loss represents the global feature.
[0097] In one or more embodiments, the identity information classification loss is the central loss between the labeled identity information and the predicted identity information.
[0098] In one or more embodiments, the local attention loss is obtained by weighted summation of the regularization loss and the alignment loss; wherein the regularization loss is the sum of the regularization losses of N local features; the alignment loss is the sum of N KL divergences; and the KL divergence of the attention map of a local feature and the KL divergence of the normalized local keypoint heatmap corresponding to the same human body region constitutes one of the KL divergences.
[0099] Specifically, the expression for the local attention loss is:
[0100] L attention =w1L intra +w2L KL ;
[0101] In the formula, w1 and w2 represent weighting coefficients; L intra L represents the regularization loss;KL This represents the alignment loss.
[0102] Taking N local features as head features, trunk features, and lower limb features as an example: the regularization loss is the sum of the regularization losses of head features, trunk features, and lower limb features; the alignment loss is the sum of three KL divergences; the three KL divergences are the KL divergence between the attention map of the head features and the normalized head keypoint heatmap, the KL divergence between the attention map of the trunk features and the normalized trunk keypoint heatmap, and the KL divergence between the attention map of the lower limb features and the normalized lower limb keypoint heatmap.
[0103] In one or more embodiments, the visual similarity loss is obtained by weighted summation of the local clustering loss and the triplet loss of local features;
[0104] The local clustering loss is the sum of the losses of N centers; the center loss of a local feature and the feature centers of the same type of feature constitutes one of the center losses.
[0105] The triplet loss of the local feature is the sum of the triplet losses of N local features; if the distance difference between two local features with different identity information, the same orientation information, and the same human body region and the feature center of their respective types is less than a preset threshold, then the triplet loss of the local feature is zero.
[0106] Specifically: the expression for the visual similarity loss is:
[0107] L visul =γ1L cluster +γ2L s-triplet ;
[0108] In the formula, γ1 and γ2 represent weighting coefficients; L cluster L represents the local clustering loss; s-triplet The triplet loss represents the local feature.
[0109] Taking N local features as head features, trunk features, and lower limb features as an example: the local clustering loss is the sum of three center losses; the three center losses are the center loss of the head feature and its similar feature, the center loss of the trunk feature and its similar feature, and the center loss of the lower limb feature and its similar feature; the triplet loss of the local feature is the sum of the triplet loss of the head feature, the triplet loss of the trunk feature, and the triplet loss of the lower limb feature; if the distance difference between two local features with different identity information, the same orientation information, and the same human body region and the feature center of their similar feature is less than a preset threshold, then the triplet loss of the local feature is zero.
[0110] The local center dynamic library is updated using all local features obtained from the current batch of training sample images.
[0111] In one or more embodiments, updating the local center dynamic library using all local features obtained from the current batch of training sample images specifically includes the following steps:
[0112] All local features obtained from the current batch of training sample images are input into the local center dynamic library, and the feature centers of various features are updated using momentum update.
[0113] Taking N local features as head features, trunk features, and lower limb features as an example:
[0114] If the current batch of training sample images only obtains one head feature A1 belonging to the same type of feature FA, then the current feature center of the same type of feature FA and the head feature A1 are weighted and summed to obtain a new feature center of the same type of feature FA.
[0115] If the current batch of training sample images yields U1 head features A1, A2, ..., A1 belonging to the same type of feature FA, then... U1 First, calculate the head features A1, A2, ..., A... U2 The average value is then summed with the current feature center of the same type of feature FA to obtain the new feature center of the same type of feature FA; where U1 is a positive integer greater than 1.
[0116] If the current batch of training sample images only obtains one trunk feature B1 belonging to the same type of feature FB, then the current feature center of the same type of feature FB and the trunk feature B1 are weighted and summed to obtain a new feature center of the same type of feature FB.
[0117] If the current batch of training sample images yields U2 torso features B1, B2, ..., B belonging to the same feature FB, then... U2 First, calculate the trunk features B1, B2, ..., B U2 The average value is then summed with the current feature center of the same type of feature FB to obtain the new feature center of the same type of feature FB; where U2 is a positive integer greater than 1.
[0118] If the current batch of training sample images only obtains one head feature C1 belonging to the same type of feature FC, then the current feature center of the same type of feature FC and the head feature C1 are weighted and summed to obtain a new feature center of the same type of feature FC.
[0119] If the current batch of training sample images yields U3 head features C1, C2, ..., Cn belonging to the same type of feature FC, then... U3 First, calculate the head features C1, C2, ..., C...U The average value is then summed with the current feature center of the same type of feature FC to obtain the new feature center of the same type of feature FC; where U3 is a positive integer greater than 1.
[0120] S3: Repeat S2 continuously to obtain the trained human feature extraction network model that integrates local attention and visual similarity.
[0121] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0122] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for pedestrian target association that integrates local attention and visual similarity, characterized in that, Includes the following steps: The pedestrian image to be associated is input into a trained human feature extraction network model that integrates local attention and visual similarity to obtain the global features and N local features of the pedestrian image to be associated; wherein, ; The global features and N local features of the pedestrian image to be associated are concatenated to obtain the overall features of the pedestrian image to be associated. The overall feature with the highest similarity to the overall feature of the pedestrian image to be associated is selected from the preset overall feature library, and the pedestrian image corresponding to the overall feature with the highest similarity is used as the associated image of the pedestrian image to be associated. The acquisition method of each overall feature in the preset overall feature library is consistent with the acquisition method of the overall features of the pedestrian image to be associated. The architecture of the human feature extraction network model that integrates local attention and visual similarity adopts the Transformer deep learning architecture for visual tasks. The human feature extraction network model that integrates local attention and visual similarity adds an overall token and N local tokens to the image block token sequence; The human feature extraction network model that integrates local attention and visual similarity retains only the attention score between the local token and the image block at a preset position in the input image when calculating the attention of each local token. The attention score between the local token and the remaining image blocks in the input image is set to negative infinity. The image block at the preset position in the input image is the image block in the input image that the local token is interested in. The trained human feature extraction network model, which integrates local attention and visual similarity, is obtained based on the following steps: S1: Initialize the local central dynamic library; specifically including the following steps: All training sample images are input into a human feature extraction network model that integrates local attention and visual similarity to obtain local features of each training sample image; each training sample image obtains N local features, and each training sample image is labeled with the pedestrian's orientation information and identity information; Local features of the same identity information, same orientation information, and same human body region are classified into the same type of features to obtain an initialized dynamic library of local centers. Each type of feature is assigned a feature center. If there is only one local feature of the same identity information, same orientation information, and same human body region, then the feature center is that local feature. If there are multiple local features of the same identity information, same orientation information, and same human body region, then the feature center is the average value of the multiple local features. S2: Input the training sample images into the human keypoint detection network model to obtain the keypoint heatmap of the training sample images; The key point heatmap of the training sample image is divided to obtain N local key point heatmaps; wherein, the human body regions of interest in the N local key point heatmaps correspond one-to-one with the human body regions of interest in the N local tokens. The training sample image and the N local key point heatmaps are subjected to the same random transformation and then input into the human feature extraction network model that integrates local attention and visual similarity to obtain the N local features of the training sample image, the attention map of the N local features, and the predicted identity information. The N local key point heatmaps are sequentially binarized and normalized at the image block level to obtain N normalized local key point heatmaps. The local attention loss is obtained by weighted summation of the regularization loss and the alignment loss; wherein the regularization loss is the sum of the regularization losses of N local features; the alignment loss is the sum of N KL divergences; one of the KL divergences is the KL divergence of the attention map of a local feature and the KL divergence of the normalized local keypoint heatmap corresponding to the same human body region. The visual similarity loss is obtained by weighted summation of the local clustering loss and the triplet loss of local features; wherein, the local clustering loss is the sum of the losses of N center features, and the center loss of a local feature and the feature center of the same type of feature constitutes one of the center losses; the triplet loss of local features is the sum of the triplet losses of N local features. The total loss function is obtained by weighted summing of the local attention loss, the visual similarity loss, the identity information classification loss, and the triplet loss of global features; Minimize the total loss function and update the network parameters of the human feature extraction network model using the backpropagation method; The local center dynamic library is updated using all local features obtained from the current batch of training sample images. S3: Repeat S2 continuously to obtain the trained human feature extraction network model that integrates local attention and visual similarity.
2. The pedestrian target association method integrating local attention and visual similarity according to claim 1, characterized in that, If the distance difference between two local features of different identity information, the same orientation information, and the same human body region and the feature center of their respective types of features is less than a preset threshold, then the triplet loss of the local feature is zero.
3. The pedestrian target association method integrating local attention and visual similarity according to claim 1, characterized in that, The local center dynamic library is updated using all local features obtained from the current batch of training sample images, specifically including the following steps: All local features obtained from the current batch of training sample images are input into the local center dynamic library, and the feature centers of various features are updated using momentum update.
4. The pedestrian target association method integrating local attention and visual similarity according to claim 1, characterized in that, The attention map is the average value of the attention maps output by all Transformer encoder layers in ViT.
Citation Information
Patent Citations
Pedestrian re-identification method based on Vision Transform attention screening
CN116682145A