Small group pedestrian re-identification method based on uncertainty second-order self-attention structure model

Abstract

The present application aims at the limitations of the prior art and proposes a small group pedestrian re-identification method based on an uncertainty second-order self-attention structure model, which mines the group structure tending to infinity by simulating the changes of members and layout based on limited samples. The member uncertainty takes the number of members in the group as a truncated Gaussian distribution instead of a fixed value, and then simulates the changes of members through dynamic sampling. The layout uncertainty constructs a random affine transformation about the position of the members to expand the fixed scheme in the training set. The present application proposes a second-order uncertainty self-attention structure model (UMSOT), which extracts a first-order token as the feature of each member, and then learns a second-order token as the group feature according to all the first-order tokens. UMSOT utilizes the structural advantages of the self-attention model to realize the explicit modeling of the layout feature and the integration with the appearance modeling. The comprehensive experiments on four datasets of CSG, SYSUGroup, RoadGroup and iLIDS-MCTS fully prove the superiority of the method.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and video surveillance technology, specifically to a method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model. Background Technology

[0002] Small group pedestrian re-identification aims to link images of the same group of pedestrians from different cameras within a non-overlapping camera network, a technique widely used in video surveillance. Compared to person re-identification, which has been developed for many years, small group pedestrian re-identification remains a nascent task and has not received much attention. Social attributes indicate that people tend to appear in groups rather than alone; therefore, small group pedestrian re-identification not only needs to overcome the difficulties of traditional person re-identification but also needs to address the unique challenges brought about by changes in group structure.

[0003] Specifically, the unique challenges of small-group pedestrian re-identification mainly include two aspects: changes in the number of members and their layout. The former refers to the fact that the number of members within a group can decrease or increase under different cameras. The latter refers to the fact that the spatial positions of members within the group may change under different cameras. These two challenges are unavoidable in real-world scenarios and need to be considered when solving small-group pedestrian re-identification problems.

[0004] Taking the Chinese invention application published on March 11, 2022, titled "A Context-Aware, Noise-Robust Pedestrian Search Method," as an example, a deep learning-based small-group pedestrian re-identification method attempts to address the aforementioned challenges. However, these methods are still imperfect when considering group structure modeling for each image, mainly due to the following two reasons: (1) These methods are all based on deterministic modeling. The features extracted by existing methods are based on fixed group structures provided by the images. Due to the diversity of group structures, these features are insufficient to describe class boundaries. (2) Technically, existing models are based on a combination of convolutional neural networks or graph neural networks. Due to the inherent defects in position modeling of the structure itself, the description of group layout features is weak, resulting in limited performance. Summary of the Invention

[0005] To address the limitations of existing technologies, this invention proposes a method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model. The technical solution adopted by this invention is as follows:

[0006] A method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model includes the following steps:

[0007] S1, acquire the image to be recognized;

[0008] S2, the image to be identified is input into a small pedestrian re-identifier trained by an uncertain second-order self-attention structure model to obtain the recognition result;

[0009] The feature extraction stage in the uncertain second-order self-attention structure model includes member feature extraction and group feature extraction. In the member feature extraction stage, the visual self-attention model is used as the extractor to extract member appearance features. In the group feature extraction stage, uncertainty modeling is performed, and relationship modeling and appearance modeling are considered as a whole to extract complete group discriminative features.

[0010] During training, supervised training constraints are simultaneously applied to the uncertain second-order self-attention structure model, including the identity categories of the group and the identity categories of each member, with the goal of minimizing the sum of the cross-entropy loss function and the hard triplet loss function.

[0011] Compared to existing technologies, this invention, based on finite samples, mines group structures that tend towards infinity by simulating changes in members and layout. Member uncertainty treats the number of members within a group as a truncated Gaussian distribution rather than a fixed value, and then simulates changes in members through dynamic sampling. Layout uncertainty constructs a random affine transformation about member positions to expand the fixed scheme in the training set. Technically, this invention proposes a second-order uncertain self-attention structure model (UMSOT), which extracts a first-order token as a feature for each member, and then learns a second-order token as a group feature based on all the aforementioned first-order tokens. UMSOT leverages the structural advantages of self-attention models to achieve explicit modeling of layout features and integration with appearance modeling. Comprehensive experiments on four datasets—CSG, SYSUGroup, RoadGroup, and iLIDS-MCTS—fully demonstrate the superiority of this method.

[0012] As a preferred approach, the training batches used during training are constructed in the following manner:

[0013] Each training batch consists of 64 group images selected from a preset training sample. Each batch has 4 randomly selected group identity categories, and 16 images are randomly selected for each group identity category. The construction process of the training batch does not impose any additional constraints on the quality of the group images, shooting angle, lighting conditions, etc. After the training batch is constructed, all members contained in each group image are cropped according to the rectangular frame position labels given in the preset image annotation file to obtain several member sub-images, and the cropped member sub-images are set to a resolution of 256×128.

[0014] Furthermore, in the member feature extraction stage:

[0015] For a group image x i All member subgraphs within For each member subgraph x i，j Perform non-overlapping block processing to evenly divide the member subgraph into 128 16×16 sub-blocks;

[0016] The sub-blocks, which are composed of two-dimensional matrices, are stacked and flattened into one-dimensional vectors. Then, a 1×1 convolution operation is performed to obtain a 768-dimensional embedding vector, which serves as the feature representation of the sub-blocks.

[0017] Add an embedding vector called the first-order token before all embedding vectors. As an integrated feature of all embedded vectors, each sub-block of the member subgraph is marked with its original position in the original graph, and a position encoding matrix is ​​preset. Based on the index of the sub-block in all the sliced ​​sub-blocks, E p The vector corresponding to the row number in the middle is used as the position feature of the current embedded vector. After being added together, the serialized input matrix of the member subgraphs in the group image is obtained.

[0018] After the serialized input matrix undergoes a 12-layer forward pass computation in the visual self-attention model, the optimized first-order token output from the last layer is used as the appearance feature representation of the current member subgraph, denoted as...

[0019] Furthermore, in the group feature extraction stage:

[0020] The optimized first-order tokens of each member are concatenated to form a serialization matrix. In the serialization matrix A second-order token is appended to the beginning, serving as the group image x. i A portion of the serialized input in the preset group feature extractor;

[0021] By modeling the uncertainty of the relationships between groups, and then considering both relationship modeling and appearance modeling as a whole, the final form of the serialized input in the group feature extractor is obtained.

[0022] The final form of the serialized input in the group feature extractor is input into the group feature extractor to extract complete group discriminative features.

[0023] Furthermore, the group feature extraction stage also includes the following process of modeling member uncertainty:

[0024] Using hyperparameters P0 and p max Solving the truncated Gaussian distribution The variance and mean can be solved as follows:

[0025]

[0026] For each subgraph within a group image, count the number of members Z. c And the maximum number of members Z contained in the group under this group identity in the entire batch. t For the truncated Gaussian distribution After making corrections, we obtain the corrected p. max :

[0027] p′ max =max(0, 1-(1-p)) max )Z t / Z c );

[0028] Using the corrected p max Reconstructing the truncated Gaussian distribution

[0029] According to Z c Construct a binary mask row vector of the corresponding dimension. The binary mask row vector Each element m in i,j All satisfy a truncated Gaussian distribution Binary results obtained by sampling on the Bernoulli distribution:

[0030]

[0031] in, This represents a probability of 1 for sampling a 1, otherwise 0. A value of 1 indicates that the current member subgraph is retained, and a value of 0 indicates that the current member subgraph is discarded. (Statistics) The number of elements in the middle that are 1 is denoted as .

[0032] Using the binary mask row vector For the serialization matrix Perform a filtering operation to The corresponding value of 0 in the middle is in Rows are removed, and a member count feature is added to the second-order token; the member count feature comes from a preset matrix E. m The This completes the modeling of the uncertainty of the members.

[0033] Furthermore, the group feature extraction stage also includes the following layout uncertainty modeling process:

[0034] For the remaining members after completing the member uncertainty modeling, the corresponding two-dimensional center coordinates are calculated based on the bounding box position coordinates of the remaining members:

[0035] Based on group image x i The width and height are normalized to the center coordinate matrix of the members:

[0036] Construct a random affine transformation matrix A1, and perform random affine transformations on the center positions of several members: Where 1 is a column vector with the same number of rows as the normalized center matrix;

[0037] The result of the random transformation is then randomly mapped back to a sub-region of the normalized interval: first, a coordinate (x... l x u ) and (y l y u A two-dimensional random subregion, where 0 ≤ x l <x u ≤1, 0≤y l <y u ≤1, then perform the transformation result mapping:

[0038]

[0039] in:

[0040]

[0041] by This is a result of modeling layout uncertainty.

[0042] Furthermore, in the group feature extraction stage, group features are extracted using the following methods:

[0043] according to Explicit extraction of layout features: The normalized imaging plane is divided into η×η square blocks using a discrete factor η; a set of layout feature matrices is preset in each of the two dimensions of the normalized imaging plane. Calculate which square block the center of each member belongs to, extract the corresponding columns from the two layout feature matrices, concatenate them to form the layout feature of the current member, and store it in the corresponding row of the layout feature matrix of the current group:

[0044]

[0045] The serialization matrix S of the current group i The construction is as follows:

[0046]

[0047] S i The optimized second-order token is extracted by the group feature extractor. As a group-discriminating feature.

[0048] Furthermore, during training, identity labels are learned from the extracted member features and group discriminative features using the following methods:

[0049] For member characteristics That is, the first-order token, which uses the sum of the cross-entropy loss function and the hard triplet loss function as the optimization objective for identity learning:

[0050]

[0051]

[0052]

[0053] Where N is the number of images in a batch. For features, a distance metric function is commonly used, such as Euclidean distance. + The function representing the maximum value is max(·, 0), x i，j , These represent the anchor sample and its difficult positive sample and difficult negative sample, respectively.

[0054] For group characteristics That is, a second-order token, which uses the sum of the cross-entropy loss function and the hard triplet loss function as the optimization objective for identity learning:

[0055]

[0056]

[0057]

[0058] The optimization goals for the two identities are summed to form the final goal:

[0059]

[0060] Where λ is a hyperparameter balancing the cross-entropy loss function and the hard triplet loss function.

[0061] This invention also provides the following:

[0062] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model.

[0063] A computer device includes a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the aforementioned method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the small-group pedestrian re-identification method based on an uncertain second-order self-attention structure model provided in an embodiment of the present invention.

[0065] Figure 2 A network diagram of an uncertain second-order self-attention structure model provided in an embodiment of the present invention. Detailed Implementation

[0066] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0067] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0068] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0069] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0070] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The invention will be further described below with reference to the accompanying drawings and embodiments.

[0071] To address the limitations of existing technologies, this embodiment provides a technical solution. The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0072] Example 1

[0073] Please refer to Figure 1 as well as Figure 2 A method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model includes the following steps:

[0074] S1, acquire the image to be recognized;

[0075] S2, the image to be identified is input into a small pedestrian re-identifier trained by an uncertain second-order self-attention structure model to obtain the recognition result;

[0076] The feature extraction stage in the uncertain second-order self-attention structure model includes member feature extraction and group feature extraction. In the member feature extraction stage, the visual self-attention model is used as the extractor to extract member appearance features. In the group feature extraction stage, uncertainty modeling is performed, and relationship modeling and appearance modeling are considered as a whole to extract complete group discriminative features.

[0077] During training, supervised training constraints are simultaneously applied to the uncertain second-order self-attention structure model, including the identity categories of the group and the identity categories of each member, with the goal of minimizing the sum of the cross-entropy loss function and the hard triplet loss function.

[0078] Compared to existing technologies, this invention, based on finite samples, mines group structures that tend towards infinity by simulating changes in members and layout. Member uncertainty treats the number of members within a group as a truncated Gaussian distribution rather than a fixed value, and then simulates changes in members through dynamic sampling. Layout uncertainty constructs a random affine transformation about member positions to expand the fixed scheme in the training set. Technically, this invention proposes a second-order uncertain self-attention structure model (UMSOT), which extracts a first-order token as a feature for each member, and then learns a second-order token as a group feature based on all the aforementioned first-order tokens. UMSOT leverages the structural advantages of self-attention models to achieve explicit modeling of layout features and integration with appearance modeling. Comprehensive experiments on four datasets—CSG, SYSUGroup, RoadGroup, and iLIDS-MCTS—fully demonstrate the superiority of this method.

[0079] Specifically, this embodiment uses the Vision Transformer, a visual self-attention model, as the extractor of member appearance features. The backbone network consists of 12 stacked self-attention modules, and the backbone network weight parameters are initialized using a model pre-trained on the ImageNet dataset.

[0080] As a preferred embodiment, the training batches used during the training process are constructed in the following manner:

[0081] Each training batch consists of 64 group images selected from a preset training sample. Each batch has 4 randomly selected group identity categories, and 16 images are randomly selected for each group identity category. The construction process of the training batch does not impose any additional constraints on the quality of the group images, shooting angle, lighting conditions, etc. After the training batch is constructed, all members contained in each group image are cropped according to the rectangular frame position labels given in the preset image annotation file to obtain several member sub-images, and the cropped member sub-images are set to a resolution of 256×128.

[0082] Furthermore, in the member feature extraction stage:

[0083] For a group image x i All member subgraphs within For each member subgraph x i，j Perform non-overlapping block processing to evenly divide the member subgraph into 128 16×16 sub-blocks;

[0084] The sub-blocks, which are composed of two-dimensional matrices, are stacked and flattened into one-dimensional vectors. Then, a 1×1 convolution operation is performed to obtain a 768-dimensional embedding vector, which serves as the feature representation of the sub-blocks. The above operation is denoted as ψ(·).

[0085] Add an embedding vector called the first-order token before all embedding vectors. As an integrated feature of all embedded vectors, each sub-block of the member subgraph is marked with its original position in the original graph, and a position encoding matrix is ​​preset. Based on the index of the sub-block in all the sliced ​​sub-blocks, E p The vector corresponding to the row number in the middle is used as the position feature of the current embedded vector. After being added together, the serialized input matrix of the member subgraphs in the group image is obtained.

[0086] The above steps can be represented as:

[0087]

[0088] Among them, S i，j x represents the subimage of the j-th member in the i-th group image. i，j The serialized input matrix, where N0 represents the largest sub-block segmentation.

[0089] After the serialized input matrix undergoes a 12-layer forward pass computation in the visual self-attention model, the optimized first-order token output from the last layer is used as the appearance feature representation of the current member subgraph, denoted as...

[0090] Furthermore, in the group feature extraction stage:

[0091] The optimized first-order tokens of each member are concatenated to form a serialization matrix. In the serialization matrix A second-order token is appended to the beginning, serving as the group image x. i A portion of the serialized input in the preset group feature extractor;

[0092] By modeling the uncertainty of the relationships between groups, and then considering both relationship modeling and appearance modeling as a whole, the final form of the serialized input in the group feature extractor is obtained.

[0093] The final form of the serialized input in the group feature extractor is input into the group feature extractor to extract complete group discriminative features.

[0094] Furthermore, the group feature extraction stage also includes the following process of modeling member uncertainty:

[0095] Using hyperparameters P0 and p max Solving the truncated Gaussian distribution The variance and mean can be solved as follows:

[0096]

[0097] For each subgraph within a group image, count the number of members Z. c And the maximum number of members Z contained in the group under this group identity in the entire batch. t For the truncated Gaussian distribution After making corrections, we obtain the corrected p. max :

[0098] p′ max =max(0, 1-(1-p)) max )Z t / Z c );

[0099] Using the corrected p max Reconstructing the truncated Gaussian distribution

[0100] According to Z c Construct a binary mask row vector of the corresponding dimension. The binary mask row vector Each element m in i,j All satisfy a truncated Gaussian distribution Binary results obtained by sampling on the Bernoulli distribution:

[0101]

[0102] in, This represents a probability of 1 for sampling a 1, otherwise 0. A value of 1 indicates that the current member subgraph is retained, and a value of 0 indicates that the current member subgraph is discarded. (Statistics) The number of elements in the middle that are 1 is denoted as .

[0103] Using the binary mask row vector For the serialization matrix Perform a filtering operation to The corresponding value of 0 in the middle is in Rows are removed, and a member count feature is added to the second-order token; the member count feature comes from a preset matrix E. m The This completes the modeling of the uncertainty of the members.

[0104] Furthermore, the group feature extraction stage also includes the following layout uncertainty modeling process:

[0105] For the remaining members after completing the member uncertainty modeling, the corresponding two-dimensional center coordinates are calculated based on the bounding box position coordinates of the remaining members:

[0106] Based on group image x i The width and height are normalized to the center coordinate matrix of the members:

[0107] Construct a random affine transformation matrix A1, and perform random affine transformations on the center positions of several members: Where 1 is a column vector with the same number of rows as the normalized center matrix;

[0108] To ensure that the result of the affine transformation remains within the normalized interval, this embodiment randomly maps the result of the random transformation back to a sub-region of the normalized interval: specifically, a coordinate system with coordinates (x... i x u ) and (y l y u A two-dimensional random subregion, where 0 ≤ x l <x u ≤1, 0≤y l <y u ≤1, then perform the transformation result mapping:

[0109]

[0110] in:

[0111]

[0112] by This is a result of modeling layout uncertainty.

[0113] Furthermore, in the group feature extraction stage, group features are extracted using the following methods:

[0114] according to Explicit extraction of layout features: The normalized imaging plane is divided into η×η square blocks using a discrete factor η; a set of layout feature matrices is preset in each of the two dimensions of the normalized imaging plane. Calculate which square block the center of each member belongs to, extract the corresponding columns from the two layout feature matrices, concatenate them to form the layout feature of the current member, and store it in the corresponding row of the layout feature matrix of the current group:

[0115]

[0116] The serialization matrix S of the current group i The construction is as follows:

[0117]

[0118] Si The optimized second-order token is extracted by the group feature extractor. As a group-discriminating feature.

[0119] Furthermore, during training, identity labels are learned from the extracted member features and group discriminative features using the following methods:

[0120] For member characteristics That is, the first-order token, which uses the sum of the cross-entropy loss function and the hard triplet loss function as the optimization objective for identity learning:

[0121]

[0122]

[0123]

[0124] Where N is the number of images in a batch. For features, a distance metric function is commonly used, such as Euclidean distance. + The function representing the maximum value is max, x i，j , These represent the anchor sample and its difficult positive sample and difficult negative sample, respectively.

[0125] For group characteristics That is, a second-order token, which uses the sum of the cross-entropy loss function and the hard triplet loss function as the optimization objective for identity learning:

[0126]

[0127]

[0128]

[0129] The optimization goals for the two identities are summed to form the final goal:

[0130]

[0131] Where λ is a hyperparameter balancing the cross-entropy loss function and the hard triplet loss function.

[0132] Example 2

[0133] A storage medium storing a computer program, which, when executed by a processor, implements the steps of the small-group pedestrian re-identification method based on an uncertain second-order self-attention structure model in Embodiment 1.

[0134] Example 3

[0135] A computer device includes a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor. When executed by the processor, the computer program implements the steps of the small-group pedestrian re-identification method based on an uncertain second-order self-attention structure model in Embodiment 1.

[0136] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model, characterized in that, Includes the following steps: S1, acquire the image to be recognized; S2, the image to be identified is input into a small pedestrian re-identifier trained by an uncertain second-order self-attention structure model to obtain the recognition result; The feature extraction stage in the uncertain second-order self-attention structure model includes member feature extraction and group feature extraction. In the member feature extraction stage, the visual self-attention model is used as the extractor to extract member appearance features. In the group feature extraction stage, uncertainty modeling is performed, and relationship modeling and appearance modeling are considered as a whole to extract complete group discriminative features. During training, supervised training constraints are simultaneously applied to the uncertain second-order self-attention structure model, including the identity categories of the group and the identity categories of each member, with the goal of minimizing the sum of the cross-entropy loss function and the hard triplet loss function.

2. The method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model according to claim 1, characterized in that, The training batches used during training are constructed in the following way: Each training batch consists of 64 group images selected from the preset training samples. Each batch has 4 random group identity categories, and 16 images are randomly selected for each group identity category. The construction process of the training batch has no additional constraints on the quality of the group images, shooting angle, lighting conditions, etc. After the training batch is constructed, all members contained in each group image are cropped according to the rectangular frame position labels given in the preset image annotation file to obtain several member sub-images, and the cropped member sub-images are set to a resolution of 256×128.

3. The method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model according to claim 2, characterized in that, In the member feature extraction stage: For a group image x i All member subgraphs within For each member subgraph x i，j Perform non-overlapping block processing to evenly divide the member subgraph into 128 16×16 sub-blocks; The sub-blocks, which are composed of two-dimensional matrices, are stacked and flattened into one-dimensional vectors. Then, a 1×1 convolution operation is performed to obtain a 768-dimensional embedding vector, which serves as the feature representation of the sub-blocks. Add an embedding vector called the first-order token before all embedding vectors. As an integrated feature of all embedded vectors; Each sub-block of the member subgraph is marked with its original position in the original graph, and a position encoding matrix is ​​preset. Based on the index of the sub-block in all the sliced ​​sub-blocks, E p The vector corresponding to the row number in the middle is used as the position feature of the current embedded vector. After being added together, the serialized input matrix of the member subgraphs in the group image is obtained. After the serialized input matrix undergoes a 12-layer forward pass computation in the visual self-attention model, the optimized first-order token output from the last layer is used as the appearance feature representation of the current member subgraph, denoted as...

4. The method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model according to claim 3, characterized in that, In the group feature extraction stage: The optimized first-order tokens of each member are concatenated to form a serialization matrix. In the serialization matrix A second-order token is appended to the beginning, serving as the group image x. i A portion of the serialized input in the preset group feature extractor; By modeling the uncertainty of the relationships between groups, and then considering both relationship modeling and appearance modeling as a whole, the final form of the serialized input in the group feature extractor is obtained. The final form of the serialized input in the group feature extractor is input into the group feature extractor to extract complete group discriminative features.

5. The method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model according to claim 4, characterized in that, The group feature extraction stage also includes the following process of modeling member uncertainty: Using hyperparameters P0 and p max Solving the truncated Gaussian distribution The variance and mean can be solved as follows: For each subgraph within a group image, count the number of members Z. c And the maximum number of members Z contained in the group under this group identity in the entire batch. t For the truncated Gaussian distribution After making corrections, we obtain the corrected p. max : p′ max =max(0,1-(1-p max )Z t / Z c ); Using the corrected p max Reconstructing the truncated Gaussian distribution According to Z c Construct a binary mask row vector of the corresponding dimension. The binary mask row vector Each element m in i,j All satisfy a truncated Gaussian distribution Binary results obtained by sampling on the Bernoulli distribution: in, This represents sampling a 1 with a probability of 1-p, otherwise it is 0; a value of 1 indicates that the current member subgraph is retained, and a value of 0 indicates that the current member subgraph is discarded; statistics. The number of elements in the middle that are 1 is denoted as . Using the binary mask row vector For the serialization matrix Perform a filtering operation to The corresponding value of 0 in the middle is in Rows are removed, and a member count feature is added to the second-order token; the member count feature comes from a preset matrix E. m The This completes the modeling of the uncertainty of the members.

6. The method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model according to claim 5, characterized in that, The group feature extraction stage also includes the following layout uncertainty modeling process: For the remaining members after completing the member uncertainty modeling, the corresponding two-dimensional center coordinates are calculated based on the bounding box position coordinates of the remaining members: Based on group image x i The width and height are normalized to the center coordinate matrix of the members: Construct a random affine transformation matrix A1, and perform random affine transformations on the center positions of several members: Where 1 is a column vector with the same number of rows as the normalized center matrix; The result of the random transformation is then randomly mapped back to a sub-region of the normalized interval: first, a coordinate (x... l x u ) and (g l y u A two-dimensional random subregion, where 0 ≤ x l <x u ≤1, 0≤y l <y u ≤1, then perform the transformation result mapping: in: by This is a result of modeling layout uncertainty.

7. The method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model according to claim 6, characterized in that, In the group feature extraction stage, group features are extracted using the following methods: according to Explicit extraction of layout features: The normalized imaging plane is divided into η×η square blocks using a discrete factor η; a set of layout feature matrices E are preset in both dimensions of the normalized imaging plane. x , Calculate which square block the center of each member belongs to, extract the corresponding columns from the two layout feature matrices, concatenate them to form the layout feature of the current member, and store it in the corresponding row of the layout feature matrix of the current group: The serialization matrix S of the current group i The construction is as follows: S i The optimized second-order feature is extracted by the group feature extractor. As a group-discriminating feature.

8. The method for re-identifying small groups of pedestrians based on an uncertain second-order self-attention structure model according to claim 7, characterized in that, During training, identity labels are learned from the extracted member features and group discriminative features using the following methods: For member characteristics That is, the first-order token, which uses the sum of the cross-entropy loss function and the hard triplet loss function as the optimization objective for identity learning: Where N is the number of images in a batch. For features, a distance metric function is commonly used, such as Euclidean distance. + The function representing the maximum value is max(·, θ), x i，j , These represent the anchor sample and its difficult positive sample and difficult negative sample, respectively. For group characteristics That is, a second-order token, which uses the sum of the cross-entropy loss function and the hard triplet loss function as the optimization objective for identity learning: The optimization goals for the two identities are summed to form the final goal: Where λ is a hyperparameter balancing the cross-entropy loss function and the hard triplet loss function.

9. A computer-readable storage medium, characterized in that: It stores a computer program that, when executed by a processor, implements the steps of the small-group pedestrian re-identification method based on an uncertain second-order self-attention structure model as described in any one of claims 1 to 8. By using rectified pulse units or through pulse units to model ANN branches, the forward computation of each layer of the spiking neural network can be represented by the ANN branches; Then, the cross-entropy loss function is applied to the output of the FC layer of the ANN branch to describe the classification error; finally, the backpropagation algorithm is used to train on the ANN branch.

10. A computer device, characterized in that: The method includes a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the small-group pedestrian re-identification method based on an uncertain second-order self-attention structure model as described in any one of claims 1 to 8.

Images