Video pedestrian re-identification method and system based on riemannian manifold second-order relationship modeling

Abstract

The application discloses a video pedestrian re-identification method and system based on a Riemannian manifold second-order relationship modeling, and relates to the technical field of video pedestrian re-identification. In view of the problem that existing methods rely on first-order features and are difficult to capture high-order space-time correlation, the method preprocesses video data, generates block feature representation and encodes into frame-level features, splices the block features to calculate an inter-channel covariance matrix and optimizes the matrix into a positive definite symmetric matrix. After a mask matrix is obtained through random masking, the mask matrix is input into a Riemannian manifold network to perform multiple rounds of bilinear mapping, nonlinear activation and cross-round feature fusion, and finally, the fusion features are projected to a tangent space to complete retrieval matching. The application mines high-order correlation information through second-order relationship modeling, combines the Riemannian manifold geometric characteristics and feature fusion mechanism, improves the robustness and recognition accuracy of the model to complex scenes, and is suitable for fields such as intelligent security and video monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video pedestrian re-identification, in particular to a video pedestrian re-identification method and system based on Riemannian manifold second-order relationship modeling. BACKGROUND

[0002] Video-based person re-identification (Video ReID) is one of the core research directions in the field of computer vision and intelligent security, and its main task is to accurately identify and match the same person identity based on video clips in a multi-camera monitoring network with non-overlapping views. With the increasing demand for cross-camera collaborative tracking in the fields of smart cities and public security, Video ReID technology has become a key support for realizing continuous search and behavior analysis of pedestrians across space and time. Compared with pedestrian re-identification based on static images, video data not only contains richer visual appearance information, but also contains continuous temporal dynamic features and motion patterns, which helps to achieve more stable and reliable identity discrimination in complex scenes.

[0003] In recent years, deep learning-based video pedestrian re-identification methods have made significant progress. Early research mainly relies on convolutional neural networks (CNN) for frame-level feature extraction, and aggregates video representations through temporal pooling, recurrent neural networks (RNN) or temporal attention mechanisms, showing good results in improving feature expression ability. Subsequently, to further exploit the structured information in pedestrian videos, some methods introduce graph convolutional networks (GCN) to explicitly model the relationships between frames or local parts of the human body to enhance feature discrimination. However, CNN methods are limited by local receptive fields and are difficult to effectively capture long-range temporal and spatial dependencies; GCN methods usually rely on pre-defined or externally constructed graph structures, and their generalization ability is constrained to some extent.

[0004] In recent years, Vision Transformer (ViT) has gradually become a research hotspot in video pedestrian re-identification due to its global self-attention mechanism, which can adaptively capture long-range dependencies and complex visual patterns, and has shown superior performance in multiple benchmark tasks. Despite this, most existing methods are still based on first-order statistical features, i.e., by extracting feature vectors of frames or segments in Euclidean space, and then aggregating the final video representation through temporal aggregation. Such methods usually follow the "space first, then time" or "time first, then space" separation processing paradigm, and fail to cooperatively model the interaction relationship between space and time dimensions, thus being unable to fully capture the inherent regional covariant relationship, channel dependency and cross-frame dynamic correlation in videos.

[0005] Although a few studies have attempted to introduce second-order statistical information (such as the covariance matrix) to enhance the feature representation capability, most of these methods only use second-order features as an auxiliary supplement to first-order features, and have not systematically mined and constructed a high-order statistical modeling framework suitable for the video pedestrian re-identification task. Therefore, the prior art generally has the following defects:

[0006] First, the first-order feature representation capability is limited: relying on first-order vector representation, it is difficult to effectively capture the complex spatio-temporal structure relationship in the video, resulting in insufficient discrimination in complex scenes such as view changes, occlusions, and light differences;

[0007] Second, the spatio-temporal modeling is separated: existing methods often process spatial and temporal information in isolation, lack a unified and coordinated spatio-temporal joint modeling mechanism, and limit the full use of multi-dimensional feature interaction;

[0008] Third, high-order statistical information is underutilized: although some research has explored second-order features, there is no systematic high-order statistical modeling system, and the mining of deep information such as regional correlation and temporal evolution patterns is still insufficient, which restricts the robustness and generalization ability of the model in complex real-world scenarios.

[0009] Therefore, how to design a pedestrian re-identification method that can fully integrate video spatio-temporal information and effectively utilize high-order statistical features has become a key research direction to improve the practicality and accuracy of the technology. SUMMARY

[0010] To this end, the present application aims to solve the problem that existing video pedestrian re-identification methods rely on first-order feature learning, use a separate processing paradigm, and cannot cooperatively depict the deep associations of video spatio-temporal information, and that existing second-order modeling methods only serve to strengthen first-order features and do not fully exploit their value, while also having the risk of feature redundancy and overfitting, resulting in insufficient discrimination and generalization of the model. Therefore, the present application provides a video pedestrian re-identification method and system based on Riemannian manifold second-order relationship modeling.

[0011] The video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling comprises the following steps:

[0012] Step S1: Obtain the RGB pedestrian video sequences in the video set to be searched and the query video set, respectively, and sequentially perform denoising, size normalization, and frame rate unification preprocessing operations on each RGB pedestrian video sequence in the two video sets to obtain preprocessed video data with consistent formats;

[0013] Step S2: Divide each frame of image contained in the preprocessed video data into multiple image blocks to generate a block feature representation containing block features, class features, and position encodings;

[0014] Step S3: encode the block feature representation to obtain a frame-level feature;

[0015] Step S4: splice each block feature in the frame-level feature to calculate an inter-channel covariance matrix, and add a regularization term to the inter-channel covariance matrix to obtain a positive definite symmetric matrix;

[0016] Step S5: perform a random mask operation on the row and column dimensions of the positive definite symmetric matrix to add a regularization term to obtain a mask matrix, specifically:

[0017] construct a binary mask matrix M with dimensions of , wherein the value of each element in the matrix is only 0 or 1, the value of 0 indicates that the corresponding position needs to be masked, and the value of 1 indicates that the feature information of the corresponding position is retained, D is the dimension of the feature label, is the spatial dimension;

[0018] perform an element-level multiplication operation on the mask matrix M and the positive definite symmetric matrix , the rows and columns of the elements marked as 0 in the matrix by the mask matrix M are masked synchronously, and a regularization term is added to the positive definite symmetric matrix after the masking operation to obtain a final mask matrix;

[0019] Step S6: input the mask matrix into a Riemannian manifold network, and sequentially perform at least two rounds of bilinear mapping and nonlinear activation operation, after each round of bilinear mapping and nonlinear activation operation is completed, fuse the output features of the current round with the output features of the previous round, and when a preset iteration calculation number is reached, obtain a final fused feature;

[0020] Step S7: project the fused feature to a tangent space to obtain a tangent space feature representation, and complete retrieval matching of pedestrian re-identification based on the tangent space feature representation.

[0021] In an embodiment of the present application, in step S2, each frame image contained in the preprocessed video data is divided into a plurality of image blocks, and the method for generating a block feature representation containing block features, class features and position encoding is as follows:

[0022] for any th target pedestrian corresponding to a preprocessed RGB video sequence , perform uniform blocking operation on each frame image of the video sequence, divide a single frame into N non-overlapping image blocks, wherein the Nth image block of the th frame is denoted as , , , and T represents the number of frames of the video sequence;

[0023] by a linear mapping function Each image block Perform feature transformation to obtain the corresponding feature labels. Simultaneously, it introduces methods to characterize the first... Category features of global comprehensive features of frames and combined with dimensions Position encoding Finally, the number was obtained. Block feature representation of the frame It satisfies the following relationship:

[0024] ;

[0025] By integrating the block feature representations of all frames in the video sequence, the final block feature representation of the video sequence is obtained. It satisfies the following relationship:

[0026] ;

[0027] in, Indicates spatial dimension, The video frame number is represented by C, the image channel number is represented by H, the image height is represented by W, the image width is represented by D, and the dimension of the feature markers in the encoder is represented by D.

[0028] In one embodiment of the present invention, the method for encoding the block feature representation to obtain frame-level features in step S3 is as follows:

[0029] The block feature representation is used as the initial input to the visual Transformer encoder, which is composed of multiple layers of identical coding units connected in series.

[0030] For the Layer coding unit, with the first Output features of layer coding units For input, perform the following operations in sequence:

[0031] For output features The execution layer performs normalization processing, and the normalized features are input into a multi-head self-attention mechanism to adaptively capture global correlations and interactions between features; the output features of the multi-head self-attention mechanism are then compared with the original input features. Perform residual joins to obtain intermediate features. ;

[0032] For the intermediate features A layer normalization process is performed, and the normalized features are input into a multilayer perceptron. Nonlinear transformations are used to map and enhance the feature dimensions. The output features of the multilayer perceptron are then compared with the original intermediate features. Performing residual connection, obtaining the output feature of the k-th layer coding unit ;

[0033] Sequentially completing the progressive processing of all coding units, outputting the frame-level feature of the video sequence through the multi-layer collaborative operation of the visual Transformer encoder.

[0034] In an embodiment of the present application, in step S4, the block features in the frame-level features are spliced, and the inter-channel covariance matrix is calculated. After adding a regularization term to the inter-channel covariance matrix, the method for obtaining a positive definite symmetric matrix is as follows:

[0035] Reserving all feature labels in the frame-level features, extracting all block feature labels corresponding to each frame-level feature, and taking the feature label corresponding to the i-th image block in the j-th frame as , , , T represents the number of frames of the video sequence, , is the number of image blocks divided by a single frame;

[0036] Integrating the block feature labels of all frames to obtain a global block feature set ;

[0037] Based on the global block feature set, the covariance matrix between feature channel dimensions is calculated to represent the statistical correlation features between different channels, and the calculation formula is:

[0038] ,

[0039] Among them, is the covariance matrix between feature channels initially calculated, is the mean of all feature channel dimensions, which is obtained by averaging the channel dimensions of all feature labels in the global block feature set;

[0040] In order to make the covariance matrix satisfy the positive definite symmetry, a positive definite regularization term is added to the diagonal line of the covariance matrix to obtain the final positive definite symmetric matrix , which is the video-level feature representation capable of representing the global high-order correlation information of the video, and the calculation formula is:

[0041] ,

[0042] Among them, is a constant greater than zero, is a unit matrix with a dimension of , and d represents the feature channel dimension.

[0043] ​In one embodiment of the present invention, in step S5, the expression of the mask matrix is ​​as follows:

[0044] ,

[0045] in, To adapt the mask matrix to the input of the Riemannian manifold network, It represents the Hadamardi (or Hadama) stack.

[0046] In one embodiment of the present invention, the method for obtaining the final fusion feature in step S6 is as follows:

[0047] The mask matrix is ​​used as the initial input to the Riemannian manifold network, which is composed of multiple consecutive feature mapping layers connected in series, with a preset iteration round of at least two rounds.

[0048] For each round's input matrix, process it according to the following steps:

[0049] The current input matrix is ​​processed through a bilinear mapping layer. Perform dimensionality transformation and feature mapping to obtain the output matrix after bilinear mapping. The calculation formula is: ,in This is a pre-defined full-rank matrix;

[0050] The output matrix after the bilinear mapping Perform singular value decomposition to obtain the left singular vector matrix. Singular value diagonal matrix and right singular vector matrix Satisfies the decomposition relation: ,in The diagonal elements are a matrix eigenvalues;

[0051] Based on the singular value diagonal matrix and preset threshold Construct a diagonal matrix A, and perform threshold filtering on the eigenvalues ​​to enhance feature discriminative power. The diagonal elements of the diagonal matrix A are defined as follows:

[0052] ,in Represents the singular value diagonal matrix The diagonal of Line 1 Column elements;

[0053] Based on the left singular vector matrix The right singular vector matrix Calculate the output features after nonlinear activation in the current round using the diagonal matrix A. : ;

[0054] After each round of bilinear mapping and nonlinear activation operations, the output features of the current round are fused. The fused feature matrix is ​​obtained by combining the output features of all previous rounds and used as the input for the next round.

[0055] Repeat the above steps until the preset number of iterations is reached to obtain the result. Frame fusion features .

[0056] In one embodiment of the present invention, after each round of bilinear mapping and nonlinear activation operations is completed, the output features of the current round are fused. The method for obtaining the fused feature matrix by combining the output features of all previous rounds is as follows:

[0057] Assuming execution has been performed Round feature mapping and activation operations define the set of features to be fused. ,in to For the front The output features after the wheel is activated. For the current number The output features after the wheel is activated;

[0058] For each output feature in the set of features to be fused Perform singular value decomposition on each side to obtain the left singular vector matrix. Singular value diagonal matrix and right singular vector matrix The decomposition form is: , ;

[0059] Based on the left singular vector matrix The singular value diagonal matrix and the right singular vector matrix Each output feature is mapped using a logarithmic mapping. Projecting from a nonlinear Riemannian manifold to its corresponding linear tangent space yields the symmetric matrix in the tangent space. : ,in, For the singular value diagonal matrix Take the natural logarithm of each diagonal element;

[0060] Since the tangent space is a Euclidean space, the symmetric matrix of all projective tangent spaces... Calculate the arithmetic mean to obtain the mean matrix in the tangent space. : ;

[0061] mapping the mean matrix back to the Riemannian manifold by exponential mapping, obtaining a fused feature matrix : wherein denotes a diagonal matrix after singular value decomposition of , is an element-wise natural exponential of diagonal elements of ; denotes a left singular vector matrix after singular value decomposition of , denotes a right singular vector matrix after singular value decomposition of .

[0062] In an embodiment of the present application, in step S7, the fused feature is projected to a tangent space, obtaining a tangent space feature representation, and the method for completing the retrieval matching of pedestrian re-identification based on the tangent space feature representation is as follows:

[0063] projecting the fused feature from the nonlinear Riemannian manifold to a linear tangent space by logarithmic mapping, obtaining a tangent space feature representation : , taking the upper triangular part of the matrix of the tangent space feature representation as the final feature representation of the i-th video sequence ;

[0064] normalizing all the final feature representations of the to-be-retrieved video set and the query video set, obtaining the normalized to-be-retrieved video set and the query video set;

[0065] for each feature in the normalized query video set , traversing all the features in the normalized to-be-retrieved video set one by one, calculating the similarity score of the two;

[0066] According to the similarity score, the normalized to-be-retrieved video set is sorted to generate a candidate video list corresponding to the query video; a matching threshold condition is set to determine whether the similarity score of any video in the candidate video list meets the matching threshold condition:

[0067] if yes, it is determined that the video and the target pedestrian corresponding to the query video are the same pedestrian;

[0068] otherwise, it is determined that there is no identity matching result of the target pedestrian corresponding to the query video in the to-be-retrieved video set; ​​​​​

[0069] aggregate the ranking candidate list of each query video and the corresponding identity matching result, to complete pedestrian re-identification retrieval.

[0070] Based on the same inventive concept, the application also provides a video pedestrian re-identification system based on Riemann manifold second-order relationship modeling, comprising a data acquisition and preprocessing module, a block feature representation generation module, a frame-level feature encoding module, a positive definite symmetric matrix construction module, a random mask processing module, a feature fusion module and a retrieval matching module.

[0071] The data acquisition and preprocessing module is configured to acquire RGB pedestrian video sequences in a to-be-retrieved video set and a query video set respectively, and sequentially perform denoising, size normalization and frame rate unification preprocessing operations on each RGB pedestrian video sequence in the two types of video sets to obtain preprocessed video data with consistent formats.

[0072] The block feature representation generation module is configured to divide each frame image contained in the preprocessed video data into a plurality of image blocks, and generate a block feature representation containing block features, class features and position encoding.

[0073] The frame-level feature encoding module is configured to encode the block feature representation to obtain frame-level features.

[0074] The positive definite symmetric matrix construction module is configured to splice the block features in the frame-level features, calculate an inter-channel covariance matrix, and obtain a positive definite symmetric matrix after adding a regularization term to the inter-channel covariance matrix.

[0075] The random mask processing module is configured to perform random mask operations on the positive definite symmetric matrix in the row and column dimensions, and obtain a mask matrix after adding a regularization term, specifically as follows.

[0076] A binary mask matrix M with a dimension of is constructed, all elements in the matrix only take values of 0 or 1, wherein an element taking a value of 0 indicates that the corresponding position needs to be masked, and a value of 1 indicates that the feature information of the corresponding position is retained, D is the dimension of the feature label, is the spatial dimension;

[0077] The mask matrix M is multiplied with the positive definite symmetric matrix in the element level, the rows and columns of the elements marked as 0 in the mask matrix M are synchronously masked, and a regularization term is added to the positive definite symmetric matrix after the mask operation to obtain a final mask matrix.

[0078] The feature fusion module is configured to input the mask matrix into a Riemannian manifold network, sequentially perform at least two rounds of bilinear mapping and nonlinear activation operation, fuse the output features of the current round and the output features of the previous round after each round of bilinear mapping and nonlinear activation operation is completed, and obtain final fused features when a preset iteration number is reached.

[0079] The search matching module is configured to project the fused features to a tangent space to obtain tangent space feature representation, and complete search matching of pedestrian re-identification based on the tangent space feature representation.

[0080] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling when executing the program.

[0081] The above technical solution of the application has the following beneficial effects compared with the prior art:

[0082] The application fully excavates high-order dynamic correlation of video cross-space area and cross-time segment by modeling inter-channel second-order relationship matrix, and has stronger discriminability and robustness compared with traditional first-order feature method; the structured constraint and geometric attribute of the second-order features are realized by relying on the Riemannian manifold network, and the middle-layer features are efficiently utilized by combining the dense connection mechanism and the fusion module based on the Fréchet mean; the positive definite symmetric matrix is processed by using the row-column synchronous mask strategy, which is different from the conventional separate row or column mask mode, can accurately retain the effective correlation structure between channels, avoid destroying the integrity of the second-order statistical information, and simultaneously cooperate with the positive definite compensation mechanism to ensure that the matrix after the mask operation still meets the requirement of symmetry and positive definiteness, and adapt to the processing requirement of the Riemannian manifold network; the matrix sparsification is realized by using the second-order random mask matrix, which not only suppresses the feature redundancy and overfitting risk, but also enhances the generalization performance and robustness of the model, so that the model has stronger pedestrian identity discrimination ability in complex scenes (such as view angle change, occlusion, and posture difference), and exhibits better performance than the existing advanced methods on multiple public data sets. BRIEF DESCRIPTION OF DRAWINGS

[0083] In order to make the content of the application more easily understood, the application will be further described in detail below according to specific embodiments of the application and in combination with the drawings.

[0084] Figure 1 is a flowchart of the video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling provided in the embodiments of the application;

[0085] Figure 2is a specific flowchart of a video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling provided in the embodiments of the present application;

[0086] Figure 3 is a brief overview of the first-order information processing scheme and the high-order information processing scheme about the mainstream video pedestrian re-identification method;

[0087] Figure 4 is a performance comparison of the video pedestrian re-identification method proposed in the present application and other mainstream video pedestrian re-identification methods on different data sets;

[0088] Figure 5 is an architecture diagram of a video pedestrian re-identification system based on Riemannian manifold second-order relationship modeling provided in the embodiments of the present application.

[0089] The description of the figures is as follows: 100, a data acquisition and preprocessing module; 200, a block feature representation generation module; 300, a frame-level feature encoding module; 400, a positive definite symmetric matrix construction module; 500, a random mask processing module; 600, a feature fusion module; and 700, a retrieval matching module. DETAILED DESCRIPTION

[0090] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not limiting to the present application.

[0091] Embodiment one:

[0092] Referring to Figure 1 and Figure 2 , the present application provides a video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling, which comprises the following steps:

[0093] Step S1: RGB pedestrian video sequences in a to-be-retrieved video set and a query video set are acquired respectively, and denoising, size normalization and frame rate unification preprocessing operations are sequentially performed on each RGB pedestrian video sequence in the two types of video sets to obtain preprocessed video data with consistent formats;

[0094] Step S2: each frame image contained in the preprocessed video data is divided into a plurality of image blocks to generate a block feature representation containing block features, class features and position encoding;

[0095] Step S3: the block feature representation is encoded to obtain frame-level features;

[0096] Step S4: each block feature in the frame-level features is spliced, a channel covariance matrix is calculated, and a positive definite symmetric matrix is obtained after adding a regularization term in the channel covariance matrix;

[0097] Step S5: Perform a random masking operation on the row and column dimensions of the positive definite symmetric matrix, and add regularization terms to obtain the mask matrix;

[0098] Step S6: Input the mask matrix into the Riemann manifold network and perform at least two rounds of bilinear mapping and nonlinear activation operations in sequence. After each round of bilinear mapping and nonlinear activation operations, fuse the output features of the current round with the output features of the previous round to reach the preset number of iterations and obtain the final fused features.

[0099] Step S7: Project the fused features onto the tangent space to obtain the tangent space feature representation, and complete the retrieval and matching of pedestrian re-identification based on the tangent space feature representation.

[0100] Further, in step S2, each frame of the preprocessed video data is divided into multiple image blocks, and the method for generating a block feature representation containing block features, category features, and location encoding is as follows:

[0101] For any 1 Preprocessed RGB video sequence corresponding to each target pedestrian Perform a uniform block division operation on each frame of the video sequence, dividing a single frame into N non-overlapping image blocks, where the th... The Nth image block of the frame is denoted as , T represents the number of frames in the video sequence;

[0102] Through linear mapping function Each image block Perform feature transformation to obtain the corresponding feature labels. ;

[0103] Introducing categorical features , used to characterize the The global integrated features of the frame supplement the deficiencies of local features of image patches;

[0104] Introducing dimensions as Position encoding It is used to characterize the spatial location information of category features and features of each image patch, avoiding positional ambiguity;

[0105] Category features Feature tags corresponding to all image patches With position encoding By merging, we obtain the first... Block feature representation of the frame It satisfies the following relationship:

[0106] ;

[0107] integrating the block feature representations of all frames in the video sequence to obtain a block feature representation of the video sequence , satisfying the following relationship:

[0108] ;

[0109] wherein, represents the spatial dimension, represents the number of video frames, C represents the number of image channels, H represents the image height, W represents the image width, and D represents the dimension of the feature mark in the subsequent visual Transformer encoder.

[0110] Further, in step S3, the block feature representation generated in step S2 is input into a visual Transformer (ViT) encoder, and through progressive processing of multiple layers of encoding units, a frame-level feature of the video sequence is output, and the specific process is as follows: The visual Transformer encoder is composed of multiple layers of encoding units with the same structure, preferably 12 layers.

[0111] For the k-th layer of encoding units (k = 1, 2, …, 12), the output feature of the (k-1)-th layer of encoding units is taken as the input (the input of the initial first layer of encoding units is the block feature representation generated in step S2), and the following operations are sequentially performed:

[0112] Layer normalization (Norm) is performed on the output feature to eliminate the difference in feature scale; the normalized feature is input into a multi-head self-attention mechanism (MHSA) to adaptively capture global correlation and interaction information between features; the output feature of the multi-head self-attention mechanism is connected with the original input feature to obtain an intermediate feature, and the calculation formula is as follows: Layer normalization is performed on the intermediate feature to input the normalized feature into a multi-layer perceptron (MLP) to realize feature dimension mapping and expression enhancement through nonlinear transformation; the output feature of the multi-layer perceptron is connected with the original intermediate feature to obtain the output feature of the k-th layer of encoding units, and the calculation formula is as follows:

[0113]

[0114]

[0115] ​​​​​​​​​​​​​​The output features of a previous layer of coding units are taken as inputs of a next layer, and progressive processing of all coding units is sequentially completed; finally, frame-level features of the video sequence are output through multi-layer collaborative operation of the visual Transformer encoder wherein corresponding to the first frame-level features of the frame, contain fusion information of category features and image block features.

[0116] Further, in step S4, to fully exploit high-order correlation information between feature channels, a video-level symmetric positive definite (SPD) matrix is constructed through the following process, and the specific steps are as follows:

[0117] All feature labels in the frame-level features output in step S3 are retained, all block feature labels corresponding to each frame-level feature are extracted, and the feature label corresponding to the i-th image block of the j-th frame is denoted as , T represents the number of frames of the video sequence, , T represents the number of frames of the video sequence, is the number of image blocks divided for a single frame;

[0118] The block feature labels of all frames are integrated to obtain a global block feature set ;

[0119] Based on the global block feature set, a covariance matrix between feature channel dimensions is calculated to represent statistical correlation features between different channels, and the calculation formula is as follows:

[0120] ,

[0121] wherein, is the covariance matrix between feature channels calculated initially, is the mean of all feature channel dimensions, which is obtained by averaging the channel dimensions of all feature labels in the global block feature set; represents the transpose of .

[0122] To make the covariance matrix satisfy the positive definite symmetry, so as to adapt to the processing requirements of the subsequent Riemannian manifold network, a positive definite regularization term is added to the diagonal line of the covariance matrix , and the final symmetric positive definite matrix is obtained, which is a video-level feature representation capable of representing global high-order correlation information of the video, and the calculation formula is as follows:

[0123] wherein, is a constant greater than zero, which is used to ensure the positive definiteness of the matrix; is a constant greater than zero, which is used to ensure the positive definiteness of the matrix; ​​​identity matrix, d represents the feature channel dimension.

[0124] Further, in step S5, to suppress feature redundancy, enhance model generalization ability, and maintain the positive definite symmetric property of the matrix, the mask matrix is generated according to the following process:

[0125] A binary mask matrix M with a dimension of is constructed, all elements of the matrix taking only 0 or 1, where an element taking 0 indicates that the corresponding position needs to be masked, and an element taking 1 indicates that the feature information of the corresponding position is retained, and D is the dimension of the feature label;

[0126] Element-level multiplication operation is performed on the mask matrix M and the positive definite symmetric matrix obtained in step S4, to realize element-level multiplication processing of the positive definite symmetric matrix. Through this operation, the rows and columns where the elements marked as 0 by the mask matrix M in the matrix are simultaneously masked, thereby shielding redundant channel-related information and retaining key feature-related information.

[0127] To avoid the matrix losing the positive definite symmetric property after the masking operation, a small positive number regularization term is added to the Hadamard product matrix, which compensates for the numerical loss caused by masking and ensures that the final output matrix still meets the positive definite symmetric requirement, obtaining the final mask matrix, and the calculation formula is:

[0128] wherein is the mask matrix adapted to the input of the Riemannian manifold network, which retains the core feature-related information and reduces the overfitting risk through masking; represents the Hadamard product.

[0129] Further, in step S6, the mask matrix is subjected to multiple rounds of feature mapping, activation, and cross-layer fusion through the densely connected Riemannian manifold network, and finally the fusion features retaining high-order correlation information are output, and the specific process is as follows:

[0130] The mask matrix obtained in step S5 is taken as the initial input of the Riemannian manifold network; the Riemannian manifold network is composed of a plurality of consecutive feature mapping layers connected in series, and the preset iteration times are at least two, to ensure sufficient transformation and fusion of features.

[0131] For the input matrix of each round (the first round input is the initial mask matrix , and the input of subsequent rounds is the feature matrix fused in the last round), the following steps are taken:

[0132] The current input matrix Perform dimensional transformation and feature mapping while preserving Riemannian geometric properties to obtain the output matrix after bilinear mapping. The calculation formula is: ,in This is a pre-defined full-rank column matrix used to achieve dimensionality compression and feature transformation;

[0133] The output matrix after the bilinear mapping Perform singular value decomposition (SVD) to obtain the left singular vector matrix. Singular value diagonal matrix and right singular vector matrix Satisfies the decomposition relation: ,in The diagonal elements are a matrix The eigenvalues ​​characterize the importance of the features;

[0134] Set preset threshold Based on the singular value diagonal matrix Construct a diagonal matrix A, and perform threshold filtering on the eigenvalues ​​to enhance feature discriminative power. The diagonal elements of the diagonal matrix A are defined as follows:

[0135] ,in Represents the singular value diagonal matrix The diagonal of Line 1 Column elements;

[0136] Based on the left singular vector matrix The right singular vector matrix Using the diagonal matrix A, calculate the output features after nonlinear activation (ReEig) in the current round. : This operation achieves a nonlinear enhancement effect similar to ReLU in Euclidean space, while maintaining the positive definite symmetry property of the matrix.

[0137] After each round of bilinear mapping and nonlinear activation operations, the Fréchet Mean method is used to fuse the output features of the current round. The fused feature matrix is ​​obtained by combining the output features of all previous rounds and used as the input for the next round.

[0138] Repeat the single-round processing and cross-round fusion steps described above until the preset number of iterations is reached, to obtain the result. Frame fusion features This feature matrix fully integrates the effective feature information from each round, and has strong discriminative power and robustness.

[0139] Specifically, after each round of bilinear mapping and nonlinear activation operations, the output features of the current round are fused. The method for obtaining the fused feature matrix by combining the output features of all previous rounds is as follows:

[0140] Assuming execution has been performed Round feature mapping and activation operations ( Define the set of features to be fused. ,in to For the front The output features after the wheel is activated. For the current number The output features after the activation of the wheel are all positive definite symmetric matrices;

[0141] For each output feature in the set of features to be fused Perform singular value decomposition on each side to obtain the left singular vector matrix. Singular value diagonal matrix and right singular vector matrix The decomposition form is: , ;

[0142] Based on the left singular vector matrix The singular value diagonal matrix and the right singular vector matrix Each output feature is mapped using a logarithmic mapping (log map). Projecting from a nonlinear Riemannian manifold to its corresponding linear tangent space yields the symmetric matrix in the tangent space. : ,in, For the singular value diagonal matrix Take the natural logarithm of each diagonal element;

[0143] Since the tangent space is a Euclidean space, the symmetric matrix of all projective tangent spaces... Calculate the arithmetic mean to obtain the mean matrix in the tangent space. : This mean matrix integrates effective information from all rounds of features while maintaining linearity.

[0144] The mean matrix is ​​obtained by exponential mapping (exp mapping). Mapping back to the Riemannian manifold yields the fused eigenma matrix. : The final output It remains a positive definite symmetric matrix, which integrates the advantages of features from each round and satisfies the subsequent processing requirements of Riemannian manifold networks;

[0145] in Indicates to The diagonal matrix after singular value decomposition. Yes The diagonal elements are each taken as their natural exponent; Indicates to The left singular vector matrix after singular value decomposition. Indicates to The right singular vector matrix after singular value decomposition.

[0146] Furthermore, in step S7, effective features are extracted through the "manifold → tangent space" projection transformation, and retrieval matching is completed based on feature similarity measurement. The specific process is as follows:

[0147] The fused features output from step S6 are obtained through logarithmic mapping. From the projection of the nonlinear Riemannian manifold to the linear tangent space, we obtain the characteristic representation of the tangent space. : .

[0148] because It is a symmetric matrix with completely redundant elements in its upper and lower triangular regions. Extract... The upper triangular part is used as the first The final feature representation of a video sequence This method retains all independent feature information of the matrix while compressing the feature dimension and reducing the computational cost of subsequent matching.

[0149] All final feature representations of the retrieved video set and the query video set Normalization and optimization are performed. The normalization method eliminates scale differences between different features, resulting in a standardized set of videos to be retrieved. and query video collection ;

[0150] For each feature in the standardized query video set Iterate through all features in the standardized video set to be retrieved one by one. The similarity score between the two videos is calculated using a metric that adapts to the linear features of the tangent space (such as Euclidean distance and cosine similarity), which quantifies the degree of matching between the features. The smaller the Euclidean distance or the closer the cosine similarity is to 1, the higher the degree of matching between the features of the two videos.

[0151] Based on the similarity scores, the standardized set of videos to be retrieved is sorted in descending order (cosine similarity) or ascending order (Euclidean distance) to generate a list of candidate videos corresponding to the query video. The matching threshold is then fine-tuned using a validation set. , whether a similarity score of any video in the candidate video list meets a matching threshold condition (cosine similarity or Euclidean distance ):

[0152] If there is a video that meets the threshold condition, it is determined that the target pedestrian corresponding to the query video and the video are the same pedestrian;

[0153] If the scores of all candidate videos do not meet the threshold condition, it is determined that there is no identity matching result of the target pedestrian corresponding to the query video in the to-be-searched video set;

[0154] The sorted candidate list and the corresponding identity matching result of each query video are summarized, and the pedestrian re-identification search process of the full query video is completed.

[0155] To fully verify the effectiveness and superiority of the video pedestrian re-identification method based on the Riemannian manifold second-order relationship modeling (hereinafter referred to as “SORSNet”) of the present application, the present embodiment compares the existing advanced methods on the public data set, and verifies the performance of the method from the quantitative point of view.

[0156] Seventeen mainstream video pedestrian re-identification methods are selected as comparison benchmarks, covering first-order information processing and high-order information processing schemes, including: first-order information processing schemes (M3D, STMN, STRF, BiCnet-TKS, MFA, CAVIT, MSTAT, PSTA, FIDN, SDCL, HASI, TF-CLIP, Climb-ReID, GAENet, MS-STI, TCViT, FAViT, PSFormer), and high-order information processing schemes (SANet, SINet). The brief overview of each comparison method is shown in Figure 3 .

[0157] Both training and testing are completed on an NVIDIA RTX 3090 GPU, ensuring sufficient computing resources to support model training and inference. The Adam optimizer is used to update the model parameters; the loss function uses a combination of label smoothing cross-entropy loss and difficult sample triplet loss to balance the optimization objectives of the classification task and the feature metric task.

[0158] The total training epoch is set to 300, and the first 10 epochs are the warm-up stage, which gradually transitions the model from a lower learning rate to the target learning rate, avoiding training shocks caused by excessive learning rate for initial random parameters. The initial learning rate of the ViT network is 5×10 -6 , and the initial learning rate of the Riemannian manifold network is 5×10 -5The learning rate is reduced by 10 times at the 60th, 100th and 200th epochs, respectively, to promote the convergence of late parameters.

[0159] Five public authoritative datasets in the field of video pedestrian re-identification are selected for testing, including a MARS dataset, a MARS-DL dataset, an LS-VID dataset, an i-LIDS-VID dataset and a PRID-2011 dataset. These datasets cover pedestrians in different scenes, perspectives and light conditions, and can comprehensively test the generalization ability and recognition performance of the model in complex environments. The sizes of the datasets are as follows: the MARS dataset and the MARS-DL dataset each include 1261 pedestrian identity IDs, the LS-VID dataset includes 3772 pedestrian identity IDs, the i-LIDS-VID dataset includes 300 pedestrian identity IDs, and the PRID-2011 dataset includes 200 pedestrian identity IDs.

[0160] The mean average precision (mAP) and Rank-1 and Rank-5 accuracy commonly used in the field are used as evaluation indexes, where the mAP reflects the overall precision of the retrieval result, the Rank-k accuracy represents the probability that the correct matching target is included in the top k results, and the higher the index value, the better the recognition performance of the model. The specific experimental results are shown in Table 1. Figure 4

[0161] From the experimental results in Table 1, it can be seen that: Figure 4

[0162] On the MARS dataset, the mAP of the SORSNet method proposed in the application reaches 89.5%, which is only slightly lower than the best Climb-ReID first-order information processing method; and the Rank-1 accuracy is as high as 92.3%.

[0163] On the MARS-DL dataset, the mAP of the SORSNet method proposed in the application reaches 93.1%, and the Rank-1 accuracy is 96.3%, which is better than all the comparison methods.

[0164] On the LS-VID dataset, the mAP of the SORSNet method proposed in the application is 86.4%, and the Rank-1 accuracy reaches 91.5%, which is significantly better than all the comparison methods, and reflects the efficient feature learning ability of the model on large-scale video data.

[0165] On the i-LIDS-VID dataset, the mAP of the SORSNet method proposed in the application is 97.9%, the Rank-1 accuracy is 96.7%, and the Rank-5 accuracy is 99.3%, which shows strong identity matching ability on this dataset, and proves the effectiveness of the dense Riemannian manifold network and the feature fusion mechanism. ​​

[0166] On the PRID-2011 dataset, the mAP of the SORSNET method proposed in the application is 97.8%, the Rank-1 accuracy is 96.6%, and the Rank-5 accuracy is 98.9%, which still maintains excellent performance in the cross-camera view change scene, highlighting the robustness of the model to complex scene factors.

[0167] In summary, the SORSNET method proposed in the application is superior to most existing advanced methods in various evaluation indicators on five public datasets, fully proving the synergistic effect of the second-order relationship modeling, the dense Riemannian manifold network, the second-order Dropout module and the feature fusion mechanism proposed in the application, which can effectively mine high-order correlation information in the video and improve the accuracy and generalization ability of pedestrian re-identification.

[0168] Embodiment two:

[0169] Based on the same inventive concept as embodiment one, the application further provides a video pedestrian re-identification system based on Riemannian manifold second-order relationship modeling, as shown in Figure 5 , which comprises a data acquisition and preprocessing module 100, a block feature representation generation module 200, a frame-level feature encoding module 300, a positive definite symmetric matrix construction module 400, a random mask processing module 500, a feature fusion module 600 and a retrieval matching module 700.

[0170] The data acquisition and preprocessing module 100 is configured to acquire RGB pedestrian video sequences in a to-be-retrieved video set and a query video set respectively, and sequentially perform denoising, size normalization and frame rate unification preprocessing operations on each RGB pedestrian video sequence in the two types of video sets to obtain preprocessed video data with consistent formats.

[0171] The block feature representation generation module 200 is configured to divide each frame image contained in the preprocessed video data into a plurality of image blocks, and generate a block feature representation containing block features, class features and position encoding.

[0172] The frame-level feature encoding module 300 is configured to encode the block feature representation to obtain frame-level features.

[0173] The positive definite symmetric matrix construction module 400 is configured to splice the block features in the frame-level features, calculate an inter-channel covariance matrix, and obtain a positive definite symmetric matrix after adding a regularization term to the inter-channel covariance matrix.

[0174] The random mask processing module 500 is configured to perform random mask operations on the row and column dimensions of the positive definite symmetric matrix to obtain a mask matrix after adding a regularization term.

[0175] The feature fusion module 600 is configured to input the mask matrix into a Riemannian manifold network, sequentially perform at least two rounds of bilinear mapping and nonlinear activation operation, fuse the output features of the current round and the output features of the previous round after each round of bilinear mapping and nonlinear activation operation is completed, and obtain final fused features when a preset iteration number is reached.

[0176] The search and matching module 700 is configured to project the fused features to a tangent space to obtain tangent space feature representation, and complete search and matching of pedestrian re-identification based on the tangent space feature representation.

[0177] The video pedestrian re-identification system based on Riemannian manifold second-order relationship modeling provided in the embodiment is used to implement the video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling, and therefore the specific embodiments of the video pedestrian re-identification system based on Riemannian manifold second-order relationship modeling can be found in the embodiment part of the video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling, for example, the data acquisition and preprocessing module 100, the block feature representation generation module 200, the frame-level feature encoding module 300, the positive definite symmetric matrix construction module 400, the random mask processing module 500, the feature fusion module 600, and the search and matching module 700, which are respectively used to implement steps S1 to S7 in the video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling in the embodiment one. Therefore, the specific embodiments can refer to the descriptions of the respective parts of the embodiment, and details are not described herein to avoid redundancy.

[0178] Embodiment three:

[0179] The application further provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the video pedestrian re-identification method based on Riemannian manifold second-order relationship modeling in the embodiment one when executing the program.

[0180] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system, or a computer program product. Therefore, the application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0181] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0182] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0183] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0184] Obviously, the above-described embodiments are only examples for clarity of description and are not limiting on the embodiments. Based on the above description, one of ordinary skill in the art can further make other different forms of changes or modifications. Here, all the embodiments are not required to be exhausted. The obvious changes or modifications derived from the above are still within the protection scope of the present application.

Claims

1. A video pedestrian re-identification method based on Riemannian manifold second-order relation modeling, characterized in that, Includes the following steps: Step S1: Obtain the RGB pedestrian video sequences from the video set to be retrieved and the video set to be queried, respectively. Perform noise reduction, size normalization and frame rate unification preprocessing operations on each RGB pedestrian video sequence in the two video sets in sequence to obtain preprocessed video data with consistent format. Step S2: Divide each frame of the preprocessed video data into multiple image blocks and generate a block feature representation that includes block features, category features and location codes; Step S3: Encode the block feature representation to obtain frame-level features; Step S4: Concatenate the features in the frame-level features, calculate the inter-channel covariance matrix, and add a regularization term to the inter-channel covariance matrix to obtain a positive definite symmetric matrix; Step S5: Perform a random masking operation on the row and column dimensions of the positive definite symmetric matrix, and add regularization terms to obtain the mask matrix, specifically: Construction dimension The binary mask matrix M is such that all elements take values ​​of 0 or 1. A value of 0 indicates that the corresponding position needs to be masked, while a value of 1 indicates that the feature information at the corresponding position is preserved. D is the dimension of the feature markers. Spatial dimension; The mask matrix M and the positive definite symmetric matrix Element-wise multiplication is performed, and the rows and columns containing elements marked as 0 by the mask matrix M are synchronously masked. Regularization terms are then added to the positive definite symmetric matrix after the masking operation. This yields the final mask matrix; Step S6: Input the mask matrix into the Riemann manifold network and perform at least two rounds of bilinear mapping and nonlinear activation operations in sequence. After each round of bilinear mapping and nonlinear activation operations, fuse the output features of the current round with the output features of the previous round to reach the preset number of iterations and obtain the final fused features. Step S7: Project the fused features onto the tangent space to obtain the tangent space feature representation, and complete the retrieval and matching of pedestrian re-identification based on the tangent space feature representation; In step S6, the method for obtaining the final fusion features is as follows: The mask matrix is ​​used as the initial input to the Riemannian manifold network, which is composed of multiple consecutive feature mapping layers connected in series, with a preset iteration round of at least two rounds. For each round's input matrix, process it according to the following steps: The current input matrix is ​​processed through a bilinear mapping layer. Perform dimensionality transformation and feature mapping to obtain the output matrix after bilinear mapping. The calculation formula is: ,in This is a pre-defined full-rank matrix; The output matrix after the bilinear mapping Perform singular value decomposition to obtain the left singular vector matrix. Singular value diagonal matrix and right singular vector matrix Satisfies the decomposition relation: ,in The diagonal elements are a matrix eigenvalues; Based on the singular value diagonal matrix and preset threshold Construct a diagonal matrix A, and perform threshold filtering on the eigenvalues ​​to enhance feature discriminative power. The diagonal elements of the diagonal matrix A are defined as follows: ,in Represents the singular value diagonal matrix The diagonal of Line 1 Column elements; Based on the left singular vector matrix The right singular vector matrix Calculate the output features after nonlinear activation in the current round using the diagonal matrix A. : ; After each round of bilinear mapping and nonlinear activation operations, the output features of the current round are fused. The fused feature matrix is ​​obtained by combining the output features of all previous rounds and used as the input for the next round. Repeat the above steps until the preset number of iterations is reached to obtain the result. Frame fusion features ; After each round of bilinear mapping and nonlinear activation operations, the output features of the current round are fused. The method for obtaining the fused feature matrix by combining the output features of all previous rounds is as follows: Assuming execution has been performed Round feature mapping and activation operations define the set of features to be fused. ,in to For the front The output features after the wheel is activated. For the current number The output features after the wheel is activated; For each output feature in the set of features to be fused Perform singular value decomposition on each side to obtain the left singular vector matrix. Singular value diagonal matrix and right singular vector matrix The decomposition form is: , ; Based on the left singular vector matrix The singular value diagonal matrix and the right singular vector matrix Each output feature is mapped using a logarithmic mapping. Projecting from a nonlinear Riemannian manifold to its corresponding linear tangent space yields the symmetric matrix in the tangent space. : ,in, For the singular value diagonal matrix Take the natural logarithm of each diagonal element; Since the tangent space is a Euclidean space, the symmetric matrix of all projective tangent spaces... Calculate the arithmetic mean to obtain the mean matrix in the tangent space. : ; The mean matrix is ​​obtained through exponential mapping. Mapping back to the Riemannian manifold yields the fused eigenma matrix. : ,in Indicates to The diagonal matrix after singular value decomposition. Yes The diagonal elements are each taken as their natural exponent; Indicates to The left singular vector matrix after singular value decomposition. Indicates to The right singular vector matrix after singular value decomposition.

2. The video pedestrian re-identification method based on Riemannian manifold second-order relation modeling according to claim 1, characterized in that: In step S2, each frame of the preprocessed video data is divided into multiple image blocks, and the method for generating a block feature representation containing block features, category features, and location codes is as follows: For any 1 Preprocessed RGB video sequence corresponding to each target pedestrian Perform a uniform block division operation on each frame of the video sequence, dividing a single frame into N non-overlapping image blocks, where the th... The Nth image block of the frame is denoted as , T represents the number of frames in the video sequence; Through linear mapping function Each image block Perform feature transformation to obtain the corresponding feature labels. Simultaneously, it introduces methods to characterize the first... Category features of global comprehensive features of frames and combined with dimensions Position encoding Finally, the number was obtained. Block feature representation of the frame It satisfies the following relationship: ； By integrating the block feature representations of all frames in the video sequence, the final block feature representation of the video sequence is obtained. It satisfies the following relationship: ； in, Indicates spatial dimension, The video frame number is represented by C, the image channel number is represented by H, the image height is represented by W, the image width is represented by D, and the dimension of the feature markers in the encoder is represented by D.

3. The video pedestrian re-identification method based on Riemannian manifold second-order relation modeling according to claim 1, characterized in that: In step S3, the block feature representation is encoded to obtain frame-level features using the following method: The block feature representation is used as the initial input to the visual Transformer encoder, which is composed of multiple layers of identical coding units connected in series. For the Layer coding unit, with the first Output features of layer coding units For input, perform the following operations in sequence: For output features The execution layer performs normalization processing, and the normalized features are input into a multi-head self-attention mechanism to adaptively capture global correlations and interactions between features; the output features of the multi-head self-attention mechanism are then compared with the original input features. Perform residual joins to obtain intermediate features. ; For the intermediate features A layer normalization process is performed, and the normalized features are input into a multilayer perceptron. Nonlinear transformations are used to map and enhance the feature dimensions. The output features of the multilayer perceptron are then compared with the original intermediate features. Perform residual connections to obtain the output features of the k-th layer coding unit. ; The progressive processing of all encoding units is completed sequentially, and the frame-level features of the video sequence are output through multi-layer collaborative operation of the visual Transformer encoder.

4. The video pedestrian re-identification method based on Riemannian manifold second-order relation modeling according to claim 1, characterized in that: In step S4, the features in the frame-level features are concatenated, the inter-channel covariance matrix is ​​calculated, and a regularization term is added to the inter-channel covariance matrix to obtain a positive definite symmetric matrix. The method is as follows: Retain all feature labels in the frame-level features, extract all block feature labels corresponding to each frame-level feature, and then... Frame number The feature labels corresponding to each image patch are denoted as follows: , T represents the number of frames in the video sequence. , The number of image blocks divided into a single frame; The block feature tags of all frames are integrated to obtain a global block feature set. ; Based on the global block feature set, the covariance matrix between feature channel dimensions is calculated to characterize the statistical correlation features between different channels. The calculation formula is as follows: ， in, This is the initial covariance matrix between the feature channels. The mean of all feature channel dimensions is obtained by averaging the channel dimensions of all feature labels in the global block feature set; To ensure that the covariance matrix satisfies positive definite symmetry, positive definite regularization terms are added to the diagonal of the covariance matrix, resulting in the final positive definite symmetric matrix. This refers to the video-level feature representation that can characterize the global high-order correlation information of a video, and its calculation formula is: ， in, A constant that is greater than zero. For dimension The identity matrix, where d represents the feature channel dimension.

5. The video pedestrian re-identification method based on Riemannian manifold second-order relation modeling according to claim 1, characterized in that: In step S5, the expression for the mask matrix is ​​as follows: ， in, To adapt the mask matrix to the input of the Riemannian manifold network, It represents the Hadamardi (or Hadama) stack.

6. The video pedestrian re-identification method based on Riemannian manifold second-order relation modeling according to claim 1, characterized in that: In step S7, the fused features are projected onto the tangent space to obtain the tangent space feature representation. The method for completing the retrieval and matching of pedestrian re-identification based on the tangent space feature representation is as follows: The fused features are obtained through logarithmic mapping. From the projection of the nonlinear Riemannian manifold to the linear tangent space, we obtain the characteristic representation of the tangent space. : Take the tangent space feature representation The upper triangular part of the matrix is ​​used as the first The final feature representation of a video sequence ; All final feature representations of the retrieved video set and the query video set Normalization is performed to obtain the standardized set of videos to be retrieved and the set of videos to be queried; For each feature in the standardized query video set Iterate through all features in the standardized video set to be retrieved one by one. Calculate the similarity score between the two. Based on the similarity scores, the standardized set of videos to be retrieved is sorted to generate a list of candidate videos corresponding to the query videos; Set a matching threshold condition to determine whether the similarity score of any video in the candidate video list meets the matching threshold condition: If so, then it is determined that the target pedestrian in the video is the same pedestrian as the one in the query video; Otherwise, it is determined that there is no identity matching result for the target pedestrian in the video set to be searched. Summarize the candidate list for each queried video and the corresponding identity matching results to complete the pedestrian re-identification retrieval.

7. A video pedestrian re-identification system based on second-order relation modeling of Riemannian manifolds, characterized in that, To implement the video pedestrian re-identification method based on Riemannian manifold second-order relation modeling as described in claim 1, the system includes the following modules: The data acquisition and preprocessing module is used to acquire RGB pedestrian video sequences from the video set to be retrieved and the video set to be queried, and to perform noise reduction, size normalization and frame rate unification preprocessing operations on each RGB pedestrian video sequence in the two video sets in sequence to obtain preprocessed video data with consistent format. The block feature representation generation module is used to divide each frame of the preprocessed video data into multiple image blocks and generate a block feature representation that includes block features, category features and position codes. A frame-level feature encoding module is used to encode the block feature representation to obtain frame-level features; A positive definite symmetric matrix construction module is used to stitch together the features in the frame-level features, calculate the inter-channel covariance matrix, and obtain a positive definite symmetric matrix by adding a regularization term to the inter-channel covariance matrix. The random masking module performs a random masking operation on the row and column dimensions of the positive definite symmetric matrix, adding regularization terms to obtain the mask matrix, specifically: Construction dimension The binary mask matrix M is such that all elements take values ​​of 0 or 1. A value of 0 indicates that the corresponding position needs to be masked, while a value of 1 indicates that the feature information at the corresponding position is preserved. D is the dimension of the feature markers. Spatial dimension; The mask matrix M and the positive definite symmetric matrix Element-wise multiplication is performed, and the rows and columns containing elements marked as 0 by the mask matrix M are synchronously masked. Regularization terms are then added to the positive definite symmetric matrix after the masking operation. This yields the final mask matrix; The feature fusion module is used to input the mask matrix into the Riemann manifold network and sequentially perform at least two rounds of bilinear mapping and nonlinear activation operations. After each round of bilinear mapping and nonlinear activation operations, the output features of the current round are fused with the output features of the previous round to reach a preset number of iterations and obtain the final fused features. The retrieval and matching module is used to project the fused features onto the tangent space to obtain the tangent space feature representation, and to complete the retrieval and matching for pedestrian re-identification based on the tangent space feature representation.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the video pedestrian re-identification method based on Riemannian manifold second-order relation modeling as described in any one of claims 1 to 6.

Images