Unsupervised pedestrian search method for raw surveillance video

Abstract

This invention relates to an unsupervised pedestrian search method for raw surveillance videos, comprising: using a pre-trained neural network to detect pedestrian regions in the raw surveillance video, and using a feature encoder to generate pedestrian features; in the raw surveillance video, based on the visual similarity and contextual similarity of pedestrians, finding the first nearest neighbor relationship of pedestrian features and organizing the chain structure of pedestrian features, determining the spatial distribution of pedestrian features, and performing unsupervised clustering based on segmented regions to generate pedestrian identity categories and their pseudo-labels; during the forward propagation process, calculating the contrast loss function of each branch from three branches: category discriminative learning, single-camera contrast learning, and multi-camera contrast learning, and updating the cluster center memory and multi-camera memory respectively according to the momentum update mechanism; jointly optimizing the contrast loss functions of the three branches, and minimizing the total loss value through multiple iterations during the backpropagation process to optimize the parameters of the feature encoder.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically an unsupervised pedestrian search method for raw surveillance videos. Background Technology

[0002] Due to the exponential growth of video data collected by surveillance cameras, analyzing this video still faces numerous challenges. On one hand, current processing of raw surveillance video relies primarily on manual analysis, supplemented by simple intelligent methods. This often leads to technical difficulties such as "video is there, but cannot be found," "can be found, but takes too long," and "service is available, but unreliable," resulting in video investigations often consuming significant time, manpower, and resources. On the other hand, manually annotating raw surveillance video faces problems such as massive data volumes, high annotation time costs, slow annotation speed, and difficulty in ensuring annotation consistency, making the workload extremely heavy and often impractical in crime investigation. Therefore, accurately locating specific pedestrian targets in raw surveillance video is not only of significant theoretical research importance in the field of computer vision but also has practical application value for maintaining public order.

[0003] In recent years, pedestrian re-identification methods have been extensively studied in academic circles both domestically and internationally. This method involves pre-cropping pedestrian images with bounding boxes, separating them from the background, and labeling the pedestrian's identity category. Then, a given pedestrian target is associated and matched against a cropped pedestrian image database. Most pedestrian re-identification methods are based on pedestrian images with labeled identity categories and use supervised learning to train pedestrian re-identification models in image databases such as CUHK, Market-150, DukeMTMC-reID, and MSMT17. However, original surveillance videos lack prior pedestrian bounding boxes and cannot pre-determine pedestrian identity categories, making it impossible to directly train pedestrian search models using supervised learning methods, thus limiting their application in practical surveillance fields. A literature search of existing technologies revealed that patent CN114821661B provides an "unsupervised pedestrian re-identification method, apparatus, device, and readable storage medium." In this method, a deep learning model is used to generate pedestrian features from cropped pedestrian images, and then pedestrian features are generated again from the preprocessed pedestrian images after clustering, thus iteratively training the deep learning model. However, this method is trained on cropped pedestrian images and requires a pre-defined number of pedestrian image categories, making it unsuitable for original surveillance videos where pedestrian bounding boxes and the number of pedestrian categories are unknown. Patent CN113065409B provides an "unsupervised pedestrian re-identification method based on camera distribution difference alignment constraints." This method belongs to the domain adaptive re-identification algorithm and also targets cropped pedestrian images. It mainly focuses on using a triplet loss function to solve the problem of consistency between the source and target domain distributions, thereby using the source domain with labeled pedestrian categories to improve the accuracy of the target domain without labeled pedestrian categories. However, directly acquired original surveillance videos do not distinguish between source and target domains, and this algorithm does not consider the spatiotemporal characteristics of the original surveillance video, thus limiting its application in practical surveillance fields.

[0004] Pedestrian search methods introduce pedestrian detection on top of pedestrian re-identification, narrowing the gap between pedestrian re-identification and practical surveillance applications. Compared with pedestrian re-identification methods based on cropped images, pedestrian search methods are more in line with the needs of actual surveillance. Based on different pedestrian detection integration methods, existing pedestrian search methods can be divided into end-to-end methods and two-stage methods. A literature search of existing technologies revealed that the paper "Joint Detection and Identification Feature Learning for Person Search" (CVPR 2017) first proposed an end-to-end pedestrian search framework, integrating pedestrian detection and pedestrian re-identification into a Fast R-CNN neural network, and using Online Instance Matching Loss (OIM) for supervised training on the CHUK-SYSU and PRW image databases. Further research revealed that the paper "Person Search via A Mask-guided Two-stream CNN Model" proposes a two-stage method that separates pedestrian detection and re-identification using a mask-guided two-stream convolutional neural network, and employs mask fusion features to enhance the discriminativeness of pedestrian features. However, it still requires supervised training on the CHUK-SYSU and PRW image databases based on labeled pedestrian identity categories. The paper "Person Search in Videos with OnePortrait Through Visual and Temporal Links" (ECCV 2018) searches for specific individuals in long videos using visual and temporal links. However, this method constructs a video dataset based on movie clips and uses manually labeled trajectory segments for supervised training. Patent CN110826424A provides a "Person Search Method Based on Person Re-identification-Driven Localization Adjustment," which constructs a person re-identification-driven localization adjustment model to supervise the output detection boxes of the person detection network based on the loss of the person re-identification network. However, this method still falls under the category of supervised person search methods.

[0005] A review of literature on pedestrian re-identification and pedestrian search reveals that while unsupervised pedestrian re-identification methods have made some progress, these methods primarily employ DBSCAN clustering to generate pseudo-labels based on pedestrian features. They then use cross-entropy loss, label smoothing, or non-parametric Softmax loss to train the pedestrian re-identification model. DBSCAN clustering partitions the feature space based on density, classifying categories through neighborhood radius and density threshold. However, it is highly sensitive to the selection of these two parameters, as different parameters significantly impact the confidence of the pseudo-labels. Furthermore, this method does not address the spatiotemporal limitations of pedestrian features, making it difficult to apply to the uneven density pedestrian feature spaces in real-world surveillance scenarios. Additionally, cross-entropy loss, label smoothing, or non-parametric Softmax loss functions struggle to address the consistency of pedestrian features across multiple surveillance camera scenarios. In contrast to unsupervised pedestrian re-identification methods, most current pedestrian search methods still utilize supervised learning, training pedestrian search models on image databases like CHUK-SYSU and PRW, which already label pedestrian identity categories. However, manually labeling pedestrians' identities in massive amounts of raw surveillance video is not feasible, and the total number of pedestrian categories is unknown, which to some extent hinders the application of supervised pedestrian search methods in actual surveillance scenarios. Summary of the Invention

[0006] This invention addresses the aforementioned shortcomings of existing technologies by providing an unsupervised pedestrian search method for raw surveillance videos. To address the problem of unknown and difficult-to-label pedestrian identities in raw surveillance videos, after extracting pedestrian features from the video, an unsupervised clustering method based on graph segmentation is used to determine pedestrian identities and generate corresponding pseudo-labels. Furthermore, to address the issue of consistent pedestrian features across multiple surveillance camera scenarios, a multi-branch contrastive learning approach is employed. This approach involves three branches: category discriminative learning, single-camera contrastive learning, and multi-camera contrastive learning. While distinguishing different pedestrian features in single-camera scenarios, it also enhances the similarity of the same pedestrian features in multi-camera scenarios, thereby improving the accuracy of pedestrian search. This invention is achieved through the following technical solutions.

[0007] An unsupervised pedestrian search method for raw surveillance video, characterized by comprising the following steps:

[0008] A pre-trained neural network is used to detect pedestrian regions in raw surveillance video, and a feature encoder is used to generate pedestrian features.

[0009] The problem of pedestrian feature clustering with an unknown total number of categories is treated as a graph segmentation problem. Each node represents a pedestrian feature, the connection between nodes represents the similarity between pedestrian features, and each segmented region represents a cluster category. In the original surveillance video, multiple pedestrians appearing in a single camera at the same time belong to different identity categories, but the same pedestrian may appear in different cameras at different times. Based on this, a weighted total similarity is calculated based on the visual similarity and contextual similarity of pedestrian features. The first nearest neighbor relationship of pedestrian features is found and the chain structure of pedestrian features is organized to determine the spatial distribution of pedestrian features and perform unsupervised clustering based on the segmented regions. The identity category of the pedestrian is labeled according to the cluster category, and pseudo-labels of the pedestrians are generated based on the cluster centers.

[0010] During the forward propagation process, the contrast loss function of each of the three branches—class discrimination learning, single-camera contrast learning, and multi-camera contrast learning—is calculated, and the cluster center memory and multi-camera memory are updated according to the momentum update mechanism.

[0011] The contrastive loss functions of the three branches are jointly optimized, and the total loss value is minimized through multiple iterations during backpropagation, thereby optimizing the parameters of the feature encoder.

[0012] Furthermore, the aforementioned method of using a pre-trained neural network to detect pedestrian regions in the original surveillance video and using a feature encoder to generate pedestrian features refers to the following: the original surveillance video consists of multiple video segments and their corresponding camera labels. In each video segment, a pre-trained neural network is used to detect pedestrian bounding boxes and determine pedestrian regions; a feature encoder is used to generate corresponding pedestrian features for each pedestrian region.

[0013] Furthermore, the process of using a pre-trained neural network to detect pedestrian regions in the original surveillance video and generating pedestrian features using a feature encoder specifically includes the following steps:

[0014] 1) For the original surveillance video V={(v i ,s j )},v i (1≤i≤M) represents the i-th video segment, M is the number of video segments, and s j (1≤j≤N) is v i The corresponding j-th camera label represents the video segment v. i By surveillance camera j Acquisition; video clips v i The pedestrian region r was detected by processing through a pre-trained neural network. i The pedestrian areas in the original surveillance video V are represented as R = {r i ,s j)};

[0015] 2) For pedestrian areas r i Through the feature encoder f θ (·) Generate pedestrian features d i =f θ (r i The pedestrian features in the original surveillance video V are represented as D = {(d i ,s j )}={(f θ (r i ),s j )}.

[0016] Furthermore, the process involves calculating a weighted total similarity based on visual and contextual similarity of pedestrian features, finding the first nearest neighbor relationships of pedestrian features and organizing the chain structure of pedestrian features, determining the spatial distribution of pedestrian features, and performing unsupervised clustering based on the segmented regions; labeling pedestrian identity categories according to cluster categories, and generating pseudo-labels for pedestrians based on cluster centers. Specifically, this includes the following steps:

[0017] 1) Among all pedestrian features D, calculate the visual similarity P(m,n) = sim(d) between two pedestrian features. i,m ,d j,n ) = sim(f θ (r i,m ),f θ (r j,n )), where d i,m Represents the i-th video segment v i The m-th pedestrian feature in the data, d j,n v represents the j-th video segment. j Find the nth pedestrian feature in the dataset; calculate the context similarity of the above pedestrian features. Where N i It is a video clip v i The number of pedestrian features, N j It is a video clip v j The number of pedestrian features; the weighted total similarity between two pedestrian features is calculated as T(m,n)=λP(m,n)+(1-λ)Q(m,n), where λ is the weighting coefficient between the two similarities; the total similarity T(m,n) is sorted in descending order, and pedestrian feature d is found. i,m First neighbor relationship

[0018] 2) Based on the first nearest neighbor relationships of pedestrian features, construct a symmetric sparse matrix A(m,n,t)∈{0,1} for graph partitioning. At time t, the matrix... in This means that the first nearest neighbor of the m-th pedestrian feature is pedestrian feature n. Let the first nearest neighbor of the nth pedestrian feature be pedestrian feature m. This indicates that the first nearest neighbor of the m-th pedestrian feature and the n-th pedestrian feature is the same, t m and t n Let A(m,n,t) represent the times when the m-th and n-th pedestrian features appear, respectively. If A(m,n,t) = 1, it indicates that there is a connection between the m-th and n-th pedestrian features. Based on the chain structure between pedestrian features, construct the connected components between pedestrian features. Determine the segmentation regions of pedestrian features through these connected components, and each segmentation region represents a cluster category.

[0019] 3) After determining the cluster categories among all pedestrian features D, the pedestrian's identity category is labeled by the cluster category. The cluster center is represented as C = {c i}(1≤i≤K), generate pseudo-labels for pedestrians based on cluster centers.

[0020] Furthermore, the aforementioned calculation of the contrast loss function for each of the three branches—category discriminative learning, single-camera contrast learning, and multi-camera contrast learning—during the forward propagation process, and updating the cluster center memory and multi-camera memory according to the momentum update mechanism, refers to the following: In the category discriminative learning branch, a category contrast loss function is calculated to ensure the discriminability between different pedestrian features; in the single-camera contrast learning branch, a single-view contrast loss function is calculated to distinguish the discriminability of different pedestrian features under the same viewpoint; in the multi-camera contrast learning branch, a multi-view contrast loss function is calculated to improve the similarity of the same pedestrian features under different viewpoints; after calculating the loss functions of the above three branches, the cluster center memory and multi-camera memory are updated according to the momentum update mechanism.

[0021] Furthermore, the steps described in the forward propagation process, which involve calculating the contrastive loss function for each of the three branches—category discriminative learning, single-camera contrastive learning, and multi-camera contrastive learning—and updating the cluster center memory and multi-camera memory according to the momentum update mechanism, include:

[0022] 1) In the category discrimination learning branch, calculate the category contrast loss function. Where q represents the pedestrian features in the training set, c + This indicates a query for the cluster center corresponding to pedestrian feature q. Let τ represent the i-th cluster center in the t-th iteration. c Represents the temperature coefficient; updates the cluster center memory M according to the momentum update mechanism. c And calculated as Where q hardIndicates the training set and its cluster centers The pedestrian feature with the lowest visual similarity, λ c Represents the momentum update coefficients of the cluster center memory;

[0023] 2) In the single-camera contrast learning branch, calculate the single-view contrast loss function. Where p + Represents q′ hard The corresponding cluster centers, q′ hard This indicates that in the single-camera training set, p + The pedestrian features with the lowest visual similarity. K represents the unrelated cluster centers that are similar to the pedestrian feature q in the query. - τ represents the number of unrelated cluster centers. c Indicates the temperature coefficient;

[0024] 3) In the multi-camera contrast learning branch, calculate the multi-view contrast loss function. in This indicates that the camera has a label s in the t-th iteration. j The i-th cluster center, q easy The corresponding cluster centers with different camera labels, q easy Indicates that in the multi-camera training set and The pedestrian features with the highest visual similarity, τ c Indicates temperature coefficient; Multi-camera memory bank M p It consists of N single-camera memory banks, with the j-th surveillance camera corresponding to a single-camera memory bank M. p,j Update the single-camera memory database M according to the momentum update mechanism. p,j And calculated as Where λ p This represents the momentum update coefficient of a single camera's memory bank.

[0025] Furthermore, the aforementioned joint optimization of the contrast loss functions of the three branches, minimizing the total loss value through multiple iterations during backpropagation to optimize the parameters of the feature encoder, refers to summing the contrast loss functions of the three branches of class discriminative learning, single-camera contrast learning, and multi-camera contrast learning, and solving for the minimum value of the total loss function through multiple iterations during backpropagation, thereby achieving the goal of optimizing the parameters of the feature encoder.

[0026] Furthermore, the step of jointly optimizing the contrastive loss functions of the three branches and minimizing the total loss value through multiple iterations during backpropagation to optimize the parameters of the feature encoder includes:

[0027] 1) For the class contrast loss function L cluster Single-view contrast loss function L Intra and multi-view contrast loss function L Inter The total loss function L is calculated as L = L cluster +L Intra +L Inter ;

[0028] 2) During backpropagation, after t iterations of training, the minimum value of the total loss function L is found to optimize the feature encoder f. θ The parameter θ in (·) is used to output pedestrian features with good discriminative power.

[0029] The beneficial effects of this invention are as follows: First, this invention employs graph segmentation to perform unsupervised clustering on all pedestrian features extracted from the original surveillance video. The clustering categories determine the pedestrian's identity category and generate pseudo-labels, solving the problem of unknown and difficult-to-label pedestrian categories in the original surveillance video. Next, it integrates unsupervised clustering and multi-branch contrastive learning. While using class discriminative learning to ensure the distinguishability of different pedestrian characteristics, single-camera contrastive learning enhances the distinguishability of different pedestrian features from the same viewpoint. Simultaneously, multi-camera contrastive learning improves the similarity of the same pedestrian features from different viewpoints. Finally, the feature encoder is trained and optimized based on the loss function in multi-branch contrastive learning, outputting highly discriminative pedestrian features. Compared with existing technologies, this invention can be directly applied to the original surveillance video, training a pedestrian search model without the need for manual labeling of pedestrian identity categories, ensuring the accuracy of pedestrian search. Attached Figure Description

[0030] Figure 1 This is a flowchart of the present invention.

[0031] Figure 2 This is a flowchart illustrating how an unsupervised clustering method based on graph segmentation determines pedestrian categories.

[0032] Figure 3 The flowchart for training the category discrimination learning branch.

[0033] Figure 4 This is a flowchart illustrating the training of the single-camera contrast learning branch and the multi-camera contrast learning branch.

[0034] Figure 5 This study compares the search accuracy of graph segmentation-based unsupervised clustering methods with that of the DBSCAN clustering method.

[0035] Figure 6 This paper compares the search accuracy of the method of this invention with that of collaborative comparison extraction method and context-guided feature learning method. Detailed Implementation

[0036] The present invention will be further described in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.

[0037] This embodiment employs an unsupervised pedestrian search method based on raw surveillance video. The specific implementation steps are as follows:

[0038] 1. First, a pre-trained YOLO neural network is used to detect pedestrian areas in the original surveillance video, and a feature encoder is used to generate pedestrian features.

[0039] (1) In the original surveillance video V={(v i ,s j In the first step, a pre-trained YOLO neural network is used in the i-th video segment v. i Pedestrian detection area r i At the same time, the corresponding camera label s is marked. j After processing the M video segments, all pedestrian areas in the original surveillance video V can be represented as R = {(r i ,s j )};

[0040] (2) Residual neural network ResNet50 is used as feature encoder f θ (·), for pedestrian area r i Generate pedestrian features d i =f θ (r i Therefore, all pedestrian features can be represented as D = {(d i ,s j )}={(f θ (r i ),s j )}.

[0041] 2. Then, an unsupervised clustering method based on graph segmentation is used to cluster all pedestrian features, and the pedestrian identity category is determined according to the cluster category and the corresponding pseudo-label is generated;

[0042] (1) Among all pedestrian features D, first calculate the visual similarity P(m,n) = sim(d) between two pedestrian features. i,m ,d j,n ) = sim(f θ (r i,m ),f θ (r j,nNext, the contextual similarity of the two pedestrian features is calculated based on visual similarity. Finally, the weighted total similarity between the two pedestrian features is calculated as T(m,n) = λP(m,n) + (1-λ)Q(m,n), where the weighting coefficient λ = 0.7. The total similarity scores T(m,n) are sorted from largest to smallest to find the pedestrian feature d. i,m First neighbor relationship

[0043] (2) Based on pedestrian characteristics d i,m First neighbor relationship Construct a symmetric sparse matrix A(m,n,t)∈{0,1} to perform graph partitioning on the feature space of pedestrian feature D. If A(m,n,t)=1, it indicates that there is a connection between the m-th and n-th pedestrian features. Based on the chain structure between pedestrian features, construct connected components between pedestrian features, thus forming segmented regions of pedestrian features. Each segmented region represents a cluster category. By counting the number of cluster categories, the total number of pedestrian categories in the original surveillance video can be determined, based on the cluster center C={c i Generate pseudo-labels for pedestrians (1≤i≤K).

[0044] 3. Then, during the forward propagation process, the contrast loss function of each of the three branches—class discrimination learning, single-camera contrast learning, and multi-camera contrast learning—is calculated, and the cluster center memory and multi-camera memory are updated according to the momentum update mechanism.

[0045] (1) In the category discrimination learning branch, calculate the category contrast loss function for the query pedestrian features q and cluster centers C in the training set. Where the temperature coefficient τ c =0.05. Correspondingly, the cluster center memory M c Initialization is performed using cluster centers C. At the t-th iteration, through... The update is performed, where the momentum update coefficient λ of the cluster center memory is... c =0.1;

[0046] (2) In the single-camera contrast learning branch, the single-camera training set and cluster centers are extracted based on the camera labels, and the single-view contrast loss function is calculated. In the multi-camera contrast learning branch, the multi-view contrast loss function is calculated based on the multi-camera training set and the cluster centers corresponding to each individual camera. Where the temperature coefficient τ c =0.05. Multi-camera memory bank M p It contains N single-camera memory libraries, and the j-th surveillance camera corresponds to a single-camera memory library M. p,jAfter initialization using the corresponding cluster centers, in the t-th iteration, through... The update is performed, where the momentum update coefficient λ of the single-camera memory library is updated. p =0.1.

[0047] 4. Finally, the contrastive loss functions of the three branches are jointly optimized. During backpropagation, the total loss value is minimized through multiple iterations, thereby optimizing the parameters of the feature encoder.

[0048] (1) In the category contrast loss function L cluster Single-view contrast loss function L Intra and multi-view contrast loss function L Inter Based on this, calculate the total loss function L = L cluster +L Intra +L Inter ;

[0049] (2) During the backpropagation process, after t iterations, the parameters in the ResNet50 neural network are optimized by minimizing the total loss function L, so that pedestrian features with good discriminativeness can be output, ensuring the accuracy of pedestrian search.

[0050] The simulation experiment of the method of the present invention is as follows:

[0051] This experiment constructed a video surveillance system to collect raw surveillance video. The system consisted of nine surveillance cameras, each capturing video from different scenarios. Two hundred thousand raw video clips were selected; 15,000 clips were used as the training dataset, and the remaining 5,000 clips were used as the test dataset to test the pedestrian search performance. Mean Average Precision (mAP) was used as the performance metric for pedestrian search.

[0052] exist Figure 5In this paper, the unsupervised clustering method based on graph segmentation proposed in this invention is compared with the DBSCAN method in terms of search accuracy. With varying the number of original surveillance video segments in the training dataset, the search accuracy of the graph segmentation-based unsupervised clustering method is higher than that of the DBSCAN method. This is because the DBSCAN clustering algorithm partitions the feature space based on density, relying on the neighborhood radius and the minimum number of points within the neighborhood to define the density threshold. The choice of these two parameters directly affects the clustering effect and the confidence of pedestrian pseudo-labels, thus impacting the subsequent pedestrian search accuracy. The graph segmentation-based unsupervised clustering method partitions the feature space through a hierarchical structure and merges or splits cluster categories by finding nearest neighbors. Compared to the DBSCAN method, it has less dependence on parameter selection and introduces constraints based on the spatiotemporal features of pedestrians, thereby generating pedestrian pseudo-labels with higher confidence and improving the accuracy of pedestrian search.

[0053] exist Figure 6 In this study, with varying the number of original surveillance video segments in the training dataset, our proposed method was compared with the collaborative contrastive refining method proposed in "Collaborative Contrastive Refining for Weakly Supervised Person Search" (IEEE Transactions on Image Processing, 2023) and the context-guided feature learning method proposed in "Exploring Visual Context for Weakly Supervised Person Search" (AAAI 2022). The results show that our proposed method has the highest mean average accuracy, while the context-guided feature learning method has the lowest. This is because the context-guided feature learning method focuses on pedestrian context detection, memory context, and scene context, but does not consider the spatiotemporal limitations of the original surveillance video or the correlation between surveillance cameras. In contrast, the collaborative contrastive refining method improves the discriminative power of pedestrian features, but it treats individual pedestrians as noise samples, which to some extent reduces the similarity of pedestrian features in multi-camera scenarios. This method not only considers the distinguishability of different pedestrian features in single-camera scenarios, but also involves the similarity of the same pedestrian features in multi-camera scenarios, thereby enhancing the expressive power of pedestrian features and improving the accuracy of pedestrian search.

Claims

1. An unsupervised pedestrian search method for raw surveillance video, characterized in that, Includes the following steps: A pre-trained neural network is used to detect pedestrian regions in raw surveillance video, and a feature encoder is used to generate pedestrian features. The problem of pedestrian feature clustering with an unknown total number of categories can be viewed as a graph segmentation problem. Each node represents a pedestrian feature, the connection between nodes represents the similarity between pedestrian features, and each segmentation region represents a cluster category. In the original surveillance video, multiple pedestrians appearing in a single camera at the same time belong to different identity categories. However, the same pedestrian may appear in different cameras at different times. Based on this, a weighted total similarity is calculated based on the visual similarity and contextual similarity of pedestrian features. The first nearest neighbor relationship of pedestrian features is found and the chain structure of pedestrian features is organized. The spatial distribution of pedestrian features is determined and unsupervised clustering is achieved according to the segmented regions. Pedestrian identity categories are labeled according to cluster categories, and pseudo-labels for pedestrians are generated based on cluster centers. The method for finding the first nearest neighbor relationships of pedestrian features and organizing the chain structure of pedestrian features is as follows: a symmetric sparse matrix is ​​constructed based on the first nearest neighbor relationships of pedestrian features. Perform graph segmentation, in t Time, matrix ,in Indicates the first m The first nearest neighbor of a pedestrian feature is the pedestrian characteristic itself. n , Indicates the first n The first nearest neighbor of a pedestrian feature is the pedestrian characteristic itself. m , Indicates the first m Pedestrian characteristics and the first n The first nearest neighbor of each pedestrian's feature is the same. and They represent the first m Pedestrian characteristics and the first n The moment when a pedestrian's characteristics appear; if , indicating the first m Pedestrian characteristics and the first n There are connections between pedestrian features; based on the chain structure between pedestrian features, connected components between pedestrian features are constructed; During the forward propagation process, the contrastive loss function is calculated for each of the three branches: category discrimination learning, single-camera contrastive learning, and multi-camera contrastive learning. The cluster center memory and multi-camera memory are then updated according to the momentum update mechanism, including: 1) In the category discrimination learning branch, calculate the category contrast loss function. ,in q This represents the characteristics of the queried pedestrians in the training set. Indicates querying pedestrian characteristics q The corresponding cluster centers, Indicates the first t In the nth iteration i Cluster centers, Represents the temperature coefficient; updates the cluster center memory based on the momentum update mechanism. And calculated as ,in Indicates the training set and its cluster centers The pedestrian feature with the lowest visual similarity. Represents the momentum update coefficients of the cluster center memory; 2) In the single-camera contrast learning branch, calculate the single-view contrast loss function. ,in express The corresponding cluster centers, Indicates that in the single-camera training set and The pedestrian features with the lowest visual similarity. Representing and querying pedestrian characteristics q Similar but unrelated cluster centers This indicates the number of unrelated cluster centers. Indicates the temperature coefficient; 3) In the multi-camera contrast learning branch, calculate the multi-view contrast loss function. ,in Indicates the first t The next iteration has a camera tag. The i Cluster centers, express Corresponding cluster centers with different camera labels Indicates that in the multi-camera training set and The pedestrian features with the highest visual similarity. Indicates temperature coefficient; multi-camera memory library M p Depend on N It consists of a single camera memory bank, the first j Each surveillance camera corresponds to a single camera memory library. Update the single-camera memory bank according to the momentum update mechanism. And calculated as ,in This represents the momentum update coefficient of a single camera's memory bank; The contrastive loss functions of the three branches are jointly optimized, and the total loss value is minimized through multiple iterations during backpropagation, thereby optimizing the parameters of the feature encoder.

2. The unsupervised pedestrian search method for original surveillance video according to claim 1, characterized in that, The aforementioned method of using a pre-trained neural network to detect pedestrian regions in the original surveillance video and using a feature encoder to generate pedestrian features refers to the following: the original surveillance video consists of multiple video segments and their corresponding camera labels. In each video segment, a pre-trained neural network is used to detect pedestrian bounding boxes and determine the pedestrian region. A feature encoder is used to generate corresponding pedestrian features for each pedestrian area.

3. The unsupervised pedestrian search method for original surveillance video according to claim 2, characterized in that, The method of using a pre-trained neural network to detect pedestrian regions in raw surveillance video and generating pedestrian features using a feature encoder specifically includes the following steps: 1) For the original surveillance video , For the first i A video clip, M The number of video clips, for The corresponding number j Each camera tag indicates a video clip. Surveillance cameras Acquisition; video clips The pedestrian area was detected by processing the data through a pre-trained neural network. Original surveillance video V All pedestrian areas in the middle are represented as ; 2) Targeting pedestrian areas The residual neural network ResNet50 is used as the feature encoder. Through the feature encoder Generate pedestrian features Original surveillance video V All pedestrian features are represented as .

4. The unsupervised pedestrian search method for original surveillance video according to claim 3, characterized in that, The method described above, based on visual and contextual similarity of pedestrian features, calculates a weighted total similarity, finds the first nearest neighbor relationships of pedestrian features and organizes the chain structure of pedestrian features, determines the spatial distribution of pedestrian features and performs unsupervised clustering based on segmented regions; labels the pedestrian identity category according to the cluster category, and generates pseudo-labels for pedestrians based on the cluster centers. Specifically, it includes the following steps: 1) In all pedestrian characteristics D In this context, the visual similarity of features between two pedestrians is calculated. ,in Indicates the first i Video clip The first in m Pedestrian characteristics Indicates the first j Video clip The first in n Pedestrian characteristics; Calculate the contextual similarity of the above pedestrian features. ,in It's a video clip. The number of pedestrian characteristics, It's a video clip. The number of pedestrian features; the weighted total similarity of two pedestrian features is calculated as follows: ,in It is a trade-off coefficient between two similarities; the total similarity is... Sort by size from largest to smallest to find traveler characteristics. First neighbor relationship ; 2) By using the connected components between pedestrian features, the segmentation regions of pedestrian features are determined, and each segmentation region represents a cluster category; 3) In all pedestrian characteristics D After determining the cluster categories, the pedestrian's identity category is labeled according to the cluster category, and the cluster center is represented as... Pseudo-labels for pedestrians are generated based on cluster centers.

5. The unsupervised pedestrian search method for original surveillance video according to claim 1, characterized in that, The aforementioned calculation of the contrast loss function for each of the three branches—class discriminative learning, single-camera contrast learning, and multi-camera contrast learning—during the forward propagation process, and updating the cluster center memory and multi-camera memory according to the momentum update mechanism, refers to: in the class discriminative learning branch, calculating the class contrast loss function to ensure the discriminability between different pedestrian features; In the single-camera contrast learning branch, the single-view contrast loss function is calculated to distinguish the features of different pedestrians under the same viewpoint. In the multi-camera contrast learning branch, a multi-view contrast loss function is calculated to improve the similarity of the same pedestrian features under different views. After calculating the loss functions of the above three branches, the cluster center memory and the multi-camera memory are updated according to the momentum update mechanism.

6. The unsupervised pedestrian search method for original surveillance video according to claim 1, characterized in that, The aforementioned method of jointly optimizing the contrast loss functions of the three branches and minimizing the total loss value through multiple iterations during backpropagation to optimize the parameters of the feature encoder refers to summing the contrast loss functions of the three branches of class discriminative learning, single-camera contrast learning, and multi-camera contrast learning, and solving for the minimum value of the total loss function through multiple iterations during backpropagation, thereby achieving the goal of optimizing the parameters of the feature encoder.

7. The unsupervised pedestrian search method for original surveillance video according to claim 1, characterized in that, The steps described above for jointly optimizing the contrastive loss functions of the three branches and minimizing the total loss value through multiple iterations during backpropagation to optimize the parameters of the feature encoder include: 1) For the class contrast loss function Single-view contrast loss function Comparison loss function with multiple perspectives Total loss function L Calculated as ; 2) During the reverse propagation process, through t Solve the total loss function after the second iteration of training. L The minimum value is obtained by using a residual neural network ResNet50 as the feature encoder. Optimize the feature encoder Parameters in This allows for the output of pedestrian features with good discriminative power.

Images