Method and device for cleaning and denoising large-scale pedestrian re-identification collection data

Abstract

The application provides a large-scale pedestrian re-identification data collection cleaning and denoising method and device, and relates to the technical field of data processing. The method comprises the following steps: obtaining an initial pedestrian re-identification data set; the initial pedestrian re-identification data set comprises a plurality of identity archives; performing preliminary screening processing based on the initial pedestrian re-identification data set to determine target clustering instances of each identity archive; and determining a target pedestrian re-identification data set from the target clustering instances of all identity archives based on a clustering instance selection strategy; wherein the clustering instance selection strategy is determined based on a distance matrix, color attribute information and perspective ratio information. Through the fusion of pedestrian instance distance analysis based on the distance of the clustering center, color attribute distribution and perspective ratio information, the application realizes multi-dimensional collaborative cleaning of large-scale noisy pedestrian data and high-quality sample screening.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for cleaning and denoising large-scale pedestrian re-identification data. Background Technology

[0002] Pedestrian re-identification technology, as a key technology for tracking pedestrians across cameras, has significant application value in fields such as intelligent security and smart retail. The performance of this technology highly depends on large-scale, high-quality labeled training data. Currently, most mainstream methods train and optimize models on a small number of publicly available, cleanly labeled academic datasets. However, the scale and scenario diversity of these datasets differ greatly from the hundreds of millions of data points collected in complex real-world security scenarios. This results in models trained on academic data generally facing severe limitations in domain generalization when directly applied to real-world scenarios, leading to a significant decline in recognition performance.

[0003] In real-world scenarios, massive amounts of pedestrian data, initially aggregated through automated methods (such as trajectory tracking and spatiotemporal correlation), are typically organized into "identity profiles" containing images of several pedestrians. However, due to environmental complexity and errors in the automated aggregation process, these profiles contain significant noise. This noise manifests primarily as follows: the same identity being incorrectly split into multiple profiles due to drastic changes in appearance, such as clothing changes, or different identities being incorrectly merged into the same profile due to occlusion or misjudgment. If such large-scale, noisy data is used directly for model training without effective cleaning, it will severely interfere with model learning, leading to difficulty in convergence or performance degradation. Current technologies lack automated cleaning solutions capable of systematically handling such massive datasets with complex noise patterns.

[0004] Therefore, how to automatically clean large-scale noisy pedestrian data and transform it into a reliable data source that can be used to train high-quality re-identification models has become an urgent technical problem to be solved. Summary of the Invention

[0005] This invention provides a method and apparatus for cleaning and denoising large-scale pedestrian re-identification data. By integrating pedestrian instance distance analysis based on cluster center distance, color attribute distribution, and viewpoint ratio information, it achieves multi-dimensional collaborative cleaning and high-quality sample screening of large-scale noisy pedestrian data.

[0006] In a first aspect, the present invention provides a method for cleaning and denoising large-scale pedestrian re-identification data, comprising the following steps: Obtain an initial pedestrian re-identification dataset; the initial pedestrian re-identification dataset includes multiple identity profiles; Based on the initial pedestrian re-identification dataset, a preliminary screening process is performed to determine the target cluster instance for each of the identity files; Based on the clustering instance selection strategy, the target pedestrian re-identification dataset is determined from the target clustering instances of all identity files; The cluster instance selection strategy is determined based on the distance matrix, color attribute information, and viewing angle ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information from the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information of the target cluster instance.

[0007] Preferably, according to the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the step of performing preliminary screening processing based on the initial pedestrian re-identification dataset to determine the target cluster instance for each identity file includes: Extract the embedding vector of each pedestrian instance from each identity profile; In each identity profile, the embedding vector of the pedestrian instance is subjected to hierarchical density space clustering to filter out noisy pedestrian instances and obtain multiple clustered instances. Evaluation metrics are calculated for each cluster instance to determine the target cluster instance for each identity file.

[0008] Preferably, according to the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the step of calculating evaluation indicators based on each cluster instance to determine the target cluster instance for each identity file includes: Calculate the average similarity between pedestrian instances within each cluster instance and the cluster center vectors of the cluster instances; The average similarity of each cluster instance is normalized to obtain the aggregation degree evaluation index of each cluster instance; Based on the aggregation degree evaluation index of each cluster instance and the preset index threshold, the target cluster instance of each identity file is determined.

[0009] Preferably, in the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the step of determining the cluster center vector of the clustered instances includes: The embedding vector of each pedestrian instance in the cluster is summed to obtain the total embedding vector; The total embedding vector and the number of pedestrian instances are calculated to obtain the cluster center vector of the clustered instances.

[0010] Preferably, according to the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, determining the target cluster instance for each identity file based on the aggregation degree evaluation index of each cluster instance and a preset index threshold includes: The aggregation degree evaluation index of each cluster instance is compared with the index threshold. Cluster instances whose aggregation degree evaluation index is greater than or equal to the index threshold are designated as target cluster instances, and cluster instances whose aggregation degree evaluation index is less than the index threshold are designated as noisy cluster instances. In each identity profile, the noisy cluster instances are filtered out to obtain the target cluster instances.

[0011] Preferably, in the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the distance matrix is ​​calculated based on the center vector of the target cluster instance, including: In each of the identity profiles, calculate the center vector distance between any two of the target cluster instances; The distance matrix is ​​constructed based on the center vector distances between all target cluster instances.

[0012] Preferably, in the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the color attribute information is determined by extracting color information based on target clustering instances, including: The color of the upper and lower body clothing of the target pedestrian instance in each target cluster instance is extracted using a pedestrian clothing color recognition model and used as the color attribute information; The target pedestrian instance is the pedestrian instance that is closest to the cluster center vector of the target cluster instance.

[0013] Preferably, in the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the viewpoint ratio information is determined statistically based on viewpoint ratio information from target clustering instances, including: The perspective information of each pedestrian instance in the target cluster is identified using a perspective attribute recognition model; the perspective information includes at least a frontal perspective, a side perspective, and a rear perspective. The perspective ratio information is calculated by statistically analyzing the perspective information corresponding to each perspective in each target cluster instance.

[0014] Preferably, in the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the clustering instance selection strategy includes: In each of the identity files, the target cluster instances are sorted using sorting rules to obtain sorted target cluster instances; A backtracking search and upper bound pruning algorithm is used to process the sorted target cluster instances and determine the largest subset of cluster instances. Initialize an empty training set, iterate through the largest cluster instance subset in each identity file, and store the largest cluster instance subset that meets the conditions into the empty training set to obtain the target pedestrian re-identification dataset.

[0015] Preferably, in the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the sorting rule is to sort according to the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance; The sorting priority of the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance decreases in that order.

[0016] Preferably, according to the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, the sum of the side view ratio and the back view ratio of the target cluster instance is calculated. The smaller of the two perspective ratios, the frontal perspective ratio and the total perspective ratio, is determined as the target perspective ratio information.

[0017] Secondly, the present invention also provides a device for cleaning and denoising large-scale pedestrian re-identification data, comprising the following modules: The acquisition module is used to acquire an initial pedestrian re-identification dataset; the initial pedestrian re-identification dataset includes multiple identity files; The filtering module is used to perform preliminary filtering based on the initial pedestrian re-identification dataset to determine the target cluster instance for each identity file; The determination module is used to determine the target pedestrian re-identification dataset from the target cluster instances of all identity files based on a cluster instance selection strategy. The cluster instance selection strategy is determined based on the distance matrix, color attribute information, and viewing angle ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information from the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information of the target cluster instance.

[0018] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cleaning and denoising method for large-scale pedestrian re-identification data as described above.

[0019] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cleaning and denoising method for large-scale pedestrian re-identification data collection as described above.

[0020] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the cleaning and denoising method for large-scale pedestrian re-identification data as described above.

[0021] This invention provides a method and apparatus for cleaning and denoising large-scale pedestrian re-identification data. The method involves acquiring an initial pedestrian re-identification dataset, which includes multiple identity profiles. Preliminary screening is performed on the initial dataset to determine target cluster instances for each identity profile. A target pedestrian re-identification dataset is then selected from the target cluster instances of all identity profiles based on a cluster instance selection strategy. This strategy is determined based on a distance matrix, color attribute information, and perspective ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instances. The color attribute information is extracted from the target cluster instances, and the perspective ratio information is statistically determined based on the perspective ratio of the target cluster instances. By integrating pedestrian instance distance analysis based on cluster center distance, color attribute distribution, and perspective ratio information, multi-dimensional collaborative cleaning and high-quality sample selection of large-scale noisy pedestrian data are achieved. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is one of the flowcharts illustrating the method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the noise identity profile captured by the present invention.

[0025] Figure 3 This is one of the schematic diagrams of pedestrian clustering examples after hierarchical density spatial clustering provided by the present invention.

[0026] Figure 4 This is the second schematic diagram of a pedestrian clustering example after hierarchical density spatial clustering provided by the present invention.

[0027] Figure 5 This is a schematic diagram of the structure of the cleaning and noise reduction device for large-scale pedestrian re-identification data provided by the present invention.

[0028] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] All actions involving the acquisition of signal information or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the device is located, and with the authorization granted by the owner of the device.

[0031] The following is combined with Figures 1-6 This invention describes a method and apparatus for cleaning and denoising large-scale pedestrian re-identification data. By integrating pedestrian instance distance analysis based on cluster center distance, color attribute distribution, and viewpoint ratio information, it achieves multi-dimensional collaborative cleaning and high-quality sample screening of large-scale noisy pedestrian data.

[0032] Figure 1 This is one of the flowcharts illustrating a method for cleaning and denoising large-scale pedestrian re-identification data provided by the present invention, such as... Figure 1 As shown, the method may include, but is not limited to, steps S100 to S300: S100, Obtain the initial pedestrian re-identification dataset; the initial pedestrian re-identification dataset includes multiple identity files; S200, based on the initial pedestrian re-identification dataset, a preliminary screening process is performed to determine the target cluster instance for each of the identity files; S300, based on a clustering instance selection strategy, determines the target pedestrian re-identification dataset from the target clustering instances of all identity files; The cluster instance selection strategy is determined based on the distance matrix, color attribute information, and viewing angle ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information from the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information of the target cluster instance.

[0033] The cleaning and denoising method for large-scale pedestrian re-identification data provided in this invention mimics the human process of processing massive amounts of chaotic information: first, preliminary classification and removal of obvious errors (clustering denoising); then, evaluation of the internal quality and external characteristics of each type of information (quality assessment and attribute analysis); and finally, selection of the most valuable and comprehensive subset of information based on multivariate rules (diversity optimization). This method is fully automated and suitable for processing tens of millions or even hundreds of millions of image data. The cleaning and denoising method for large-scale pedestrian re-identification data provided in this invention is suitable for cleaning large-scale datasets, not small datasets.

[0034] In step S100 of some embodiments, an initial pedestrian re-identification dataset is obtained; the initial pedestrian re-identification dataset includes multiple identity profiles.

[0035] Understandably, the initial pedestrian re-identification dataset can come from actual security monitoring systems. Through preliminary technical means such as trajectory analysis and face comparison, massive amounts of pedestrian images are aggregated into hundreds of thousands to millions of "identity profiles". Each identity profile may contain several to hundreds of pedestrian images (i.e., "pedestrian instances"), but these images are not guaranteed to belong to the same person, and are mixed with the aforementioned noise.

[0036] In step S200 of some embodiments, a preliminary screening process is performed based on the initial pedestrian re-identification dataset to determine the target cluster instance for each identity file.

[0037] Understandably, this step involves internal "refinement" of each rough identity profile. Since a single profile may contain different clothing styles of the same person or include different individuals, a more refined unsupervised segmentation is required. Specifically, using a clustering algorithm described later, each profile is split into multiple subsets with highly consistent internal visual features, called "cluster instances," and outliers (noise) that clearly do not belong to any subset are removed. Then, through quality evaluation, reliable cluster instances are selected as "target cluster instances."

[0038] Combination Figure 2 The image shows a pedestrian instance diagram for each identity file before denoising. It includes pedestrian instances from various angles and pedestrian instances of various colors. The number of noisy pedestrian instances is relatively large. If used for model training, it will result in poor model robustness, poor training accuracy, and poor generalization.

[0039] In step S300 of some embodiments, the target pedestrian re-identification dataset is determined from the target cluster instances of all identity files based on a cluster instance selection strategy. The cluster instance selection strategy is determined based on the distance matrix, color attribute information, and viewing angle ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information from the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information of the target cluster instance.

[0040] Understandably, this step is a global optimization process. The cluster instance selection strategy comprehensively considers the visual distinguishability between cluster instances (reflected by the distance matrix), clothing color distribution (color attribute information), and pedestrian posture distribution (viewpoint ratio information).

[0041] The distance matrix is ​​derived from the "center vectors" of the target cluster instances (i.e., the average representation of their features). It quantifies the visual differences between different instances within the same archive. The greater the difference, the more likely they represent different identities.

[0042] The color attribute information describes the typical clothing color of each target cluster instance.

[0043] The perspective ratio information describes the distribution of pedestrians from the front, side, and back angles in each target cluster instance.

[0044] By comprehensively utilizing these three factors, a clustering instance selection strategy was formulated. The goal was to select a set of images from millions of data points globally, characterized by a moderate number of images, high distinguishability of identities, and diverse and balanced distribution of colors and viewpoints—the final "target pedestrian re-identification dataset." This systematically solved the core challenge of automatically constructing a high-quality training set from hundreds of millions of noisy data points, providing a reliable data foundation for subsequent model training.

[0045] Combination Figure 3 and Figure 4 The image shows a target pedestrian re-identification dataset processed according to the embodiments provided by this invention, i.e., a clean identity profile. Each identity profile contains pedestrian instances of the same color or similar colors. Figure 3 and Figure 4 Using this target pedestrian re-identification dataset to train the model can significantly improve the model's generalization ability and robustness.

[0046] In some embodiments of the present invention, the preliminary screening process based on the initial pedestrian re-identification dataset to determine the target cluster instance for each identity file includes: Extract the embedding vector of each pedestrian instance from each identity profile; In each identity profile, the embedding vector of the pedestrian instance is subjected to hierarchical density space clustering to filter out noisy pedestrian instances and obtain multiple clustered instances. Evaluation metrics are calculated for each cluster instance to determine the target cluster instance for each identity file.

[0047] Understandably, a pre-trained "base pedestrian re-identification model" is used as the feature extractor. This base pedestrian re-identification model is preferably a deep convolutional neural network (such as IBN-Net or BAU) trained on public datasets of multiple different scenarios (i.e., multiple "domains") using "domain generalization" techniques. Domain generalization is a training strategy aimed at enabling the model to learn universal pedestrian feature representations that are unaffected by specific scene styles.

[0048] When a pedestrian image is input into the pre-trained "basic pedestrian re-identification model," a fixed-length numerical vector (e.g., 512-dimensional) is extracted from the network's end. This vector is called the "embedding vector." The essence of the embedding vector is to compress the high-dimensional pixel information of the image into a low-dimensional point containing semantic features. In the feature space, the embedding vectors of similar images are closer together.

[0049] Within each identity profile, the embedding vectors of pedestrian instances are subjected to hierarchical density space clustering. The HDBSCAN algorithm is used here.

[0050] HDBSCAN is an advanced hierarchical, density-based clustering algorithm. Its advantage lies in its ability to automatically discover clusters of arbitrary shapes without requiring a pre-specified number of clusters, and its robustness to noise (outliers), making it ideal for processing data with unknown internal structures.

[0051] The specific implementation steps include: 1) Calculating the Euclidean distance between all pairwise embedding vectors in the identity file; 2) Constructing a "cross-distance" graph based on the distances and generating a minimum spanning tree to represent the hierarchical relationships between data points; 3) Pruning the tree according to a preset "minimum cluster size" parameter (e.g., set to 4, meaning the minimum acceptable cluster must contain at least 4 samples) to form the final planar clustering result. Each cluster output by the algorithm is a "cluster instance," while points marked as noise are filtered out as "noise pedestrian instances."

[0052] Evaluation metrics are calculated for each cluster instance to determine the target cluster instance. Specifically, each cluster instance generated by HDBSCAN is scored, and low-scoring, noisy pedestrian instances are filtered out.

[0053] This embodiment combines highly generalizable feature extraction with robust density clustering. This step enables automated, fine-grained data cleaning and reorganization within each original noise profile, providing structured, higher-quality intermediate data units (clustering instances) for subsequent processing.

[0054] In some embodiments of the present invention, the step of determining the cluster center vector of the clustering instance includes: The embedding vector of each pedestrian instance in the cluster is summed to obtain the total embedding vector; The total embedding vector and the number of pedestrian instances are calculated to obtain the cluster center vector of the clustered instances.

[0055] Understandably, the first step is to calculate the cluster center vector. For a cluster containing k embedding vectors {v1, v2, ..., v...}, ... k The cluster center vector c of the cluster instances of} is calculated by the arithmetic mean: c = (v1 + v2 + ... + v k ) / k. The cluster center vector c can be regarded as the "average feature" of all samples in the cluster.

[0056] In some embodiments of the present invention, the step of calculating evaluation metrics based on each cluster instance to determine the target cluster instance for each identity file includes: Calculate the average similarity between pedestrian instances within each cluster instance and the cluster center vectors of the cluster instances; The average similarity of each cluster instance is normalized to obtain the aggregation degree evaluation index of each cluster instance; Based on the aggregation degree evaluation index of each cluster instance and the preset index threshold, the target cluster instance of each identity file is determined.

[0057] It is understandable that the sample vector v of each cluster instance is calculated. i The cosine similarity with the cluster center vector c. Cosine similarity is an index that measures the consistency of the directions of two vectors. The value ranges from [-1, 1]. The larger the value, the closer the directions are and the more similar the features are.

[0058] The calculation formula is: sim(v i c) = (v i · c) / (||v i || ||c||). Where, sim(v) i c) represents cosine similarity, v i denoted by , represents the embedding vector of pedestrian instances within each cluster instance, and c represents the cluster center vector of the corresponding cluster instance.

[0059] Finally, the arithmetic mean of the similarities of all clustered instance samples is calculated to obtain the average similarity between pedestrian instances within each cluster and the cluster center vector of the cluster instance.

[0060] Further, the average similarity is normalized to obtain an aggregation degree evaluation index. Since the cosine similarity is bounded by itself, this average value can be directly used as the index, or it can be mapped to the interval [0, 1] through linear scaling (e.g., (average similarity + 1) / 2) to obtain the aggregation degree evaluation index. This index intuitively reflects the "clustering" tightness of the samples within the clustering instance. The higher the index, the better the internal consistency and the higher the quality of the clustering.

[0061] This embodiment provides an objective and computable clustering quality evaluation criterion. Through the central vector and cosine similarity, the abstract concept of clustering quality is transformed into specific numerical values, making automated filtering possible and further improving the accuracy of data cleaning.

[0062] In some embodiments of the present invention, determining the target clustering instance of each identity profile based on the aggregation degree evaluation index of each clustering instance and a preset index threshold includes: Comparing the aggregation degree evaluation index of each clustering instance with the index threshold respectively; Regarding the clustering instance whose aggregation degree evaluation index is greater than or equal to the index threshold as the target clustering instance, and regarding the clustering instance whose aggregation degree evaluation index is less than the index threshold as the noise clustering instance; In each identity profile, screening out the noise clustering instances to obtain the target clustering instances.

[0063] It can be understood that an empirical index threshold T_q is set (for example, with a value of 0.65). Comparing the aggregation degree evaluation index score calculated for each clustering instance with T_q.

[0064] If score ≥ T_q, it is determined that the clustering instance is internally compact and has reliable quality, and it is marked as the target clustering instance; if score < T_q, it is determined that the clustering instance is loose, may still contain noise or has inconsistent features, and it is marked as the noise clustering instance.

[0065] In each identity profile, deleting the entire cluster marked as the noise clustering instance and all pedestrian images included therein, and only retaining the target clustering instances to enter the subsequent process.

[0066] This embodiment realizes the secondary purification of the clustering result. HDBSCAN may generate some clusters that meet the scale requirements but are not pure internally. This step eliminates them through a quantization threshold, ensuring that each retained target clustering instance has high internal quality and laying a solid foundation for constructing a high-quality final data set.

[0067] In some embodiments of the present invention, the distance matrix is ​​calculated based on the center vector of the target cluster instance, including: In each of the identity profiles, calculate the center vector distance between any two of the target cluster instances; The distance matrix is ​​constructed based on the center vector distances between all target cluster instances.

[0068] It is understandable that, after the above steps to remove noisy clustering instances twice, M target clustering instances are retained, and each target clustering instance has its corresponding cluster center vector c1, c2, ..., c_M.

[0069] Calculate any two cluster center vectors c i and c j The Euclidean distance between them. Euclidean distance, which is the straight-line distance in multidimensional space, is the most intuitive measure of difference.

[0070] The calculation formula is: Among them, dist(c i , c j Let ) represent any two center vectors c i and c j The Euclidean distance and the center vector distance between them. i and c j d represents the different cluster center vectors, where d is the dimension of the feature vector.

[0071] The larger the center vector distance value, the farther apart the two target cluster instances are in the feature space, and the greater their visual appearance difference.

[0072] Based on the pairwise center vector distances of all M target cluster instances, construct an M × M symmetric matrix, namely the distance matrix D, where the element in the i-th row and j-th column is D. ij = dist(c i , c j This distance matrix fully characterizes the differences between all reliable clustered instances within the same archive.

[0073] This embodiment encodes the visual differences between clustered instances in a global and structured manner. The distance matrix is ​​the core basis for subsequent judgment of "which instances can be regarded as different training identities", transforming subjective visual judgment into objective numerical comparison.

[0074] In some embodiments of the present invention, the color attribute information is determined by extracting color information based on target clustering instances, including: The color of the upper and lower body clothing of the target pedestrian instance in each target cluster instance is extracted using a pedestrian clothing color recognition model and used as the color attribute information; The target pedestrian instance is the pedestrian instance that is closest to the cluster center vector of the target cluster instance.

[0075] Understandably, a pedestrian clothing color recognition model is used to extract the colors of the upper and lower body clothing of target pedestrian instances within each target cluster instance. This pedestrian clothing color recognition model is a classification model trained on a deep convolutional neural network (such as ResNet-50). Its structure and function are as follows: the pedestrian clothing color recognition model is trained on a large amount of data labeled with the colors of the upper and lower body of pedestrians. It can receive a pedestrian image and predict the predefined color categories most likely to belong to the upper and lower body regions, respectively. The color categories are a finite set, such as {black, white, gray, red, blue, green, yellow, brown, purple, pink, orange, others}.

[0076] For each target cluster instance, a representative sample needs to be selected to characterize its color. The selection principle is: find the pedestrian instance in the cluster whose pedestrian instance embedding vector has the highest cosine similarity to the cluster center vector c. This pedestrian instance is called the target pedestrian instance because it is closest to the "central feature" of the cluster and best represents the general appearance of the cluster.

[0077] The target pedestrian instance is input into the color recognition model to obtain its upper body color label C_top and lower body color label C_bottom. This pair of color labels (C_top, C_bottom) is used as the color attribute information for the entire cluster instance. This is an efficient and reasonable approximation, using the most representative sample color to summarize the color tendency of the entire cluster.

[0078] This embodiment assigns an intuitive, semantic color label to each abstract cluster instance. This allows subsequent algorithms to "understand" the distribution of data along the color dimension, thereby implementing targeted balancing strategies to prevent severe color bias in the final dataset (e.g., too many "white on top, black on bottom" combinations) and improving the robustness of the trained model.

[0079] In some embodiments of the present invention, the viewpoint ratio information is determined by statistical analysis of viewpoint ratio information based on target cluster instances, including: The perspective information of each pedestrian instance in the target cluster is identified using a perspective attribute recognition model; the perspective information includes at least a frontal perspective, a side perspective, and a rear perspective. The perspective ratio information is calculated by statistically analyzing the perspective information corresponding to each perspective in each target cluster instance.

[0080] Understandably, a viewpoint attribute recognition model is used, which is also a CNN-based classification model. Its function is to determine the shooting angle of pedestrian images. It classifies the input image into a limited number of categories, typically including at least: frontal view (pedestrian facing the camera), side view (pedestrian sideways to the camera), and rear view (pedestrian with their back to the camera).

[0081] For a target cluster instance containing N pedestrian instances, the view attribute recognition model is used to identify all N instances, and the number of instances classified as front, side, and back is counted and denoted as n_front, n_side, and n_back.

[0082] Calculate the proportions of each viewpoint: p_front = n_front / N, p_side = n_side / N, p_back = n_back / N.

[0083] Where p_front represents the frontal view ratio, p_side represents the side view ratio, and p_back represents the rear view ratio.

[0084] The vector (p_front, p_side, p_back) composed of these three proportions represents the viewpoint proportion information of this cluster instance. It describes the distribution of pedestrians represented by this cluster under different shooting angles.

[0085] This embodiment quantifies the pose diversity of each identity data unit. Model training requires exposure to samples from different perspectives to learn comprehensive features. This information allows the algorithm to prioritize cluster instances that contain more non-frontal (more challenging and valuable) perspectives, thereby guiding the constructed dataset to better improve the model's robustness to pose changes.

[0086] In some embodiments of the present invention, the clustering instance selection strategy includes: In each of the identity files, the target cluster instances are sorted using sorting rules to obtain sorted target cluster instances; A backtracking search and upper bound pruning algorithm is used to process the sorted target cluster instances and determine the largest subset of cluster instances. Initialize an empty training set, iterate through the largest cluster instance subset in each identity file, and store the largest cluster instance subset that meets the conditions into the empty training set to obtain the target pedestrian re-identification dataset.

[0087] Understandably, within each identity profile, the instances of the target cluster are sorted using ranking rules. This sorting is to prioritize higher-quality instances with greater training value.

[0088] In some embodiments of the present invention, the sorting rule is to sort according to the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance; The sorting priority of the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance decreases in that order.

[0089] In some embodiments of the present invention, the sum of the side view ratio and the back view ratio of the target cluster instance is calculated; The smaller of the two perspective ratios, the frontal perspective ratio and the total perspective ratio, is determined as the target perspective ratio information.

[0090] Understandably, the first step is to calculate the target view ratio information P_target for each target cluster instance. P_target = min(p_front, p_side + p_back). Here, p_front represents the proportion of the frontal view, p_side represents the proportion of the side view, and p_back represents the proportion of the rear view.

[0091] The larger the value of min(proportion of frontal view and the sum of proportions of non-frontal view), the more balanced the view distribution of the instance is, and the less it is biased towards the easily accessible frontal view, thus the higher the data value.

[0092] The ranking is based on a triple: (P_target, N, Score_compact). Here, P_target represents the target viewpoint ratio information, N is the number of pedestrian instances within the instance (the more samples, the stronger the statistical significance), and Score_compact is the aforementioned aggregation degree evaluation metric (the higher the internal quality, the better).

[0093] During sorting, the priority is arranged in descending order: first, sort by P_target in descending order; if P_target is the same, sort by N in descending order; if they are still the same, sort by Score_compact in descending order. This way, the sorted target cluster instances can be obtained.

[0094] Furthermore, a backtracking search and upper bound pruning algorithm is used to determine the largest subset of clustered instances. The goal of this step is to select the largest possible subset from the sorted list of target clustered instances within the same identity profile, while ensuring that any two pedestrian instances within the subset are sufficiently distinct. The specific implementation is as follows: Backtracking search with upper bound pruning is a depth-first search algorithm for solving combinatorial optimization problems. It systematically attempts to add pedestrian instances to the current candidate subset and uses an "upper bound" function to estimate the best possible outcome for the current branch. If this estimate is unlikely to exceed the currently known optimal solution, the search for that branch is terminated early (i.e., "pruning"), thus significantly improving search efficiency.

[0095] Specific implementation steps: Assume the current identity file has M sorted pedestrian instances, and a preset distance threshold T_dist (e.g., 0.8). The backtracking search and upper bound pruning algorithm starts a depth-first search from the first pedestrian instance. When attempting to add the i-th pedestrian instance to the current subset S_current, it must be checked whether its distance (read from the distance matrix D) to the center vector of each existing instance in S_current is greater than T_dist. Only if this "mutually distinguishable" condition is met can it be added. At the same time, the algorithm maintains a globally optimal solution S_best (the largest subset currently found). When searching any branch, an upper bound is calculated: upper bound = |S_current| + (M - current index). If the upper bound ≤ |S_best|, it means that even if all subsequent instances are added (although this is practically impossible), the record cannot be broken, so the branch is pruned immediately, and the algorithm backtracks to the previous level. Finally, the algorithm returns the subset with the largest number of elements that satisfies the constraints, called the maximum clustering instance subset. This subset represents the set of "identities" that are most different from each other that can be extracted from the current file.

[0096] Furthermore, an empty training set is initialized, and the largest subset of clustered instances in each identity file is traversed. Instances that meet the criteria are stored to obtain the target pedestrian re-identification dataset. This is the final global filtering and aggregation step.

[0097] Initialize a global, empty set `Final_Training_Set`. Iterate through the largest subset generated from all identity profiles. For each candidate cluster instance in the subset, perform a two-step check: Color bias correction check: Query the frequency of the color attribute (C_top, C_bottom) of the candidate instance among all existing instances in the Final_Training_Set. If the frequency of this color combination is significantly higher than the average frequency (e.g., frequency > average frequency + standard deviation), the instance is rejected with a probability P_reject that is positively correlated with the frequency. This balances the color distribution of the final dataset and prevents the model from overfitting to common clothing colors.

[0098] Cross-file identity deduplication check: For candidate instances that pass the color check, calculate the maximum cosine similarity between their cluster center vector and the cluster center vectors of all instances in the Final_Training_Set. If this maximum value exceeds a preset similarity threshold T_sim (e.g., 0.85), the candidate instance is considered highly similar to existing instances and is likely to correspond to the same real identity (just a mis-aggregation from different files). To avoid label conflicts, it is discarded.

[0099] Candidate instances that pass the above two layers of checks will be formally added to the Final_Training_Set. Finally, the Final_Training_Set contains all pedestrian images and their corresponding new identity labels (each cluster instance is assigned a unique ID), which constitute the cleaned and optimized target pedestrian re-identification dataset.

[0100] This embodiment ensures identity separability through a distance matrix, favors high-value data through viewpoint ranking, ensures balanced distribution through color de-biasing, and guarantees identity uniqueness through cross-archive deduplication. Ultimately, it can automatically mine and construct an ideal training dataset with clean identities, highly challenging samples, and balanced attribute distribution from massive amounts of noisy archives, greatly improving the utilization value and transformation efficiency of noisy data.

[0101] This invention successfully implements a highly efficient and automated large-scale pedestrian noise data cleaning solution by constructing a complete technical chain, including feature extraction and fine-grained clustering, quality quantification evaluation, multimodal attribute analysis, and global optimization selection that integrates discriminability, diversity, and balance constraints. This method can transform hundreds of millions of unreliable pedestrian profiles generated in real-world scenarios into high-quality, highly generalizable pedestrian re-identification datasets that can be directly used for training. It fundamentally solves the data bottleneck problem that restricts the deployment and application of this technology in real-world scenarios, demonstrating significant practical value and industrial prospects.

[0102] In some embodiments of the present invention, the target pedestrian re-identification dataset obtained by the aforementioned method can be applied to the training of the pedestrian re-identification model.

[0103] This target pedestrian re-identification dataset contains two core components: a high-quality collection of pedestrian images and pseudo-identity labels automatically generated by a cleaning process for each image. Each pseudo-identity label corresponds to a target cluster instance, ensuring that images under the same label have highly consistent identity characteristics.

[0104] The specific steps for constructing and initializing a pedestrian re-identification model are as follows: Select a pedestrian re-identification model architecture as the base model. This model is usually a deep convolutional neural network, such as ResNet-50 or Vision Transformer as the core backbone network. Add an IBN layer to the ResNet-50 network to enhance the network's generalization ability, and add a fully connected layer (classification head) at the end to calculate the identity classification loss, as well as an embedding layer to calculate the metric learning loss (such as triplet loss).

[0105] The model is trained using a target pedestrian re-identification dataset. The dataset is divided into a training set and a validation set. The standard pedestrian re-identification model training workflow is followed. Forward propagation involves batching images into the network, extracting features through the backbone network, and obtaining the identity classification probability and feature embedding vector through the classification head and embedding layer, respectively.

[0106] Loss calculation, or calculating the combined loss function, typically includes: Identity classification loss, such as cross-entropy loss, encourages the model to predict images with the same pseudo-identity label as belonging to the same category.

[0107] Metric learning losses, such as triplet loss or contrastive loss, directly constrain the vectors of similar images to be closer and the vectors of dissimilar images to be further apart in the feature embedding space.

[0108] Backpropagation and optimization: Based on the calculated total loss, the gradient is calculated using the backpropagation algorithm, and the model weights are updated using an optimizer (such as AdamW).

[0109] Repeat the above steps in an iterative loop until the model's performance on the validation set (such as Rank-1 accuracy, mAP) converges or reaches the preset number of iterations.

[0110] By using the target pedestrian re-identification dataset obtained through the cleaning process of this invention for training, the final technical effect of this invention can be directly verified and achieved. Because this dataset possesses three core advantages—high identity purity (guaranteed through clustering and deduplication), rich intra-class diversity (guaranteed through viewpoint proportion preference and color de-biasing), and high inter-class discriminability (guaranteed through distance matrix constraints)—the pedestrian re-identification model trained on it can: It converges faster and avoids gradient interference and model oscillation caused by noise labels.

[0111] Learning more discriminative features and clear positive-negative sample pairs makes metric learning more effective.

[0112] Achieving stronger generalization ability, the balanced and diverse data distribution reduces the model's overfitting to specific attributes (such as common colors and single perspectives), making it more robust when facing the complex changes in the real world.

[0113] The cleaning and denoising apparatus for large-scale pedestrian re-identification data collection provided by the present invention is described below. The cleaning and denoising apparatus for large-scale pedestrian re-identification data collection described below can be referred to in correspondence with the cleaning and denoising method for large-scale pedestrian re-identification data collection described above.

[0114] like Figure 5 The diagram shown is a structural schematic of a cleaning and denoising device for large-scale pedestrian re-identification data collection provided by the present invention. The device includes the following modules: The acquisition module 510 is used to acquire an initial pedestrian re-identification dataset; the initial pedestrian re-identification dataset includes multiple identity files; The filtering module 520 is used to perform preliminary filtering based on the initial pedestrian re-identification dataset to determine the target cluster instance for each identity file. The determination module 530 is used to determine the target pedestrian re-identification dataset from the target cluster instances of all identity files based on a cluster instance selection strategy. The cluster instance selection strategy is determined based on the distance matrix, color attribute information, and viewing angle ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information from the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information of the target cluster instance.

[0115] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to extract the embedding vector of each pedestrian instance in each identity file; In each identity profile, the embedding vector of the pedestrian instance is subjected to hierarchical density space clustering to filter out noisy pedestrian instances and obtain multiple clustered instances. Evaluation metrics are calculated for each cluster instance to determine the target cluster instance for each identity file.

[0116] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to calculate the average similarity between pedestrian instances within each cluster instance and the cluster center vector of the cluster instance. The average similarity of each cluster instance is normalized to obtain the aggregation degree evaluation index of each cluster instance; Based on the aggregation degree evaluation index of each cluster instance and the preset index threshold, the target cluster instance of each identity file is determined.

[0117] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to sum the embedding vectors of each pedestrian instance in the clustering instance to obtain the total embedding vector. The total embedding vector and the number of pedestrian instances are calculated to obtain the cluster center vector of the clustered instances.

[0118] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to compare the aggregation degree evaluation index of each cluster instance with the index threshold. Cluster instances whose aggregation degree evaluation index is greater than or equal to the index threshold are designated as target cluster instances, and cluster instances whose aggregation degree evaluation index is less than the index threshold are designated as noisy cluster instances. In each identity profile, the noisy cluster instances are filtered out to obtain the target cluster instances.

[0119] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to calculate the center vector distance between any two target cluster instances in each identity file; The distance matrix is ​​constructed based on the center vector distances between all target cluster instances.

[0120] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to extract the upper and lower body clothing colors of target pedestrian instances in each target cluster instance using a pedestrian clothing color recognition model, as the color attribute information; The target pedestrian instance is the pedestrian instance that is closest to the cluster center vector of the target cluster instance.

[0121] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to identify the perspective information of each pedestrian instance in the target cluster instance using a perspective attribute recognition model; the perspective information includes at least a frontal perspective, a side perspective, and a rear perspective. The perspective ratio information is calculated by statistically analyzing the perspective information corresponding to each perspective in each target cluster instance.

[0122] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to sort the target cluster instances in each identity file using sorting rules to obtain sorted target cluster instances. A backtracking search and upper bound pruning algorithm is used to process the sorted target cluster instances and determine the largest subset of cluster instances. Initialize an empty training set, iterate through the largest cluster instance subset in each identity file, and store the largest cluster instance subset that meets the conditions into the empty training set to obtain the target pedestrian re-identification dataset.

[0123] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to sort the data according to the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance. The sorting priority of the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance decreases in that order.

[0124] Preferably, the cleaning and denoising device for large-scale pedestrian re-identification data provided by the present invention is specifically used to calculate the sum of the side view ratio and the back view ratio of the target cluster instance. The smaller of the two perspective ratios, the frontal perspective ratio and the total perspective ratio, is determined as the target perspective ratio information.

[0125] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a method for cleaning and denoising large-scale pedestrian re-identification data. This method includes: acquiring an initial pedestrian re-identification dataset; the initial pedestrian re-identification dataset includes multiple identity files; performing preliminary screening processing based on the initial pedestrian re-identification dataset to determine a target cluster instance for each identity file; and determining a target pedestrian re-identification dataset from the target cluster instances of all identity files based on a cluster instance selection strategy; wherein the cluster instance selection strategy is determined based on a distance matrix, color attribute information, and viewing angle ratio information; the distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information based on the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information based on the target cluster instance.

[0126] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cleaning and denoising method for large-scale pedestrian re-identification data collection provided by the above methods. The method includes: acquiring an initial pedestrian re-identification dataset; the initial pedestrian re-identification dataset includes multiple identity files; performing preliminary screening processing based on the initial pedestrian re-identification dataset to determine a target cluster instance for each identity file; and determining a target pedestrian re-identification dataset from the target cluster instances of all identity files based on a cluster instance selection strategy; wherein the cluster instance selection strategy is determined based on a distance matrix, color attribute information, and viewing angle ratio information; the distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information based on the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information based on the target cluster instance.

[0128] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for cleaning and denoising large-scale pedestrian re-identification data collection provided by the methods described above. The method includes: acquiring an initial pedestrian re-identification dataset; the initial pedestrian re-identification dataset includes multiple identity files; performing preliminary screening processing based on the initial pedestrian re-identification dataset to determine a target cluster instance for each identity file; and determining a target pedestrian re-identification dataset from the target cluster instances of all identity files based on a cluster instance selection strategy; wherein the cluster instance selection strategy is determined based on a distance matrix, color attribute information, and viewing angle ratio information; the distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information based on the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information based on the target cluster instance.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for cleaning and denoising large-scale pedestrian re-identification data, characterized in that, include: Obtain the initial pedestrian re-identification dataset; The initial pedestrian re-identification dataset includes multiple identity profiles; Based on the initial pedestrian re-identification dataset, a preliminary screening process is performed to determine the target cluster instance for each of the identity files; Based on the clustering instance selection strategy, the target pedestrian re-identification dataset is determined from the target clustering instances of all identity files; The cluster instance selection strategy is determined based on the distance matrix, color attribute information, and viewing angle ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information from the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information of the target cluster instance.

2. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 1, characterized in that, The preliminary screening process based on the initial pedestrian re-identification dataset to determine the target cluster instance for each identity file includes: Extract the embedding vector of each pedestrian instance from each identity profile; In each identity profile, the embedding vector of the pedestrian instance is subjected to hierarchical density space clustering to filter out noisy pedestrian instances and obtain multiple clustered instances. Evaluation metrics are calculated for each cluster instance to determine the target cluster instance for each identity file.

3. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 2, characterized in that, The step of calculating evaluation metrics based on each cluster instance to determine the target cluster instance for each identity file includes: Calculate the average similarity between pedestrian instances within each cluster instance and the cluster center vectors of the cluster instances; The average similarity of each cluster instance is normalized to obtain the aggregation degree evaluation index of each cluster instance; Based on the aggregation degree evaluation index of each cluster instance and the preset index threshold, the target cluster instance of each identity file is determined.

4. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 3, characterized in that, The steps for determining the cluster center vector of the clustering instance include: The embedding vector of each pedestrian instance in the cluster is summed to obtain the total embedding vector; The total embedding vector and the number of pedestrian instances are calculated to obtain the cluster center vector of the clustered instances.

5. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 3, characterized in that, The determination of the target cluster instance for each identity file based on the aggregation degree evaluation index of each cluster instance and a preset index threshold includes: The aggregation degree evaluation index of each cluster instance is compared with the index threshold. Cluster instances whose aggregation degree evaluation index is greater than or equal to the index threshold are designated as target cluster instances, and cluster instances whose aggregation degree evaluation index is less than the index threshold are designated as noisy cluster instances. In each identity profile, the noisy cluster instances are filtered out to obtain the target cluster instances.

6. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 5, characterized in that, The distance matrix is ​​calculated based on the center vectors of the target cluster instances, and includes: In each of the identity profiles, calculate the center vector distance between any two of the target cluster instances; The distance matrix is ​​constructed based on the center vector distances between all target cluster instances.

7. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 6, characterized in that, The color attribute information is determined based on color information extraction from the target cluster instance, including: The color of the upper and lower body clothing of the target pedestrian instance in each target cluster instance is extracted using a pedestrian clothing color recognition model and used as the color attribute information; The target pedestrian instance is the pedestrian instance that is closest to the cluster center vector of the target cluster instance.

8. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 7, characterized in that, The viewpoint ratio information is determined based on viewpoint ratio statistics of target cluster instances, including: The perspective information of each pedestrian instance in the target cluster is identified using a perspective attribute recognition model; the perspective information includes at least a frontal perspective, a side perspective, and a rear perspective. The perspective ratio information is calculated by statistically analyzing the perspective information corresponding to each perspective in each target cluster instance.

9. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 8, characterized in that, The clustering instance selection strategy includes: In each of the identity files, the target cluster instances are sorted using sorting rules to obtain sorted target cluster instances; A backtracking search and upper bound pruning algorithm is used to process the sorted target cluster instances and determine the largest subset of cluster instances. Initialize an empty training set, iterate through the largest cluster instance subset in each identity file, and store the largest cluster instance subset that meets the conditions into the empty training set to obtain the target pedestrian re-identification dataset.

10. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 9, characterized in that, The method includes: The sorting rule is based on the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance. The sorting priority of the target viewpoint ratio information, the number of pedestrian instances in the target cluster instance, and the aggregation degree evaluation index of the target cluster instance decreases in that order.

11. The method for cleaning and denoising large-scale pedestrian re-identification data according to claim 10, characterized in that, The method includes: Calculate the sum of the side view ratio and the back view ratio of the target cluster instance; The smaller of the two perspective ratios, the frontal perspective ratio and the total perspective ratio, is determined as the target perspective ratio information.

12. A device for cleaning and denoising large-scale pedestrian re-identification data collection, characterized in that, include: The acquisition module is used to acquire the initial pedestrian re-identification dataset; The initial pedestrian re-identification dataset includes multiple identity profiles; The filtering module is used to perform preliminary filtering based on the initial pedestrian re-identification dataset to determine the target cluster instance for each identity file; The determination module is used to determine the target pedestrian re-identification dataset from the target cluster instances of all identity files based on a cluster instance selection strategy. The cluster instance selection strategy is determined based on the distance matrix, color attribute information, and viewing angle ratio information. The distance matrix is ​​calculated based on the center vector of the target cluster instance, the color attribute information is determined by extracting color information from the target cluster instance, and the viewing angle ratio information is determined by statistically analyzing the viewing angle ratio information of the target cluster instance.

Images