Cross-view pedestrian re-identification method and device based on Gaussian modeling

Abstract

The application discloses a cross-view pedestrian re-identification method and device based on Gaussian modeling, which comprises the following steps: remodeling the features of the current batch of class centers and the uncertainty of predicting the features of the current batch of class centers, randomly sampling a random noise irrelevant to model parameters, equivalently re-describing the visual features, and measuring the similarity of Gaussian distribution between two domains by using JS divergence; modeling the new description of the multi-view of the target domain by using the features of the multi-view extracted by the neural network and the uncertainty between the views; introducing a regularization term, measuring by using KL divergence through explicit constraint close to the standard normal distribution; generating new Gaussian feature representation through a mixed Gaussian model by using the Gaussian distribution reserved in the Gaussian prototype level path through a memory network, and then outputting the final recognition result by using a normalized ternary loss to constrain the overall distribution. The device comprises a processor and a memory.

Description

Technical Field

[0001] This invention relates to the field of cross-view pedestrian re-identification, and in particular to a cross-view pedestrian re-identification method and apparatus based on Gaussian modeling. Background Technology

[0002] With the widespread application of deep learning in various fields, object retrieval tasks have attracted attention from the multimedia, biometrics, and computer vision communities. [1] Especially in the practical application of person re-identification. Learning highly accurate person re-identification models relies on the availability of large-scale annotated datasets, but the performance of these models drops sharply when evaluated on datasets lacking true labels for different images. Much of the progress in person re-identification models is due to deep convolutional neural networks. [2] While there has been progress in this area, most deep convolutional neural networks are trained in a supervised manner, making transfer learning difficult. As a result, unsupervised domain adaptation methods for person re-identification models trained on labeled source domain datasets and unlabeled target domain datasets have received widespread attention. In particular, cross-domain multi-view person re-identification technology has gained increasing importance by leveraging images on unlabeled datasets to retrieve corresponding multi-view targets.

[0003] Most existing methods focus on adversarial approaches. [3-5] Instance-based [6,7] Based on prototype [8,9] and based on fused view features [10,11] This eliminates inter-domain differences and completes the domain adaptation task. On one hand, the prototype-based domain adaptation method DLEA... [9] It compensates for the semantic information ignored by adversarial domain adaptation methods and narrows the domain gap, but it does not pay attention to the data uncertainty brought about by the mapping of images or multi-view targets to the feature space. It cannot handle intra-class differences and similarities well, resulting in inaccurate pedestrian re-identification. Moreover, due to the problem of inter-domain differences, the authenticity of noisy pseudo-labels is still not high.

[0004] On the other hand, these methods all neglect the learning of uncertain information, such as the fused view feature-based and instance-based domain adaptation method GVCNN.

[11] and HIFA [6] It ignores the uncertainty of individual instance representation caused by multiple perspectives, making it difficult to grasp the subtle differences in information brought by each perspective.

[0005] In summary, existing cross-view pedestrian re-identification techniques face the following three shortcomings:

[0006] 1. Existing methods directly use features extracted from images or multiple views for semantic alignment, but do not learn the uncertain information after mapping to the feature space, which reduces the accuracy of pedestrian re-identification.

[0007] 2. Conventional prototype-based methods simply apply pseudo-labels without proposing how to improve the authenticity of pseudo-labels. By using new feature representations of high-order Gaussian prototypes of samples, high-order prototypes can be obtained without using pseudo-labels. This allows the model to be constrained after each training session, thereby improving the authenticity of predicted pseudo-labels and avoiding the decrease in the accuracy of pedestrian re-identification caused by directly using pseudo-labels.

[0008] 3. Using only image features ignores the uncertain information in instance representation caused by multiple viewpoints. Existing methods for fusing view features are prone to overfitting in pedestrian re-identification, reducing the accuracy of pedestrian re-identification when using new instances. Summary of the Invention

[0009] This invention provides a cross-view pedestrian re-identification method and apparatus based on Gaussian modeling. It utilizes mobile class centers to address the problem of certain classes being absent from a batch due to small batch sizes, thus reducing the adverse effects of false labels. It employs prototype uncertainty to handle inherent data noise, eliminating the drawbacks of data uncertainty in continuous mapping spaces. An image enhancement module is designed to improve the robustness of the pedestrian re-identification model, while simultaneously exploring the uncertainty information brought about by multiple images and multiple views. Inspired by triplet loss, the global Gaussian prototype recorded in the memory network is constrained by loss, resulting in compact intra-class data and dispersed heterogeneous data, improving the accuracy of pedestrian re-identification using new instances, thereby enhancing the precision of pedestrian re-identification. This method still meets the needs in diverse application scenarios, as detailed below:

[0010] Firstly, a cross-view pedestrian re-identification method based on Gaussian modeling, the method comprising:

[0011] We remodel using the features of the current batch's category center and the uncertainty of predicting the features of the current batch's category center, and randomly sample a random noise that is independent of the model parameters to equivalently reformulate the visual features. We then use JS divergence to measure the similarity of the Gaussian distributions between the two domains.

[0012] By utilizing the features of multiple views extracted by neural networks and the uncertainties between views, a new representation of multiple views of the target domain is formed through modeling.

[0013] A regularization term is introduced to approximate the standard normal distribution through explicit constraints, and the KL divergence is used as a measure.

[0014] The Gaussian distributions preserved in the Gaussian prototype-level path are processed by a memory network and then processed by a Gaussian mixture model to generate new Gaussian feature representations. The overall distribution is constrained by a standardized ternary loss, and the final recognition result is output.

[0015] In a second aspect, a cross-view pedestrian re-identification device based on Gaussian modeling is characterized in that the device comprises: a processor and a memory, wherein the memory stores program instructions, and the processor invokes the program instructions stored in the memory to cause the device to perform the steps of the method described in any one of the first aspects.

[0016] Third aspect, a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the steps of the method described in any of the first aspects.

[0017] The beneficial effects of the technical solution provided by this invention are:

[0018] 1. In the vision and prototype paths, a mobile semantic transfer network is used to move and align the category centers to solve the problem of incorrect labels in unsupervised domain adaptation pseudo-labels. Instead of directly treating pseudo-label samples as real samples, each category center is aligned in the source and target domains. The correct category center is used to neutralize the adverse effects of the incorrect category center. Due to the small batch size in the experiment, moving the category center can achieve more accurate semantic representation learning, thereby alleviating the problem of pedestrian re-identification matching deviation caused by noisy pseudo-labels.

[0019] 2. In the visual and Gaussian prototype paths, at the prototype level, a Gaussian distribution is modeled using the features m of the current batch's class center and the intra-class uncertainty s to handle the inherent noise of the dataset. This eliminates the data uncertainty inherent in the continuously mapped feature space, addressing the data uncertainty problem caused by image and view mapping. The features are reformulated as z = m + ωs, where ω ∈ N(0,1). A regularization term is introduced during the optimization process, explicitly constraining z(m,ω,s) to approximate the standard normal distribution N(0,1). The KL divergence between these two distributions is used to measure this, preventing the new feature representation z from degenerating into the original deterministic representation z = m + c (c is a constant). Simultaneously, JS divergence is used to measure the similarity of the Gaussian distributions between the two domains, reducing the degree of difference between domains. This improves the data representation ability of pedestrian images and views mapped to the feature space under unsupervised domain adaptation, thereby improving the accuracy of pedestrian re-identification.

[0020] 3. In the Gaussian vision and Gaussian mixture prototype path, in order to solve the uncertainty problem of multiple views in cross-view pedestrian re-identification at the instance level, this invention transforms the original simple target domain multiple views taking the maximum generated feature into multiple view modeling to form a Gaussian distribution, thus obtaining the connection between pedestrian multi-view information. Correspondingly, this invention uses image enhancement technology to form multiple different images to model and form a Gaussian distribution for each image in the source domain, which improves the robustness of the pedestrian re-identification model. At the same time, it uses KL divergence to avoid the degradation of new feature representation z and uses domain adversarial similarity loss to guide the global alignment of the source domain and the target domain, effectively enhancing the domain invariant feature expression ability of the pedestrian re-identification visual feature extraction network.

[0021] 4. A high-order Gaussian feature representation is generated through Gaussian mixture model. JS divergence is used to measure inter-domain divergence, which better learns the uncertain information of instance representation brought about by multiple perspectives. At the same time, by contrast center loss, the distance between Gaussian instances of the same class and high-order Gaussian prototypes in the Gaussian feature space is narrowed, which solves the problem of overfitting in pedestrian re-identification and improves the accuracy of pedestrian re-identification when using new instances.

[0022] 5. In the global Gaussian mixture path, this embodiment of the invention utilizes a memory network to retain the Gaussian features at the pedestrian image and view prototype level after each round of training. These features are then processed by a Gaussian mixture model to generate new global Gaussian mixture feature representations. Similarly, KL divergence constraints are used to prevent the degradation of the new feature representations. Inspired by triplet loss, the global Gaussian mixture prototype recorded in the memory network is constrained by triplet normalization loss, making the global category center more accurate in semantic representation learning, the same category more compact in space, and different categories more dispersed in space, thereby improving the accuracy of pedestrian re-identification. Attached Figure Description

[0023] Figure 1 This is a flowchart of a cross-view pedestrian re-identification method based on Gaussian modeling;

[0024] Figure 2 This is a network structure diagram for cross-view pedestrian re-identification based on Gaussian modeling. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0026] Example 1

[0027] A cross-view pedestrian re-identification method based on Gaussian modeling, see [link to relevant documentation]. Figure 1 The method includes the following steps:

[0028] 101: Utilizing a two-dimensional convolutional neural network to extract visual features from an image (source domain) and multiple views of a target pedestrian (target domain);

[0029] In this embodiment of the invention, visual features are divided into two different processing methods: one is to process them at the prototype level after obtaining the visual features, namely steps 103 and 104; the other is to perform data augmentation and instance-level processing on the image, namely step 105.

[0030] 102: Based on the unsupervised adversarial domain adaptation learning strategy, under the condition of covariate transfer, the source domain classification error and the difference between the source domain and the target domain are minimized to train the classifier and domain discriminator, thereby guiding the global alignment of the source domain and the target domain.

[0031] This step effectively enhances the domain-invariant feature representation capability of the pedestrian re-identification visual feature extraction network.

[0032] 103: Calculate the moving category center using the category centers of the previous batch and the current batch. Through loss constraints, the local semantic representation becomes more accurate. Use a domain discriminator to guide the global alignment of the source and target domains.

[0033] In the visual and prototype paths, the mobile semantic transfer network is used to obtain the mobile category center from the category centers of the previous batch and the current batch. The square of the Euclidean distance is used as the loss to align the source domain and the target domain with each category center. The correct category center is used to neutralize the adverse effects of the incorrect category center.

[0034] Among them, due to the small batch size in the experiment, the moving average class center can achieve more accurate local semantic representation learning. The mobile semantic transfer network solves the problem of erroneous labels in the unsupervised domain adaptation pseudo-labels, thereby alleviating the pedestrian re-identification matching deviation problem caused by noisy pseudo-labels.

[0035] Domain adversarial similarity loss is used to measure inter-domain divergence. In this embodiment of the invention, an additional domain discriminator is used to determine whether these features come from the source domain or the target domain, while the visual feature extraction network is used to deceive the domain discriminator. The goal is to reach a balance in the game between the two parties, thereby guiding the global alignment of the source domain and the target domain.

[0036] 104: Use the features m and uncertainty s of the current batch's category center to model a Gaussian distribution, and reformulate the visual features as z = m + ωs, where ω ∈ N(0,1). At the same time, use JS divergence to measure the similarity of the Gaussian distributions between the two domains.

[0037] In the visual and Gaussian prototype paths, this embodiment of the invention eliminates the drawback of data uncertainty in the continuously mapped feature space after obtaining the features extracted by the neural network, thus solving the problem of data uncertainty caused by image and view mapping. For the first time, it proposes to use the features m of the current batch's category center and the uncertainty s within the class to model and form a Gaussian distribution, and to reformulate the visual features as z = m + ωs, where ω ∈ N(0,1). At the same time, it uses JS divergence to measure the similarity of Gaussian distributions between the two domains, thereby improving the data representation ability of pedestrian images mapped to the feature space under unsupervised domain adaptation and improving the accuracy of pedestrian re-identification.

[0038] Compared to previous prototype-based unsupervised domain adaptation methods, this method not only utilizes the new features of the current batch class centers to represent the uncertainties introduced during mapping, but also uses the features of the class centers of previous and current batches to address the adverse effects of small experimental batches and incorrect class centers.

[0039] 105: Features m extracted from multiple views using neural networks v Uncertainty between views v Modeling forms a Gaussian distribution z of multiple views of the target domain. views =m v +ωs v Each image in the source domain is processed using an image enhancement algorithm to create multiple images, and features m are extracted using a neural network. imgs And calculate the uncertainty s between multiple images. imgs Modeling forms a Gaussian distribution z of multiple images in the source domain. imgs =m imgs +ωs imgs For the first time, the uncertainties within multiple source domain images and multiple target domain views of pedestrians are utilized to represent the features of a single pedestrian, thereby improving the robustness of the pedestrian re-identification model.

[0040] A new Gaussian distribution representation z generated from instances of the source and target domains. imgs and z views By fitting Gaussian mixture models and iteratively applying the expectation-maximization algorithm, the features and uncertainties of each class in these distributions are obtained, thus yielding new high-order Gaussian prototype feature representations for the source and target domains. This method avoids directly using pseudo-labels to obtain these new high-order Gaussian prototype feature representations, allowing for model constraint after each training iteration to improve the realism of predicted pseudo-labels. This avoids the decrease in accuracy caused by directly using pseudo-labels. Furthermore, JS divergence is used to constrain inter-domain distributions and reduce inter-domain differences.

[0041] In the Gaussian vision and Gaussian mixture prototype paths, to address the uncertainty in the representation of a single instance during cross-view pedestrian re-identification at the instance level, a Gaussian distribution is modeled. This embodiment of the invention transforms the original simple target domain multi-view max pooling for generating compact descriptor features into:

[0042] Features m extracted from multiple views using neural networks v Uncertainty between views v Modeling to form a Gaussian distribution z of multiple views of the target domain views =m v +ωs v This allows the network to learn the uncertainties between different views during training, thereby improving the accuracy of multi-view representation of a particular instance and obtaining the connections between pedestrian multi-view information. Correspondingly, in this embodiment of the invention, each image in the source domain is used to form multiple images through an image enhancement algorithm, and features m are extracted through a neural network. imgs And calculate the uncertainty s between multiple images. imgs Modeling forms a Gaussian distribution z of multiple images in the source domain. imgs =m imgs +ωs imgs For the first time, the uncertainties within multiple source domain images and multiple target domain views of pedestrians are utilized to represent the features of a single pedestrian, thereby enhancing the robustness of the pedestrian re-identification network.

[0043] Next, we will present a new Gaussian distribution representation z generated from the source and target domain instances. imgs and z views By fitting Gaussian mixture models and iteratively applying the Expectation-Maximum (EM) algorithm, the features and uncertainties of each class in these distributions are obtained, thus yielding new high-order Gaussian prototype feature representations for the source and target domains. This approach avoids directly using pseudo-labels to obtain these new high-order Gaussian prototype feature representations, allowing for model constraint after each training iteration and improving the realism of predicted pseudo-labels. This prevents the decrease in accuracy caused by directly using pseudo-labels. Furthermore, inter-domain distribution constraints are implemented using JS divergence and contrast center loss to reduce inter-domain differences. Contrast center loss also narrows the distance between Gaussian instances of the same class and high-order Gaussian prototypes in the Gaussian feature space, addressing the overfitting problem in person re-identification and improving the accuracy when using new instances for person re-identification.

[0044] 106: Introduce a regularization term to explicitly constrain z(m,ω,s) to approximate the standard normal distribution N(0,1), and use the KL divergence between these two distributions as a measure;

[0045] Among them, KL divergence is used to measure the degeneration of the new feature representation z after prototype-level and instance-level Gaussian modeling into the original deterministic representation z = m + c (c is a constant).

[0046] 107: The Gaussian distributions preserved in the Gaussian prototype path are processed by a memory network and then processed by a Gaussian mixture model to generate new Gaussian feature representations. The overall distribution is constrained by a standardized ternary loss.

[0047] In the global Gaussian mixture path, this embodiment of the invention utilizes a memory network to retain the Gaussian features of the pedestrian image and view in the Gaussian prototype path after each round of training. These features are then processed by a Gaussian mixture model to generate new global Gaussian mixture feature representations. Inspired by triplet loss, the global Gaussian mixture prototype recorded in the memory network is constrained by triplet normalization loss, making the category center more accurate in semantic representation learning, the same category more compact in space, and different categories more dispersed in space, thereby improving the accuracy of pedestrian re-identification.

[0048] 108: The final constraint result from step 107 is applied to pedestrian re-identification, and the final identification result is output, which improves the accuracy of pedestrian re-identification.

[0049] Step 108 includes:

[0050] Using a cross-view pedestrian re-identification database as input, and through multiple rounds of training using steps 101-107, a complete and highly accurate cross-view pedestrian re-identification technology is obtained. Next, a cross-view pedestrian re-identification test is conducted: by inputting an image of a single pedestrian, features are extracted using the technology described in this invention, and then the JS divergence between the input pedestrian image and other pedestrian features in the database is calculated. This measures the similarity between the input pedestrian image and all cross-view views of pedestrians in the database. Since the JS divergence result ranges from 0 to 1, it effectively describes the degree of difference between two different distributions in the feature space, making it easy to select pedestrian cross-view views with smaller differences and higher rankings.

[0051] In summary, the embodiments of the present invention propose a novel method for cross-view pedestrian re-identification and design a novel network structure, thereby improving the performance of cross-view pedestrian re-identification.

[0052] Example 2

[0053] The solution in Example 1 will be further described below with specific examples:

[0054] 201: A two-dimensional neural network extracts visual features from an image (source domain) and visual features from multiple views of a target pedestrian (target domain). The AlexNet architecture is used as the visual feature network, which includes a five-layer convolutional network and a three-layer fully connected network.

[0055] 202: Based on an unsupervised adversarial domain adaptation learning strategy, under the condition of covariate transfer, the classifier C and domain discriminator D are trained by minimizing the source domain classification error and the difference between the source and target domains, thereby guiding the global alignment of the source and target domains. This effectively enhances the accuracy of the pedestrian re-identification visual feature extraction network and its domain-invariant feature representation ability. The following is the objective function:

[0056]

[0057] Where F is a visual feature network that shares parameters between the source and target domains, x s Represents the source domain instance, x t L represents the target domain instance. CE For cross-entropy loss, and These are the sets of all samples in the source and target domains, respectively.

[0058] 203: In the visual and prototype paths, after obtaining visual features, a mobile semantic transfer network is used at the prototype level to align mobile category centers to address the problem of erroneous labels in unsupervised domain adaptation pseudo-labels. This utilizes previously batched category centers. and the current batch category center Features to obtain the center of the mobile category This is used to address the adverse effects of small experimental batches and incorrect category centers, thereby mitigating the pedestrian re-identification matching bias caused by noisy pseudo-labels. The update of the moving category center is as follows:

[0059]

[0060] Where ρ is the hyperparameter of the mobile category center, which is usually set to 0.3.

[0061] By aligning the centroids of each class in the source and target domains, and due to the small batch size in the experiment, the squared Euclidean distance d is used. ED Limit the distance between centroids of products with the same category label but different domains to ensure that features of the same category are similar, as shown in the following formula:

[0062]

[0063] in, and Let J and J represent the j-th class center of the source domain and the target domain, respectively, where J is the number of class centers in the source domain and the target domain.

[0064] 204: Utilizing the feature C of the current batch's category center jThe uncertainty of predicting the current batch category center features is remodeled to improve the data representation ability of pedestrian images mapped to the feature space under unsupervised domain adaptation, thereby improving the accuracy of pedestrian re-identification;

[0065] In the visual and Gaussian prototype paths, this embodiment of the invention, after obtaining the features extracted by the neural network, eliminates the drawback of data uncertainty in the feature space of continuous mapping, solving the data uncertainty problem brought about by image and view mapping. It defines that each image or view is no longer a deterministic representation in the feature space, but rather follows a Gaussian distribution z(m,ω,s). This embodiment of the invention uses reparameterization techniques to allow the model to still function as in the usual model training process, thereby enabling backpropagation. Specifically, it utilizes the feature m=C of the current batch's class center. j The uncertainty s in predicting the current batch category center feature is remodeled as z = m + ωs, and a random noise ω∈N(0,1) independent of the model parameters is randomly sampled, thus equivalently restating the visual features as:

[0066]

[0067] Where, n class For a certain category C in the current batch j Quantity, For the features of the i-th instance of the j-th class, the feature C of the class center of the current batch j As shown in the following formula:

[0068]

[0069] By restating features that are continuously mapped to the feature space, the problem of data uncertainty caused by image and view mapping is solved. For the first time, Gaussian distribution is formed by modeling the features of the current batch's class center and the uncertainty within the class. This overcomes the problem of previous methods that directly use features extracted from images or multiple views for semantic alignment without learning the uncertainty information within the class after mapping to the feature space. It improves the data representation ability of pedestrian images mapped to the feature space under unsupervised domain adaptation, and improves the accuracy of pedestrian re-identification.

[0070] Since the Gaussian distribution is modeled using the features of the current batch's class centers and intra-class uncertainties for the first time, the JS divergence is used to measure the similarity of the Gaussian distributions between the two domains. Because the JS divergence ranges from 0 to 1, it effectively describes the degree of difference between the two Gaussian distributions, as shown in the following formula:

[0071]

[0072] Among them, z s and z tThe new representations of Gaussian distributions for the source and target domains are given, with JS representing JS divergence and KL representing KL divergence. Previous methods typically use JS divergence between instances to measure the similarity between instances in the current batch. This invention enhances the feature representation capability of categories by measuring similarity at the Gaussian prototype level between domains, improves the semantic representation capability of instance-based methods, reduces the domain margin, and significantly reduces model runtime. By measuring the similarity of Gaussian distributions between domains through JS divergence, it strengthens the global semantic alignment between the source and target domains, effectively enhancing the person re-identification network to be more compact for similar classes and more discrete for different classes in the Gaussian feature space, while reducing the degree of difference between domains.

[0073] The visual feature processing required in steps 203 and 204 is as follows: visual features are extracted from the cross-view pedestrian view through a convolutional network, the features are numerically compared, the maximum value of multiple views is obtained, and then a single feature representation of the cross-view pedestrian multi-view is obtained.

[0074] 205: A new representation of multi-view objects in the target domain is formed by modeling the features of multi-view objects extracted by neural networks and the uncertainties between views. views ;

[0075] The visual feature processing required in step 205 is as follows: First, in the source domain, inspired by semi-supervised consistency regularization, the input image is perturbed to different degrees, and the final classification result of the model is still similar to the correct result. To increase the perturbation, the following changes are used to enhance the image in this invention: maximizing image contrast, rotation, pixel value inversion, adjusting image brightness, color, sharpness, saturation, and clarity, shearing and translating along the horizontal or vertical direction of the image, equalizing the image histogram, converting to an inverted color image, reducing the number of bits in each color channel of the image, and masking information in a certain area of ​​the image, totaling 16 different operations. In each training, N operations are selected from the 16 operations (which can be consistent with the number of pedestrian views across perspectives). To ensure the robustness of the model, the intensity M of the image change can be adjusted independently.

[0076] Through the above operations, a single image is transformed and stitched together to form multiple images in the source domain for the first time, which improves the robustness of cross-view pedestrian re-identification. Multiple images and multiple view features are obtained by using a visual feature extraction network with multiple views in the target domain.

[0077] In the Gaussian vision and Gaussian mixture prototype path, for the first time at the instance level, to address the uncertainty of single instance representation in cross-view pedestrian re-identification, the connections between pedestrian multi-view information are obtained. Similar to step 204, the multi-view visual features are modeled into a Gaussian distribution and then equivalently reformulated as z. views =m v +ωs vIn this embodiment of the invention, the original simple target domain multi-view max pooling to generate compact descriptor features is transformed into:

[0078] A new multi-view representation of the target domain is formed by modeling the features of multiple views extracted by neural networks and the uncertainties between views. views This allows the network to learn the uncertainties between different views during training, thereby improving the accuracy of multi-view representation of a particular instance. After the multi-view visual features of the target domain are modeled into a Gaussian distribution, they are equivalently reformulated as:

[0079]

[0080] in, k views The number of target domain views, The features extracted from the k-th view of the target domain after passing through the visual feature network.

[0081] Existing methods neglect the uncertainty of individual instance representations due to multiple viewpoints, making it difficult to grasp the subtle differences brought by each viewpoint. Methods based on fused view features are prone to overfitting in person re-identification, reducing the accuracy when using new instances for person re-identification. In the Gaussian vision and Gaussian mixture prototype path, this invention, for the first time, transforms the original simple method of maximizing the generated features from multiple views of the target domain into a method of modeling multiple views to form a Gaussian distribution z. views This allowed us to obtain the connections between pedestrian multiview information.

[0082] Correspondingly, inspired by semi-supervised consistency regularization, this invention, for the first time, transforms each image in the source domain into multiple images using an image enhancement algorithm. Similarly, it uses a neural network to extract features and calculate the uncertainty between the multiple images, modeling a new Gaussian distribution z for the source domain's multiple images. imgs =m imgs +ωs imgs This enhances the robustness of the person re-identification network, improves the robustness of the person re-identification model, and consequently improves the accuracy of representing a specific instance from multiple images. (Feature m) imgs and uncertainty imgs To achieve consistency with the target domain, further details are omitted.

[0083] Similar to step 202, based on the unsupervised adversarial domain adaptation learning strategy, a classifier and a domain discriminator are trained to guide the alignment of the source and target domains. Next, a new Gaussian distribution representation z of the source and target domain instances is generated. imgs and z viewsBy fitting a Gaussian mixture model and constraining the person re-identification model, the realism of predicted pseudo-labels is improved. Instead of directly using pseudo-labels, a new feature representation of the higher-order Gaussian prototype is obtained. This allows for constraining the model after each training iteration, thereby improving the realism of predicted pseudo-labels and avoiding the decrease in accuracy caused by directly using pseudo-labels. The probability distribution of the Gaussian mixture model is as follows:

[0084]

[0085] in, Let m represent the parameters of the j-th class in the mixture model: expectation, variance, and probability, respectively. j s j For each instance, the expected value and variance are iteratively calculated using the EM algorithm to obtain the parameters of the Gaussian mixture model. This allows the acquisition of the features and uncertainties of each class in these distributions, thereby obtaining new high-order Gaussian prototype feature representations of the source and target domains. Simultaneously, JS divergence is used to constrain inter-domain distributions and reduce inter-domain differences. The specific implementation is similar to equation (6) and will not be elaborated further. At the same time, the contrast center loss L is used to obtain the new features of the source and target domains. ccl To narrow the distance between Gaussian instances of the same category and higher-order Gaussian prototypes in the Gaussian feature space, the following equation is used:

[0086]

[0087] Where, N class d represents the number of instances in the current batch. ED The square of the Euclidean distance. For the Gaussian features of the nth instance of the jth class, For the current batch of higher-order Gaussian prototype features, n class This represents the total number of higher-order Gaussian prototypes in the current batch. `l≠j` indicates that the higher-order Gaussian prototype label `l` is not equal to the Gaussian instance. The corresponding label j, γ is a fixed hyperparameter used to limit the maximum value of the second term in equation (9) to prevent the pedestrian re-identification network from entering an unstable state during training, and θ is the contrast center loss L. ccl The hyperparameters γ and θ need to be manually adjusted.

[0088] Using JS divergence and contrast center loss L cclThis invention addresses the issues of discrepancies in the pedestrian re-identification domain and the tendency to overfit, improving the accuracy of pedestrian re-identification using new instances. It overcomes the shortcomings of previous instance-based and prototype-based methods that did not combine instances and prototypes. Furthermore, this invention utilizes a Gaussian distribution, making the distance between Gaussian instances and higher-order Gaussian prototypes of the same category more concentrated in the feature space. This increases the distributional similarity between Gaussian instances and higher-order Gaussian prototypes of the same category, thereby strengthening the connection between instances and prototypes. At the same time, it discretizes Gaussian instances and higher-order Gaussian prototypes of different categories in the feature space, improving the accuracy of pedestrian re-identification.

[0089] 206: By explicitly constraining z(m,ω,s) to approximate the standard normal distribution N(0,1), the KL divergence between the two distributions is used as a measure;

[0090] In the model optimization process, a regularization term is introduced. Regardless of whether it is the visual and Gaussian prototype path or the Gaussian visual and Gaussian mixture prototype path, the regularization term is explicitly constrained to approximate the standard normal distribution N(0,1). The KL divergence between these two distributions is used as a measure to avoid the new feature representation z=m+ωs after Gaussian modeling at the prototype level and instance level from degenerating into the original deterministic representation z=m+c (c is a constant). The simplified expression is as follows:

[0091] L sim =(m 2 +s 2 -logs 2 -1) / 2 (10)

[0092] 207: Obtaining New Gaussian Feature Representations Similarly, using the constraints of equation (9), and inspired by the triplet loss, we utilize L... norm Constraining the overall distribution addresses the overfitting problem in pedestrian re-identification, improving the accuracy when using new instances for pedestrian re-identification;

[0093] In the global Gaussian mixture path, this embodiment of the invention utilizes a memory network to ensure that each training round retains previous semantic information. It preserves the Gaussian features at the pedestrian image and view prototype level after each training round, processes them through a Gaussian mixture model to obtain Gaussian mixture model parameters, and generates a new global Gaussian mixture feature representation. Similarly, using Equation (9) as a constraint, the distance between the current batch of Gaussian instances and the new global mixture Gaussian prototypes of the same class is made denser in the feature space, while the distance between the current batch of Gaussian instances and the new global mixture Gaussian prototypes of different classes is made discrete in the feature space. For the first time in cross-view pedestrian re-identification, each round of training has previous semantic information, improving the accuracy of pedestrian re-identification. Inspired by the triplet loss, this invention constrains the overall distribution by using the standardized current batch of Gaussian instances and the new high-order Gaussian prototypes of different classes, obtaining the standardized triplet loss L. norm As shown in the following formula:

[0094]

[0095] Where η is the minimum distance between the current Gaussian prototype and the global mixture Gaussian prototypes of different categories, and J is the number of categories. d represents the new Gaussian representations of the same and different categories at the current prototype level, respectively. ED z is the square of the Euclidean distance, and z2 is L2 normalization, which makes the new Gaussian representations of different categories belong to the same order of magnitude. To avoid the aggregation of new Gaussian feature representations of category centers into very close spaces, restrictions are imposed. and The distance is at least To compare The distance from η makes the category center more accurate in semantic representation learning, and makes the same category more compact in space, while different categories are more dispersed in space. Since each round of training has previous semantic information, the accuracy of cross-view pedestrian re-identification can be greatly improved.

[0096] 208: The final constraint results of step 207 are applied to pedestrian re-identification, which improves the recognition accuracy of pedestrian re-identification.

[0097] Step 208 includes:

[0098] Using a cross-view pedestrian re-identification database as input, and through multiple rounds of training using the 201-207 method, a complete and highly accurate cross-view pedestrian re-identification technology is obtained. Next, cross-view pedestrian re-identification testing is conducted: by inputting an image of a single pedestrian, features are extracted using the technology described in this invention, and then the JS divergence between the input pedestrian image and the features of other pedestrians in the database is calculated. This measures the similarity between the input pedestrian image and all cross-view views of pedestrians in the database. Since the JS divergence result ranges from 0 to 1, it effectively describes the degree of difference between two different distributions in the feature space, thus easily selecting pedestrian cross-view views with smaller differences and higher rankings. The specific operation is as follows:

[0099] The pedestrian re-identification test first involves using the pedestrian re-identification model trained on the cross-view pedestrian re-identification database to input the pedestrian image. This step requires uploading the pedestrian image to be identified as input on the recognition page, and the backend obtains the features of the input pedestrian image after model calculation.

[0100] The second step involves measuring the similarity between the pedestrian image features obtained from the backend and the cross-view features of all pedestrians in the database. Prior to this step, all features of the cross-view features of all pedestrians in the database have been obtained through model calculations and saved. This allows for the extraction of only the pedestrian image features to be identified after input, eliminating the need to repeatedly extract image features from the cross-view pedestrian recognition database. A simple matrix multiplication is performed between the input pedestrian image features and the saved cross-view features to calculate similarity using JavaScript divergence, meeting the practical needs of pedestrian image recognition and saving time.

[0101] The third step, the recognition results page, will display the recognition results of the input single pedestrian image. You will see cross-view pedestrian views with different perspectives, minimal differences from the input pedestrian image, and sorted by similarity.

[0102] In summary, this invention, through steps 201-208, utilizes a mobile semantic transfer network to move and align category centers, addressing the issue of erroneous labels in unsupervised domain adaptation pseudo-labels. This achieves more accurate semantic representation learning, mitigating the pedestrian re-identification matching bias caused by noisy pseudo-labels. Simultaneously, by eliminating the data uncertainty inherent in the feature space of continuous mapping, it solves the problems of inherent noise in the dataset and data uncertainty arising from image and view mapping. Using JS divergence to measure the similarity of Gaussian distributions between two domains reduces the degree of difference between domains, improving the data representation capability of pedestrian images and views mapped to the feature space under unsupervised domain adaptation, thus enhancing the accuracy of pedestrian re-identification. Furthermore, by utilizing the uncertainty information of multi-path mapping, it explores the data representation capability of pedestrian images and cross-viewpoint views mapped to the feature space, better learning the instance representation uncertainty information brought about by multi-viewpoint mapping, resolving the overfitting problem in pedestrian re-identification, and improving the accuracy of pedestrian re-identification using new instances in practical applications. It also enhances the robustness of pedestrian re-identification, thereby improving the performance of cross-viewpoint pedestrian re-identification.

[0103] Example 3

[0104] A cross-view pedestrian re-identification device based on Gaussian modeling, comprising a processor and a memory. The processor and memory store program instructions, and the processor invokes the program instructions stored in the memory to cause the device to execute the following method steps:

[0105] We remodel using the features of the current batch's category center and the uncertainty of predicting the features of the current batch's category center, and randomly sample a random noise that is independent of the model parameters to equivalently reformulate the visual features. We then use JS divergence to measure the similarity of the Gaussian distributions between the two domains.

[0106] By utilizing the features of multiple views extracted by neural networks and the uncertainties between views, a new representation of multiple views of the target domain is formed through modeling.

[0107] A regularization term is introduced to approximate the standard normal distribution through explicit constraints, and the KL divergence is used as a measure.

[0108] The Gaussian distributions preserved in the Gaussian prototype-level path are processed by a memory network and then processed by a Gaussian mixture model to generate new Gaussian feature representations. The overall distribution is constrained by a ternary normalization loss, and the final recognition result is output.

[0109] Specifically, the new representation of the target domain's multi-views is modeled by utilizing the features extracted from the multi-views by neural networks and the uncertainties between the views:

[0110] The Gaussian distribution of the target domain multi-view is formed by modeling the features of the multi-view extracted by the neural network and the uncertainty between the views. Each image in the source domain is transformed into multiple images through image enhancement. Features are extracted by the neural network and the uncertainty between the multiple images is calculated to model the Gaussian distribution of the source domain multi-image. The uncertainty within the multi-view of the pedestrian in the source domain and the multi-view of the target domain is used to represent the features of a single pedestrian.

[0111] The new Gaussian distribution representations generated from the source and target domain instances are respectively fitted with Gaussian mixture models, and the features and uncertainties of each class are obtained by iterative computation using the expectation-maximization algorithm.

[0112] Obtain new high-order Gaussian prototype feature representations of source and target domain categories, and constrain inter-domain distribution using JS divergence.

[0113] Specifically, the equivalent restatement of visual features is as follows:

[0114]

[0115] Category center feature C j As shown in the following formula:

[0116]

[0117] Where, n class For a certain category C in the current batch j Quantity, Let i be the feature of the i-th instance of class j.

[0118] Furthermore, the new expression for the target domain multi-view is:

[0119]

[0120] in, k views The number of target domain views, The features extracted from the k-th view of the target domain after passing through the visual feature network.

[0121] The Gaussian distribution generated from the source and target domain instances is now described as follows:

[0122] z imgs =m imgs +ωs imgs

[0123] Where, m imgs As a feature, s imgs For uncertainty, ω∈N(0,1).

[0124] Furthermore, a regularization term is introduced to explicitly constrain the approximation of the standard normal distribution, specifically measured by KL divergence:

[0125] L sim =(m 2 +s 2 -logs 2 -1) / 2

[0126] Where m represents the features of the category centers, and s represents the uncertainty in predicting the features of the category centers in the current batch.

[0127] The overall distribution is constrained by a standardized ternary loss as follows:

[0128] The overall distribution is constrained by the standardized Gaussian instances of the current batch and new higher-order Gaussian prototypes of different categories, as follows:

[0129]

[0130] in, d represents the new Gaussian representations of the same and different categories at the current prototype level, respectively. ED Let |z|² be the square of the Euclidean distance, |z|² be L² normalization, which makes the new Gaussian representations of different categories belong to the same order of magnitude, and η be the minimum distance between the current Gaussian prototype and the global mixture Gaussian prototypes of different categories. For the global Gaussian mixture prototype, J is the number of categories.

[0131] The final recognition result is output as follows:

[0132] Input an image of a single pedestrian, extract features, and then calculate the JS divergence between the input pedestrian image and the features of other pedestrians in the database. Measure the similarity between the input pedestrian image and all cross-view views of pedestrians in the database, and filter out the cross-view views of pedestrians with smaller differences and higher ranking.

[0133] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0134] The aforementioned processor and memory can be devices with computing capabilities, such as computers, microcontrollers, or single-chip microcomputers. In specific implementations, this embodiment of the invention does not limit the execution entity and allows selection based on the needs of the actual application. Data signals are transmitted between the memory and processor via a bus, which will not be elaborated upon in this embodiment.

[0135] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0136] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0137] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0138] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0139] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0140] References:

[0141] [1]Oza P,Sindagi VA,Sharmini V V,et al.Unsupervised domain adaptationofobject detectors:A survey[J].IEEE Transactions on PatternAnalysis andMachine Intelligence,2023.

[0142] [2]LeCun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied todocument recognition[J].Proceedings ofthe IEEE,1998,86(11):2278-2324.

[0143] [3]Ganin Y,Lempitsky V.Unsupervised domain adaptation bybackpropagation[C] / / International conference on machine learning.PMLR,2015:1180-1189.

[0144] [4]Long M,Zhu H,Wang J,et al.Deep transfer learning with jointadaptation networks[C] / / International conference on machine learning.PMLR,2017:2208-2217.

[0145] [5]Long M,Cao Z,Wang J,et al.Conditional adversarial domainadaptation[J].Advances in neural information processing systems,2018,31.

[0146] [6]Zhou H,Nie W,Li W,et al.Hierarchical instance feature alignmentfor 2D image-based 3Dshape retrieval[C] / / Proceedings of the Twenty-NinthInternational Conference on International Joint Conferences onArtificialIntelligence.2021:839-845.

[0147] [7]Zhou H,Nie W,Song D,et al.Semantic consistency guided instancefeature alignment for 2Dimage-based 3D shape retrieval[C] / / Proceedings of the28th ACM International Conference on Multimedia.2020:925-933.

[0148] [8]Xie S,Zheng Z,Chen L,et al.Learning semantic representations forunsupervised domain adaptation[C] / / International conference on machinelearning.PMLR,2018:5423-5432.

[0149] [9]Zhou H,Liu AA,Nie W.Dual-level embedding alignment network for 2Dimage-based 3Dobject retrieval[C] / / Proceedings ofthe 27th ACM InternationalConference on Multimedia.2019:1667-1675.

[0150]

[10] Long M, Wang J, Ding G, et al.Transfer feature learning with jointdistribution adaptation[C] / / Proceedings of the IEEE international conference on computer vision.2013:2200-2207.

[0151]

[11] Feng Y, Zhang Z, Zhao X, et al.Group-view convolutional neural networks for 3D shape recognition[C] / / Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition(CVPR).264-272.

[0152] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for cross-view pedestrian re-identification based on Gaussian modeling, characterized in that, The method comprises: The visual features are equivalently re-expressed by using the features of the category center of the current batch and the uncertainty of predicting the features of the category center of the current batch, and a random noise unrelated to the model parameters is randomly sampled to equivalently re-express the visual features, and the similarity between the Gaussian distributions of the two domains is measured by using the JS divergence; The features of the multi-view extracted by the neural network and the uncertainty between the views are modeled to form a new expression of the multi-view of the target domain; A regularization term is introduced to measure the proximity to the standard normal distribution by explicit constraint, and the KL divergence is used for measurement; The Gaussian distribution remaining in the Gaussian prototype level path is respectively generated into a new Gaussian feature expression by a memory network, the overall distribution is constrained by a normalized ternary loss, and finally the recognition result is output; The features of the multi-view extracted by the neural network and the uncertainty between the views are modeled to form a new expression of the multi-view of the target domain, and the specific process is as follows: The features of the multi-view extracted by the neural network and the uncertainty between the views are modeled to form a Gaussian distribution of the multi-view of the target domain, each image of the source domain is formed into multiple images by image enhancement, the features are extracted by the neural network, and the uncertainty between the multiple images is calculated to model the Gaussian distribution of the multiple images of the source domain; the uncertainty between the multiple images of the cross-view pedestrian source domain and the multi-view of the target domain is used to represent the features of a single pedestrian; The Gaussian distribution new expression of the source domain and the target domain is respectively fitted into a Gaussian mixture model, and the expectation maximization algorithm is iteratively operated to obtain the features and the uncertainty of each class; The high-order Gaussian prototype new feature expression of the source domain and the target domain is obtained, and the distribution between the domains is constrained by the JS divergence; The visual features are equivalently re-expressed by using the features of the category center of the current batch and the uncertainty of predicting the features of the category center of the current batch, and a random noise unrelated to the model parameters is randomly sampled to equivalently re-express the visual features, and the similarity between the Gaussian distributions of the two domains is measured by using the JS divergence; ； Features of the class centers The following formula: ； wherein, the number of instances of the class, the feature of the instance of the class.​ 2.The cross-view pedestrian re-identification method based on Gaussian modeling according to claim 1, wherein, The expression of the new expression of the multi-view of the target domain is: ； wherein, , is the number of target domain views, is the target domain first feature extracted by the visual feature network from the target domain view. 3.The cross-view pedestrian re-identification method based on Gaussian modeling according to claim 1, wherein, The Gaussian distribution new expression of the source domain and the target domain is: ； wherein is a feature, is an uncertainty, . 4.The cross-view pedestrian re-identification method based on Gaussian modeling according to claim 1, wherein, The regularization term is introduced to measure the proximity to the standard normal distribution by explicit constraint, and the KL divergence is used for measurement, and the specific process is as follows: ； Wherein, m is the feature of the category center, and s is the uncertainty of predicting the feature of the category center of the current batch.

5. The cross-view pedestrian re-identification method based on Gaussian modeling according to claim 1, characterized in that, The overall distribution is constrained by the ternary normalized loss, and the specific process is as follows: The overall distribution is constrained by the normalized current batch Gaussian instance and the new different class high-order Gaussian prototype, and the specific process is as follows: ； where, , respectively denote the current prototype level of the same class and different class Gaussian new representation, is the square of the Euclidean distance, is the standardized so that the different class Gaussian new representation belongs to the same order of magnitude, is the minimum distance of the current Gaussian prototype and the global mixed Gaussian prototype of different classes, is the global mixed Gaussian prototype, is the number of classes.

6. The cross-view pedestrian re-identification method based on Gaussian modeling according to claim 1, characterized in that, The final recognition result is output, and the specific process is as follows: An image of a single pedestrian is input, the features are extracted, the JS divergence between the features of the input pedestrian and other pedestrians in the database is calculated, the similarity between the input pedestrian image and all pedestrian cross-view views in the database is measured, and the pedestrian cross-view views with small differences and high ranking are screened out.

7. A device for cross-view pedestrian re-identification based on Gaussian modeling, comprising: The device comprises a processor and a memory, the memory stores program instructions, and the processor invokes the program instructions stored in the memory to make the device execute the method steps of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, the computer program comprises program instructions, and the program instructions are executed by the processor to make the processor execute the method steps of any one of claims 1-6.

Images