Video pedestrian re-identification method and system based on hyperbolic uncertainty recovery

Abstract

The present invention relates to the field of computer vision technology, and in particular to a method and system for video pedestrian re-identification based on hyperbolic uncertainty recovery. The method comprises: converting a pedestrian target into a visual feature patch block to obtain video features, inputting a text prompt to obtain text features, calculating the similarity with the visual feature patch block and sorting the text features, taking the mean of the patch features before the index with the largest gradient change and fusing them to obtain pedestrian component features; locally adding noise to the features, and using a diffusion model to predict the features before adding noise; during denoising, using the occluded features after adding noise as input and the unoccluded original features as conditions to guide learning. The network results are mapped to a hyperbolic space modeling prediction, the mean of the sampling results is calculated and the result with the highest confidence is selected, and the pedestrian component features are input into the diffusion model prediction to obtain the pedestrian re-identification results. This solution can weaken the influence of occlusion on video pedestrian re-identification and improve the recognition accuracy when pedestrians are occluded.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a video pedestrian re-identification method and system based on hyperbolic uncertainty recovery. Background Art

[0002] In the field of computer vision, person re-identification (ReID) aims to accurately identify the same person from images or video sequences captured by different cameras. Depending on the type of input data, person re-identification can be divided into image-based and video-based methods.

[0003] In practical applications of person re-identification, occlusion remains a critical challenge that needs to be addressed. Whether for image-based or video-based person re-identification, occlusion significantly negatively impacts recognition performance. In image-based person re-identification, occlusion can lead to a partial loss of information about a person's appearance, making it difficult for the model to accurately extract and match their features, thereby reducing recognition accuracy. For example, when a person is partially obscured by other objects or people, key features such as their face and clothing markings may not be fully captured, resulting in increased feature similarity between different people and a higher misrecognition rate.

[0004] In video-based person re-identification, occlusion problems often lead to a serious performance degradation. Because video data has temporal continuity, the rich temporal information it contains is crucial for accurately identifying pedestrians. However, occlusion will destroy the integrity and continuity of video features, causing a series of problems such as misalignment and missing video features. Specifically, in a video sequence, some key frames of pedestrians may lose important feature information due to occlusion, making the entire video feature representation unable to accurately reflect the true identity of the pedestrian. Moreover, due to the dynamic changes in video content, the occlusion situation may vary in different frames, further increasing the difficulty of feature extraction and matching. For example, in a surveillance video, a pedestrian may be obscured by a vehicle at some moments and return to normal at other moments. This discontinuous occlusion will make it difficult for the model to construct a stable and accurate pedestrian feature representation, which seriously affects the performance of the re-identification task.

[0005] To address the occlusion problem, existing technologies have proposed some solutions, but these solutions still have many limitations. Some methods attempt to simulate occlusion situations through data augmentation techniques to improve the model's robustness to occlusion. However, these methods can generally only simulate limited occlusion patterns and cannot cover the complex and varied occlusion scenarios in actual applications. Therefore, the effect in actual applications is not ideal. Other methods use attention mechanisms to guide the model to focus on the non-occluded areas of pedestrians, but when dealing with severe occlusion situations, these methods still find it difficult to achieve satisfactory recognition results due to the lack of available effective information. In addition, most existing methods do not fully consider the temporal information of the video and fail to fully utilize the correlation between different frames in the video sequence to alleviate the uncertainty caused by occlusion, thereby limiting the further improvement of recognition performance. Summary of the Invention

[0006] The purpose of the present invention is to propose a video pedestrian re-identification method and system based on hyperbolic uncertainty recovery, which can weaken the impact of occlusion on video pedestrian re-identification and improve the recognition accuracy when pedestrians are occluded.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a method for video person re-identification based on hyperbolic uncertainty recovery, comprising:

[0008] Input video for feature extraction, convert pedestrian targets in the image into corresponding visual feature patches, and obtain the video features corresponding to each frame of the image;

[0009] Select pedestrian features, including:

[0010] Input the designed text prompt into CLIP's text encoder to obtain text features;

[0011] Calculate the similarity between each text feature and each visual feature patch and sort them;

[0012] Calculate the gradient of its similarity, find the index with the largest gradient change, take out all patch features before this index, take the mean and fuse them into component features to obtain pedestrian component features;

[0013] Hyperbolic uncertainty recovery, including:

[0014] By training the diffusion model, the features are locally denoised, the content of the occluded area is inferred from the visible area, and the features before denoising are predicted using the diffusion model.

[0015] In the denoising process, the occluded features after adding noise are used as input, and the original features without occlusion are used as conditional information to guide the diffusion model to learn denoising. An explicit supervision signal is used to supervise the denoising process of the model by minimizing the mean square error loss between the predicted noise and the actual noise.

[0016] Map the results obtained by the network to the hyperbolic space, perform modeling and prediction, calculate the mean of all sampling results in the hyperbolic space, and select the sampling result with the highest confidence as the result;

[0017] Perform model training and inference, input pedestrian component features into the diffusion model for prediction, and obtain pedestrian re-identification results.

[0018] The basic solution's beneficial effects: By integrating textual cues with the CLIP text encoder, the model can be precisely guided to focus on the core semantic information of pedestrians. Textual cues (such as "pedestrian," "head," and "clothing") are matched against visual feature patches for similarity, prioritizing the model's focus on key areas related to pedestrians and significantly reducing the impact of background interference, such as irrelevant objects, on feature extraction. This semantically guided attention allows the model to reliably lock onto pedestrians even in complex and changing real-world scenarios (such as crowded streets, dynamic backgrounds, and sudden changes in lighting). This provides purer, more discriminative feature input for subsequent re-identification, improving the model's recognition performance in complex environments.

[0019] By fully leveraging the generation and reasoning capabilities of the diffusion model and simulating real occlusion scenarios through a local noise addition strategy, the model can deeply learn the correlation between occluded and visible areas during training. Through the innovative "noise-substitute-occlusion" training method, the diffusion model can accurately infer the semantic information of occluded areas from local visible features, significantly enhancing the feature recovery capabilities for partial occlusion, severe occlusion, and other situations. Compared to the traditional global noise addition diffusion model, local noise addition better fits the local characteristics of actual occlusion, making the model's adaptation to occlusion more targeted, effectively improving the robustness of re-identification in occluded scenarios, and reducing matching errors caused by occlusion.

[0020] The prediction results of the diffusion model are mapped into hyperbolic space for hierarchical modeling, leveraging hyperbolic space's ability to efficiently represent hierarchical relationships in high-dimensional data. Firstly, the geometric properties of hyperbolic space more accurately capture subtle differences in pedestrian characteristics, strengthening the distinction between individual pedestrians. Secondly, by calculating the mean and confidence level of multiple sampling results in hyperbolic space, a quantitative assessment of the model's "decision confidence" is achieved. This mechanism of selecting the optimal result based on confidence allows the model to make more reliable matching decisions in scenarios with ambiguous features and high uncertainty, further improving the accuracy and stability of re-ID results.

[0021] By replacing the global noise of traditional diffusion models with localized noise, only certain feature regions are perturbed during training, significantly reducing the amount of training data and computational overhead. During inference, the feature recovery task of localized noise can complete the denoising process in fewer time steps, significantly shortening the inference time for a single sample. This lightweight design reduces reliance on hardware computing power while maintaining model performance, making the model easier to deploy in resource-constrained scenarios such as actual monitoring systems, and improving the engineering practicality of the technical solution.

[0022] This method optimizes key challenges in video person re-identification (RID), such as background interference, occlusion, and generalization in complex scenes. It can be reliably applied to practical tasks such as public security surveillance, cross-camera suspect tracking, and crowd flow analysis in large venues. It maintains high RID performance, particularly under complex conditions such as long video sequences, dynamic clothing changes, and multiple occlusions. This method provides reliable technical support for intelligent security, smart cities, and other fields, and has significant engineering application value and widespread potential.

[0023] As an implementable preferred solution, a visual encoder based on ViT's CLIP is used to extract features from the extracted image frames; the visual encoder performs convolution operations and Transformer structure processing on the input image frames to extract local features and global semantic information of the image.

[0024] As an implementable preferred solution, calculate the similarity between each text feature and each visual feature patch block The calculation formula is:

[0025]

[0026] in, is the text feature vector, is the visual feature patch block feature vector, and They represent the magnitude of the corresponding vectors respectively.

[0027] As an implementable and preferred solution, the process of locally adding noise to features through the training of a diffusion model includes:

[0028] Randomly select feature areas and generate masks, where 1 indicates occlusion and 0 indicates visibility, to obtain the occluded features.

[0029] Inject Gaussian noise , is gradually injected through T time steps. The specific noise addition process is expressed as:

[0030]

[0031] in is the noise attenuation multiplicative factor; is the noise intensity at time step s; is the feature before noise addition, is the feature after adding occlusion, is the feature after adding noise.

[0032] As an implementable optimal solution, the learning objectives of the denoising process are:

[0033]

[0034] in, is the actual noise, is the noise predicted by the model, and c is the conditional information;

[0035] By predicting the noise through the network, the recovered features can be derived, and the formula is expressed as:

[0036]

[0037] The uncertainty in the noise prediction directly corresponds to the uncertainty in the recovered features.

[0038] As an implementable and preferred solution, the process of mapping the network results into a hyperbolic space and performing modeling and prediction includes:

[0039] Collect multiple results from the prediction and calculate their distance from the origin in hyperbolic space:

[0040]

[0041] in, is the origin of hyperbolic space, For the prediction results, k is the curvature of the hyperbolic model;

[0042] Use spherical radius It constructs a bounded normalized distance itself to obtain a normalized metric;

[0043] Setting the threshold , when the threshold is exceeded When , the model is considered to be confident, and the sampling result with the highest confidence is selected as the final result; when it does not exceed the threshold When , the model is considered uncertain;

[0044] Calculate the mean of all sampled results in the dual space to capture the semantic abstraction shared by multiple predictions and achieve a balance between specificity and generalization:

[0045]

[0046]

[0047] in, yes The weight of is the shape-preserving factor of the hyperbolic space, is Möbius addition.

[0048] As an implementable and optimal solution, during the training phase, cross entropy loss and triplet loss are used for supervised training to supervise the selected local pedestrian features to enhance the model's ability to select the correct target pedestrian;

[0049] The formula for calculating cross entropy loss is:

[0050]

[0051] in is the true category label, Predict the probability that the sample belongs to a certain category for the model;

[0052] The triplet loss is calculated as:

[0053]

[0054] in, is the anchor sample, is a positive sample, is a negative sample, It is the distance measure between samples, and margin is a set value.

[0055] As an implementable optimal solution, a penalty term for the correctness of the selection of local pedestrian features is added to the loss function. If the selected local pedestrian features deviate significantly from the true target pedestrian features, the corresponding penalty value is added to the loss function.

[0056] As an implementable optimal solution, the preprocessed video data is input into the model for training according to the batch size. According to the value of the loss function, the gradient of the model parameters is calculated using the back propagation algorithm, and the model parameters are updated according to the calculated gradient using the optimization algorithm.

[0057] In a second aspect, the present invention also provides a video pedestrian re-identification system based on hyperbolic uncertainty recovery, which utilizes the above-mentioned video pedestrian re-identification method based on hyperbolic uncertainty recovery, including a pedestrian feature selection module and a hyperbolic uncertainty recovery module. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Schematic diagram of the hyperbolic uncertainty recovery framework proposed in this invention.

[0059] Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0060] Reference numerals: electronic device 500 , processor 501 , communication interface 502 , memory 503 , bus 504 . DETAILED DESCRIPTION

[0061] In order to make the technical solution and advantages of the present application clearer, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. It will be understood that the specific embodiments described herein are only partial embodiments of the present invention, which are only used to explain the present application, rather than to limit the present application. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered to be isolated, and they can be combined with each other to achieve better technical effects. The same reference numerals appearing in the drawings of the following embodiments represent the same features or components, which can be applied to different embodiments.

[0062] In addition, unless otherwise defined, technical or scientific terms used in the description of the present invention should have the common meanings understood by those skilled in the art in the art to which the present invention belongs.

[0063] The present invention will be further described in detail below with reference to the accompanying drawings:

[0064] The video pedestrian re-identification method based on hyperbolic uncertainty recovery includes the following steps.

[0065] Step S100: Input video and perform feature extraction, including:

[0066] Step S101: Video preprocessing: obtaining an input video sequence. This video sequence can be a video containing pedestrians captured from a surveillance camera or other device. A frame extraction operation is performed on the video sequence, decomposing the video into a series of image frames at a certain frame rate.

[0067] Step S102 , extracting visual features, includes: performing feature extraction on the extracted image frames using a CLIP (Contrastive Language - Image Pretraining) visual encoder based on ViT (Vision Transformer).

[0068] CLIP's visual encoder is a pre-trained neural network model that converts images into feature vectors rich in semantic information. When processing each frame, the visual encoder converts objects such as pedestrians into corresponding visual feature patches. These patches are small regions into which the image is divided, and each patch contains the visual feature information of that region.

[0069] Specifically, the visual encoder performs a series of complex computations on the input image frames, including convolution operations and Transformer structure processing. Convolution operations extract local image features, while the Transformer structure captures global semantic information. After these processes, the corresponding video features for each image frame are obtained.

[0070] Step S200, selecting pedestrian features, including:

[0071] In step S201, the designed text prompt "A {cls} part of a person." is input into CLIP's text encoder to obtain text features, where "{cls}" represents various parts of a person, such as the head, torso, and limbs. CLIP's text encoder is a pre-trained neural network model that converts the input text into corresponding text feature vectors. The text encoder performs word embedding on each word in the text, then processes it through a multi-layer neural network to obtain a vector representing the text's semantic features.

[0072] Step S202: Calculate the similarity between each text feature and each visual feature patch. And sort them, the calculation formula is:

[0073]

[0074] in, is the text feature vector, is the visual feature patch block feature vector, and They represent the magnitude of the corresponding vectors respectively.

[0075] Step S203: Calculate the gradient of the similarity and find the one with the largest gradient change. Index. Take all patch features before this index, take the mean and fuse them into component features, and finally obtain the number of pedestrian component features corresponding to {cls}. Locate the target area and eliminate the influence of background, etc. For example, if {cls} represents the head, then obtain the pedestrian component features representing the head features.

[0076] Step S300, hyperbolic uncertainty recovery, includes:

[0077] In step S301, after obtaining pedestrian features, the features are locally noisyed using a diffusion model training method to simulate occlusion. The content of the occluded area is inferred from the visible area (as a condition). Using the diffusion model to predict the pre-noised features compensates for the lack of corresponding real-world label supervision and significantly reduces computational resources.

[0078] Specifically, randomly select feature regions and generate masks , where 1 indicates occlusion and 0 indicates visibility, and the features after occlusion are obtained ,in is the feature before adding noise. Inject Gaussian noise , is gradually injected through T time steps. The specific noise addition process can be expressed as:

[0079]

[0080] in is the noise attenuation multiplicative factor, is the noise intensity at time step s. is the feature before noise addition, is the feature after adding occlusion, is the feature after adding noise.

[0081] from arrive The transformation is expressed as:

[0082]

[0083] Will Substituting into the above formula, we can see that the area ,because , and the occluded area ,because , so we have:

[0084]

[0085] in represents the true characteristics of the noise disturbance, and Represents pure noise, no information, and needs to be restored.

[0086] Step S302: In the denoising process, the occlusion feature after adding noise is used As input, the original features are complete and unobstructed The conditional information c is used to guide the diffusion model to learn denoising, thereby making up for the lack of corresponding real label supervision and significantly reducing computing resources.

[0087] The denoising process is the inverse process of the denoising process and aims to learn a conditional model to reconstruct the true distribution from the noise distribution:

[0088]

[0089] in are model parameters.

[0090] In order to effectively restore the occluded feature area, an explicit supervision signal is used, which compares the predicted output of the diffusion model at each time step with the corresponding real noise. To constrain.

[0091] Specifically, during training, the denoising process of the model is supervised by minimizing the mean squared error (MSE) loss between the predicted noise and the actual noise:

[0092]

[0093] in, is the actual noise, is the noise predicted by the model, and c is the conditional information.

[0094] Although the training goal of the diffusion model is to predict the injected noise , but the variability of the noise prediction also reflects the uncertainty of the underlying feature recovery. Therefore, by predicting the noise through the network, the recovered features can be derived, which can be expressed as:

[0095]

[0096] This means that the uncertainty of the noise prediction directly corresponds to the uncertainty of the recovered features. The distribution of distribution.

[0097] Step S303: Map the results obtained by the network into a hyperbolic space, and perform modeling and prediction.

[0098] Collect multiple results from the prediction and calculate their distance from the origin in hyperbolic space:

[0099]

[0100] in, is the origin of hyperbolic space, For the prediction results, k is the curvature of the hyperbolic model;

[0101] ,and when , it is not bounded.

[0102] Use spherical radius itself to construct a bounded normalized distance, because It is a monotonically increasing function, so it can be expressed as: .

[0103] We get a simple, equivalent, and interpretable normalized metric: the closer the point is to the boundary, the more , the larger the value, the more "confident" the model is.

[0104] Set a threshold , when the threshold is exceeded When , the model is considered to be confident, and the sampling result with the highest confidence is selected as the final result; when it does not exceed the threshold , the model is considered uncertain.

[0105] Calculate the mean of all sampling results in the hyperbolic space, namely gyromidpoint. This operation can capture the semantic abstraction shared by multiple predictions and achieve a balance between specificity and generalization:

[0106]

[0107]

[0108] in, yes The weight of is the shape-preserving factor of the hyperbolic space, is Möbius addition.

[0109] Through the above-mentioned sampling and aggregation mechanism, this technical solution not only significantly enhances the robustness of the recovery of occluded features, but also provides clear uncertainty modeling capabilities, approximately simulating the process of human cognitive reasoning under uncertain conditions.

[0110] Step S400, performing model training, includes:

[0111] Step S401: During the training phase, cross entropy loss and triplet loss are used to perform supervised training.

[0112] The formula for calculating cross entropy loss is:

[0113]

[0114] in is the true category label (0 or 1 indicates whether it belongs to a certain category), The model predicts the probability that the sample belongs to a certain category. The calculation formula for triplet loss is:

[0115]

[0116] in, is the anchor sample (anchor), is a positive sample (positive), is a negative sample (negative), It is the distance measure between samples, and margin is a set value.

[0117] The selected local pedestrian features are supervised to enhance the model's ability to select the correct target pedestrian. This is achieved by adding a penalty term to the loss function to determine the correctness of the local pedestrian feature selection. For example, if the selected local pedestrian feature deviates significantly from the true target pedestrian feature, a corresponding penalty value is added to the loss function.

[0118] In step S402, the preprocessed video data is fed into the model in batches of a specified size for training. The model performs a forward propagation calculation based on the input data to obtain predictions. A loss function is then calculated based on the predictions and the true labels. Based on the loss function, the gradients of the model parameters are calculated using a backpropagation algorithm. The backpropagation algorithm uses the chain rule to calculate gradients layer by layer from the output layer to the input layer, determining the contribution of each parameter to the loss function. An optimization algorithm is then used to update the model parameters based on the calculated gradients.

[0119] In step S500, during the inference phase, since the feature recovery patterns have been learned from a large amount of data during the training phase, the inference phase does not require the multiple iterative time-stepping operations required during the training phase. The diffusion model only requires a very small time step to complete predictions and achieve good results. Based on this small time step, the features processed in the previous steps are input into the diffusion model for prediction.

[0120] The diffusion model makes predictions based on the input features and learned parameters according to its internal calculation logic to obtain the final pedestrian re-identification results.

[0121] This method was fully experimentally validated on two large-scale video person re-identification datasets, achieving excellent recognition performance. As shown in Table 1, on the MARS dataset, the Rank-1 accuracy reached 94.1%, with a mAP of 89.0%; on the LS-VID dataset, the Rank-1 accuracy reached 90.7%, with a mAP of 83.9%; and on the iLIDS-VID dataset, the Rank-1 accuracy reached 96.7%, with a mAP of 98.1%.

[0122] Table 1 Experimental results comparison table

[0123]

[0124] The above results show that the method of the present invention can still maintain stable and accurate identity recognition capabilities under complex conditions such as multiple viewpoints, multiple scenes, lighting changes, and occlusion (there are a large number of occluded videos in the MARS and iLIDS-VID datasets), verifying its robustness and wide applicability in practical applications.

[0125] This embodiment also provides a video pedestrian re-identification system based on hyperbolic uncertainty recovery, which can execute the above-mentioned video pedestrian re-identification method based on hyperbolic uncertainty recovery.

[0126] Reference Figure 1 The hyperbolic uncertainty recovery framework proposed by the present invention is shown. The framework mainly has two stages: positioning and recovery. In the positioning stage, text features are obtained through a text encoder through manually designed text prompts, and the similarity is calculated with the visual features to obtain a similarity matrix. Then, the pedestrian selection module obtains pedestrian features to remove the influence of interference such as background. In the recovery stage, local noise is added to the pedestrian features obtained in the positioning stage to simulate occlusion, and then it is input into the diffusion model. The diffusion model consists of N Transformer blocks, each of which contains a self-attention layer (Self-Attn), a cross-attention layer (Cross-Attn) and a feedforward network layer (MLP FFN); then the output of the Transformer block is mapped to the hyperbolic space, and the mean and variance are predicted by the hyperbolic multi-head self-attention layer (H-Multi-Head Self-Attn), and then multiple prediction results are sampled through the mean and variance, and then their "confidence" is calculated in the hyperbolic space, and different prediction results are generated according to the degree of "confidence". Finally, the entire framework is trained and optimized by cross entropy loss and triplet loss, specifically including:

[0127] The pedestrian feature selection module uses the textual cue (A {cls} part of a person) to generate textual features for the corresponding body parts. This module then calculates similarity with the visual feature patches extracted by the visual encoder, sorts them, and calculates the gradient of the similarity. The patch with the largest gradient change is found. The patch preceding this patch is then considered the visual feature patch most similar to the textual cue. The average of these selected patches is then taken to obtain the region corresponding to the textual cue, i.e., the body part features. This module calculates the similarity between the image and text and selects the most relevant patches, allowing the model to locate the target area and eliminate background influences.

[0128] The hyperbolic uncertainty recovery module simulates occlusion by adding Gaussian noise to the located pedestrian features, and inputs it into the improved diffusion model of the present invention for recovery; multiple results are sampled from the prediction and mapped to the hyperbolic space for hierarchical modeling. Due to the characteristics of the hyperbolic space, it is naturally suitable for modeling hierarchical data. The degree to which the prediction results are close to the edge in the hyperbolic space is calculated to judge the "confidence" of the model. When the model is more "confident", it will generate specific predictions. When the model is less "confident", it will generate abstract predictions. For example, when the occlusion is small, the model can be The model uses context and other information to infer the occluded part, thus generating a specific prediction result. However, when the occlusion is more serious, the model cannot accurately predict the occluded part, so it will generate an abstract prediction. For example, if the lower body of a person is completely occluded, the model cannot make inferences through the context, but at least the model knows that it is the lower body that is occluded through the existing part (upper body), at least the specific details are not clear. Therefore, the model can calculate an abstract prediction based on the existing prediction results to represent the lower body without the need for complete details; in this way, it can make up for the problem of missing features caused by occlusion.

[0129] The present application also provides an electronic device 500 that utilizes the aforementioned method for video person re-identification based on hyperbolic uncertainty recovery. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the aforementioned method for video person re-identification based on hyperbolic uncertainty recovery are implemented. In the present application, the processor serves as the control center of the computer method and can be a processor of a physical machine or a processor of a virtual machine.

[0130] Reference Figure 2 The electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503, and at least one bus 504. The bus 504 is used to implement connection and communication between these components, the communication interface 502 is used to communicate signaling or data with other node devices, and the memory 503 stores machine-readable instructions executable by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 via the bus 504, and when the machine-readable instructions are called by the processor 501, the steps of the video pedestrian re-identification method based on hyperbolic uncertainty recovery as described above are executed.

[0131] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor of an electronic device, it can implement the steps of the video pedestrian re-identification method based on hyperbolic uncertainty recovery as described above.

[0132] Those skilled in the art will understand that all or part of the processes in the method for video pedestrian re-identification based on hyperbolic uncertainty recovery can be implemented by instructing related hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of various embodiments of the method for video pedestrian re-identification based on hyperbolic uncertainty recovery. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0133] The above contents are merely embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. A person of ordinary skill in the art is aware of all common technical knowledge in the technical field to which the invention belongs before the filing date or priority date, is able to obtain all existing technologies in the field, and has the ability to apply conventional experimental means before that date. A person of ordinary skill in the art can, under the guidance of this application, improve and implement this scheme in combination with his or her own abilities. Some typical known structures or known methods should not become an obstacle for a person of ordinary skill in the art to implement this application. It should be pointed out that for a person of ordinary skill in the art, several variations and improvements can be made without departing from the structure of the present invention, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection claimed in this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A video pedestrian re-identification method based on hyperbolic uncertainty recovery, characterized by: include: Input video for feature extraction, convert pedestrian targets in the image into corresponding visual feature patches, and obtain the video features corresponding to each frame of the image; Select pedestrian features, including: Input the designed text prompt into CLIP's text encoder to obtain text features; Calculate the similarity between each text feature and each visual feature patch and sort them; Calculate the gradient of its similarity, find the index with the largest gradient change, take out all patch features before this index, take the mean and fuse them into component features to obtain pedestrian component features; Hyperbolic uncertainty recovery, including: By training the diffusion model, the features are locally denoised, the content of the occluded area is inferred from the visible area, and the features before denoising are predicted using the diffusion model. In the denoising process, the occluded features after adding noise are used as input, and the original features without occlusion are used as conditional information to guide the diffusion model to learn denoising. An explicit supervision signal is used to supervise the denoising process of the model by minimizing the mean square error loss between the predicted noise and the actual noise. Map the results obtained by the network into hyperbolic space and perform modeling and prediction, including: Collect multiple results from the prediction and calculate their distance from the origin in hyperbolic space: in, is the origin of hyperbolic space, For the prediction results, k is the curvature of the hyperbolic model; Use spherical radius It constructs a bounded normalized distance itself to obtain a normalized metric; Setting the threshold , when the threshold is exceeded When , the model is considered to be confident, and the sampling result with the highest confidence is selected as the final result; when it does not exceed the threshold When , the model is considered uncertain; Calculate the mean of all sampled results in the dual space to capture the semantic abstraction shared by multiple predictions and achieve a balance between specificity and generalization: in, yes The weight of is the shape-preserving factor of the hyperbolic space, It is Möbius addition; Calculate the mean of all sampling results in the hyperbolic space and select the sampling result with the highest confidence as the result; Perform model training and inference, input pedestrian component features into the diffusion model for prediction, and obtain pedestrian re-identification results.

2. The video person re-identification method based on hyperbolic uncertainty recovery according to claim 1 is characterized in that The visual encoder based on ViT's CLIP is used to extract features from the extracted image frames. The visual encoder performs convolution operations and Transformer structure processing on the input image frames to extract local features and global semantic information of the image.

3. The video person re-identification method based on hyperbolic uncertainty recovery according to claim 1 is characterized in that Calculate the similarity between each text feature and each visual feature patch The calculation formula is: in, is the text feature vector, is the visual feature patch block feature vector, and They represent the magnitude of the corresponding vectors respectively.

4. The video person re-identification method based on hyperbolic uncertainty recovery according to claim 1 is characterized in that The process of locally adding noise to features through the training of the diffusion model includes: Randomly select feature areas and generate masks, where 1 indicates occlusion and 0 indicates visibility, to obtain the occluded features; Inject Gaussian noise , is gradually injected through T time steps. The specific noise addition process is expressed as: in is the noise attenuation multiplicative factor, is the noise intensity at time step s; is the feature before noise addition, is the feature after adding occlusion, is the feature after adding noise.

5. The video pedestrian re-identification method based on hyperbolic uncertainty recovery according to claim 4 is characterized in that The learning objectives of the denoising process are: in, is the actual noise, is the noise predicted by the model, and c is the conditional information; By predicting the noise through the network, the recovered features can be derived, and the formula is expressed as: The uncertainty in the noise prediction directly corresponds to the uncertainty in the recovered features.

6. The video person re-identification method based on hyperbolic uncertainty recovery according to claim 5 is characterized in that During the training phase, cross entropy loss and triplet loss are used for supervised training to supervise the selected local pedestrian features to enhance the model's ability to select the correct target pedestrian; The formula for calculating cross entropy loss is: in is the true category label, Predict the probability that the sample belongs to a certain category for the model; The triplet loss is calculated as: in, is the anchor sample, is a positive sample, is a negative sample, It is the distance measure between samples, and margin is a set value.

7. The video person re-identification method based on hyperbolic uncertainty recovery according to claim 6 is characterized in that A penalty term for the correct selection of local pedestrian features is added to the loss function.

8. The video person re-identification method based on hyperbolic uncertainty recovery according to claim 7 is characterized in that The preprocessed video data is input into the model for training according to the batch size. According to the value of the loss function, the gradient of the model parameters is calculated using the back propagation algorithm, and the model parameters are updated according to the calculated gradient using the optimization algorithm.

9. A video person re-identification system based on hyperbolic uncertainty recovery, characterized by: A video pedestrian re-identification method based on hyperbolic uncertainty recovery as described in any one of claims 1 to 8 is used, comprising a pedestrian feature selection module and a hyperbolic uncertainty recovery module.

Images