A multi-view online enhanced pedestrian re-identification method, device and medium

Abstract

The present application relates to the technical field of computer vision and artificial intelligence, in particular to a multi-view online enhanced pedestrian re-identification method, device and medium, the method comprising: obtaining target images and space-time information of pedestrians to be identified, generating at least one completed image consistent in identity and different in view angle, inputting the target images and the completed image into a feature extraction network respectively and fusing them into joint feature representation, matching with a dynamic retrieval database to obtain a candidate matching feature set, if a preset trigger condition is met, performing review decision and determining a final identification result, otherwise taking the optimal matching result as the final identification result, recording the final identification result and decision information as training samples, and training and optimizing the feature extraction network for subsequent pedestrian re-identification tasks; the present application continuously improves the cross-view recognition accuracy by online completion of multi-view information of target images, combined with human-computer collaborative decision and closed-loop feedback mechanism.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a multi-view online enhanced pedestrian re-identification method, device, and medium. Background Technology

[0002] Person re-identification (ReID) aims to identify the same pedestrian across different cameras and is an important research direction in the field of computer vision. Existing methods typically extract pedestrian appearance features based on deep convolutional neural networks and achieve cross-camera retrieval and matching by calculating the feature similarity between the query image and the base database images.

[0003] However, pedestrian images in real-world surveillance scenarios are mostly captured from a single perspective, and the perspective of the same pedestrian varies significantly under different cameras. When the query image is from the back or side, the discriminative feature information is insufficient, leading to a significant decrease in cross-view matching accuracy. Some existing technologies alleviate this problem by augmenting images through data augmentation or generative models, but generated images are usually only used for sample expansion during the training phase and fail to actively participate in matching decisions during online inference. In addition, existing systems lack effective human-machine collaboration mechanisms, and cannot be intervened and corrected in a timely manner when the model makes a misjudgment; they also lack online closed-loop learning capabilities, and cannot use feedback data generated in actual business operations to continuously optimize the model, making it difficult for system performance to gradually improve with data accumulation.

[0004] To address the aforementioned problems, this invention provides a multi-view online enhanced pedestrian re-identification method. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a multi-view online enhanced pedestrian re-identification method, device, and medium. It completes the multi-view information of the target image online through a generative model, and combines human-machine collaborative decision-making and a closed-loop feedback mechanism to achieve continuous improvement in cross-view recognition accuracy.

[0006] According to a first aspect of the present invention, a multi-view online enhanced pedestrian re-identification method is provided, comprising the following steps:

[0007] S1, based on the target image of the pedestrian to be identified and the timestamp and camera identifier of the acquired target image, generate at least one complete image with the same identity label as the target image but a different perspective.

[0008] S2, the target image and each completed image are input into the pre-trained feature extraction network to extract image features, and all extracted image features are fused into a joint feature representation.

[0009] S3, calculate the similarity between the joint feature representation and the candidate image features in the dynamic retrieval base, and obtain the candidate matching feature set according to the similarity ranking result; the dynamic retrieval base is constructed based on the timestamp and camera identifier.

[0010] S4. Determine whether the candidate matching feature set meets the preset triggering conditions. If it does, introduce external information to review the candidate matching feature set and determine the final recognition result; otherwise, take the best matching result in the candidate matching feature set as the final recognition result.

[0011] S5. Record the final recognition result and the corresponding decision information as training samples. When the number of accumulated training samples reaches a preset threshold or a preset time node is reached, perform training optimization on the feature extraction network and use the optimized network parameters for subsequent pedestrian re-identification tasks.

[0012] According to a second aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the above-described multi-view online enhanced pedestrian re-identification method.

[0013] According to a third aspect of the present invention, an electronic device is provided, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0014] The present invention has at least the following beneficial effects:

[0015] This invention provides a multi-view online enhanced pedestrian re-identification method. It acquires the target image and spatiotemporal information of the pedestrian to be identified, and generates at least one complete image with the same identity as the target image but a different viewpoint. This effectively compensates for the lack of a person's viewpoint in actual monitoring scenarios, improving the reliability of subsequent cross-view matching. The target image and the complete image are then input into a feature extraction network for image feature extraction and fused into a joint feature representation. This enables a single query image to have multi-view discrimination capabilities, significantly improving the robustness and accuracy of cross-view matching. Finally, the joint feature representation is matched with a dynamic retrieval database constructed based on spatiotemporal information to obtain a candidate matching feature set, limiting the retrieval scope to... The system extracts a subset of candidate features within a specific spatiotemporal window, significantly reducing the size of the database and improving retrieval efficiency and accuracy, providing a reliable initial screening basis for subsequent collaborative review decisions. Next, it determines whether the candidate matching feature set meets preset triggering conditions. If so, external information is introduced for review decisions and the final recognition result is determined; otherwise, the optimal matching result is used as the final recognition result, effectively reducing the false positive rate and achieving an adaptive balance between operational efficiency and recognition reliability. Finally, the final recognition result and decision information are recorded as training samples to train and optimize the feature extraction network for subsequent pedestrian re-identification tasks, achieving system self-evolution and continuous improvement in cross-view recognition accuracy. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The flowchart illustrates the multi-view online enhanced pedestrian re-identification method provided in this embodiment of the invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention provides a multi-view online enhanced pedestrian re-identification method, such as... Figure 1 As shown, the method includes the following steps:

[0020] S1, based on the acquired target image of the pedestrian to be identified, the timestamp of the acquired target image, and the camera identifier, generates at least one complete image with the same identity label as the target image but a different perspective; this can be understood as: when re-identifying pedestrians in a single target image, generating complete images from multiple angles to supplement pedestrian information. The identity label of the target image refers to the identity identifier of the pedestrian in the target image, used to distinguish different pedestrians.

[0021] Specifically, the completed image is generated through the following steps:

[0022] S101, perform human pose key point detection on the target image to obtain the initial set of pose key point coordinates of the pedestrian to be identified in the target image. In specific implementation, a pose estimation algorithm, such as the OpenPose algorithm, can be used to locate several key points of the human body in the target image and combine them into the head, shoulders, elbows, wrists, hips, knees, and ankles to form an explicit skeleton similar to a stick figure.

[0023] S102, Obtain preset target posture parameters; the target posture parameters include the target viewing direction and the target human body posture; the target viewing direction is at least one of a frontal view, a left-side view, a right-side view, or a rear view; the target human body posture is at least one of a preset standard standing posture, a walking posture, or a sitting posture; it can be understood that the target posture parameters are a pre-set generated target. In specific implementations, the target viewing direction and the target human body posture are discrete posture codes, and implicit posture codes can also be used as an aid to achieve smoother posture adjustment through implicit coding in a continuous vector space.

[0024] S103, the target image, the initial set of key point coordinates of the pose, and the target pose parameters are input into the given generative model, and the given generative model is driven to generate at least one complete image that is consistent with the identity label of the target image and meets the prior constraints of the target pose parameters, with the identity features of the target image being preserved as a constraint; wherein, the given generative model is embedded with prior constraints of human body structure.

[0025] Specifically, the aforementioned prior constraints on human anatomy refer to the knowledge of human physical structure that the generative model learns in advance during the training phase, including but not limited to constraints on the range of motion of human joints, limb proportions, and joint linkages. For example, the bending angle of the arm does not exceed its physiological limits, and the relative positions of the shoulder, elbow, and wrist conform to the geometric constraints of the human skeletal structure. During the generation process, the prior constraints on human anatomy act as spatial regularization terms or generation constraints in the decoding phase of the generative model to prevent ergonomically incorrect images such as joint distortions, disproportionate limbs, or unnatural postures from appearing in the generated images, ensuring the pose rationality and visual realism of the generated completed images.

[0026] Preferably, the given generative model employs a diffusion model or a Generative Adversarial Network (GAN) to perform pose transformation and multi-angle image generation for pedestrians in the target image. Pose transformation refers to changing the person's body posture, such as changing from walking with arms outstretched to standing upright. The given generative model, by learning the latent spatial representation of human pose, can generate images that retain the person's identity but feature entirely new poses and different perspectives, based on specified target pose parameters. Multi-angle image generation refers to the generative model actively supplementing perspectives not captured in the original surveillance footage but potentially appearing in subsequent searches. For example, given a frontal and left-side view image as input, it generates a supplementary right-side and / or rear-view image.

[0027] As described above, by detecting key points of human posture and setting target posture parameters, the precise and controllable generation of the posture and viewpoint of the completed image is achieved, transforming the original single-view query into a multi-view query. This effectively compensates for the lack of human viewpoints in actual monitoring scenarios and reduces the risk of missed detection due to poor target orientation. At the same time, identity preservation constraints and prior human structure constraints are introduced to ensure that the generated completed image conforms to ergonomic rationality while maintaining the original human identity characteristics, avoiding the generation of distorted or deformed images, thereby improving the reliability of subsequent cross-view matching.

[0028] S2, the target image and each completed image are input into a pre-trained feature extraction network to extract image features, and all extracted image features are fused into a joint feature representation. In specific implementations, when extracting image features from different viewpoints, a shared encoder or an independent encoder can be used; using a shared encoder means using the same neural network to extract features from all images, while using an independent encoder means using different neural networks to extract image features from different viewpoints.

[0029] As described above, features are extracted from the original image and the generated completed image and fused into a joint feature representation, enabling a single query image to have multi-view discrimination capability. The fused features can simultaneously characterize the appearance characteristics of the same pedestrian under different viewpoints. For example, it makes it possible to match the front query image with the back database image, and the viewpoint change is transformed from the original interference information into complementary information, which significantly improves the robustness and accuracy of cross-viewpoint matching.

[0030] In one implementation, when using a shared encoder, the training process of the pre-trained feature extraction network includes:

[0031] S10: Obtain the training set for the pedestrian re-identification task containing multiple identity labels, extract the features of the corresponding image samples of each pedestrian in the training set, and perform supervised training using a preset loss function to obtain the preliminarily trained feature extraction network.

[0032] Specifically, the preset loss function employs a joint training approach using a classification loss function and a loss function from metric learning. The classification loss function, such as the cross-entropy loss function, is used to constrain the feature extraction network to output correct identity category predictions. The loss function from metric learning is used to narrow the distance between image sample features of the same identity and widen the distance between sample features of different identities, such as the triplet loss function, thereby obtaining a feature extraction network with preliminary pedestrian discrimination capabilities. Those skilled in the art are familiar with the specific implementation methods of the cross-entropy loss function and the triplet loss function, which will not be elaborated upon here.

[0033] S20: Using a given generative model, generate corresponding multi-view completion images for each image sample in the training set. Construct a cross-view positive sample pair set based on each image sample and its corresponding multi-view completion image. This can be understood as follows: A cross-view positive sample pair refers to forming a positive sample pair by combining the same image sample with each of its generated completion images. That is, when an image sample generates M completion images, a total of M independent cross-view positive sample pairs are constructed. The two images in each cross-view positive sample pair have the same identity label but different perspectives. All cross-view positive sample pairs of all image samples together constitute the cross-view positive sample pair set.

[0034] S30, the feature extraction network initially trained by inputting the cross-view positive sample pair set is optimized using a contrastive learning paradigm. The optimization objective is to ensure that the feature similarity between image samples with the same identity label and their corresponding multi-view completion images approaches 1, and the feature similarity between image samples with different identity labels approaches 0, thus obtaining the trained feature extraction network. Alternatively, the optimization objective can be to ensure that the similarity between the feature representations of image samples with the same identity and their multi-view completion images is not lower than a first preset similarity threshold, and the similarity between the feature representations of image samples with different identities is not higher than a second preset similarity threshold. Those skilled in the art can set the first and second preset similarity thresholds according to actual needs.

[0035] Preferably, the contrastive learning paradigm employs a supervised contrastive loss function. In a specific implementation, for each image sample used as a reference in the cross-view positive sample pair set, the positive sample is any image from any viewpoint with the same identity label as that image sample, and the negative sample is any image with a different identity label than that image sample. The optimization objective is to achieve a feature distribution that "brings similar images closer together and pushes dissimilar images further apart" by minimizing the supervised contrastive loss function. During training, all image sample pairs in the cross-view positive sample pair set participate in the loss calculation. The network parameters of the initially trained feature extraction network are updated through gradient backpropagation, and iterative optimization is performed until the loss function converges, resulting in the trained feature extraction network. Those skilled in the art can set the number of iterations or convergence conditions according to actual needs, which will not be elaborated here.

[0036] As described above, by using a generative model to construct cross-view positive sample pairs for the training set and optimizing them using a contrastive learning paradigm, the feature extraction network learns identity discrimination features from changing viewpoints during the training phase. This significantly improves the network's generalization ability to viewpoint changes and pose differences. Moreover, this training process does not require additional collection of real samples from multiple viewpoints; it can enhance the model's cross-view robustness at low cost by relying solely on generated data, laying a high-quality feature extraction foundation for subsequent multi-view fusion matching.

[0037] Furthermore, all extracted image features are fused into a joint feature representation using one of the following methods:

[0038] Preset fixed weight coefficients are assigned to image features from different perspectives, and the weighted sum of each image feature is obtained to obtain a joint feature representation. This can be understood as: the preset fixed weight coefficients are the proportions of each perspective image feature in the fusion process. For example, the weight coefficients of the target image feature, the frontal completion image feature, the left side completion image feature, the right side completion image feature, and the back completion image feature are preset to 0.3, 0.25, 0.2, 0.15, and 0.1, respectively, and the sum of each weight is 1.

[0039] Each image feature is input into an attention network, which automatically learns the contribution of each viewpoint's image features to the current pedestrian re-identification task. Based on these contributions, fusion weights are generated, and the image features are weighted and summed to obtain a joint feature representation. In a specific implementation, the attention network uses a lightweight neural network module, whose input is the feature vector of each viewpoint image, and whose output is the attention weight score corresponding to each feature.

[0040] Specifically, the contribution refers to the amount of useful information provided by image features from different perspectives in distinguishing pedestrian identities. For example, frontal features typically provide richer details of the face and clothing, thus contributing more, while back features contain relatively less information and contribute less. The attention network automatically learns this contribution assessment during training and generates fusion weights accordingly, allowing high-contribution features to occupy a larger proportion in the fusion process, while reducing the proportion of low-contribution features, thereby achieving adaptive feature fusion.

[0041] The image features are concatenated into a joint feature vector and then input into a multi-layer fully connected network. After nonlinear transformation, the joint feature representation is output. This can be understood as the last fully connected layer of the multi-layer fully connected network outputting a single fused feature vector of a preset dimension.

[0042] The above provides three fusion methods suitable for different scenario requirements: fixed weight fusion has high computational efficiency, attention fusion can adaptively highlight high-contribution perspectives, and splicing mapping fusion can capture complex interaction relationships between perspectives, improving business flexibility. By integrating the features of the original perspective and the completed perspective into a unified joint feature representation, a single query image can have multi-view discrimination capability, significantly improving the robustness of cross-view matching.

[0043] S3, the joint feature representation is compared with the candidate image features in the dynamic retrieval database to calculate similarity, and a candidate matching feature set is obtained based on the similarity ranking result; this can be understood as: according to a preset number of candidates, such as selecting 3 candidate image features to form a candidate matching feature set. When calculating similarity, Euclidean distance or cosine distance can be used.

[0044] Specifically, the dynamic retrieval database is constructed based on the timestamp and camera identifier.

[0045] Furthermore, the construction steps of the dynamic retrieval base database include:

[0046] S301, a spatial retrieval range is determined with the spatial location corresponding to the camera identifier as the center and a preset spatial distance as the radius. A temporal retrieval range is determined with the acquisition time corresponding to the timestamp as the center and a preset time window as the span. Those skilled in the art can preset the spatial distance and preset time window according to the type of the current task and the actual scenario. For example, in a security monitoring scenario, the preset spatial distance can be set to 200 meters, meaning the maximum spatial range that the target person can theoretically move within the preset time window; beyond this range, it is physically impossible for them to appear. The preset time window can be set to 5 minutes, that is, a time span of 2.5 minutes before and after the acquisition time, used to limit the time range of candidate images participating in the retrieval. By limiting the spatial retrieval range and the temporal retrieval range, a spatiotemporal retrieval box is constructed, limiting the retrieval range to a specific spatiotemporal region to filter out candidate images that are spatiotemporally unreasonable.

[0047] S302, Select all pedestrian feature vectors collected by cameras within the spatial retrieval range and whose timestamps are within the temporal retrieval range from the pre-built global feature database, and construct a dynamic retrieval base database; this can be understood as: the dynamic retrieval base database is a temporary retrieval base database dynamically constructed according to the current task.

[0048] Specifically, the global feature database refers to a set of feature vectors of at least one sample image corresponding to each of multiple pedestrian identities that are stored in advance; it can be understood that each record in the global feature database contains at least a pedestrian identity label, an image feature vector, the camera identifier that captured the image, and a timestamp.

[0049] The above-mentioned method constructs a dynamic retrieval database based on spatiotemporal information, which limits the retrieval scope to a subset of candidate features within a specific spatiotemporal window. This significantly reduces the size of the database and improves retrieval efficiency and accuracy. The candidate matching feature set is obtained by ranking similarity, providing a reliable initial screening basis for subsequent collaborative review and decision-making.

[0050] S4. Determine whether the candidate matching feature set meets the preset triggering conditions. If it does, introduce external information to review the candidate matching feature set and determine the final recognition result; otherwise, take the best matching result in the candidate matching feature set as the final recognition result.

[0051] Specifically, the preset triggering condition is at least one of the following:

[0052] The similarity difference between each candidate matching result in the candidate matching feature set is less than a preset difference threshold. The similarity difference is the maximum absolute value of the similarity difference between adjacent candidate matching results in each candidate matching result. That is, when the maximum value is less than the preset difference threshold, it indicates that the similarity difference between two adjacent candidate matching results is small enough that the model cannot make a unique and certain choice, making it difficult to automatically determine the optimal result. In this case, collaborative decision-making is triggered, introducing external information for verification.

[0053] The matching confidence level of the best matching result in the candidate matching feature set is lower than a preset confidence threshold. Those skilled in the art can set the preset confidence threshold according to actual needs, for example, 0.7.

[0054] The current pedestrian re-identification application scenarios are marked as key business scenarios; this can be understood as business application scenarios that require high recognition accuracy and have low error tolerance. Examples include scenarios such as missing persons early warning in elderly care institutions, abnormal patient departure detection in medical care facilities, deployment in key security areas, and exclusive services for VIP customers.

[0055] As described above, by setting trigger conditions, the system can accurately intervene in collaborative decision-making when the model's self-assessment is insufficient or the business risk is high. While ensuring the operational efficiency of normal scenarios, it provides a reliable safety net mechanism for high-risk scenarios. The setting of trigger conditions not only avoids excessively frequent triggering that affects efficiency, but also introduces external information for review and decision-making when the model is unable to make a reliable judgment, effectively reducing the misjudgment rate and achieving an adaptive balance between operational efficiency and recognition reliability.

[0056] Furthermore, the external information includes the contextual reasoning results based on spatiotemporal trajectory constraints and the confirmation or selection instructions input by the human reviewer through the interactive interface based on the contextual reasoning results; this can be understood as: sending the candidate matching feature set optimized after contextual reasoning to the interactive interface for the human reviewer to review on this basis.

[0057] In a specific embodiment, the context reasoning result is obtained through the following steps:

[0058] S401, based on the camera identifier and timestamp corresponding to the target image, and the camera identifier and timestamp corresponding to each candidate matching result in the acquired candidate matching feature set, calculate the reachability probability of each candidate matching result under the preset road network model; the reachability probability is calculated as follows:

[0059] P i =1 / (1+e^(-α×(ΔT i -D i / V))), where Pi Let ΔT be the reachability probability of the i-th candidate matching result. i D is the absolute value of the time difference between the timestamp corresponding to the target image and the timestamp corresponding to the i-th candidate matching result. i Let V be the shortest travel distance between the camera corresponding to the target image and the camera corresponding to the i-th candidate matching result under the preset road network model, where V is the preset average pedestrian speed, α is the preset adjustment parameter, and e is the natural constant.

[0060] Preferably, the preset average pedestrian walking speed is the normal walking speed of pedestrians set according to different application scenarios. For example, in a security monitoring scenario, it can be set to 1.2 meters per second, corresponding to normal walking speed, while in a nursing home care scenario, it can be set to 0.8 meters per second, corresponding to the walking speed of the elderly.

[0061] Specifically, α is related to V, camera density, and the type of the current time period. The formula for calculating α is as follows:

[0062] Where α0 is the preset baseline value, λ1, λ2, and λ3 are preset influence factors that sum to 1, V0 is the preset standard speed, ρ is the current camera spatial density, ρ0 is the preset baseline density, and q is the adjustment factor corresponding to the current time period type. For example, since pedestrian speed is usually greater during the day than at night, the adjustment factor corresponding to the daytime period is greater than the adjustment factor corresponding to the nighttime period.

[0063] As described above, by dynamically linking the steepness of the accessibility probability transition α with the average pedestrian speed, camera spatial density, and time period factor, the spatiotemporal constraint determination can adaptively adapt to the pedestrian movement characteristics, monitoring coverage density, and day-night traffic differences in different scenarios. This avoids misjudgments caused by overly strict or lenient fixed parameters in different scenarios. Without relying on repeated manual parameter adjustments, this dynamic adjustment mechanism enables the system to maintain optimal spatiotemporal inference performance in various application scenarios such as nursing homes, commercial streets, and security monitoring, significantly improving the cross-scenario generalization ability and accuracy of spatiotemporal constraint determination.

[0064] S402, candidate matching results with reachability probabilities lower than a preset reachability threshold are removed from the candidate matching feature set. The reachability probabilities of the remaining candidate matching results are used as weight coefficients and weighted and fused with the similarity corresponding to each candidate matching result to obtain a fusion score for each candidate matching result. This can be understood as multiplying the reachability probability by the similarity to obtain the fusion result. The preset reachability threshold is set in advance by those skilled in the art based on actual needs, for example, 0.4.

[0065] S403, the candidate results are reordered from high to low according to the fusion score to obtain a candidate matching feature set optimized by contextual reasoning. In specific implementation, the optimized candidate matching feature set is used as the input basis for the final confirmation by the human reviewer. The human reviewer simultaneously views the visual comparison and spatiotemporal reasoning annotations of the images corresponding to the candidate matching features on the interactive interface, such as the distance between cameras, time difference, and accessibility probability. Based on the complete contextual information, the human reviewer inputs a confirmation or selection command to achieve the final decision-making through human-machine collaboration.

[0066] The above-mentioned approach, by introducing accessibility probability calculation based on a road network model to perform soft screening and ranking optimization of candidate matching results, enables candidates that are physically reachable in time to receive higher ranking weights, while effectively eliminating candidates that are spatiotemporally inaccessible. This significantly improves the spatiotemporal rationality of cross-camera matching. Furthermore, the approach combines contextual reasoning results with manual review, providing intuitive spatiotemporal information for human decision-making. This forms a multi-level review mechanism of machine initial screening, spatiotemporal filtering, and manual confirmation, which greatly improves the accuracy and interpretability of decision-making in key scenarios while ensuring recognition efficiency.

[0067] S5. Record the final recognition result and the corresponding decision information as training samples. When the number of accumulated training samples reaches a preset threshold or a preset time node is reached, perform training optimization on the feature extraction network and use the optimized network parameters for subsequent pedestrian re-identification tasks.

[0068] Specifically, the final identification result and corresponding decision information are all the supervised information generated during the output process in step S4, including at least one of the following: confirmed correct matching pairs, misjudged cases corrected after manual review, the distribution and similarity data of candidate matching feature sets when triggering the review decision, and the discrimination criteria related to spatiotemporal constraints. The above information together constitutes training samples with explicit supervision labels, where confirmed correct matching pairs are used as positive samples, and candidate matching results incorrectly selected by the model in corrected misjudged cases are used as hard negative samples. That is, confirmed correct matching pairs refer to image pairs consisting of target images and images corresponding to candidate matching features that are confirmed to belong to the same identity; misjudged cases refer to image pairs consisting of images corresponding to candidate matching features that are manually excluded in the review decision, have a similarity to the target image higher than a preset confusion threshold, but have different identities.

[0069] Furthermore, the preset threshold for the number of accumulated training samples is set according to the actual application scenario. For example, it can be set to trigger training optimization once every 500 accumulated feedback samples. If the sample accumulation speed is slow, a timed triggering strategy can be adopted, such as performing training optimization once every 24 hours. The two triggering methods are independent of each other, and the training process is triggered when either condition is met.

[0070] The above-mentioned approach uses the final identification results and decision information as training samples and triggers incremental optimization of the feature extraction network when conditions are met. This enables the system to continuously learn and evolve from actual business feedback, and the optimized network parameters are directly used for subsequent pedestrian re-identification tasks. This forms a closed-loop mechanism of identification, feedback, optimization, and re-identification, realizing the self-evolutionary capability of the system performance to continuously improve with the accumulation of business data, thereby gradually improving the accuracy of pedestrian re-identification.

[0071] In one specific embodiment, step S5 further includes updating the feature layer and the decision layer; wherein,

[0072] The feature layer update includes: adding the feature vector of the target image corresponding to the final recognition result to the corresponding identity label in the dynamic retrieval base database as new data for the corresponding pedestrian, and / or correcting the incorrectly labeled identity labels and corresponding feature vectors in the dynamic retrieval base database. For example, when the same pedestrian is confirmed to be identified by multiple cameras and at multiple time points, the number of its feature vectors in the global feature database continuously increases, enriching the features from different perspectives, thereby improving the recall rate of subsequent retrievals; when the identity label of any candidate image in the dynamic retrieval base database is confirmed to be incorrect by manual review, the identity label corresponding to the feature vector of that image is corrected to the correct identity label, and its corresponding feature vector index is updated to avoid the erroneous sample from continuously causing mismatches in subsequent retrievals.

[0073] The update of the decision layer includes: adjusting the preset difference threshold and / or preset reliability threshold based on the statistical analysis results of historical cases that trigger review decisions and confirmed misjudgment cases in historical cases; and increasing the weight of the spatiotemporal information in the ranking of candidate matching results based on the contribution of spatiotemporal information to the discrimination of candidate matching results in the statistical analysis results.

[0074] In one implementation, the contribution of the spatiotemporal information is the difference between the ranking accuracy based solely on visual similarity and the ranking accuracy after fusing spatiotemporal information, obtained by statistically analyzing historically manually confirmed cases.

[0075] Specifically, based on the statistical analysis results, the preset difference threshold and / or preset reliability threshold are adjusted according to preset adjustment methods. These preset adjustment methods include, but are not limited to: manual parameter tuning; adaptive algorithms, which automatically adjust the thresholds up or down according to a preset step size based on the historical trend of the false positive rate or trigger rate, such as ±0.02 each time; and optimization target-driven parameter tuning, which uses a preset optimization target, such as a trigger rate ≤20% and a false positive rate ≤5%, to automatically find the optimal threshold combination using grid search or Bayesian optimization. One application scenario is as follows: If, after analyzing all cases that triggered collaboration in the past month, it is found that 80% of the misjudgments occurred between 0.6 and 0.7, the preset confidence threshold is increased from 0.5 to 0.7 to cover more high-risk cases; if the similarity difference between the best candidate matching result and the second-ranked candidate matching result in the misjudged cases is found to be concentrated between 0.05 and 0.1, the preset difference threshold is increased from 0.03 to 0.1 to capture more cases where the model output is uncertain; if it is found that the misjudgment rate is much higher at night than during the day, the preset confidence threshold for this scenario is adjusted accordingly.

[0076] As described above, the feature layer updates enable real-time correction and enrichment of the base database, allowing correct matching results to take effect immediately in subsequent searches. The decision layer updates adaptively adjust the trigger threshold and spatiotemporal weights based on historical case statistical analysis, continuously optimizing the trigger accuracy and ranking rationality of collaborative decision-making. Through fast feedback from the feature layer, medium feedback from the decision layer, and slow feedback from the model layer, a complementary feedback mechanism is formed, collaboratively driving the continuous improvement of system performance at different time scales.

[0077] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiments.

[0078] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0079] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A multi-view online augmented pedestrian re-identification method, characterized in that, The method includes the following steps: S1, based on the acquired target image of the pedestrian to be identified and the timestamp and camera identifier of the captured target image, generate at least one complete image that is consistent with the identity label of the target image but has a different perspective; S2, input the target image and each completed image into the pre-trained feature extraction network to extract image features, and fuse all extracted image features into a joint feature representation; S3, calculate the similarity between the joint feature representation and the candidate image features in the dynamic retrieval base database, and obtain the candidate matching feature set based on the similarity ranking result; The dynamic retrieval database is constructed based on the timestamp and camera identifier; S4. Determine whether the candidate matching feature set meets the preset triggering conditions. If it does, introduce external information to review and decide on the candidate matching feature set and determine the final recognition result. Otherwise, the optimal matching result in the candidate matching feature set is taken as the final recognition result; the external information includes the context reasoning result based on spatiotemporal trajectory constraints and the confirmation or selection instruction input by the human reviewer through the interactive interface based on the context reasoning result; wherein, the context reasoning result is obtained through the following steps: S401, based on the camera identifier and timestamp corresponding to the target image, and the camera identifier and timestamp corresponding to each candidate matching result in the obtained candidate matching feature set, the accessibility probability of each candidate matching result is calculated under the preset road network model; the accessibility probability is calculated as follows: P i =1 / (1+e^(-α×(ΔT i -D i / V))), where P i Let ΔT be the reachability probability of the i-th candidate matching result. i D is the absolute value of the time difference between the timestamp corresponding to the target image and the timestamp corresponding to the i-th candidate matching result. i Let V be the shortest travel distance between the camera corresponding to the target image and the camera corresponding to the i-th candidate matching result under the preset road network model, V be the preset average pedestrian speed, α be the preset adjustment parameter, and e be the natural constant; S402, remove candidate matching results with reachability probabilities lower than the preset reachability threshold from the candidate matching feature set, and use the reachability probabilities of each remaining candidate matching result as weight coefficients to perform weighted fusion with the similarity corresponding to each candidate matching result to obtain the fusion score of each candidate matching result; S403, reorder each candidate result according to the fusion score from high to low to obtain the candidate matching feature set optimized by context reasoning; S5. Record the final recognition result and the corresponding decision information as training samples. When the number of accumulated training samples reaches a preset threshold or a preset time node is reached, perform training optimization on the feature extraction network and use the optimized network parameters for subsequent pedestrian re-identification tasks.

2. The multi-view online enhanced pedestrian re-identification method according to claim 1, characterized in that, In step S1, the completed image is generated through the following steps: S101, Perform human posture key point detection on the target image to obtain the initial set of posture key point coordinates of the pedestrian to be identified in the target image; S102, Obtain preset target posture parameters; the target posture parameters include target viewpoint direction and target human posture; The target viewing angle is at least one of a frontal view, a left-side view, a right-side view, or a rear view; the target human posture is at least one of a preset standard standing posture, walking posture, or sitting posture. S103, the target image, the initial set of key point coordinates of the pose, and the target pose parameters are input into the given generative model, and the given generative model is driven to generate at least one complete image that is consistent with the identity label of the target image and meets the prior constraints of the target pose parameters, with the identity features of the target image being preserved as a constraint; wherein, the given generative model is embedded with prior constraints of human body structure.

3. The multi-view online enhanced pedestrian re-identification method according to claim 1, characterized in that, The training process of the pre-trained feature extraction network includes: S10: Obtain the training set for the pedestrian re-identification task containing multiple identity labels, extract the features of the corresponding image samples of each pedestrian in the training set, and use a preset loss function for supervised training to obtain the preliminary trained feature extraction network. S20: Using a given generative model, generate corresponding multi-view completion images for each image sample in the training set, and construct a cross-view positive sample pair set based on each image sample and the corresponding multi-view completion image. S30. Input the cross-view positive sample pair set into the initially trained feature extraction network, and optimize it using the contrastive learning paradigm. The optimization objective is to make the feature similarity between image samples with the same identity label and the corresponding multi-view completion image approach 1, and the feature similarity between image samples with different identity labels approach 0, so as to obtain the trained feature extraction network.

4. The multi-view online enhanced pedestrian re-identification method according to claim 1, characterized in that, In step S2, all extracted image features are fused into a joint feature representation using one of the following methods: Preset fixed weight coefficients are assigned to image features from different perspectives, and the weighted sum of each image feature is obtained to obtain a joint feature representation; Each image feature is input into an attention network, which automatically learns the contribution of each viewpoint's image features to the current pedestrian re-identification task. Based on the contribution, a fusion weight is generated, and each image feature is weighted and summed to obtain a joint feature representation. The image features are concatenated into a joint feature vector and then input into a multi-layer fully connected network. After nonlinear transformation, the joint feature representation is output.

5. The multi-view online enhanced pedestrian re-identification method according to claim 1, characterized in that, The steps for constructing the dynamic retrieval base database include: S301, the spatial retrieval range is determined with the spatial location corresponding to the camera identifier as the center and the preset spatial distance as the radius, and the time retrieval range is determined with the collection time corresponding to the timestamp as the center and the preset time window as the span; S302, select all pedestrian feature vectors collected by cameras within the spatial retrieval range and whose timestamps are within the temporal retrieval range from the pre-constructed global feature database, and construct a dynamic retrieval base database; the global feature database refers to a set of feature vectors of at least one sample image corresponding to each of multiple pedestrian identities that are pre-stored.

6. The multi-view online enhanced pedestrian re-identification method according to claim 1, characterized in that, The preset triggering condition mentioned in step S4 is at least one of the following: The similarity difference between each candidate matching result in the candidate matching feature set is less than a preset difference threshold; wherein, the similarity difference is the maximum value among the absolute values ​​of the similarity differences between adjacent ranked candidate matching results in each candidate matching result. The matching confidence score corresponding to the best matching result in the candidate matching feature set is lower than the preset confidence threshold. The current pedestrian re-identification application scenario is marked as a key business scenario.

7. The multi-view online enhanced pedestrian re-identification method according to claim 6, characterized in that, Step S5 also includes updating the feature layer and the decision layer; among which, The feature layer update includes: adding the feature vector of the target image corresponding to the final recognition result to the corresponding identity tag in the dynamic retrieval base as new data for the corresponding pedestrian, and / or correcting the incorrectly labeled identity tags and corresponding feature vectors in the dynamic retrieval base; The update of the decision layer includes: adjusting the preset difference threshold and / or preset reliability threshold based on the statistical analysis results of historical cases that trigger review decisions and confirmed misjudgment cases in historical cases; and increasing the weight of the spatiotemporal information in the ranking of candidate matching results based on the contribution of spatiotemporal information to the discrimination of candidate matching results in the statistical analysis results.

8. A non-transitory computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the multi-view online enhanced pedestrian re-identification method as described in any one of claims 1-7.

9. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 8.

Images