Online pedestrian re-identification method based on multi-feature cyclic confirmation and system thereof

Abstract

The application provides an online pedestrian re-identification method based on multi-feature cycle confirmation and a system thereof, which comprises a target tracking step: obtaining historical coordinate information of pedestrian movement in a monitoring video; correlating and tracking detection results, and outputting a prediction result containing a possible position of the pedestrian at a next time; a pedestrian re-identification step: performing feature extraction on a pedestrian image to obtain a feature vector of human multi-dimensional features, performing feature matching on the newly extracted feature vector and a feature vector stored in a database, and returning an optimal matching result; a cycle confirmation step: maintaining a matching result queue Q and a counter dictionary C for each pedestrian, and updating the occurrence frequency of each matching ID; setting a continuous matching threshold T1 and a maximum matching number threshold T2, confirming the identity of the pedestrian according to the count in C and the continuous matching condition in Q. The application can improve the reliability and accuracy of pedestrian re-identification in a real-time video monitoring system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent video monitoring, in particular to an online pedestrian re-identification method based on multi-feature cyclic confirmation and a system applying the method. BACKGROUND

[0002] In the modern urban security monitoring system, with the expansion of the city size and the increase of public security demand, video monitoring has become an important means to maintain social order. However, the traditional video monitoring system mainly relies on manual monitoring, which is not only inefficient, but also easy to miss key information. In order to improve the monitoring efficiency and accuracy, pedestrian re-identification (ReID) technology has gradually become a research hotspot in this field. Pedestrian re-identification technology aims to accurately and quickly identify the same pedestrian in the pedestrian images captured under different monitoring cameras through computer vision and pattern recognition algorithms, which is of great significance for criminal tracking, missing person searching, abnormal behavior monitoring and other application scenarios.

[0003] Currently, pedestrian re-identification technology mainly relies on deep learning models. These models learn the feature representation of pedestrians by training on large-scale pedestrian image datasets, and identify pedestrians through feature matching in the test phase. However, although deep learning models have made significant progress in pedestrian re-identification tasks, there are still many challenges and limitations. First, pedestrians in the real world have high diversity and unpredictability, including changes in clothing, posture, perspective, lighting conditions and other factors, which greatly reduce the performance of the trained model in actual application. In addition, in a multi-camera real-time video monitoring system, a pedestrian may be captured by multiple cameras at the same time, and the image lighting conditions, perspectives under each camera are different, in addition, there are complex backgrounds and pedestrian occlusions, which increase the difficulty of identification. Even through the naked eye, it needs to be compared several times to accurately judge, and the recognition effect of the traditional recognition method in the real-time dynamic environment will be greatly reduced. Therefore, it is urgent to develop a new method to improve the accuracy of pedestrian re-identification and meet the actual needs of modern urban security monitoring. SUMMARY

[0004] In order to solve the problem of low recognition accuracy of traditional pedestrian re-identification algorithm in real-time video monitoring environment, the purpose of the present application is to provide an online pedestrian re-identification method based on multi-feature cyclic confirmation and a system thereof, which improves the reliability and accuracy of pedestrian re-identification in real-time video monitoring system through a series of continuous and dynamic identification and verification steps.

[0005] The present application achieves the above-mentioned purposes through the following technical solutions:

[0006] An online pedestrian re-identification method based on multi-feature cyclic confirmation, comprising:

[0007] Target tracking step: Obtain the historical coordinate information of the pedestrian motion in the monitoring video through the YOLO target detection algorithm, which includes the bounding box coordinates (x, y, w, h), the confidence score and the category; use the target tracking algorithm ByteTrack to associate and track the detection results to realize the prediction of the continuous motion trajectory of the pedestrian in the video, and output the prediction result containing the possible position of the pedestrian at the next moment;

[0008] Pedestrian re-identification step: receive the pedestrian image frame transmitted by the target tracking step, extract the features of the pedestrian image by using the deep learning model, obtain the feature vector of the multi-dimensional features of the human body, match the newly extracted feature vector with the feature vector stored in the database, and return the matching result most similar to the feature vector in the database as the optimal matching result;

[0009] Cycle confirmation step: set the sliding window size N for recording the last N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the frequency of each matching ID; set the continuous matching threshold T1 and the maximum matching number threshold T2, confirm the identity of the pedestrian according to the count in C and the continuous matching in Q; wherein the continuous matching threshold T1 is used to measure the stability of a matching ID in continuous matching for multiple times, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

[0010] According to the online pedestrian re-identification method based on multi-feature cycle confirmation provided by the application, when the target tracking algorithm ByteTrack is used to predict the coordinates at the next moment, the following steps are specifically executed:

[0011] Define the detection result set D={d1,d2,...,dm}, wherein each detection result di is represented as a five-tuple (xi,yi,wi,hi,si), which respectively represents the center point horizontal coordinate, the center point vertical coordinate, the width, the height and the confidence score of the detection frame;

[0012] Define the tracking result set T={t1,t2,...,tn}, wherein each tracking result ti is also represented as a five-tuple (xi,yi,wi,hi,vi), which respectively represents the center point horizontal coordinate, the center point vertical coordinate, the width, the height and the speed or state information of the target of the tracking frame;

[0013] Calculate the association matrix C, wherein the element Cij=IoU(di,tj) represents the intersection over union IoU between the detection result di and the tracking result tj;

[0014] The Hungarian algorithm is used for matching the association matrix C to obtain an optimal association result, that is, optimal pairing of the detection result and the tracking result;

[0015] For the unmatched detection result, if the confidence score si is greater than a preset threshold, a new tracking track is created, otherwise, secondary matching is performed.

[0016] The online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the application further comprises:

[0017] The secondary matching comprises:

[0018] For the detection result that fails to be successfully paired in the first matching, similarity evaluation is performed on the existing track in the tracking result set T;

[0019] A similarity threshold β is set, wherein 0<β<1, which is used to determine whether the similarity between the detection result and the existing track is sufficient to establish association;

[0020] If the similarity between a certain detection result and a certain existing track is greater than or equal to the similarity threshold β, the detection result is associated with the track, the state information of the track is updated, and the detection result is removed from the matching queue;

[0021] If after the secondary matching, the detection result fails to be successfully associated with any track, whether to create a new track based on the detection result or ignore it as noise data is determined according to the system strategy.

[0022] In the pedestrian re-identification step, the deep learning model OSNet is used for feature extraction of the pedestrian image, comprising:

[0023] The input image is subjected to initial convolutional layer for preliminary feature extraction;

[0024] The image features pass through four OSNet blocks, each of which contains multiple lightweight convolution units LCA, and the lightweight convolution unit LCA extracts multi-scale features of the image through the combination of point-by-point convolution, depth separable convolution and 1x1 convolution;

[0025] The output of each OSNet block is subjected to channel attention mechanism processing to enhance the attention degree to important features;

[0026] After global average pooling and full connection layer processing, a feature vector with a dimension of 512 is obtained, which includes the overall appearance, local details and semantic information of the pedestrian;

[0027] The extracted feature vector is used to compare with existing features in the database, and the similarity or distance between the feature vectors is calculated to realize the re-identification of pedestrians.

[0028] According to the online pedestrian re-identification method based on multi-feature cycle confirmation provided by the application, for the newly extracted feature vector, first, normalization processing is performed to ensure the comparability between different feature vectors.

[0029] The cosine similarity or other measurement methods are used to calculate the similarity between the new feature vector and the feature vectors in the database.

[0030] According to the size of the similarity, the feature vectors in the database are sorted, and several feature vectors with the highest similarity or the smallest distance are selected as candidate matching results.

[0031] The candidate matching results are further verified, and after verification, the new feature vector is associated with the corresponding pedestrian identity information.

[0032] According to the online pedestrian re-identification method based on multi-feature cycle confirmation provided by the application, the cosine similarity is used to calculate the similarity between the feature vectors, which specifically includes the following steps:

[0033] The stored feature vector V1 and the newly extracted feature vector V2 are obtained from the database, wherein V1 and V2 are vectors with the same dimension;

[0034] The dot product of the feature vectors V1 and V2, i.e. V1·V2, is calculated, which is used to reflect the directional similarity of the two vectors in the multi-dimensional space;

[0035] The norm of the feature vectors V1 and V2, i.e. ||V1|| and ||V2||, is calculated, which represents the length of the vector in the multi-dimensional space;

[0036] The cosine similarity formula is used to calculate the similarity between V1 and V2, which is represented by the following formula:

[0037] similarity=cos(θ)=(V1·V2) / (||V1||*||V2||)

[0038] Wherein, θ is the included angle between V1 and V2.

[0039] According to the calculated similarity value, all feature vectors in the database are sorted, the closer the similarity value is to 1, the more similar the two feature vectors are, i.e. the corresponding pedestrian identities are more likely to be the same; the feature vector with the highest similarity is selected as the optimal matching result.

[0040] The application provides an online pedestrian re-identification method based on multi-feature cyclic confirmation, and the cyclic confirmation step specifically comprises the following steps:

[0041] A sliding window size N is set to record and store the latest N matching results of each pedestrian;

[0042] A matching result queue Q is maintained for each pedestrian, and the queue is used to sequentially store the latest matching results; meanwhile, a counter dictionary C is maintained to record the number of times each matching ID appears in the queue Q;

[0043] When a new matching result is generated, it is added to the matching result queue Q, and the length of the queue Q is checked; if the length of the queue Q exceeds the set sliding window size N, the oldest matching result in the queue is automatically removed to keep the length of the queue constant;

[0044] The number of times the corresponding matching ID appears in the counter dictionary C is updated according to the new matching result;

[0045] Two judgment thresholds are set: a continuous matching threshold T1 and a maximum matching number threshold T2, and the identity of the pedestrian is confirmed according to the following judgment conditions:

[0046] If there is a matching ID in the counter dictionary C whose count is greater than or equal to the continuous matching threshold T1, and the ID is the continuous matching result of the latest T1 matching results, the ID is directly confirmed as the identity of the pedestrian; if the continuous matching condition is not met, but the length of the matching result queue Q reaches the maximum matching number threshold T2, the matching ID with the highest count in the counter dictionary C is selected as the identity of the pedestrian for confirmation.

[0047] According to the online pedestrian re-identification method based on multi-feature cyclic confirmation, the confirmation result is fed back to the system after the identity of the pedestrian is confirmed, and the historical re-identification record of the pedestrian is updated;

[0048] For the matching IDs with lower counts or long time of non-appearance in the counter dictionary C, periodic cleaning or reduction of the priority is performed to reduce the false matching and improve the confirmation efficiency;

[0049] According to the requirements of the actual application scene, the values of the sliding window size N, the continuous matching threshold T1 and the maximum matching number threshold T2 are dynamically adjusted to adapt to the pedestrian re-identification requirements in different scenes;

[0050] During the identity confirmation, other auxiliary information is combined to further improve the accuracy of the identity confirmation;

[0051] For the case where there are multiple matching IDs with high similarity, an additional verification method is used to assist in determining the final identity confirmation result.

[0052] According to the online pedestrian re-identification method based on multi-feature cycle confirmation provided by the application, the historical coordinate information of pedestrian movement in the monitoring video is obtained through a YOLO target detection algorithm, and the method comprises the following steps:

[0053] Video frame images are obtained from the monitoring video in time sequence frame by frame or at a predetermined interval;

[0054] A YOLO target detection algorithm is applied to each frame of image to detect pedestrians in the image;

[0055] For each detected pedestrian, the boundary box coordinates thereof are extracted, the coordinates comprising the position (x, y) of the pedestrian in the image and the width w and height h of the boundary box; the confidence score corresponding to each detected pedestrian is recorded; the category of each detected target is confirmed as "pedestrian", and the category information is recorded;

[0056] The boundary box coordinates, confidence scores and category information of the detected pedestrians in each frame are combined in time sequence to construct a historical coordinate information sequence of the pedestrians;

[0057] The constructed historical coordinate information sequence is stored in a database, and an interface or method is provided to retrieve the historical coordinate information of a specific pedestrian or a specific time period as needed.

[0058] An online pedestrian re-identification system based on multi-feature cycle confirmation comprises:

[0059] A video stream analysis and target tracking module: historical coordinate information of pedestrian movement in the monitoring video is obtained through a YOLO target detection algorithm, and the information comprises boundary box coordinates (x, y, w, h), confidence scores and categories; a target tracking algorithm ByteTrack is used to associate and track the detection results to realize the prediction of the continuous movement track of the pedestrian in the video, and a prediction result containing the possible position of the pedestrian at the next time is output;

[0060] A pedestrian re-identification module: receives the pedestrian image frame transmitted by the target tracking step, extracts features of the pedestrian image by using a deep learning model to obtain a feature vector of the multi-dimensional features of the human body, matches the newly extracted feature vector with the feature vector stored in the database, and returns the matching result most similar to the feature vector in the database as the optimal matching result;

[0061] The cycle confirmation module sets a sliding window size N to record the last N matching results, maintains a matching result queue Q and a counter dictionary C for each pedestrian, and updates the occurrence frequency of each matching ID; a continuous matching threshold T1 and a maximum matching number threshold T2 are set, and the identity of the pedestrian is confirmed according to the count in C and the continuous matching in Q; wherein the continuous matching threshold T1 is used to measure the stability of a certain matching ID in continuous matching, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

[0062] Therefore, compared with the prior art, the present application has the following beneficial effects:

[0063] 1、The present application can comprehensively utilize the appearance, texture, color and other feature information of pedestrians by adopting multi-feature matching technology, effectively dealing with complex situations such as illumination changes, occlusions and posture changes of pedestrians under different camera angles. At the same time, the cycle confirmation mechanism further enhances the stability and accuracy of recognition, so that the accuracy of pedestrian recognition in the actual deployed system can be stably reached above 95%, far exceeding the average level of the prior art.

[0064] 2、The present application optimizes the algorithm logic and data processing flow in depth, ensuring that even in a large monitoring system containing a large number of cameras, it can also process and analyze massive video data in real time. This efficient data processing capability makes the present application particularly suitable for resource-limited and fast-response security monitoring scenarios such as city streets, public places and transportation hubs.

[0065] 3、By accurately identifying pedestrians and tracking their movement trajectories, the present application provides a more comprehensive and accurate information base for the monitoring system, which not only helps to timely discover and respond to potential security threats, but also provides strong data support for urban management and planning. Therefore, the implementation of the present application significantly enhances the overall performance of the monitoring system, making it better serve the security monitoring needs of modern cities.

[0066] 4、Since the present application does not depend on a specific hardware or software platform and can be easily integrated into existing monitoring systems, it has high adaptability and flexibility. This makes the present application widely applicable to various types of monitoring scenarios, providing more reliable and efficient solutions for urban safety, public safety and other fields.

[0067] In summary, by introducing multi-feature matching and cycle confirmation mechanism, the present application not only significantly improves the accuracy of pedestrian recognition, but also realizes real-time and efficient data processing, enhances the overall performance of the monitoring system, and enhances the adaptability and flexibility of the system.

[0068] The present application will be further described in detail below in conjunction with the drawings and specific embodiments. Attached Figure Description

[0069] Figure 1 This is a flowchart of an embodiment of an online pedestrian re-identification method based on multi-feature cyclic confirmation according to the present invention.

[0070] Figure 2 This is a flowchart illustrating an embodiment of an online pedestrian re-identification method based on multi-feature cyclic confirmation according to the present invention.

[0071] Figure 3 This is a schematic diagram of an embodiment of an online pedestrian re-identification system based on multi-feature cyclic confirmation according to the present invention. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0073] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0074] An Example of an Online Pedestrian Re-identification Method Based on Multi-Feature Cyclic Confirmation

[0075] See Figure 1 and Figure 2 This embodiment provides an online pedestrian re-identification method based on multi-feature cyclic verification, including the following steps:

[0076] Target tracking steps: Obtain historical coordinate information of pedestrian movement in the surveillance video using the YOLO target detection algorithm. This information includes bounding box coordinates (x, y, w, h), confidence score, and category. Use the ByteTrack target tracking algorithm to correlate and track the detection results to predict the continuous movement trajectory of pedestrians in the video and output the prediction result containing the possible position of pedestrians at the next moment.

[0077] The pedestrian re-identification step: receiving the pedestrian image frame transmitted by the target tracking step, using a deep learning model to extract features of the pedestrian image, obtaining a feature vector of the human multi-dimensional feature, performing feature matching between the newly extracted feature vector and the feature vector stored in the database, and returning the most similar matching result in the database as the optimal matching result.

[0078] The cycle confirmation step: setting a sliding window size N for recording the last N matching results, maintaining a matching result queue Q and a counter dictionary C for each pedestrian, and updating the frequency of each matching ID; setting a continuous matching threshold T1 and a maximum matching number threshold T2, confirming the identity of the pedestrian according to the count in C and the continuous matching in Q; wherein the continuous matching threshold T1 is used to measure the stability of a matching ID in continuous matching, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

[0079] In the above target tracking step, when using the target tracking algorithm ByteTrack to predict the coordinates of the next time, the following steps are specifically executed:

[0080] Define a detection result set D = {d1, d2,..., dm}, where each detection result di is represented as a five-tuple (xi, yi, wi, hi, si), representing the center point horizontal coordinate, center point vertical coordinate, width, height, and confidence score of the detection box, respectively;

[0081] Define a tracking result set T = {t1, t2,..., tn}, where each tracking result ti is also represented as a five-tuple (xi, yi, wi, hi, vi), representing the center point horizontal coordinate, center point vertical coordinate, width, height, and speed or state information of the target, respectively;

[0082] Calculate the association matrix C, where the element Cij = IoU(di, tj) represents the intersection over union IoU between the detection result di and the tracking result tj;

[0083] Use the Hungarian algorithm to match the association matrix C to obtain the optimal association result, i.e., the optimal pairing of detection results and tracking results;

[0084] For unmatched detection results, if the confidence score si is greater than a preset threshold, a new tracking track is created; otherwise, secondary matching is performed.

[0085] It can be seen that the core idea of the ByteTrack in the embodiment is to use low-confidence detection results to associate the occluded targets. The specific formula is as follows: the detection result is D={d1, d2,..., dm}, wherein di=(xi, yi, wi, hi, si), the tracking result is T={t1, t2,..., tn}, wherein ti=(xi, yi, wi, hi, vi), and the association matrix C is obtained by calculating the IOU distance of the detection result and the tracking result: Cij=IoU(di, tj). ByteTrack uses the Hungarian algorithm to match C to obtain the optimal association result. For the unmatched detection result, if si is greater than a threshold, a new track is created; otherwise, secondary matching is performed.

[0086] In the embodiment, the secondary matching specifically includes the following steps:

[0087] For the detection results that fail to be successfully paired in the first matching, similarity evaluation is performed on the existing tracks in the tracking result set T;

[0088] A similarity threshold β is set, wherein 0<β<1, which is used to judge whether the similarity between the detection result and the existing track is sufficient to establish association;

[0089] If the similarity of a certain detection result and a certain existing track is greater than or equal to the similarity threshold β, the detection result is associated with the track, the state information of the track is updated, and the detection result is removed from the matching queue;

[0090] If there are still detection results that fail to be successfully associated with any track after secondary matching, it is decided according to the system strategy whether to create a new track based on these detection results or to ignore them as noise data.

[0091] In the pedestrian re-identification step, the pedestrian re-identification module receives the uploaded pedestrian image frame, and extracts a feature vector of multiple features such as faces and bodies by using a deep learning model such as OSNet. The OSNet has the characteristics of using a lightweight attention mechanism and multi-scale feature fusion, and specifically includes:

[0092] The input image is subjected to initial convolutional layer for preliminary feature extraction;

[0093] The image features pass through 4 OSNet blocks, and each OSNet block internally includes a plurality of lightweight convolution units LCA. The lightweight convolution unit LCA extracts multi-scale features of the image by combining point-by-point convolution, depth separable convolution and 1x1 convolution;

[0094] The output of each OSNet block is subjected to channel attention mechanism processing to enhance the attention degree to important features;

[0095] After global average pooling and full connection layer processing, a feature vector with a dimension of 512 is obtained, which includes the overall appearance, local details and semantic information of the pedestrian;

[0096] After successful extraction, the extracted feature vector is used to compare with the existing features in the database, and the similarity or distance between the feature vectors is calculated to realize pedestrian re-identification.

[0097] In the comparison process, efficient indexing and retrieval technology is adopted to speed up the matching process of the feature vector and improve the real-time performance and accuracy of pedestrian re-identification.

[0098] In this embodiment, for the newly extracted feature vector, first, normalization processing is performed to ensure comparability between different feature vectors;

[0099] Cosine similarity, Euclidean distance or Hamming distance and other measurement methods are used to calculate the similarity between the new feature vector and the feature vectors in the database;

[0100] According to the size of the similarity, the feature vectors in the database are sorted, and the feature vectors with the highest similarity or the smallest distance are selected as the candidate matching results;

[0101] The candidate matching results are further verified, such as through face comparison, clothing color and texture analysis, etc., to improve the accuracy of pedestrian re-identification; After verification, the new feature vector is associated with the corresponding pedestrian identity information and updated to the database for subsequent tracking and identification tasks.

[0102] In this embodiment, cosine similarity is used to calculate the similarity between feature vectors, which includes the following steps:

[0103] Get the stored feature vector V1 from the database and the newly extracted feature vector V2, where V1 and V2 are vectors with the same dimension;

[0104] Calculate the dot product of feature vectors V1 and V2, i.e. V1·V2, which reflects the directional similarity of two vectors in multi-dimensional space;

[0105] Calculate the norm of feature vectors V1 and V2, i.e. ||V1|| and ||V2||, which represents the length of the vector in multi-dimensional space;

[0106] Calculate the similarity between V1 and V2 using the cosine similarity formula, represented by the following formula:

[0107] similarity=cos(θ)=(V1·V2) / (||V1||*||V2||)

[0108] wherein θ is the angle between V1 and V2;

[0109] According to the calculated similarity values, all the feature vectors in the database are sorted, and the closer the similarity value is to 1, the more similar the two feature vectors are, that is, the corresponding pedestrian identities are more likely to be the same; the feature vector with the highest similarity is selected as the optimal matching result.

[0110] Through the above steps, the embodiment can efficiently and accurately calculate the similarity between the feature vectors and return the optimal pedestrian matching result, thereby providing reliable technical support for the pedestrian re-identification task.

[0111] In the embodiment, the loop confirmation step specifically includes:

[0112] A sliding window size N is set to record and store the last N matching results of each pedestrian; a matching result queue Q is maintained for each pedestrian, which is used to store the latest matching results in order, and a counter dictionary C is maintained to record the number of occurrences of each matching ID in the queue Q;

[0113] When a new matching result is generated, it is added to the matching result queue Q, and the length of the queue Q is checked; if the length of the queue Q exceeds the set sliding window size N, the oldest matching result in the queue is automatically removed to keep the queue length constant; the number of occurrences of the corresponding matching ID in the counter dictionary C is updated according to the new matching result; two judgment thresholds are set: a continuous matching threshold T1 and a maximum matching number threshold T2, and the identity of the pedestrian is confirmed according to the following judgment conditions:

[0114] If there is a matching ID in the counter dictionary C whose count is greater than or equal to the continuous matching threshold T1, and the ID is the continuous matching result of the last T1 matching results, the ID is directly confirmed as the identity of the pedestrian; if the continuous matching condition is not met, but the length of the matching result queue Q reaches the maximum matching number threshold T2, the matching ID with the highest count in the counter dictionary C is selected as the identity of the pedestrian for confirmation. This strategy considers the reliability of continuous matching, and limits the decision time through the maximum matching number, thereby balancing the accuracy and real-time performance.

[0115] After confirming the identity of the pedestrian, the confirmation result is fed back to the system, and the historical re-identification record of the pedestrian is updated; for the matching IDs with lower counts or long time of non-appearance in the counter dictionary C, periodic cleaning or reduction of their priority is performed to reduce false matching and improve confirmation efficiency; according to the requirements of the actual application scene, the values of the sliding window size N, the continuous matching threshold T1 and the maximum matching number threshold T2 are dynamically adjusted to adapt to the pedestrian re-identification requirements in different scenes; when performing identity confirmation, other auxiliary information such as the motion trajectory, speed and direction of the pedestrian is combined to further improve the accuracy of identity confirmation; for the case where there are multiple matching IDs with high similarity, additional verification methods such as face comparison and clothing feature analysis are used to assist in determining the final identity confirmation result.

[0116] In the above target tracking step, the historical coordinate information of the pedestrian motion in the monitoring video is obtained by the YOLO target detection algorithm, including:

[0117] The video frame images are obtained from the monitoring video in time sequence frame by frame or at a predetermined interval;

[0118] The YOLO target detection algorithm is applied to each frame of image to detect the pedestrian in the image;

[0119] For each detected pedestrian, the boundary box coordinates thereof are extracted, including the position (x, y) of the pedestrian in the image and the width w and height h of the boundary box; the confidence score corresponding to each detected pedestrian is recorded; the class of each detected target is confirmed as “pedestrian”, and the class information is recorded;

[0120] The boundary box coordinates, confidence score and class information of the detected pedestrian in each frame are combined in time sequence to construct a historical coordinate information sequence of the pedestrian;

[0121] The constructed historical coordinate information sequence is stored in the database and effectively managed for subsequent pedestrian trajectory analysis, re-identification or other monitoring analysis tasks; an interface or method is provided to retrieve the historical coordinate information of a specific pedestrian or a specific time period as needed for pedestrian behavior analysis, anomaly detection, trajectory prediction and other applications.

[0122] Through the above method, the embodiment can efficiently and accurately obtain the historical coordinate information of the pedestrian motion from the monitoring video, providing rich data support for pedestrian re-identification, monitoring analysis and behavior research.

[0123] An online pedestrian re-identification system embodiment based on multi-feature cyclic confirmation

[0124] As shown in Figure 3 The online pedestrian re-identification system based on multi-feature cyclic confirmation provided by the embodiment comprises:

[0125] Video stream analysis and target tracking module: Obtain the historical coordinate information of the pedestrian motion in the monitoring video through the YOLO target detection algorithm, which includes the boundary box coordinates (x, y, w, h), the confidence score and the category; use the target tracking algorithm ByteTrack to associate and track the detection results to realize the prediction of the continuous motion trajectory of the pedestrian in the video, and output the prediction result containing the possible position of the pedestrian at the next moment. It can be seen that this module is connected to the monitoring video source and is responsible for receiving and analyzing the video stream in real time. It can identify pedestrians in the video and track their motion trajectories, and predict the possible position of the pedestrians at the next moment, thereby realizing continuous tracking.

[0126] Pedestrian re-identification module: receives the pedestrian image frames transmitted by the target tracking step, extracts the features of the pedestrian image using a deep learning model to obtain a feature vector of the multi-dimensional features of the human body, matches the newly extracted feature vector with the feature vectors stored in the database, and returns the matching result most similar to the feature vectors in the database as the optimal matching result. It can be seen that this module receives the pedestrian image frames transmitted by the target tracking module and extracts multi-feature vectors such as faces and bodies from them, which will be used for subsequent matching with the features in the database.

[0127] Cycle confirmation module: set the sliding window size N to record the last N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the frequency of each matching ID; set the continuous matching threshold T1 and the maximum matching number threshold T2, confirm the identity of the pedestrian according to the count in C and the continuous matching in Q; wherein the continuous matching threshold T1 is used to measure the stability of a matching ID in continuous matching for multiple times, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts. It can be seen that this module performs cycle confirmation according to the historical re-identification results provided by the pedestrian re-identification module. Specifically, the system maintains a sliding window matching result queue Q and a counter dictionary C for each detected pedestrian. When a certain pedestrian is matched to the same number for T1 times (such as T1 = 3), the identity of the pedestrian is confirmed. If the number changes during matching and reaches the maximum matching number T2 (such as T2 = 5), the number with the highest count in C is selected as the confirmed number. Otherwise, add a new matching result to the queue Q and wait for the next round of matching. At the same time, for the confirmed pedestrians, only tracking is performed without repeated matching. In addition, the system also dynamically adjusts the T1 and T2 thresholds according to the scene complexity. The strategy proposed in this embodiment balances accuracy and efficiency and adapts to the pedestrian re-identification needs in different scenarios.

[0128] The database module is mainly responsible for storing the feature data of the recorded pedestrians, and provides data support for subsequent feature comparison. As can be seen, the extracted feature vector is transmitted to the database module and matched with the features stored in the database. The pedestrian re-identification module calculates an optimal average matching result based on the matching result to preliminarily determine the identity of the pedestrian.

[0129] In summary, the core of the embodiment is to propose a pedestrian re-identification mechanism based on cycle confirmation. The mechanism selects the highest credibility identity by analyzing the historical identification records, thereby significantly improving the pedestrian re-identification accuracy. The embodiment focuses on realizing cross-domain pedestrian re-identification, and the target object is a real pedestrian in an open world, rather than relying on a model trained on a dataset for re-identification, which is another important technical point that the application intends to protect. In view of the real-time requirement of the video monitoring system, the embodiment introduces the concept of the maximum matching number to balance the identification accuracy and efficiency of the system, which is very important for the video monitoring system with limited resources in practical applications.

[0130] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0131] The above embodiments are only preferred embodiments of the present application, and cannot be used to limit the scope of protection of the present application. Any non-essential changes and replacements made by those skilled in the art based on the present application shall fall within the scope of protection of the present application.

Claims

1. An online pedestrian re-identification method based on multi-feature cycle confirmation, characterized in that, The method comprises the following steps: Target tracking step: obtain the historical coordinate information of the pedestrian motion in the monitoring video through the YOLO target detection algorithm, which includes the boundary box coordinates (x, y, w, h), the confidence score and the category; use the target tracking algorithm ByteTrack to associate and track the detection results to realize the prediction of the continuous motion trajectory of the pedestrian in the video, and output the prediction result containing the possible position of the pedestrian at the next moment; Pedestrian re-identification step: receive the pedestrian image frame transmitted by the target tracking step, extract the features of the pedestrian image by using the deep learning model, obtain the feature vector of the multi-dimensional features of the human body, match the newly extracted feature vector with the feature vector stored in the database, and return the matching result most similar to the feature vector in the database as the optimal matching result; Loop confirmation step: set a sliding window size N for recording the last N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the frequency of each matching ID; Set a continuous matching threshold T1 and a maximum matching number threshold T2, confirm the identity of the pedestrian according to the count in C and the continuous matching in Q; wherein the continuous matching threshold T1 is used to measure the stability of a matching ID in continuous matching for multiple times, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts; When the target tracking algorithm ByteTrack is used to predict the coordinates at the next moment, the following steps are specifically performed: Define a detection result set D = {d1, d2,..., dm}, wherein each detection result di is represented as a five-tuple (xi, yi, wi, hi, si), which respectively represents the center point horizontal coordinate, the center point vertical coordinate, the width, the height and the confidence score of the detection box; Define a tracking result set T = {t1, t2,..., tn}, wherein each tracking result ti is also represented as a five-tuple (xi, yi, wi, hi, vi), which respectively represents the center point horizontal coordinate, the center point vertical coordinate, the width, the height and the speed or state information of the target; Calculate the association matrix C, wherein the element Cij = IoU(di, tj) represents the intersection over union IoU between the detection result di and the tracking result tj; Match the association matrix C using the Hungarian algorithm to obtain the optimal association result, that is, to optimally pair the detection results and the tracking results; For the unmatched detection results, if the confidence score si is greater than a preset threshold, a new tracking trajectory is created; otherwise, secondary matching is performed; The loop confirmation step specifically comprises: Set a sliding window size N for recording and storing the last N matching results of each pedestrian; Maintain a matching result queue Q for each pedestrian, which is used to store the latest matching results in order, and maintain a counter dictionary C for recording the number of times each matching ID appears in the queue Q; When a new matching result is generated, it is added to the matching result queue Q, and the length of the queue Q is checked; if the length of the queue Q exceeds the set sliding window size N, the oldest matching result in the queue is automatically removed to keep the queue length constant; According to the new matching result, the number of occurrences of the corresponding matching ID in the counter dictionary C is updated; Two judgment thresholds are set: a continuous matching threshold T1 and a maximum matching number threshold T2, and the identity of the pedestrian is confirmed according to the following judgment conditions: If there is a matching ID in the counter dictionary C whose count is greater than or equal to the continuous matching threshold T1, and the ID is the continuous matching of the last T1 matching results, the ID is directly confirmed as the identity of the pedestrian; if the continuous matching condition is not met, but the length of the matching result queue Q reaches the maximum matching number threshold T2, the matching ID with the highest count in the counter dictionary C is selected as the identity of the pedestrian for confirmation.

2. The method of claim 1, wherein, Further comprising: The secondary matching includes: For detection results that fail to successfully pair in the first matching, similarity evaluation is performed with existing trajectories in the tracking result set T; A similarity threshold β is set, where 0 < β < 1, to judge whether the similarity between the detection result and the existing trajectory is sufficient to establish an association; If the similarity between a certain detection result and a certain existing trajectory is greater than or equal to the similarity threshold β, the detection result is associated with the trajectory, the state information of the trajectory is updated, and the detection result is removed from the to-be-matched queue; If after the secondary matching, there are still detection results that fail to successfully associate with any trajectory, it is decided according to the system strategy whether to create a new trajectory based on these detection results or to ignore them as noise data.

3. The method of claim 1, wherein, In the pedestrian re-identification step, a deep learning model OSNet is used to extract features from pedestrian images, including: The input image is subjected to initial convolutional layer for preliminary feature extraction; The image features pass through 4 OSNet blocks, each of which contains multiple lightweight convolution units LCA, which extract multi-scale features of the image through the combination of point-wise convolution, depthwise separable convolution and 1x1 convolution; The output of each OSNet block is processed by a channel attention mechanism to enhance the attention to important features; After global average pooling and full connection layer processing, a feature vector with a dimension of 512 is obtained, which includes the overall appearance, local details and semantic information of the pedestrian; The extracted feature vector is used to compare with existing features in the database, and the similarity or distance between the feature vectors is calculated to realize the re-identification of the pedestrian.

4. The method of claim 3, characterized in that: For the newly extracted feature vector, first normalization processing is performed to ensure comparability between different feature vectors; Cosine similarity or other measurement methods are used to calculate the similarity between the new feature vector and the feature vectors in the database; According to the size of the similarity, the feature vectors in the database are sorted, and a number of feature vectors with the highest similarity or the smallest distance are selected as candidate matching results; Further verification is performed on the candidate matching result, and after the verification passes, the new feature vector is associated with the corresponding pedestrian identity information.

5. The method of claim 4, wherein: The cosine similarity is used to calculate the similarity between the feature vectors, specifically including the following steps: Obtain the stored feature vector V1 from the database, and the newly extracted feature vector V2, wherein V1 and V2 are vectors of the same dimension; Calculate the dot product of the feature vectors V1 and V2, i.e. V1 · V2, which reflects the directional similarity of the two vectors in the multi-dimensional space; Calculate the norm of the feature vectors V1 and V2, respectively, i.e. ||V1|| and ||V2||, which represents the length of the vector in the multi-dimensional space; Calculate the similarity between V1 and V2 using the cosine similarity formula, denoted as the following formula: similarity = cos(θ) = (V1 · V2) / (||V1|| * ||V2||) Where θ is the angle between V1 and V2; According to the calculated similarity value, sort all feature vectors in the database, the closer the similarity value is to 1, the more similar the two feature vectors are, i.e. the corresponding pedestrian identities are more likely to be the same; select the feature vector with the highest similarity as the optimal matching result.

6. The method of claim 1, wherein: After confirming the pedestrian identity, the confirmation result is fed back to the system, and the historical re-identification record of the pedestrian is updated; For matching IDs with low counts or long time of non-appearance in the counter dictionary C, periodic cleaning or priority reduction is performed to reduce false matching and improve confirmation efficiency; According to the requirements of the actual application scenario, dynamically adjust the values of the sliding window size N, the continuous matching threshold T1 and the maximum matching number threshold T2 to adapt to the pedestrian re-identification requirements in different scenarios; When performing identity confirmation, combine other auxiliary information to further improve the accuracy of identity confirmation; For the case where there are multiple matching IDs with high similarity, additional verification means are used to assist in determining the final identity confirmation result.

7. The method according to any one of claims 1 to 6, characterized in that, The historical coordinate information of the pedestrian motion in the monitoring video obtained by the YOLO target detection algorithm includes: Obtain video frame images from the monitoring video in time sequence frame by frame or at a predetermined interval; Apply the YOLO target detection algorithm to each frame of image to detect pedestrians in the image; For each detected pedestrian, extract its bounding box coordinates, which include the position (x, y) of the pedestrian in the image and the width w and height h of the bounding box; record the confidence score corresponding to each detected pedestrian; confirm the class of each detected target as "pedestrian" and record this class information; Combine the bounding box coordinates, confidence scores and class information of the detected pedestrians in each frame in time sequence to construct a historical coordinate information sequence of the pedestrian; Store the constructed historical coordinate information sequence in the database and provide an interface or method to retrieve the historical coordinate information of a specific pedestrian or a specific time period as needed.

8. An online pedestrian re-identification system based on multi-feature cycle confirmation, characterized in that, The system is applied to the online pedestrian re-identification method based on multi-feature cycle confirmation as claimed in any one of claims 1 to 7, and the system comprises: a video stream analysis and target tracking module: obtaining the historical coordinate information of the pedestrian motion in the monitoring video through the YOLO target detection algorithm, the information including the boundary box coordinates (x, y, w, h), the confidence score and the category; using the target tracking algorithm ByteTrack to associate and track the detection results to realize the prediction of the continuous motion track of the pedestrian in the video, and output the prediction results containing the possible position of the pedestrian at the next time; a pedestrian re-identification module: receiving the pedestrian image frame transmitted by the target tracking step, using a deep learning model to extract the features of the pedestrian image, obtaining the feature vector of the multi-dimensional features of the human body, matching the newly extracted feature vector with the feature vector stored in the database, and returning the matching result most similar to the feature vector in the database as the optimal matching result; a cycle confirmation module: setting the sliding window size N for recording the last N matching results, maintaining a matching result queue Q and a counter dictionary C for each pedestrian, and updating the frequency of occurrence of each matching ID; setting the continuous matching threshold T1 and the maximum matching number threshold T2, confirming the identity of the pedestrian according to the count in C and the continuous matching in Q; wherein the continuous matching threshold T1 is used to measure the stability of a certain matching ID in continuous matching for multiple times, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

Images