Online pedestrian re-identification method and system based on multi-feature cyclic confirmation

Abstract

The invention provides an online pedestrian re-identification method and system based on multi-feature cyclic confirmation. The method comprises a target tracking step: obtaining historical coordinate information of pedestrian motion in a monitoring video; the detection results are correlated and tracked, and a prediction result containing the possible position of the pedestrian at the next moment is output; a pedestrian re-identification step: carrying out feature extraction on the pedestrian image to obtain a feature vector of a human body multi-dimensional feature, carrying out feature matching on the newly extracted feature vector and a feature vector stored in a database, and returning an optimal matching result; a cycle confirmation step: maintaining a matching result queue Q and a counter dictionary C for each pedestrian, and updating the occurrence frequency of each matching ID; and setting a continuous matching threshold T1 and a maximum matching frequency threshold T2, and confirming the identity of the pedestrian according to the count in the C and the continuous matching condition in the Q. According to the invention, the reliability and accuracy of pedestrian re-identification in a real-time video monitoring system can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent video surveillance, and in particular to an online pedestrian re-identification method based on multi-feature cyclic confirmation and a system applying the method. Background Art

[0002] In the modern urban security monitoring system, with the expansion of urban scale and the increase in public security needs, video surveillance has become an important means of maintaining social order. However, traditional video surveillance systems mainly rely on manual monitoring, which is not only inefficient but also prone to missing key information. In order to improve monitoring efficiency and accuracy, pedestrian re-identification (ReID) technology has gradually become a research hotspot in this field. Pedestrian re-identification technology aims to accurately and quickly identify the same pedestrian in pedestrian images captured by different surveillance cameras through computer vision and pattern recognition algorithms. This is of great significance for application scenarios such as crime tracking, missing persons search, and abnormal behavior monitoring.

[0003] At present, pedestrian re-identification technology mainly relies on deep learning models, which are trained on large-scale pedestrian image datasets to learn pedestrian feature representations and identify pedestrians through feature matching in the test phase. However, although deep learning models have made significant progress in pedestrian re-identification tasks, there are still many challenges and limitations. First, pedestrians in the real world are highly diverse and unpredictable, including changes in clothing, posture, perspective, lighting conditions and other factors. These changes greatly reduce the performance of trained models in practical applications. In addition, in a multi-camera real-time video surveillance system, pedestrians may be captured by multiple cameras at the same time. The image lighting conditions and perspectives under each camera are different. In addition, factors such as complex backgrounds and pedestrian occlusion increase the difficulty of recognition. Even if compared by naked eye, it takes multiple times to make an accurate judgment. The recognition effect of traditional recognition methods in real-time dynamic environments will be greatly reduced. Therefore, it is urgent to develop a new method to improve the accuracy of pedestrian re-identification and meet the actual needs of modern urban security monitoring. Summary of the invention

[0004] In order to solve the problem of low recognition accuracy faced by traditional pedestrian re-identification algorithms in real-time video surveillance environments, the purpose of the present invention is to provide an online pedestrian re-identification method and system based on multi-feature cyclic confirmation, which improves the reliability and accuracy of pedestrian re-identification in real-time video surveillance systems by implementing a series of continuous and dynamic recognition and verification steps.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] An online person re-identification method based on multi-feature cyclic confirmation, comprising:

[0007] Target tracking step: Use the YOLO target detection algorithm to obtain the historical coordinate information of pedestrian movement in the surveillance video, which includes the bounding box coordinates (x, y, w, h), confidence score and category; use the target tracking algorithm ByteTrack to associate and track the detection results to predict the continuous movement trajectory of pedestrians in the video, and output the prediction results including the possible position of pedestrians at the next moment;

[0008] Pedestrian re-identification step: receive the pedestrian image frame transmitted by the target tracking step, use the deep learning model to extract the features of the pedestrian image, obtain the feature vector of the multi-dimensional features of the human body, perform feature matching on the newly extracted feature vector with the feature vector stored in the database, and return the matching result that is most similar to the feature vector in the database as the optimal matching result;

[0009] Loop confirmation step: Set the sliding window size N to record the most recent N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the frequency of occurrence of each matching ID; set the continuous matching threshold T1 and the maximum matching number threshold T2, and confirm the identity of the pedestrian based on the count in C and the continuous matching in Q; the continuous matching threshold T1 is used to measure the stability of a matching ID in multiple consecutive matches, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

[0010] According to an online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the present invention, when using the target tracking algorithm ByteTrack to predict the coordinates at the next moment, the following steps are specifically performed:

[0011] Define the detection result set D = {d1, d2, ..., dm}, where each detection result di is represented by a five-tuple (xi, yi, wi, hi, si), which respectively represents the horizontal coordinate of the center point of the detection box, the vertical coordinate of the center point, the width, the height and the confidence score;

[0012] Define a tracking result set T = {t1, t2, ..., tn}, where each tracking result ti is also represented as a five-tuple (xi, yi, wi, hi, vi), which respectively represents the horizontal coordinate of the center point of the tracking box, the vertical coordinate of the center point, the width, the height, and the speed or state information of the target;

[0013] Calculate the correlation matrix C, where the element Cij = IoU (di, tj) represents the intersection over union (IoU) between the detection result di and the tracking result tj;

[0014] Use the Hungarian algorithm to match the correlation matrix C to obtain the optimal correlation result, that is, to optimally pair the detection result with the tracking result;

[0015] For unmatched detection results, if their confidence score si is greater than the preset threshold, a new tracking trajectory is created; otherwise, a secondary matching is performed.

[0016] According to the present invention, an online pedestrian re-identification method based on multi-feature cyclic confirmation further includes:

[0017] The secondary matching includes:

[0018] For the detection results that failed to be successfully paired in the first matching, similarity evaluation is performed with the existing tracks in the tracking result set T;

[0019] Set a similarity threshold β, where 0<β<1, to determine whether the similarity between the detection result and the existing trajectory is sufficient to establish an association;

[0020] If the similarity between a certain detection result and an existing trajectory is greater than or equal to the similarity threshold β, the detection result is associated with the trajectory, the status information of the trajectory is updated, and the detection result is removed from the queue to be matched;

[0021] If after the second matching, there are still detection results that cannot be successfully associated with any track, the system strategy will decide whether to create new tracks based on these detection results or ignore them as noise data.

[0022] According to an online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the present invention, in the pedestrian re-identification step, a deep learning model OSNet is used to extract features from pedestrian images, including:

[0023] The input image passes through the initial convolutional layer for preliminary feature extraction;

[0024] The image features pass through 4 OSNet blocks. Each OSNet block contains multiple lightweight convolutional units (LCAs). The lightweight convolutional unit (LCA) extracts multi-scale features of the image through a combination of point-by-point convolution, depth-separable convolution, and 1x1 convolution.

[0025] The output of each OSNet block is processed by a channel attention mechanism to enhance the focus on important features;

[0026] After global average pooling and fully connected layer processing, a feature vector with a dimension of 512 is obtained, which includes the overall appearance, local details and semantic information of the pedestrian;

[0027] The extracted feature vectors are used to compare with existing features in the database, and pedestrian re-identification is achieved by calculating the similarity or distance between the feature vectors.

[0028] According to an online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the present invention, the newly extracted feature vectors are first normalized to ensure comparability between different feature vectors;

[0029] Using cosine similarity or other measurement methods, calculate the similarity between the new feature vector and the feature vector in the database;

[0030] According to the size of the similarity, the feature vectors in the database are sorted, and several feature vectors with the highest similarity or the smallest distance are selected as candidate matching results;

[0031] The candidate matching results are further verified, and after the verification is passed, the new feature vector is associated with the corresponding pedestrian identity information.

[0032] According to an online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the present invention, the similarity between feature vectors is calculated using cosine similarity, which specifically includes the following steps:

[0033] Obtain the stored feature vector V1 and the newly extracted feature vector V2 from the database, where V1 and V2 are both vectors of the same dimension;

[0034] Calculate the dot product of the feature vectors V1 and V2, that is, V1·V2. This dot product is used to reflect the directional similarity of two vectors in multidimensional space.

[0035] Calculate the norms of the eigenvectors V1 and V2, namely ||V1|| and ||V2||, respectively. The norm represents the length of the vector in the multidimensional space.

[0036] The cosine similarity formula is used to calculate the similarity between V1 and V2, which is expressed as the following formula:

[0037] similarity=cos(θ)=(V1·V2) / (||V1||*||V2||)

[0038] Where θ is the angle between V1 and V2;

[0039] According to the calculated similarity value, all feature vectors in the database are sorted. The closer the similarity value is to 1, the more similar the two feature vectors are, that is, the more likely the corresponding pedestrian identities are to be the same; the feature vector with the highest similarity is selected as the optimal matching result.

[0040] According to an online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the present invention, the cyclic confirmation step specifically includes:

[0041] Set a sliding window size N to record and store the most recent N matching results of each pedestrian;

[0042] Maintain a matching result queue Q for each pedestrian, which is used to store the most recent matching results in order, and maintain a counter dictionary C to record the number of times each matching ID appears in the queue Q;

[0043] When a new matching result is generated, it is added to the matching result queue Q and the length of the queue Q is checked; if the length of the queue Q exceeds the set sliding window size N, the oldest matching result in the queue is automatically removed to keep the queue length constant;

[0044] Update the number of occurrences of the corresponding matching ID in the counter dictionary C according to the new matching results;

[0045] Two judgment thresholds are set: the continuous matching threshold T1 and the maximum matching number threshold T2, and the identity of the pedestrian is confirmed according to the following judgment conditions:

[0046] If there is a matching ID whose count in the counter dictionary C is greater than or equal to the continuous matching threshold T1, and the ID is a continuous match of the most recent T1 matching results, then the ID is directly confirmed as the identity of the pedestrian; if the continuous matching condition is not met, but the length of the matching result queue Q reaches the maximum matching number threshold T2, then the matching ID with the highest count in the counter dictionary C is selected as the pedestrian's identity for confirmation.

[0047] According to an online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the present invention, after confirming the identity of the pedestrian, the confirmation result is fed back to the system and the historical re-identification record of the pedestrian is updated;

[0048] For matching IDs with low counts or those that have not appeared for a long time in the counter dictionary C, regular cleanup or lowering of their priority is performed to reduce mismatches and improve confirmation efficiency;

[0049] According to the requirements of actual application scenarios, the sliding window size N, the continuous matching threshold T1 and the maximum matching number threshold T2 are dynamically adjusted to meet the requirements of pedestrian re-identification in different scenarios;

[0050] When confirming identity, other auxiliary information is combined to further improve the accuracy of identity confirmation;

[0051] In the case where there are multiple matching IDs with high similarity, additional verification measures are used to assist in determining the final identity confirmation result.

[0052] According to an online pedestrian re-identification method based on multi-feature cyclic confirmation provided by the present invention, the historical coordinate information of pedestrian movement in the surveillance video is obtained by using the YOLO target detection algorithm, including:

[0053] Acquire video frame images from the surveillance video frame by frame or at predetermined intervals in time sequence;

[0054] Apply the YOLO object detection algorithm to each frame image to detect pedestrians in the image;

[0055] For each pedestrian detected, extract its bounding box coordinates, which include the position (x, y) of the pedestrian in the image and the width w and height h of the bounding box; record the confidence score corresponding to each detected pedestrian; confirm that the category of each detected target is "pedestrian" and record this category information;

[0056] Combine the bounding box coordinates, confidence scores, and category information of the pedestrians detected in each frame in chronological order to construct a historical coordinate information sequence of the pedestrians;

[0057] The constructed historical coordinate information sequence is stored in a database, and an interface or method is provided to retrieve the historical coordinate information of a specific pedestrian or a specific time period as needed.

[0058] An online person re-identification system based on multi-feature cyclic confirmation, comprising:

[0059] Video stream analysis and target tracking module: The YOLO target detection algorithm is used to obtain the historical coordinate information of pedestrian movement in the surveillance video, which includes the bounding box coordinates (x, y, w, h), confidence score and category; the target tracking algorithm ByteTrack is used to associate and track the detection results to predict the continuous movement trajectory of pedestrians in the video, and output the prediction results including the possible position of pedestrians at the next moment;

[0060] Pedestrian re-identification module: receives the pedestrian image frame transmitted by the target tracking step, uses the deep learning model to extract features from the pedestrian image, obtains the feature vector of the multi-dimensional features of the human body, performs feature matching on the newly extracted feature vector with the feature vector stored in the database, and returns the matching result that is most similar to the feature vector in the database as the optimal matching result;

[0061] Loop confirmation module: Set the sliding window size N to record the most recent N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the frequency of occurrence of each matching ID; set the continuous matching threshold T1 and the maximum matching number threshold T2, and confirm the identity of the pedestrian based on the count in C and the continuous matching in Q; the continuous matching threshold T1 is used to measure the stability of a matching ID in multiple consecutive matches, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

[0062] It can be seen that compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The present invention uses multi-feature matching technology to comprehensively utilize multiple feature information such as the appearance, texture, and color of pedestrians, and effectively deal with complex situations such as lighting changes, occlusion, and posture changes of pedestrians under different camera viewing angles. At the same time, the cyclic confirmation mechanism further enhances the stability and accuracy of recognition, so that in the actual deployed system, the accuracy of pedestrian recognition can stably reach more than 95%, far exceeding the average level of existing technologies.

[0064] 2. The present invention has deeply optimized the algorithm logic and data processing flow, ensuring that even in a large-scale monitoring system containing a large number of cameras, massive video data can be processed and analyzed in real time. This efficient data processing capability makes the present invention particularly suitable for security monitoring scenarios with limited resources and requiring rapid response, such as urban streets, public places, transportation hubs, etc.

[0065] 3. By accurately identifying pedestrians and tracking their movement trajectories, the present invention provides a more comprehensive and accurate information basis for the monitoring system, which not only helps to timely discover and respond to potential security threats, but also provides strong data support for urban management and planning. Therefore, the implementation of the present invention significantly enhances the comprehensive performance of the entire monitoring system, enabling it to better serve the security monitoring needs of modern cities.

[0066] 4. Since the present invention does not rely on a specific hardware or software platform and can be easily integrated into an existing monitoring system, it has extremely high adaptability and flexibility. This enables the present invention to be widely used in various types of monitoring scenarios, providing a more reliable and efficient solution for urban security, public safety and other fields.

[0067] In summary, the present invention not only significantly improves the accuracy of pedestrian recognition by introducing multi-feature matching and a cyclic confirmation mechanism, but also realizes real-time and efficient data processing, improves the comprehensive performance of the monitoring system, and enhances the adaptability and flexibility of the system.

[0068] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flow chart of an embodiment of an online pedestrian re-identification method based on multi-feature cyclic confirmation of the present invention.

[0070] Figure 2 It is a flow chart of an embodiment of an online pedestrian re-identification method based on multi-feature cyclic confirmation of the present invention.

[0071] Figure 3 It is a schematic diagram of an embodiment of an online pedestrian re-identification system based on multi-feature cyclic confirmation of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0074] An embodiment of an online pedestrian re-identification method based on multi-feature cyclic confirmation

[0075] See also Figure 1 and Figure 2 , this embodiment provides an online pedestrian re-identification method based on multi-feature cyclic confirmation, comprising the following steps:

[0076] Target tracking steps: The YOLO target detection algorithm is used to obtain the historical coordinate information of pedestrian movement in the surveillance video, which includes the bounding box coordinates (x, y, w, h), confidence score and category; the target tracking algorithm ByteTrack is used to associate and track the detection results to predict the continuous motion trajectory of pedestrians in the video, and output the prediction results containing the possible positions of pedestrians at the next moment.

[0077] Pedestrian re-identification step: receive the pedestrian image frame transmitted by the target tracking step, use the deep learning model to extract features of the pedestrian image, obtain the feature vector of the multi-dimensional features of the human body, match the newly extracted feature vector with the feature vector stored in the database, and return the matching result that is most similar to the feature vector in the database as the optimal matching result.

[0078] Loop confirmation step: Set the sliding window size N to record the most recent N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the frequency of occurrence of each matching ID; set the continuous matching threshold T1 and the maximum matching number threshold T2, and confirm the identity of the pedestrian based on the count in C and the continuous matching in Q; the continuous matching threshold T1 is used to measure the stability of a matching ID in multiple consecutive matches, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

[0079] In the above target tracking step, when using the target tracking algorithm ByteTrack to predict the coordinates of the next moment, the following steps are specifically performed:

[0080] Define the detection result set D = {d1, d2, ..., dm}, where each detection result di is represented by a five-tuple (xi, yi, wi, hi, si), which respectively represents the horizontal coordinate of the center point of the detection box, the vertical coordinate of the center point, the width, the height and the confidence score;

[0081] Define a tracking result set T = {t1, t2, ..., tn}, where each tracking result ti is also represented as a five-tuple (xi, yi, wi, hi, vi), which respectively represents the horizontal coordinate of the center point of the tracking box, the vertical coordinate of the center point, the width, the height, and the speed or state information of the target;

[0082] Calculate the correlation matrix C, where the element Cij = IoU (di, tj) represents the intersection over union (IoU) between the detection result di and the tracking result tj;

[0083] Use the Hungarian algorithm to match the correlation matrix C to obtain the optimal correlation result, that is, to optimally pair the detection result with the tracking result;

[0084] For unmatched detection results, if their confidence score si is greater than the preset threshold, a new tracking trajectory is created; otherwise, a secondary matching is performed.

[0085] It can be seen that the core idea of ​​ByteTrack in this embodiment is to use low-confidence detection results to associate occluded targets. The specific formula is as follows: detection result: D = {d1, d2, ..., dm}, where di = (xi, yi, wi, hi, si), tracking result: T = {t1, t2, ..., tn}, where ti = (xi, yi, wi, hi, vi), the association matrix C is obtained by calculating the IOU distance between the detection result and the tracking result: Cij = IoU (di, tj). ByteTrack uses the Hungarian algorithm to match C and obtain the optimal association result. For unmatched detection results, if si is greater than the threshold, a new track is created; otherwise, a secondary match is performed.

[0086] In this embodiment, the secondary matching specifically includes the following steps:

[0087] For the detection results that failed to be successfully paired in the first matching, similarity evaluation is performed with the existing tracks in the tracking result set T;

[0088] Set a similarity threshold β, where 0<β<1, to determine whether the similarity between the detection result and the existing trajectory is sufficient to establish an association;

[0089] If the similarity between a certain detection result and an existing trajectory is greater than or equal to the similarity threshold β, the detection result is associated with the trajectory, the status information of the trajectory is updated, and the detection result is removed from the queue to be matched;

[0090] If after the second matching, there are still detection results that cannot be successfully associated with any track, the system strategy will decide whether to create new tracks based on these detection results or ignore them as noise data.

[0091] In the person re-identification step, the person re-identification module receives the uploaded pedestrian image frames and uses deep learning models such as OSNet to extract feature vectors of multiple features such as face and body. OSNet is characterized by the use of lightweight attention mechanism and multi-scale feature fusion, including:

[0092] The input image passes through the initial convolutional layer for preliminary feature extraction;

[0093] The image features pass through 4 OSNet blocks. Each OSNet block contains multiple lightweight convolutional units (LCAs). The lightweight convolutional unit (LCA) extracts multi-scale features of the image through a combination of point-by-point convolution, depth-separable convolution, and 1x1 convolution.

[0094] The output of each OSNet block is processed by a channel attention mechanism to enhance the focus on important features;

[0095] After global average pooling and fully connected layer processing, a feature vector with a dimension of 512 is obtained, which includes the overall appearance, local details and semantic information of the pedestrian;

[0096] After successful extraction, the extracted feature vector is used to compare with the existing features in the database, and the pedestrian re-identification is achieved by calculating the similarity or distance between the feature vectors.

[0097] During the comparison process, efficient indexing and retrieval techniques are used to accelerate the matching process of feature vectors and improve the real-time and accuracy of pedestrian re-identification.

[0098] In this embodiment, the newly extracted feature vectors are first normalized to ensure comparability between different feature vectors;

[0099] Calculate the similarity between the new feature vector and the feature vector in the database using metrics such as cosine similarity, Euclidean distance, or Hamming distance;

[0100] According to the size of the similarity, the feature vectors in the database are sorted, and several feature vectors with the highest similarity or the smallest distance are selected as candidate matching results;

[0101] The candidate matching results are further verified, such as through face comparison, clothing color and texture analysis, to improve the accuracy of pedestrian re-identification; after the verification, the new feature vector is associated with the corresponding pedestrian identity information and updated to the database for subsequent tracking and recognition tasks.

[0102] In this embodiment, the cosine similarity is used to calculate the similarity between feature vectors, which specifically includes the following steps:

[0103] Obtain the stored feature vector V1 and the newly extracted feature vector V2 from the database, where V1 and V2 are both vectors of the same dimension;

[0104] Calculate the dot product of the feature vectors V1 and V2, that is, V1·V2. This dot product is used to reflect the directional similarity of two vectors in multidimensional space.

[0105] Calculate the norms of the eigenvectors V1 and V2, namely ||V1|| and ||V2||, respectively. The norm represents the length of the vector in the multidimensional space.

[0106] The cosine similarity formula is used to calculate the similarity between V1 and V2, which is expressed as the following formula:

[0107] similarity=cos(θ)=(V1·V2) / (||V1||*||V2||)

[0108] Where θ is the angle between V1 and V2;

[0109] According to the calculated similarity value, all feature vectors in the database are sorted. The closer the similarity value is to 1, the more similar the two feature vectors are, that is, the more likely the corresponding pedestrian identities are to be the same; the feature vector with the highest similarity is selected as the optimal matching result.

[0110] Through the above steps, this embodiment can efficiently and accurately calculate the similarity between feature vectors and return the optimal pedestrian matching result, providing reliable technical support for the pedestrian re-identification task.

[0111] In this embodiment, the cycle confirmation step specifically includes:

[0112] Set a sliding window size N to record and store the most recent N matching results for each pedestrian; maintain a matching result queue Q for each pedestrian, which is used to store the most recent matching results in order, and maintain a counter dictionary C to record the number of times each matching ID appears in the queue Q;

[0113] When a new matching result is generated, it is added to the matching result queue Q and the length of the queue Q is checked. If the length of the queue Q exceeds the set sliding window size N, the oldest matching result in the queue is automatically removed to keep the queue length constant. The number of occurrences of the corresponding matching ID in the counter dictionary C is updated according to the new matching result. Two judgment thresholds are set: the continuous matching threshold T1 and the maximum matching number threshold T2, and the identity of the pedestrian is confirmed according to the following judgment conditions:

[0114] If there is a matching ID whose count in the counter dictionary C is greater than or equal to the continuous matching threshold T1, and the ID is a continuous match of the most recent T1 matching results, then the ID is directly confirmed as the identity of the pedestrian; if the continuous matching condition is not met, but the length of the matching result queue Q reaches the maximum matching number threshold T2, then the matching ID with the highest count in the counter dictionary C is selected as the pedestrian's identity for confirmation. This strategy not only takes into account the reliability of continuous matching, but also limits the decision time through the maximum matching number, balancing accuracy and real-time performance.

[0115] After confirming the identity of the pedestrian, the confirmation result is fed back to the system and the pedestrian's historical re-identification record is updated; for matching IDs with low counts or those that have not appeared for a long time in the counter dictionary C, they are regularly cleaned up or their priorities are lowered to reduce false matches and improve confirmation efficiency; according to the needs of actual application scenarios, the values ​​of the sliding window size N, the continuous matching threshold T1 and the maximum matching number threshold T2 are dynamically adjusted to adapt to the requirements of pedestrian re-identification in different scenarios; when performing identity confirmation, other auxiliary information, such as the pedestrian's movement trajectory, speed, direction, etc., is combined to further improve the accuracy of identity confirmation; in the case of multiple matching IDs with high similarity, additional verification methods, such as face comparison, clothing feature analysis, etc., are used to assist in determining the final identity confirmation result.

[0116] In the above target tracking step, the YOLO target detection algorithm is used to obtain the historical coordinate information of pedestrian movement in the surveillance video, including:

[0117] Acquire video frame images from the surveillance video frame by frame or at predetermined intervals in time sequence;

[0118] Apply the YOLO object detection algorithm to each frame image to detect pedestrians in the image;

[0119] For each pedestrian detected, extract its bounding box coordinates, which include the position (x, y) of the pedestrian in the image and the width w and height h of the bounding box; record the confidence score corresponding to each detected pedestrian; confirm that the category of each detected target is "pedestrian" and record this category information;

[0120] Combine the bounding box coordinates, confidence scores, and category information of the pedestrians detected in each frame in chronological order to construct a historical coordinate information sequence of the pedestrians;

[0121] The constructed historical coordinate information sequence is stored in the database and effectively managed for subsequent pedestrian trajectory analysis, re-identification or other monitoring and analysis tasks; an interface or method is provided to retrieve the historical coordinate information of a specific pedestrian or a specific time period as needed for applications such as pedestrian behavior analysis, anomaly detection, and trajectory prediction.

[0122] Through the above method, this embodiment can efficiently and accurately obtain the pedestrian's movement history coordinate information from the surveillance video, providing rich data support for pedestrian re-identification, surveillance analysis, behavior research, etc.

[0123] An embodiment of an online pedestrian re-identification system based on multi-feature cyclic confirmation

[0124] like Figure 3 As shown, this embodiment provides an online pedestrian re-identification system based on multi-feature cyclic confirmation, including:

[0125] Video stream analysis and target tracking module: The YOLO target detection algorithm is used to obtain the historical coordinate information of pedestrian movement in the surveillance video, which includes the bounding box coordinates (x, y, w, h), confidence score and category; the target tracking algorithm ByteTrack is used to associate and track the detection results to predict the continuous motion trajectory of pedestrians in the video, and output the prediction results containing the possible position of pedestrians at the next moment. It can be seen that this module is connected to the surveillance video source and is responsible for receiving and analyzing the video stream in real time. It can identify pedestrians in the video, track their motion trajectory, and predict the possible position of pedestrians at the next moment, thereby achieving continuous tracking.

[0126] Pedestrian re-identification module: Receives pedestrian image frames transmitted by the target tracking step, uses the deep learning model to extract features from pedestrian images, obtains feature vectors of multi-dimensional features of the human body, performs feature matching between the newly extracted feature vectors and the feature vectors stored in the database, and returns the matching result that is most similar to the feature vector in the database as the optimal matching result. It can be seen that this module receives pedestrian image frames transmitted by the target tracking module, and extracts multiple feature vectors such as face and body from them, which will be used for subsequent matching with features in the database.

[0127] Circular confirmation module: Set the sliding window size N to record the latest N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the frequency of occurrence of each matching ID; set the continuous matching threshold T1 and the maximum matching number threshold T2, and confirm the identity of the pedestrian according to the count in C and the continuous matching in Q; among them, the continuous matching threshold T1 is used to measure the stability of a matching ID in multiple consecutive matches, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts. It can be seen that this module performs cyclic confirmation based on the historical re-identification results provided by the pedestrian re-identification module. Specifically, the system maintains a sliding window matching result queue Q and a counter dictionary C for each detected pedestrian. When a pedestrian is matched with the same number for T1 consecutive times (such as T1=3), the identity of the pedestrian is confirmed. If the number changes during the matching process and reaches the maximum matching number T2 (such as T2=5), the number with the highest count in C is selected as the confirmation number. Otherwise, add a new matching result to the queue Q and wait for the next round of matching. At the same time, for confirmed pedestrians, only tracking is performed without repeated matching. In addition, the system also dynamically adjusts the T1 and T2 thresholds according to the complexity of the scene. The strategy proposed in this embodiment balances accuracy and efficiency and adapts to the needs of pedestrian re-identification in different scenes.

[0128] Database module,The database module is mainly responsible for storing the recorded pedestrian feature data,providing data support for subsequent feature comparison.,It can be seen that the extracted feature vector is transmitted to the,database module and matched with the features stored in the database.,Based on the matching results, the pedestrian re-identification module,calculates an optimal average matching result to preliminarily determine the,identity of the pedestrian.

[0129] In summary, the core of this embodiment is to propose a pedestrian re-identification mechanism based on cyclic confirmation, which analyzes historical identification records to select the identity with the highest credibility, thereby significantly improving the accuracy of pedestrian re-identification. This embodiment focuses on achieving cross-domain pedestrian re-identification, and its target object is real pedestrians in the open world, rather than relying on a model pre-trained on a data set for re-identification, which is another important technical point that the present invention intends to protect. In response to the real-time requirements of the video surveillance system, this embodiment introduces the concept of the maximum number of matches to balance the recognition accuracy and efficiency of the system, which is very important for video surveillance systems with limited resources in practical applications.

[0130] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. An online person re-identification method based on multi-feature cyclic confirmation, characterized in that: The following steps are involved: Target tracking step: Use the YOLO target detection algorithm to obtain the historical coordinate information of pedestrian movement in the surveillance video, which includes the bounding box coordinates (x, y, w, h), confidence score and category; use the target tracking algorithm ByteTrack to associate and track the detection results to predict the continuous movement trajectory of pedestrians in the video, and output the prediction results including the possible position of pedestrians at the next moment; Pedestrian re-identification step: receive the pedestrian image frame transmitted by the target tracking step, use the deep learning model to extract the features of the pedestrian image, obtain the feature vector of the multi-dimensional features of the human body, perform feature matching on the newly extracted feature vector with the feature vector stored in the database, and return the matching result that is most similar to the feature vector in the database as the optimal matching result; Loop confirmation step: Set the sliding window size N to record the most recent N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the occurrence frequency of each matching ID; Set the continuous matching threshold T1 and the maximum matching number threshold T2, and confirm the identity of the pedestrian based on the count in C and the continuous matching in Q; the continuous matching threshold T1 is used to measure the stability of a matching ID in multiple consecutive matches, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

2. The method according to claim 1, characterized in that: When using the target tracking algorithm ByteTrack to predict the coordinates of the next moment, the following steps are performed: Define the detection result set D = {d1, d2, ..., dm}, where each detection result di is represented by a five-tuple (xi, yi, wi, hi, si), which respectively represents the horizontal coordinate of the center point of the detection box, the vertical coordinate of the center point, the width, the height and the confidence score; Define a tracking result set T = {t1, t2, ..., tn}, where each tracking result ti is also represented as a five-tuple (xi, yi, wi, hi, vi), which respectively represents the horizontal coordinate of the center point of the tracking box, the vertical coordinate of the center point, the width, the height, and the speed or state information of the target; Calculate the correlation matrix C, where the element Cij = IoU (di, tj) represents the intersection over union (IoU) between the detection result di and the tracking result tj; Use the Hungarian algorithm to match the correlation matrix C to obtain the optimal correlation result, that is, to optimally pair the detection result with the tracking result; For unmatched detection results, if their confidence score si is greater than the preset threshold, a new tracking trajectory is created; otherwise, a secondary matching is performed.

3. The method according to claim 2, characterized in that Also includes: The secondary matching includes: For the detection results that failed to be successfully paired in the first matching, similarity evaluation is performed with the existing tracks in the tracking result set T; Set a similarity threshold β, where 0<β<1, to determine whether the similarity between the detection result and the existing trajectory is sufficient to establish an association; If the similarity between a certain detection result and an existing trajectory is greater than or equal to the similarity threshold β, the detection result is associated with the trajectory, the status information of the trajectory is updated, and the detection result is removed from the queue to be matched; If after the second matching, there are still detection results that cannot be successfully associated with any track, the system strategy will decide whether to create new tracks based on these detection results or ignore them as noise data.

4. The method according to claim 1, characterized in that: In the pedestrian re-identification step, the deep learning model OSNet is used to extract features from pedestrian images, including: The input image passes through the initial convolutional layer for preliminary feature extraction; The image features pass through 4 OSNet blocks. Each OSNet block contains multiple lightweight convolution units (LCAs). The lightweight convolution unit (LCA) extracts multi-scale features of the image through a combination of point-by-point convolution, depth-separable convolution, and 1x1 convolution. The output of each OSNet block is processed by a channel attention mechanism to enhance the focus on important features; After global average pooling and fully connected layer processing, a feature vector with a dimension of 512 is obtained, which includes the overall appearance, local details and semantic information of the pedestrian; The extracted feature vectors are used to compare with existing features in the database, and pedestrian re-identification is achieved by calculating the similarity or distance between the feature vectors.

5. The method according to claim 4, characterized in that: For the newly extracted feature vectors, normalization is first performed to ensure comparability between different feature vectors; Using cosine similarity or other measurement methods, calculate the similarity between the new feature vector and the feature vector in the database; According to the size of the similarity, the feature vectors in the database are sorted, and several feature vectors with the highest similarity or the smallest distance are selected as candidate matching results; The candidate matching results are further verified, and after the verification is passed, the new feature vector is associated with the corresponding pedestrian identity information.

6. The method according to claim 5, characterized in that: The cosine similarity is used to calculate the similarity between feature vectors, which specifically includes the following steps: Obtain the stored feature vector V1 and the newly extracted feature vector V2 from the database, where V1 and V2 are both vectors of the same dimension; Calculate the dot product of the feature vectors V1 and V2, that is, V1·V2. This dot product is used to reflect the directional similarity of two vectors in multidimensional space. Calculate the norms of the eigenvectors V1 and V2, namely ||V1|| and ||V2||, respectively. The norm represents the length of the vector in the multidimensional space. The cosine similarity formula is used to calculate the similarity between V1 and V2, which is expressed as the following formula: similarity=cos(θ)=(V1·V2) / (||V1||*||V2||) Where θ is the angle between V1 and V2; According to the calculated similarity value, all feature vectors in the database are sorted. The closer the similarity value is to 1, the more similar the two feature vectors are, that is, the more likely the corresponding pedestrian identities are to be the same; the feature vector with the highest similarity is selected as the optimal matching result.

7. The method according to claim 1, characterized in that The cycle confirmation step specifically includes: Set a sliding window size N to record and store the most recent N matching results of each pedestrian; Maintain a matching result queue Q for each pedestrian, which is used to store the most recent matching results in order, and maintain a counter dictionary C to record the number of times each matching ID appears in the queue Q; When a new matching result is generated, it is added to the matching result queue Q and the length of the queue Q is checked; if the length of the queue Q exceeds the set sliding window size N, the oldest matching result in the queue is automatically removed to keep the queue length constant; Update the number of occurrences of the corresponding matching ID in the counter dictionary C according to the new matching results; Two judgment thresholds are set: the continuous matching threshold T1 and the maximum matching number threshold T2, and the identity of the pedestrian is confirmed according to the following judgment conditions: If there is a matching ID whose count in the counter dictionary C is greater than or equal to the continuous matching threshold T1, and the ID is a continuous match of the most recent T1 matching results, then the ID is directly confirmed as the identity of the pedestrian; if the continuous matching condition is not met, but the length of the matching result queue Q reaches the maximum matching number threshold T2, then the matching ID with the highest count in the counter dictionary C is selected as the pedestrian's identity for confirmation.

8. The method according to claim 7, characterized in that: After confirming the identity of the pedestrian, the confirmation result is fed back to the system and the pedestrian's historical re-identification record is updated; For matching IDs with low counts or those that have not appeared for a long time in the counter dictionary C, regular cleanup or lowering of their priority is performed to reduce mismatches and improve confirmation efficiency; According to the requirements of actual application scenarios, the sliding window size N, the continuous matching threshold T1 and the maximum matching number threshold T2 are dynamically adjusted to meet the requirements of pedestrian re-identification in different scenarios; When confirming identity, other auxiliary information is combined to further improve the accuracy of identity confirmation; In the case where there are multiple matching IDs with high similarity, additional verification measures are used to assist in determining the final identity confirmation result.

9. The method according to any one of claims 1 to 8, characterized in that: The method of obtaining the historical coordinate information of pedestrian movement in the surveillance video by using the YOLO target detection algorithm includes: Acquire video frame images from the surveillance video frame by frame or at predetermined intervals in time sequence; Apply the YOLO object detection algorithm to each frame image to detect pedestrians in the image; For each detected pedestrian, extract its bounding box coordinates, which include the position (x, y) of the pedestrian in the image and the width w and height h of the bounding box; record the confidence score corresponding to each detected pedestrian; confirm that the category of each detected target is "pedestrian" and record this category information; Combine the bounding box coordinates, confidence scores, and category information of the pedestrians detected in each frame in chronological order to construct a historical coordinate information sequence of the pedestrians; The constructed historical coordinate information sequence is stored in a database, and an interface or method is provided to retrieve the historical coordinate information of a specific pedestrian or a specific time period as needed.

10. An online person re-identification system based on multi-feature cyclic confirmation, characterized in that: include: Video stream analysis and target tracking module: The YOLO target detection algorithm is used to obtain the historical coordinate information of pedestrian movement in the surveillance video, which includes the bounding box coordinates (x, y, w, h), confidence score and category; the target tracking algorithm ByteTrack is used to associate and track the detection results to predict the continuous movement trajectory of pedestrians in the video, and output the prediction results including the possible position of pedestrians at the next moment; Pedestrian re-identification module: receives the pedestrian image frame transmitted by the target tracking step, uses the deep learning model to extract features from the pedestrian image, obtains the feature vector of the multi-dimensional features of the human body, performs feature matching on the newly extracted feature vector with the feature vector stored in the database, and returns the matching result that is most similar to the feature vector in the database as the optimal matching result; Loop confirmation module: Set the sliding window size N to record the most recent N matching results, maintain a matching result queue Q and a counter dictionary C for each pedestrian, and update the occurrence frequency of each matching ID; Set the continuous matching threshold T1 and the maximum matching number threshold T2, and confirm the identity of the pedestrian based on the count in C and the continuous matching in Q; the continuous matching threshold T1 is used to measure the stability of a matching ID in multiple consecutive matches, and the maximum matching number threshold T2 is used to limit the maximum number of matching attempts.

Images