A pedestrian re-identification device and method combined with YOLOv8 model
By combining the YOLOv8 model and the camera-perceived pedestrian re-identification method, the problems of low target detection accuracy and slow training speed in the prior art are solved, real-time video stream processing and pedestrian monitoring are achieved, and the generalization ability of the model is improved.
Patent Information
- Application Number
- CN202411289167.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-14
AI Technical Summary
In the prior art, the accuracy of target detection is low and the training speed is slow, so real-time video streaming processing cannot be achieved, resulting in poor pedestrian monitoring effect.
Using the combined YOLOv8 model, combining the wireless camera group module, main control module and PC terminal, the YOLOv8s multi-scale precise class detection model and camera-perceived pedestrian re-identification method, the European-style distance and K-nearest neighbor algorithm are used to perform pedestrian image similarity. Compute to realize real-time video stream processing and pedestrian tracking.
It improves the accuracy and training speed of pedestrian detection, realizes real-time video streaming processing, and enhances the effect of pedestrian monitoring and the generalization ability of the model in different camera environments.
Smart Images

Figure CN118799660B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a pedestrian re-identification device and method combined with a YOLOv8 model. Background Art
[0002] Campus safety is an important security issue, but security forces are mostly limited to specific times and places, and cannot follow up on visitors' whereabouts on campus, resulting in a lack of behavioral monitoring of the person; the existing technology generally uses Yolov5 for target detection, but the current accuracy of Yolov5 is lower than that of the advanced Yolov8 algorithm, and the training speed is slow, which has a great impact on the project tuning process. In terms of deployment, due to the defects of Yolov5 and the inability to effectively push real-time video streams to the host computer, later video processing can only be achieved through reserved videos, and the effect of real-time processing cannot be achieved. Summary of the invention
[0003] Purpose of the invention: The purpose of the present invention is to provide a pedestrian re-identification device and method combined with a YOLOv8 model to solve the problems of low target detection accuracy, slow training speed and inability to realize real-time video stream processing in the prior art.
[0004] Technical solution: The pedestrian re-identification device combined with the YOLOv8 model described in the present invention includes: a wireless camera group module, a main control module, a PC end, and a two-dimensional corner module; the wireless camera group module, the main control module, the PC end, and the two-dimensional corner module communicate through wireless signals; the wireless camera group module obtains real-time video streams and transmits them to the PC end through the main control module; the PC end uses the YOLOv8s multi-scale precise pedestrian category detection model to return the dynamic coordinates of a specific pedestrian to the main control module; the main control module performs PID calculations, controls the two-dimensional corner module to make angles, and two-dimensionally tracks specific pedestrians and uploads them to the PC end, and stores them in a query set.
[0005] The pedestrian re-identification method of the present invention is implemented by a pedestrian re-identification device of the YOLOv8 model, and includes the following steps:
[0006] (1) Using the pedestrian re-identification device combined with the YOLOv8 model, pedestrian data is obtained and stored as a query set;
[0007] (2) Construct a YOLOv8s-based multi-scale precise pedestrian category detection model, including a YOLOv8s model and a multi-scale precise pedestrian category detection module; the YOLOv8s model passes the processed image to the multi-scale precise pedestrian category detection module in a serial manner;
[0008] (3) Use the camera-aware pedestrian re-identification method to calculate the similarity between the image output by the YOLOv8s model and the pedestrian images in the query set; if similar, update the query set; if not similar, store them in the query set; the camera-aware pedestrian re-identification method is as follows: first, use the Euclidean distance to obtain the original distance matrix of all pedestrian images; second, measure the distance between pedestrian images based on the overlap of the K-nearest neighbor algorithm, encode the neighboring pedestrian image set of the pedestrian image as a weighted unit vector, and convert the neighboring pedestrian image set comparison problem into a pure vector calculation.
[0009] Furthermore, in step (2), the YOLOv8s model processing process is as follows: YOLOv8s performs target detection on the input video frame. In the detection results, only the detection frame of the category "person", i.e., pedestrians, is further processed; for each detected "person" object, the corresponding detection frame coordinates (X min , Y min , X max , Y max ); use the coordinates to crop the pedestrian image on the original image, and then resize the image; input the cropped and processed pedestrian image and the query set into the multi-scale precise pedestrian category detection module for feature extraction, and pass the features to the detection head for target prediction.
[0010] Furthermore, in step (2), the multi-scale precise pedestrian category detection module is specifically as follows: it is used to extract pedestrian image features and retrieve specific pedestrian IDs, and the backbone network is ResNet50; wherein, the basic learning rate function is replaced by a staged learning rate scheduler, and by combining with a warm-up function, polynomial decay is used for learning rate decay during training, and in the subsequent stage, it switches to cosine annealing to smoothly adjust the learning rate; an adaptive representation joint attention module is added after the third layer of the ResNet50 module, including: a spatial transformer attention block STAB and a squeeze and excitation attention block SEAB; the loss function part is called after ResNet50, and a regularized center loss is added.
[0011] Furthermore, in step (3), the specific formula is as follows:
[0012] Set sample , the sequence formula is:
[0013] ;
[0014] in, Representative samples After sorting, the first The nearest neighbor samples;
[0015] Then use the sample and sequence Represents the K-nearest neighbor set The formula is as follows:
[0016] ;
[0017] in, For samples The number of training samples closest to the Indicates that from the sequence Extract the elements from the first to the elements, including A subsequence of elements;
[0018] K mutual neighbor sets It is expressed as:
[0019] ;
[0020] in, is the number of training samples with the shortest distance after constraint;
[0021] The determination of K mutual nearest neighbor sets is as follows:
[0022] ;
[0023] The constraints are as follows:
[0024] ;
[0025] in, Indicates the number of samples in the set;
[0026] sample The stable mutual neighbor set of is encoded as:
[0027] ;
[0028] in, is an N-dimensional unit vector, if exist In the nearest neighbor set of is based on and The original distance between them is calculated, otherwise it is 0. The formula is as follows:
[0029] ;
[0030] in, for and Considering that similar samples share similar features and neighbors, LQE is used to of The weighted mutual neighbor set encoding vectors of the nearest neighbors are averaged to generate a weighted expansion vector:
[0031] ;
[0032] in, represent The mutual neighbor set encoding vector of for and The number of training samples with the closest distance under strong matching, and finally and The optimized Jaccard distance between them is calculated by vectorized overlap:
[0033] ;
[0034] in, and It is the intersection and union operation in vector form.
[0035] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: To address the performance issues of traditional person re-identification in a multi-camera environment, a camera-aware person re-identification method is introduced. By adaptively adjusting sample features under different cameras, the problem of negative samples introduced between cameras is overcome, and the generalization ability of the model under different cameras is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flow chart of the present invention;
[0037] Figure 2 It is a schematic diagram of the device of the present invention;
[0038] Figure 3 It is a schematic diagram of the model of the present invention. DETAILED DESCRIPTION
[0039] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0040] like Figure 1 As shown, an embodiment of the present invention provides a pedestrian re-identification device combined with a YOLOv8 model, such as Figure 2As shown, it includes: a wireless camera group module, a main control module, a PC terminal, and a two-dimensional corner module; the wireless camera group module, the main control module, the PC terminal, and the two-dimensional corner module communicate through wireless signals; the wireless camera group module obtains real-time video streams and transmits them to the PC terminal through the main control module; the PC terminal uses the YOLOv8s multi-scale precise pedestrian category detection model to return the dynamic coordinates of specific pedestrians to the main control module; the main control module performs PID calculations, controls the two-dimensional corner module to make angles, and tracks specific pedestrians in two-dimensional steering and uploads them to the PC terminal, and stores them in the query set.
[0041] The embodiment of the present invention also provides a pedestrian re-identification method combined with a YOLOv8 model, which is implemented by a pedestrian re-identification device combined with a YOLOv8 model. Figure 3 As shown, the following steps are included:
[0042] (1) Using the pedestrian re-identification device combined with the YOLOv8 model, pedestrian data is obtained and stored as a query set;
[0043] (2) Construct a multi-scale precise pedestrian category detection model based on YOLOv8s, including: YOLOv8s model and multi-scale precise pedestrian category detection module; the YOLOv8s model passes the processed image to the multi-scale precise pedestrian category detection module in a serial manner; the YOLOv8s model processing process is as follows: YOLOv8s performs target detection on the input video frame, and in the detection results, only the detection frame of the category "person", i.e., pedestrian, is further processed; for each detected "person" object, the corresponding detection frame coordinates (X min , Y min , X max , Y max ); use the coordinates to crop the pedestrian image on the original image, and then resize the image; input the cropped and processed pedestrian image and the query set into the multi-scale precise pedestrian category detection module for feature extraction, and pass the features to the detection head for target prediction.
[0044] The multi-scale precise pedestrian category detection module is as follows: it is used to extract pedestrian image features and retrieve specific pedestrian IDs, and the backbone network is ResNet50; the basic learning rate function is replaced by a staged learning rate scheduler, and by combining it with a warm-up function, polynomial attenuation is used for learning rate decay during training, and in subsequent stages it is switched to cosine annealing to smoothly adjust the learning rate; an adaptive representation joint attention module is added after the third layer of the ResNet50 module, including: spatial transformer attention block STAB and squeeze and excitation attention block SEAB; the loss function part is called after ResNet50, and a regularized center loss is added.
[0045] (3) Use the camera-aware pedestrian re-identification method to calculate the similarity between the image output by the YOLOv8s model and the pedestrian images in the query set; if similar, update the query set; if not similar, store it in the query set; the camera-aware pedestrian re-identification method is as follows: First, use the Euclidean distance to obtain the original distance matrix of all pedestrian images; second, measure the distance between pedestrian images based on the overlap of the K-nearest neighbor algorithm, set the neighboring pedestrian image set encoding of the pedestrian image as a weighted unit vector, and convert the neighboring pedestrian image set comparison problem into a pure vector calculation. The specific formula is as follows:
[0046] Set sample , the sequence formula is:
[0047] ;
[0048] in, Representative samples After sorting, the first The nearest neighbor samples;
[0049] Then use the sample and sequence Represents the K-nearest neighbor set The formula is as follows:
[0050] ;
[0051] in, For samples The number of training samples closest to the Indicates that from the sequence Extract the elements from the first to the elements, including A subsequence of elements;
[0052] K mutual neighbor sets It is expressed as:
[0053] ;
[0054] in, is the number of training samples with the shortest distance after constraint;
[0055] The determination of K mutual nearest neighbor sets is as follows:
[0056] ;
[0057] The constraints are as follows:
[0058] ;
[0059] in, Indicates the number of samples in the set;
[0060] sample The stable mutual neighbor set of is encoded as:
[0061] ;
[0062] in, is an N-dimensional unit vector, if exist In the nearest neighbor set of is based on and The original distance between them is calculated, otherwise it is 0. The formula is as follows:
[0063] ;
[0064] in, for and Considering that similar samples share similar features and neighbors, LQE is used to of The weighted mutual neighbor set encoding vectors of the nearest neighbors are averaged to generate a weighted expansion vector:
[0065] ;
[0066] in, represent The mutual neighbor set encoding vector of for and The number of training samples with the closest distance under strong matching, and finally and The optimized Jaccard distance between them is calculated by vectorized overlap:
[0067] ;
[0068] in, and It is the intersection and union operation in vector form.
[0069] A comparative experiment was conducted on the pedestrian re-identification methods under the strong baseline, using Rank1 and MAP as evaluation index parameters. Rank1 can represent the accuracy of the first retrieval target of the image, and MAP represents the accuracy of all retrieval results. As shown in Table 1, compared with the existing methods, the camera-aware pedestrian re-identification method proposed in this invention has excellent performance.
[0070] Table 1 Comparative test of existing methods of camera-aware pedestrian re-identification
[0071] .
[0072] The collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. It is not allowed to illegally collect, use, process, or transmit unauthorized or unprotected data, or to illegally buy, sell, provide, or disclose unauthorized or unprotected data.
Claims
1. A pedestrian re-identification method combined with a YOLOv8 model, characterized in that: The implementation of the pedestrian re-identification device based on the joint YOLOv8s model includes the following steps: (1) The pedestrian re-identification device combined with the YOLOv8s model is used to obtain pedestrian data and store it as a query set; wherein the pedestrian re-identification device combined with the YOLOv8s model includes: a wireless camera group module, a main control module, a PC terminal, and a two-dimensional corner module; the wireless camera group module, the main control module, the PC terminal, and the two-dimensional corner module communicate through wireless signals; the wireless camera group module obtains real-time video streams and transmits them to the PC terminal through the main control module; the PC terminal uses the YOLOv8s multi-scale precise pedestrian category detection model to return the dynamic coordinates of a specific pedestrian to the main control module; the main control module performs PID calculations, controls the two-dimensional corner module to make angles, and two-dimensionally tracks specific pedestrians and uploads them to the PC terminal, and stores them in the query set; (2) Construct a multi-scale precise pedestrian category detection model based on YOLOv8s, including: YOLOv8s model and multi-scale precise pedestrian category detection module; the YOLOv8s model passes the processed image to the multi-scale precise pedestrian category detection module in a serial form; the YOLOv8s model processing process is as follows: YOLOv8s performs target detection on the input video frame, and in the detection results, only the detection frame of the category "person", i.e., pedestrian, is further processed; for each detected "person" object, the corresponding detection frame coordinates (X min , Y min , X max , Y max ); use the coordinates to crop the pedestrian image on the original image, and then adjust the image size; input the cropped and processed pedestrian image and the query set together into the multi-scale precision pedestrian category detection module for feature extraction, and pass the features to the detection head for target prediction; the multi-scale precision pedestrian category detection module is as follows: it is used to extract pedestrian image features and retrieve specific pedestrian IDs, and the backbone network is ResNet50; among them, the basic learning rate function is replaced by a staged learning rate scheduler, and by combining with the warm-up function, polynomial attenuation is used for learning rate attenuation during training, and cosine annealing is switched to smoothly adjust the learning rate in the subsequent stage; an adaptive representation joint attention module is added after the third layer of the ResNet50 module, including: spatial transformer attention block STAB and squeeze and excitation attention block SEAB; the loss function part is called after ResNet50, and the regularized center loss is added; (3) Use the camera-aware pedestrian re-identification method to calculate the similarity between the image output by the YOLOv8s model and the pedestrian images in the query set; if similar, update the query set; if not similar, store them in the query set; the camera-aware pedestrian re-identification method is as follows: first, use the Euclidean distance to obtain the original distance matrix of all pedestrian images; second, measure the distance between pedestrian images based on the overlap of the K-nearest neighbor algorithm, set the neighboring pedestrian image set encoding of the pedestrian image as a weighted unit vector, and convert the neighboring pedestrian image set comparison problem into a pure vector calculation; the specific formula is as follows: Set sample , the sequence formula is: ; in, Representative samples After sorting, the first The nearest neighbor samples; Then use the sample and sequence Represents the K-nearest neighbor set The formula is as follows: ; in, For samples The number of training samples closest to the Indicates that from the sequence Extract the elements from the first to the elements, including A subsequence of elements; K mutual neighbor sets It is expressed as: ; in, is the number of training samples with the shortest distance after constraint; The determination of K mutual nearest neighbor sets is as follows: ; The constraints are as follows: ; in, Indicates the number of samples in the set; sample The stable mutual neighbor set of is encoded as: ; in, is an N-dimensional unit vector, if exist In the nearest neighbor set of is based on and The original distance between them is calculated, otherwise it is 0. The formula is as follows: ; in, for and Considering that similar samples share similar features and neighbors, LQE is used to of The weighted mutual neighbor set encoding vectors of the nearest neighbors are averaged to generate a weighted expansion vector: ; in, represent The mutual neighbor set encoding vector of for and The number of training samples with the closest distance under strong matching, and finally and The optimized Jaccard distance between them is calculated by vectorized overlap: ; in, and It is the intersection and union operation in vector form.
