A pedestrian attribute retrieval system based on fuzzy features

Abstract

The present invention discloses a pedestrian attribute retrieval system based on fuzzy features, comprising a pedestrian recognition module, a pedestrian tracking module and a pedestrian attribute detection module connected in sequence. Compared with the prior art, the present invention realizes lightweight, efficient and fast pedestrian attribute retrieval through the design of lightweight convolution operation, multi-scale feature representation and computing resource adaptability.

Description

Technical Field

[0001] The invention relates to the field of security, and in particular to a pedestrian attribute retrieval system based on fuzzy features. Background Art

[0002] In some scenarios in the security field, it is impossible to obtain reliable visual information of the object to be searched, especially when there are no photos of the object to be searched, and the only way to search is to rely on the fuzzy pedestrian attribute features described by eyewitnesses (such as gender, whether wearing a hat, whether carrying a backpack, etc.). Therefore, it is necessary to design a retrieval system that can retrieve pedestrians based on fuzzy pedestrian attribute features to find pedestrians with the characteristic attributes described by eyewitnesses. Compared with the traditional pedestrian relocalization problem that directly uses pictures or videos as search information, retrieving target pedestrians based on fuzzy feature attributes is a more challenging task because the language description may not be so accurate or easy to miss details. This method requires higher accuracy and reliability. To achieve this retrieval, we need to combine deep learning technology and computer vision technology.

[0003] The functions of security monitoring systems on the market are relatively simple, mainly focusing on face detection, pedestrian recognition, and pedestrian trajectory tracking. However, there are only a few feasible solutions for searching pedestrians using fuzzy feature attributes. These solutions are based on pedestrian attribute retrieval from a single image and cannot meet the requirements of high-speed real-time processing of videos. The latest deep learning models have made some progress in this area of ​​research, but there are still some limitations, such as limited ability to process videos and inability to effectively and continuously track target pedestrians with certain characteristic attributes. In addition, the movement of target pedestrians in the video will cause changes in the characteristics of pedestrians captured by the camera, such as changes in lighting, pedestrians being blocked, changes in pedestrian posture, and changes in orientation, which increase the difficulty of retrieval. In summary, these challenges mainly include three aspects that make the search unable to achieve satisfactory results:

[0004] (1) Real-time challenge: Pedestrian attribute retrieval usually needs to be performed in a real-time environment, such as real-time video surveillance or security systems. In pedestrian attribute retrieval, real-time is a key issue, mainly reflected in time constraints and video data processing. In a real-time environment, the algorithm needs to process and analyze the input video within a limited time to quickly and accurately retrieve the attributes of pedestrians. In addition, real-time pedestrian attribute retrieval usually requires efficient processing of video data, so the algorithm must be able to effectively process this data to ensure that the task is completed in real time.

[0005] (2) Robustness challenge: The detection of pedestrian attributes needs to be robust under various environmental conditions, including changes in lighting and perspective. Under different lighting conditions, the appearance of pedestrians will be affected. For example, shadows, highlights, or color distortion may appear. At the same time, pedestrians may appear in different postures and angles, or be partially occluded. The model needs to adapt to these changes, be able to recognize different perspectives, and accurately detect pedestrian attributes.

[0006] (3) Diversity and complexity challenges: Pedestrian attribute detection is a field with diversity and complexity challenges. The task involves multiple attributes, such as pedestrian orientation, clothing, gender, age, etc., and there are complex associations and changes between these attributes. Therefore, in order to effectively handle the associations and changes between different attributes, an effective model is needed to meet these challenges. Summary of the invention

[0007] In order to overcome the deficiencies of the prior art, the present invention proposes a pedestrian attribute retrieval system based on fuzzy features, so as to retrieve target pedestrians through fuzzy feature attributes.

[0008] The technical solution of the present invention is implemented as follows: a pedestrian attribute retrieval system based on fuzzy features, comprising a pedestrian recognition module, a pedestrian tracking module and a pedestrian attribute detection module connected in sequence, wherein

[0009] The pedestrian recognition module is an improved YOLOv5 network, including MBConv, FusedMBConv modules and a channel attention mechanism. The improved YOLOv5 network training process includes:

[0010] S11, data set preparation, data preprocessing and enhancement. The data set includes training set and validation set, as well as annotation files;

[0011] S12, improved YOLOv5 network construction, adding MBConv and Fused-MBConv modules to the YOLOv5 network architecture;

[0012] S13, training the improved YOLOv5 for multiple rounds of iterative training until the model converges;

[0013] The pedestrian tracking module uses the pedestrian image obtained by the pedestrian recognition module as input data. The pedestrian tracking method includes the following steps:

[0014] S21, target initialization. In the initial frame V0, the target detection module marks the ones with confidence greater than τ2 as T high , mark the confidence level between τ1 and τ2 as T low, where τ2>τ1>0, all the detection frames of the input images and the objects in the detection frames are considered as untracked. The detection frames set as untracked will skip matching with other targets, and the detection frames and objects in the detection frames with confidence greater than τ (τ=τ1+τ2) will be initialized as tracked;

[0015] S22, each time pedestrian tracking is performed, the previously input V0~V t-1 The frame undergoes initialization target, which is the same as the current frame V t To match, the current frame V t The target is also initialized when input, as the next frame V t+1 The matching object;

[0016] S23, the previously input V0~V t-1 Frame T high And marked as the tracked detection box, use Kalman filter to predict the detection box mean and residual covariance matrix, and convert V t The input detection box and the previously input frames V0~V t-1 The detection frame calculates the intersection over union (Intersection over Union: Calculate the ratio of the intersection and union between the detection result and the true target to measure the accuracy of the detection result, which can reflect the relationship between the detection frame and the predicted frame). Calculate the cost matrix based on the intersection over union, and then take the score of the detection frame into account. By fusing the cost matrix and the detection frame score, the intersection over union similarity is finally obtained (the result of fusing the cost matrix and the detection frame score). Input the intersection over union similarity into the linear assignment algorithm to obtain the matched index, unmatched target ① and unmatched detection frame ②, and use the Kalman gain for the detection frame in the matched index whose status is tracking. By using the Kalman gain, the mean and residual covariance matrix obtained by the Kalman filter can be updated to the latest estimated values, thereby updating the position of the detection frame, and finally reactivating the detection frame whose status is lost;

[0017] S24, use the unmatched target ① in the state of tracking and V0~V t-1 Frame input and marked as T low The detection frame calculates the intersection-over-union ratio and calculates the cost matrix based on the intersection-over-union ratio. Then the score of the detection frame is taken into account. By fusing the cost matrix and the detection frame score, the intersection-over-union ratio similarity is finally obtained. The intersection-over-union ratio similarity is input into the linear assignment algorithm to obtain the matched index, the unmatched target ③ and the unmatched detection frame ④. The detection frame whose status is tracked in the index of this match is used with the Kalman gain. By using the Kalman gain, the mean and residual covariance matrix obtained by the Kalman filter can be updated to the latest estimated values, thereby updating the position of the detection frame and setting the status of the unmatched target ③ to lost;

[0018] S25, using the unmatched detection frame ② and the previously input V0~V t-1 The frame is marked as T high And the detection frame in the state of untracked calculates the intersection and union ratio. The cost matrix is ​​calculated based on the intersection and union ratio, and then the score of the detection frame is taken into account. By fusing the cost matrix and the detection frame score, the intersection and union ratio similarity is finally obtained. The intersection and union ratio similarity is input into the linear assignment algorithm to obtain the matching index, the unmatched target ⑤ and the unmatched detection frame ⑥. The detection frame in the index of this match with the tracking status uses the Kalman gain. By using the Kalman gain, the mean and residual covariance matrix obtained by the Kalman filter can be updated to the latest estimated values, so as to update the position of the detection frame, set the status of the unmatched target ⑤ to delete, and if the confidence of the unmatched detection frame ⑥ is greater than τ, where τ=τ1+τ2, then the detection target is set to tracked. If the detection frame with the status set to untracked has no match for more than 30s, it is deleted;

[0019] The pedestrian attribute detection module includes an improved VOVNet network, and the improved VOVNet network training process includes:

[0020] S31, preparation of data set and data preprocessing and enhancement, wherein the data set includes a training set and a validation set, and also includes image files and corresponding label files;

[0021] S32, construction of the improved VOVNet network, selecting the VOVNet network as the backbone network for pedestrian attribute detection, and connecting an adaptive flat pooling and a one-dimensional convolution operation at the back;

[0022] S33, training the improved VOVNet network, inputting training data into the improved VOVNet network, calculating the improved VOVNet network output and loss function, and updating the parameters of the improved VOVNet network using a back propagation algorithm.

[0023] Furthermore, in the improved YOLOv5 network training process, the data preprocessing and enhancement in S11 includes the following steps:

[0024] S111, loading label data, for each image, parsing the label data, obtaining bounding box coordinates and category labels, and converting the bounding box coordinates and category labels into a format required by the model;

[0025] S112, image resizing, image normalization and image enhancement.

[0026] Furthermore, the image enhancement in S112 includes but is not limited to: random cropping, rotation or flipping, mosaic data enhancement, copy and paste data enhancement and affine transformation.

[0027] Furthermore, in the improved YOLOv5 network training process, S13 also includes the step of using the trained YOLOv5 network to perform target detection on the test set, and calculating the target detection indicators, including accuracy, recall rate and average precision mean.

[0028] Furthermore, the evaluation tool for evaluating the improved YOLOv5 network is COCO API, and mAP is calculated using COCOAPI.

[0029] Furthermore, the improved VOVNet network training process S31 also includes the following steps:

[0030] S311, load the images and label data of the training set and the validation set, and convert the label data into the format required by the model;

[0031] S312, image resizing, image normalization and image enhancement.

[0032] Furthermore, the image enhancement in S312 includes but is not limited to random cropping, random flipping, random rotation, random scaling, random brightness and contrast adjustment, random color transformation and random noise addition.

[0033] Furthermore, in S33, the loss function for training the improved VOVNet network uses a cross entropy loss function, and its optimizer is SGD. The stochastic gradient descent algorithm is used to perform multiple rounds of iterative training on the training set, and the model parameters are optimized through back propagation and gradient update.

[0034] Furthermore, the S33 also includes the steps of: evaluating the trained model using a validation set, and using an evaluation tool to evaluate the performance of the improved VOVNet network on the PA100K dataset.

[0035] Furthermore, the evaluation tool is Top-1 or Top-5 accuracy.

[0036] The beneficial effect of the present invention is that, compared with the prior art, the present invention realizes lightweight, efficient and fast pedestrian attribute detection through the design of lightweight convolution operation, multi-scale feature representation and computing resource adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the structure of a pedestrian attribute retrieval system based on fuzzy features of the present invention;

[0038] Figure 2 It is a schematic diagram of the structure of the pedestrian recognition module in the present invention;

[0039] Figure 3It is a schematic diagram of the improved YOLOv5 network structure in the present invention;

[0040] Figure 4 It is a schematic diagram of the structure of the pedestrian tracking module in the present invention;

[0041] Figure 5 It is a schematic diagram of the improved VOVNet network structure in the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] See also Figure 1 The present invention discloses a pedestrian attribute retrieval system based on fuzzy features, comprising a pedestrian recognition module, a pedestrian tracking module and a pedestrian attribute detection module which are connected in sequence.

[0044] See also Figure 2 and Figure 3 , the present invention improves the backbone network of CSPDarknet53 and optimizes the network structure system of the model. The backbone of the original CSPDarknet53 replaces the CBS (Conv+Batch Normalization+SiLU) in the network with other convolution modules. We use an efficient convolutional neural network, the structure of which includes MBConv and FusedMBConv modules and a channel attention mechanism (SE: Squeeze-and-Excitation Networks block), which can effectively reduce the number of parameters and the amount of calculation. By using MBConv and FusedMBConv modules, the number of parameters and the amount of calculation of the CSPDarknet53 network can be effectively reduced, the requirements for computing resources are reduced, and the speed and efficiency of locating pedestrians are improved. The introduction of the channel attention mechanism (SE) can enhance the feature representation ability of the network, so that the improved YOLOv5 has a wider range of applicability and stronger performance advantages in practical applications. By optimizing the network structure of the model and improving performance, the improved YOLOv5 can better cope with the task of locating pedestrians in practical applications and has a wider range of applicability.

[0045] The training process of the pedestrian recognition module includes:

[0046] Dataset preparation: Download and decompress the COCO dataset, including the training set and the validation set. Make sure that the dataset contains a label file (such as a label file in JSON format) that contains the location and category information of the target object.

[0047] Data preprocessing and enhancement: First, the input COCO dataset needs to be preprocessed and enhanced. Load the label data, usually a file containing bounding box coordinates and category labels. For each image, parse the label data to obtain the bounding box coordinates and category labels. Convert the bounding box coordinates and category labels to the format required by the model, such as the following two common formats: Bounding box format: Four coordinate values ​​are usually used to represent the position of the bounding box (such as the coordinates of the upper left corner and the lower right corner). Category label format: One-hot encoding can be used to represent the category label, such as "1" for pedestrians. Image resizing, image normalization, data enhancement (such as random cropping, rotation or flipping, Mosaic data enhancement, copy and paste data enhancement, affine transformation, etc.) to enhance the robustness and generalization ability of the model.

[0048] Model preparation: Download the YOLOv5 codebase, which can be obtained from the official GitHub repository. Install the required dependencies, such as PyTorch, OpenCV, etc. We are improving YOLOv5n, so we need to modify the YOLOv5n yaml file.

[0049] Network construction: In the network architecture of YOLOv5, MBConv and Fused-MBConv modules are added to make full use of the feature extraction capabilities of the new modules to reduce the amount of calculation while increasing the training speed as much as possible.

[0050] Parameter setting: Use genetic algorithm hyperparameter evolution strategies, such as learning rate, batch size, network structure, etc. Genetic algorithms can improve the performance of the model by optimizing the values ​​of these hyperparameters. Genetic algorithms continuously iterate to generate new hyperparameter combinations by performing operations such as mutation, crossover, and selection on hyperparameters, and screen out the optimal hyperparameter combination by evaluating its performance on the validation set. Because the hyperparameters provided by YOLOv5 are for CSPDarknet53, and we are using the modified network, we need to set appropriate hyperparameters. Including learning rate, batch size, regularization parameters, etc., finally ensure that the parameter settings can be fully trained and optimized.

[0051] Training process: Run the training script, specify the path of the training set and validation set, batch size, learning rate and other parameters. Use appropriate optimization algorithms such as and loss functions to train the improved YOLOv5. According to the characteristics and goals of the training data set, set the appropriate number of iterations and training strategies. You can perform multiple rounds of iterative training as needed until the model converges, ensuring that the new network can converge in YOLOv5 and provide excellent detection performance.

[0052] When evaluating the model, CIoU (Complete Intersection over Union: a common indicator for measuring the performance of target detection algorithms) is used to measure the difference between the predicted box and the true box. The composite loss function used in training includes classification loss, positioning loss, and target confidence loss. Classification loss is used to measure the difference between the target category in the predicted box and the true category, and is usually calculated using the cross entropy loss function. Positioning loss is used to measure the position accuracy of the predicted box, and the positioning loss function used here is the square loss function. Target confidence loss is used to measure the IoU value between the predicted box and the true box, using the binary cross entropy loss function.

[0053] In terms of optimizer, YOLOv5 uses an optimizer called AdamW. AdamW is a gradient-based optimization algorithm that combines the Adam optimizer with the weight decay method. It has a faster convergence speed and better generalization performance, and can effectively optimize network parameters.

[0054] Evaluation and testing: Use the trained model to perform target detection on the test set and calculate target detection indicators such as Precision, Recall, and mAP (Mean Average Precision). You can use evaluation tools such as COCO API to calculate mAP. Compared with YOLOv5 or other related algorithms based on traditional backbone networks, the improved YOLOv5 has increased the number of network layers from 213 to 388, the number of parameters has decreased from 1.86 million to 1.49 million, and the time to infer a batch (5,000 images) has decreased from 18.91s to 11.07s (the GPU used is NVIDIA T4).

[0055] See also Figure 4The input of the pedestrian tracking module is the video sequence V, and the final output is the trajectory T of the pedestrian, that is, each trajectory is the bounding box of the object in each frame. Kalman filtering is a commonly used state estimation method. Its basic principle is to estimate the state of the system by jointly inferring the dynamic model and observation model of the system. By jointly inferring the dynamic model and observation model of the system, the state of the system can be estimated in an uncertain environment. The cost matrix can be calculated using the intersection-over-union ratio and the detection frame, and then the cost matrix and the detection frame are used for score fusion to calculate their IoU similarity. The IoU similarity is input into the linear assignment algorithm to filter out the qualified detection frames, and finally the trajectory is updated using the Kalman filter.

[0056] The use of the pedestrian tracking module can effectively solve the robustness challenge of pedestrian attribute detection and tracking. The tracking capability provided by this module can also help reduce false detections and missed detections. By associating the detection box with the historical trajectory of the target, the identity and position of the target can be judged more accurately, avoiding misjudging adjacent targets or backgrounds as targets. This enables the model to better cope with situations such as rapid movement, occlusion, and scale changes of targets in the video in consecutive frames.

[0057] The pedestrian tracking method in the pedestrian tracking module includes the following steps:

[0058] Data preparation: The previous target detection module can detect multiple pedestrian targets in an image or video in real time. Use the pedestrian array obtained by the previous module and use the pedestrian images stored in it as input data.

[0059] Target initialization: In the initial frame (V0), the target detection module marks the objects with confidence greater than τ2 as T high , mark the confidence level between τ1 and τ2 as T low (τ2>τ1>0) Then, the detection frames of all input images and the object status in the detection frames are regarded as untracked. The detection frames set as untracked will skip matching with other targets, and the detection frames and objects in the detection frames with confidence greater than τ (τ=τ1+τ2) will be initialized to tracked.

[0060] Pedestrian tracking: Each time pedestrian tracking is performed, the previously input frames (V0~V t-1 ) undergoes initialization, and the current frame (V t ) to match, the current frame (V t ) input also undergoes target initialization as the next frame (V t+1 ) of the target.

[0061] Specific implementation: The previously input frame (V0~V t-1) The detection box with high confidence and marked as tracked uses Kalman filtering to predict the detection box mean and residual covariance matrix, and V t The input detection box and the previously input frames V0~V t-1 The detection frame calculates the intersection over union ratio and the cost matrix based on the intersection over union ratio. Then the score of the detection frame is taken into account. The IoU similarity is finally obtained by fusing the cost matrix and the detection frame score. The IoU similarity is input into the linear assignment algorithm to obtain the matched index, the unmatched target ① and the unmatched detection frame ②. The detection frame in the matched index with the tracking status is used with the Kalman gain. By using the Kalman gain, the mean and residual covariance matrix obtained by the Kalman filter can be updated to the latest estimated values, thereby updating the position of the detection frame and finally reactivating the detection frame with the lost status.

[0062] We use the unmatched target① in the state of tracking and V0~V t-1 Frame input and marked as T low The detection frame calculates the intersection over union ratio and calculates the cost matrix based on the intersection over union ratio. Then the score of the detection frame is taken into account. The IoU similarity is finally obtained by fusing the cost matrix and the detection frame score. The IoU similarity is input into the linear assignment algorithm to obtain the matched index, the unmatched target ③ and the unmatched detection frame ④. The detection frame whose status is tracked in the index of this match is used with the Kalman gain. By using the Kalman gain, the mean and residual covariance matrix obtained by the Kalman filter can be updated to the latest estimated value, thereby updating the position of the detection frame and setting the status of the unmatched target ③ to lost.

[0063] Finally, use the unmatched detection box ② and the previously input V0~V t-1 The frame is marked as T high And the detection frame in the state of untracked calculates the intersection and union ratio. According to the intersection and union ratio, the cost matrix is ​​calculated, and then the score of the detection frame is taken into account. By fusing the cost matrix and the detection frame score, the IoU similarity is finally obtained (the result of fusing the cost matrix and the detection frame score). The IoU similarity is input into the linear assignment algorithm to obtain the matched index, the unmatched target ⑤ and the unmatched detection frame ⑥. The detection frame in the state of tracking in the index of this match is used with the Kalman gain. By using the Kalman gain, the mean and residual covariance matrix obtained by the Kalman filter can be updated to the latest estimated value, so as to update the position of the detection frame, set the state of the unmatched target ⑤ to delete, and if the confidence of the unmatched detection frame ⑥ is greater than τ, where τ=τ1+τ2, then the detection target is set to tracked. If the detection frame with the state set to untracked has no match for more than 30s, it is deleted.

[0064] See also Figure 5,Current technologies have some difficulties and limitations in dealing with the diversity and complexity challenges in the field of pedestrian attribute detection. For example, there are interdependencies and correlations between different attributes. The orientation of pedestrians may affect the system's judgment of clothing appearance, gender, and age, which in turn will affect the detection accuracy. Therefore, when performing pedestrian attribute detection, it is necessary to consider the association between different attributes to obtain accurate results. However, current technologies often find it difficult to effectively model and capture the complex associations between these attributes, resulting in a decrease in detection accuracy when dealing with multiple attributes.

[0065] Traditional pedestrian attribute detection methods usually rely on hand-designed feature extractors, using color histograms, texture features, and shape features to represent human attribute features. These feature extractors often fail to fully capture the rich attribute information of pedestrians, are difficult to generalize, and have poor versatility. Simple metrics such as Euclidean distance or cosine similarity are used for matching. These metrics may not be accurate or effective enough in some cases, especially in high-dimensional spaces, non-uniformly distributed data, or sensitive to outliers, resulting in low accuracy and robustness of detection results. The attribute classifiers used in existing methods often face the ambiguity and diversity of attribute semantics, resulting in low accuracy of classification results. In addition, the definition and understanding of attributes vary greatly, especially in the case of certain clothing attributes, which makes attribute classification difficult, further reducing the robustness of the model.

[0066] The present invention improves the VOVNet network, and adopts a series of lightweight convolution operations in the network, such as 1x1 convolution, 3x3 convolution and CBS (Conv+Batch Normalization+SiLU). By introducing multiple branches and multiple scales of convolution operations, it is possible to simultaneously extract feature representations of different scales. Finally, in order to enhance the ability to extract features, an attention mechanism (ESE: Effective Squeeze-and-Excitation Block) is added. At the same time, 1x1 convolution is used to reduce the number of channels of the feature map and merge feature maps in the channel dimension. This approach is very beneficial for the detection of fine-grained features. It can better capture and process the associations and changes between different attributes, thereby improving the performance of pedestrian attribute detection. Using the VOVNet network in the system solves the problem of interdependence and association between different attributes in pedestrian attribute detection.

[0067] When detecting attributes, it is necessary to detect each pedestrian in the image, which requires a large number of parameters to represent and process this information. Pedestrian attribute detection (such as gender, age, direction, etc.) involves a lot of calculations. Therefore, in order to better adapt to the actual situation and improve performance, the design fully considers the characteristics of the task of detecting pedestrian attributes. In order to reduce the number of parameters, we use GAP (Global Average Pooling) and 1x1 convolution to replace the traditional fully connected layer in CNN, thereby achieving the purpose of parameter reduction. The use of GAP is based on a more natural transformation of the convolution structure to enhance the correspondence between feature maps and categories.

[0068] The training process of the pedestrian attribute recognition module includes:

[0069] Dataset preparation: Download and decompress the PA100K dataset, including the training set and the validation set. Make sure the dataset contains image files and corresponding label files. The label file can be the category label corresponding to each image.

[0070] Data preprocessing and enhancement: Before using VOVNet, the input PA100k dataset is preprocessed and enhanced. Load the images and label data of the training set and validation set. Convert the label data to the format required by the model, such as encoding the label One-Hot and representing the pedestrian attributes with key-value pairs. Image resize, image normalization, data enhancement (such as random cropping, random flipping, random rotation, random scaling, random brightness and contrast adjustment, random color transformation, random noise addition, etc.) to enhance the robustness and generalization ability of the model.

[0071] Model preparation: Download the VOVNet codebase, which can be obtained from the official GitHub repository. Install the required dependencies, such as PyTorch, OpenCV, etc. We improve the network and modify the code of the provided network structure.

[0072] Network construction: VOVNet is selected as the backbone network for pedestrian attribute detection. VOVNet is a deep convolutional neural network architecture for image classification and target detection. It has a strategy for capturing "multi-scale feature representation" and efficient information transfer. However, since the number of stacked parameters of the network cannot be underestimated, an adaptive flat pooling and a one-dimensional convolution operation are connected at the back to convert the feature map into a one-dimensional tensor, greatly reducing the parameters.

[0073] Training process: input training data into the network, calculate network output and loss function, and use the back-propagation algorithm to update network parameters. The default learning rate and batch size can be used. The loss function for training VOVNet uses the cross-entropy loss function, which can measure the difference between the predicted category and the true category. The optimizer is SGD (Stochastic Gradient Descent): a stochastic gradient descent algorithm that performs multiple rounds of iterative training on the training set to optimize model parameters through back-propagation and gradient updates. In each round of iteration, forward propagation and back-propagation are performed using the images and label data of the training set to calculate the loss and update the model parameters. The number of training rounds can be selected by observing the performance of the validation set.

[0074] Evaluation and testing: Use the validation set to evaluate the trained model and calculate indicators such as classification accuracy. Evaluation tools such as Top-1 and Top-5 accuracy can be used to evaluate the performance of the model on the PA100K dataset. Performance comparison with classification models based on the VOVNet network or other related algorithms shows that the improved VOVNet network has a reduced number of model parameters and floating-point operations trained on the PA100K dataset, with a Top-1 accuracy of 81%, while the baseline is only 78%.

[0075] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A pedestrian attribute retrieval system based on fuzzy features, characterized in that: It includes a pedestrian recognition module, a pedestrian tracking module and a pedestrian attribute detection module connected in sequence, wherein The pedestrian recognition module is an improved YOLOv5 network, including MBConv, FusedMBConv modules and a channel attention mechanism. The improved YOLOv5 network training process includes: S11, data set preparation, data preprocessing and enhancement. The data set includes training set and validation set, as well as annotation files; S12, improved YOLOv5 network construction, adding MBConv and Fused-MBConv modules to the YOLOv5 network architecture; S13, training the improved YOLOv5 for multiple rounds of iterative training until the model converges; The pedestrian tracking module uses the pedestrian image obtained by the pedestrian recognition module as input data. The pedestrian tracking method includes the following steps: S21, target initialization. In the initial frame V0, the target detection module marks the ones with confidence greater than τ2 as T high , mark the confidence level between τ1 and τ2 as T low , where τ2>τ1>0, all the detection frames of the input images and the objects in the detection frames are considered as untracked. The detection frames set as untracked will skip matching with other targets, and the detection frames and objects in the detection frames with confidence greater than τ will be initialized as tracked, where τ=τ1 + τ2; S22, each time pedestrian tracking is performed, the previously input V0~V t-1 The frame undergoes initialization target, which is the same as the current frame V t To match, the current frame V t The target is also initialized when input, as the next frame V t+1 The matching object; S23, the previously input V0~V t-1 Frame T high And marked as the tracked detection box, use Kalman filter to predict the detection box mean and residual covariance matrix, and convert V t The input detection box and the previously input frame V0~V t-1 The detection frame calculates the intersection-and-union ratio, calculates the cost matrix based on the intersection-and-union ratio, obtains the intersection-and-union ratio similarity by fusing the cost matrix and the detection frame score, inputs the intersection-and-union ratio similarity into the linear assignment algorithm to obtain the matched index, the unmatched target a and the unmatched detection frame b, uses the Kalman gain for the detection frame in the matched index whose state is tracking, and updates the mean and residual covariance matrix obtained by the Kalman filter to the latest estimated value by using the Kalman gain, thereby updating the position of the detection frame and reactivating the detection frame whose state is lost; S24, use the unmatched target a in the state of tracking and V0~V t-1 Frame input and marked as T low The detection frame calculates the intersection-and-union ratio and calculates the cost matrix based on the intersection-and-union ratio. The intersection-and-union ratio similarity is finally obtained by fusing the cost matrix and the detection frame score. The intersection-and-union ratio similarity is input into the linear assignment algorithm to obtain the matched index, the unmatched target c and the unmatched detection frame d. The detection frame whose status is tracked in the index of this match is used with the Kalman gain. The Kalman gain can be used to update the mean and residual covariance matrix obtained by the Kalman filter to the latest estimated values, so as to update the position of the detection frame and set the status of the unmatched target c to lost. S25, using the unmatched detection box b and the previously input V0~V t-1 The frame is marked as T high The detection frame whose status is untracked calculates the intersection-and-union ratio. The cost matrix is ​​calculated based on the intersection-and-union ratio. The intersection-and-union ratio similarity is obtained by fusing the cost matrix and the detection frame score. The intersection-and-union ratio similarity is input into the linear assignment algorithm to obtain the matching index, the unmatched target e and the unmatched detection frame f. The detection frame whose status is tracked in the index of this match uses the Kalman gain. The mean and residual covariance matrix obtained by the Kalman filter are updated to the latest estimated value by using the Kalman gain, so as to update the position of the detection frame. The status of the unmatched target e is set to deleted. If the confidence of the unmatched detection frame f is greater than τ, where τ=τ1 +τ2, then the detection target is set to tracked. If the detection frame whose status is set to untracked has no match for more than 30s, it is deleted. The pedestrian attribute detection module includes an improved VOVNet network, and the improved VOVNet network training process includes: S31, preparation of data set and data preprocessing and enhancement, wherein the data set includes a training set and a validation set, and also includes image files and corresponding label files; S32, construction of the improved VOVNet network, selecting the VOVNet network as the backbone network for pedestrian attribute detection, and connecting an adaptive flat pooling and a one-dimensional convolution operation at the back; S33, training the improved VOVNet network, inputting training data into the improved VOVNet network, calculating the improved VOVNet network output and loss function, and updating the parameters of the improved VOVNet network using a back propagation algorithm.

2. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 1, characterized in that: During the improved YOLOv5 network training process, the data preprocessing and enhancement in S11 includes the following steps: S111, loading label data, for each image, parsing the label data, obtaining bounding box coordinates and category labels, and converting the bounding box coordinates and category labels into a format required by the model; S112, image resizing, image normalization and image enhancement.

3. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 2, characterized in that: The image enhancement in S112 includes but is not limited to: random cropping, rotation or flipping, Mosaic data enhancement, copy and paste data enhancement and affine transformation.

4. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 2, characterized in that: In the improved YOLOv5 network training process, S13 also includes the steps of: using the trained YOLOv5 network to perform target detection on the test set, and calculating the target detection indicators, including accuracy, recall rate and average precision mean.

5. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 4, characterized in that: The evaluation tool for evaluating the improved YOLOv5 network is COCO API, and mAP is calculated using COCO API. Furthermore, the improved VOVNet network training process S31 also includes the following steps: S311, load the images and label data of the training set and the validation set, and convert the label data into the format required by the model; S312, image resizing, image normalization and image enhancement.

6. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 5, characterized in that: The image enhancement in S312 includes but is not limited to random cropping, random flipping, random rotation, random scaling, random brightness and contrast adjustment, random color transformation, and random noise addition.

7. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 1, characterized in that: In S33, the loss function for training the improved VOVNet network uses the cross entropy loss function, and its optimizer is SGD. The stochastic gradient descent algorithm is used to perform multiple rounds of iterative training on the training set, and the model parameters are optimized through back propagation and gradient update.

8. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 7, characterized in that: The S33 also includes the steps of: evaluating the trained model using a validation set, and using an evaluation tool to evaluate the performance of the improved VOVNet network on the PA100K dataset.

9. The pedestrian attribute retrieval system based on fuzzy features as claimed in claim 8, characterized in that: The evaluation tool is Top-1 or Top-5 accuracy.

Images