An improved pedestrian attribute monitoring and recognition method

Abstract

The present invention discloses an improved pedestrian attribute recognition method, relating to the field of visual image processing technology. The present invention includes the following steps: obtaining an image or video containing a human body or face from a camera, wherein the camera is equipped with a HiSilicon chip, and a trained model is transplanted into the HiSilicon chip, wherein the model includes a detection model based on YOLOv5 and a classification model based on ResNet; labeling pedestrians based on data collected from a pedestrian attribute dataset, wherein the YOLOv5-based detection model detects the facial region and the entire body region from the image or video information and frames them respectively; and the ResNet-based classification model identifies and classifies the specific attributes of the framed facial region and the entire body region. The present invention improves the accuracy of pedestrian recognition by providing a new pedestrian attribute recognition technical solution, and transplants this technical solution to a camera to achieve all-day real-time intelligent monitoring of suspicious persons.

Description

Technical Field

[0001] The invention belongs to the technical field of visual image processing and relates to an improved pedestrian attribute monitoring and recognition method. Background Art

[0002] Traditional approaches to pedestrian attribute recognition rely on manual feature extraction, classifiers, or the development of robust feature representations from an attribute perspective. Methods like HOG and LBP can only mine low-level, deep image features and are generally insufficient for practical applications. With the continuous advancement of computer vision, deep learning technologies have emerged in recent years, enabling the extraction of high-level semantic information. For example, face detection and recognition technologies have matured and are widely used. However, the estimation of key attributes of faces or bodies, such as age and gender, remains a research hotspot in pattern recognition and machine learning, with significant potential.

[0003] Pedestrian attribute recognition involves examining or identifying the visual semantic features within a given human frame by analyzing the mapping relationship between human attribute features and low-level image features. Existing deep learning-based pedestrian attribute recognition methods fall into two main categories: one that independently learns each attribute, treating each attribute as a binary classification; the other that uses a single model to evaluate all attributes, a multi-label or multi-classification task. However, existing technologies suffer from various issues, including low pedestrian attribute recognition accuracy:

[0004] Data issues:

[0005] 1) Each pedestrian has different attributes, so the number of attributes is variable, which will lead to the problem of imbalanced distribution.

[0006] 2) The problem of pedestrian perspective diversity caused by sampling from different camera angles.

[0007] 3) The camera resolution in actual scenes is generally low. When a person moves, the human body captured by the camera may be blurred.

[0008] Network structure issues:

[0009] 1) Ignoring the analysis of the relationship between different attributes

[0010] 2) The classification model does not use the latest backbone. Summary of the Invention

[0011] In order to overcome the shortcomings of the existing technology, the present invention provides an improved pedestrian attribute monitoring and recognition method. By setting a new pedestrian attribute recognition technical solution, the accuracy of pedestrian recognition is improved, and the technical solution is transplanted to a camera to achieve real-time intelligent monitoring of suspicious persons throughout the day, thereby reducing the occurrence of safety hazards. The present invention is achieved through the following technical solution: the present invention provides an improved pedestrian attribute monitoring and recognition method, comprising the following steps: obtaining a picture or video with a human body or face from a camera, the camera reading the picture and video information and transmitting the information to the NNIE module of the HiSilicon chip inside the camera for data loading, the camera being installed with a HiSilicon chip, the HiSilicon chip being transplanted with a trained data model, the data model including a detection model based on YOLOv5 and a classification model based on ResNetSt; performing label recognition on pedestrians based on data collected from the pedestrian attribute dataset, firstly detecting the face area and the whole body area from the picture or video information based on the YOLOv5 detection model and framing them respectively; then identifying and classifying the specific attributes of the framed face area and the whole body area based on the ResNetSt classification model.

[0012] As a preferred solution, dataset preparation is performed before pedestrian attribute recognition. The LabelImg open source annotation tool is used to label the pedestrian video frames or images captured by the camera. The labels are the person pedestrian dataset, and the division rule is that the training dataset accounts for 80% and the verification dataset accounts for 20%. The pedestrian attribute dataset uses the open source PA-100k pedestrian attribute dataset, which includes 23 labels such as age, gender, wearing glasses, and backpack.

[0013] As a preferred solution, the YOLOv5-based detection model includes the following training steps:

[0014] (1) Loading the pre-trained model: For a single obvious label such as a pedestrian, the yolov5s pre-trained model with the fastest model inference time is selected and placed in the CUDA thread;

[0015] (2) Loading data: Load the training and validation datasets in tensor format and place them in the CUDA thread;

[0016] (3) Select the optimizer: Select the batch random descent SGD optimizer or the adaptive Adam optimizer, and design the optimizer's learning strategy;

[0017] (4) Select loss function: YOLOv5 uses fixed IOU Loss and Obj Loss;

[0018] (5) Start iterative training: Design the number of iterations (epochs) and batch size (batch), and design the precise, recall, and mAP evaluation indicators on the validation dataset after each iteration. Update the model weights and learning rate after each iteration until the model's indicators on the validation dataset no longer increase, indicating that the model training is complete.

[0019] (6) Model transplantation: transplant the trained model to the camera HiSilicon chip and start real-time testing.

[0020] As a preferred solution, the classification model based on ResneSt includes the following training steps:

[0021] (1) Loading the model: Load the ResneSt model architecture and place it in a CUDA thread;

[0022] (2) Data augmentation: Data augmentation techniques mainly include image rotation, cropping, and changing the brightness, contrast, and saturation of images, with the aim of expanding data diversity and reducing the overfitting problem of the model;

[0023] (3) Loading data: Load the tensor-formatted training and validation datasets with a data ratio of 8:2 and place them in the CUDA thread;

[0024] (4) Select the optimizer: Select the batch random descent SGD optimizer or the adaptive Adam / AdamW optimizer, and design the optimizer's learning strategy;

[0025] (5) Select the loss function: use the classification loss function BCE Loss or Focal Loss;

[0026] (6) Start iterative training: Design the number of iterations (epochs) and the batch size (batch). The specific training steps are: in the first stage, calculate the loss value and accuracy of each attribute during the training process; in the second stage, establish a separate network branch for the attribute with a large average loss and a small accuracy, while the other attributes remain on the original branch, and then the two branches jointly predict all attributes; in the third stage, create two new network branches with the same structure as the branches in the second stage, and optimize the parameters of the new branches to make their attribute recognition performance better than the second stage. After each iteration of each stage, the precision and recall evaluation indicators are used on the validation dataset. After each iteration, the model weights and learning rate are updated once until the model training is completed. Finally, the model in the third stage is used for attribute prediction;

[0027] (7) Model transplantation: transplant the trained model to the camera HiSilicon chip and start real-time testing.

[0028] As a preferred solution, the Focal Loss loss function is:

[0029]

[0030] The beneficial effects of the present invention are:

[0031] The solution of the present invention improves the overall pedestrian attribute recognition capability, and combined with facial attribute recognition assistance, it achieves a breakthrough from face scanning to person recognition. By setting up a new pedestrian attribute recognition technology solution, the present invention aims to improve the accuracy of pedestrian recognition and transplant it to the camera to achieve real-time intelligent monitoring of suspicious persons throughout the day, reducing the occurrence of safety hazards.

[0032] By identifying pedestrians' gender, age, clothing, and appearance, cameras can quickly identify them and assist in locating and tracking specific people, such as suspicious people in shopping malls, effectively assisting the police in deploying security or solving cases, and reducing safety hazards.

[0033] The present invention optimizes the training model, mainly including:

[0034] (1) To address data imbalance or camera perspective issues, the present invention designs specific image enhancement technologies, such as image flipping, cropping, image affine transformation, and image scaling transformation with specific hyperparameters, which effectively increase the diversity of human attributes and angles and enhance the robustness of the training model.

[0035] (2) Focusing on the analysis of the relationship between attributes, the present invention improves the performance of pedestrian attribute recognition in video surveillance scenarios by mining the correlation between attributes. To this end, the present invention proposes a multi-stage pedestrian attribute recognition method based on deep learning theory, while exploring the positive and negative correlations between attributes.

[0036] (3) An improved loss function that increases the difference between positive and negative samples is constructed and applied to the three-stage training to further improve the model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention will be further described below with reference to the accompanying drawings.

[0038] Figure 1 This is a flow chart of pedestrian attribute recognition according to the present invention. DETAILED DESCRIPTION

[0039] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0040] Example:

[0041] The present invention provides an improved pedestrian attribute monitoring and recognition method, comprising the following steps: obtaining a picture or video with a human body or a face from a camera, the camera reading the picture and video information and transmitting the information to an NNIE module of a HiSilicon chip inside the camera for data loading, wherein the camera is installed with a HiSilicon chip, and a trained data model is transplanted into the HiSilicon chip, wherein the data model includes a detection model based on YOLOv5 and a classification model based on ResneSt; performing label recognition on pedestrians according to data collected from a pedestrian attribute data set, firstly detecting a face area and an entire body area from the picture or video information based on the YOLOv5 detection model, and framing them respectively; then recognizing and classifying the specific attributes of the framed face area and entire body area based on the ResneSt classification model.

[0042] As attached Figure 1 As shown in FIG, the specific process of pedestrian attribute recognition of the present invention is as follows: (1) data model loading, including pre-processing operations such as data transformation of the image or video information in the camera, and importing the trained YOLOv5-based detection model and ResneSt-based classification model into the HiSilicon chip; (2) human shape detection, the YOLOv5-based detection model is responsible for detecting the human shape, that is, the face area and the whole body area, and framing the detected human shape. At this time, there are two detection results: result one detects the whole body area; result two detects the face area and the whole body area at the same time. If the face area cannot be detected, the recognition of the face attribute model will not be triggered; the ResneSt-based classification model classifies the human attributes of the detection results of the YOLOv5-based detection model, and specifically identifies them according to the 26 tags in the pedestrian attribute data set; (3) locking a specific person, locking a specific person through the recognition result, and achieving the purpose of monitoring suspicious people. An alarm device is also installed on the camera, which will be triggered when a specific person is detected.

[0043] Preparation of the training environment for training models: at least one 6G GPU NVIDIA graphics card and Linux operating system, build appropriate versions of Python, Pytorch, OpenCV and other open source libraries, install the matching versions of CUDA and CUDNN environment, which will help accelerate model training.

[0044] Dataset Preparation: Before pedestrian attribute recognition, we prepare the dataset. We use the LabelImg open-source annotation tool to label the video frames or images of pedestrians captured by the camera. The labels are the person pedestrian dataset, and the training dataset accounts for 80% and the validation dataset accounts for 20%. The pedestrian attribute dataset uses the open-source PA-100k pedestrian attribute dataset, which includes 23 labels such as age, gender, wearing glasses, and backpacks. The specific 23 labels are shown in Table 1 below.

[0045] The YOLOv5-based detection model includes the following training steps:

[0046] (1) Loading the pre-trained model: For a single obvious label such as a pedestrian, the yolov5s pre-trained model with the fastest model inference time is selected and placed in the CUDA thread;

[0047] (2) Loading data: Load the training and validation datasets in tensor format and place them in the CUDA thread;

[0048] (3) Select the optimizer: Select the batch random descent SGD optimizer or the adaptive Adam optimizer, and design the optimizer's learning strategy;

[0049] (4) Select loss function: YOLOv5 uses fixed IOU Loss and Obj Loss;

[0050] (5) Start iterative training: Design the number of iterations (epochs) and batch size (batch), and design the precise, recall, and mAP evaluation indicators on the validation dataset after each iteration. Update the model weights and learning rate after each iteration until the model's indicators on the validation dataset no longer increase, indicating that the model training is complete.

[0051] (6) Model transplantation: transplant the trained model to the camera HiSilicon chip and start real-time testing.

[0052] The classification model based on ResneSt includes the following training steps:

[0053] (1) Loading the model: Load the ResneSt model architecture and place it in a CUDA thread;

[0054] (2) Data augmentation: Data augmentation techniques mainly include image rotation, cropping, and changing the brightness, contrast, and saturation of images, with the aim of expanding data diversity and reducing the overfitting problem of the model;

[0055] (3) Loading data: Load the tensor-formatted training and validation datasets with a data ratio of 8:2 and place them in the CUDA thread;

[0056] (4) Select the optimizer: Select the batch random descent SGD optimizer or the adaptive Adam / AdamW optimizer, and design the optimizer's learning strategy;

[0057] (5) Select the loss function: use the classification loss function BCE Loss or Focal Loss;

[0058] (6) Start iterative training: Design the number of iterations (epochs) and the batch size (batch). The specific training steps are: in the first stage, calculate the loss value and accuracy of each attribute during the training process; in the second stage, establish a separate network branch for the attribute with a large average loss and a small accuracy, while the other attributes remain on the original branch, and then the two branches jointly predict all attributes; in the third stage, create two new network branches with the same structure as the branches in the second stage, and optimize the parameters of the new branches to make their attribute recognition performance better than the second stage. After each iteration of each stage, the precision and recall evaluation indicators are used on the validation dataset. After each iteration, the model weights and learning rate are updated once until the model training is completed. Finally, the model in the third stage is used for attribute prediction;

[0059] (7) Model transplantation: transplant the trained model to the camera HiSilicon chip and start real-time testing.

[0060] The Focal Loss loss function is:

[0061]

[0062] The model evaluation of the improved pedestrian attribute recognition scheme of the present invention on the pedestrian attribute recognition dataset PA-100k shows that the performance is enhanced, as shown in Table 1 below.

[0063] Table 1

[0064]

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. An improved pedestrian attribute recognition method, characterized by: The method includes the following steps: obtaining a picture or video with a human body or face from a camera; the camera reads the picture and video information and transmits the information to the NNIE module of the HiSilicon chip inside the camera for data loading; the HiSilicon chip is installed in the camera, and a trained data model is transplanted into the HiSilicon chip, which includes a detection model based on YOLOv5 and a classification model based on ResneSt; Pedestrians are identified based on data collected from the pedestrian attribute dataset. First, the YOLOv5 detection model is used to detect the face and body regions from the image or video information and frame them respectively. Then, the ResneSt classification model is used to identify and classify the specific attributes of the framed face and body regions. The YOLOv5-based detection model includes the following training steps: (1) Loading the pre-trained model: For a single obvious label such as a pedestrian, the yolov5s pre-trained model with the fastest model inference time is selected and placed in the CUDA thread; (2) Loading data: Load the training and validation datasets in tensor format and place them in the CUDA thread; (3) Select the optimizer: Select the batch random descent SGD optimizer or the adaptive Adam optimizer, and design the optimizer's learning strategy; (4) Select loss function: YOLOv5 uses fixed IOU Loss and Obj Loss; (5) Start iterative training: Design the number of iterations (epochs) and batch size (batch), and design the precise, recall, and mAP evaluation indicators on the validation dataset after each iteration. Update the model weights and learning rate after each iteration until the model's indicators on the validation dataset no longer increase, indicating that the model training is complete. (6) Model transplantation: transplant the trained model to the HiSilicon chip of the camera and start real-time testing; The classification model based on ResneSt includes the following training steps: (1) Loading the model: Load the ResneSt model architecture and place it in a CUDA thread; (2) Data augmentation: Data augmentation techniques mainly include image rotation, cropping, and changing the brightness, contrast, and saturation of images. The purpose is to expand data diversity and reduce the overfitting problem of the model; (3) Loading data: Load the tensor-formatted training and validation datasets with a data ratio of 8:2 and place them in the CUDA thread; (4) Select the optimizer: Select the batch random descent SGD optimizer or the adaptive Adam / AdamW optimizer, and design the optimizer's learning strategy; (5) Select the loss function: use the classification loss function BCE Loss or Focal Loss; (6) Start iterative training: Design the number of iterations epoch and batch size batch. The specific training steps are as follows: in the first stage, calculate the loss value and accuracy of each attribute during the training process; in the second stage, establish a separate network branch for the attribute with a large average loss and a small accuracy, and keep the other attributes on the original branch. Then, the two branches jointly predict all attributes; in the third stage, create two new network branches with the same structure as the branches in the second stage, and optimize the parameters of the new branches so that their attribute recognition performance is better than that of the second stage; After each iteration of each stage, the precision and recall evaluation indicators are calculated on the validation dataset. The model weights and learning rate are updated after each iteration until the model training is completed. Finally, the model of the third stage is used for attribute prediction. (7) Model transplantation: transplant the trained model to the camera HiSilicon chip and start real-time testing.

2. The improved pedestrian attribute recognition method according to claim 1, characterized in that: Before pedestrian attribute recognition, we prepare the dataset and use the LabelImg open source annotation tool to label the pedestrian video frames or images captured by the camera. The label is the person pedestrian dataset, and the division rule is that the training dataset accounts for 80% and the validation dataset accounts for 20%; The pedestrian attribute dataset uses the open source PA-100k pedestrian attribute dataset, which includes 23 labels: age, gender, wearing glasses, and backpack.

3. The improved pedestrian attribute recognition method according to claim 1, characterized in that: The Focal Loss loss function is: , where N is the number of pedestrian images and L is the number of attributes. Which attribute is it? It is The loss weight of each attribute, The first The occurrence ratio of the attribute, is a tuning parameter, It is In the picture The truth value of an attribute, It is the predicted In the picture The probability of an attribute.

Images