Pedestrian attribute recognition method, device, equipment, storage medium and computer program product

Abstract

The present application relates to the technical field of computer vision target detection, and discloses a method, device, equipment, storage medium and computer program product for pedestrian attribute recognition, including: constructing a target detection network based on the yolov8 network; acquiring the collected initial detection image, and preprocessing the initial detection image to obtain a target detection image; performing pedestrian attribute recognition on the target detection image according to the target detection network, determining the location area of ​​the pedestrian in the image, and outputting the attributes of the pedestrian in the location area. Using yolov8 as the basic network for improvement, the pedestrian detection task is combined with the structured attribute recognition task, and the attributes of each pedestrian target in the image are directly output end-to-end, which significantly improves the robustness and reliability of the pedestrian detection network.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision target detection, and in particular to a method, device, equipment, storage medium and computer program product for pedestrian attribute recognition. Background Art

[0002] At present, the conventional pedestrian attribute recognition method is divided into two stages. First, pedestrians are detected in the images taken by the surveillance camera through target detection technology. Secondly, the area containing pedestrians is further recognized by structured attributes, so as to output the detection box and attribute values ​​of the pedestrian area. However, in the face of relatively dense pedestrian scenes, a single pedestrian area cropped out by detection may include information of multiple pedestrians, and structured attributes can easily cause recognition conflicts. Summary of the invention

[0003] The main purpose of this application is to provide a pedestrian attribute recognition method, device, equipment, storage medium and computer program product, aiming to solve the technical problem that the current conventional pedestrian attribute recognition method faces relatively dense pedestrian scenes, the single pedestrian area detected and cropped may include multiple pedestrian information, and the structured attributes are prone to cause recognition conflicts.

[0004] To achieve the above objectives, the present application proposes a pedestrian attribute recognition method, the pedestrian attribute recognition method comprising:

[0005] Build a target detection network based on the yolov8 network;

[0006] Acquire the collected initial detection image, and preprocess the initial detection image to obtain a target detection image;

[0007] The target detection image is subjected to pedestrian attribute recognition according to the target detection network, a location area of ​​the pedestrian in the image is determined, and attributes of the pedestrian in the location area are output.

[0008] Optionally, the step of constructing a target detection network based on the yolov8 network includes:

[0009] Use deformable convolution to build DC2F module;

[0010] Use depthwise separable convolution to build the SPPFD module;

[0011] According to the DC2F module and the SPPFD module, the C2F module and the SPPF module in the yolov8 network are optimized to obtain an improved yolov8 network model;

[0012] The pedestrian structured attribute classification head is connected to the detection head of the improved yolov8 network model to obtain the target detection network.

[0013] Optionally, the step of building a DC2F module using deformable convolution includes:

[0014] Determine structural parameters of a deformable convolution kernel according to preset attribute recognition requirements, and construct a deformable convolution block using deformable convolution based on the structural parameters;

[0015] Determine the stacking quantity and order of the deformable convolution blocks based on the preset attribute recognition requirement, and form a target bottleneck layer;

[0016] The first convolution block and the bottleneck layer of the C2F module in the yolov8 network are replaced with the deformable convolution block and the target bottleneck layer to generate a DC2F module.

[0017] Optionally, the step of using depthwise separable convolution to build an SPPFD module includes:

[0018] Construct a DWCBL unit based on depthwise separable convolution, batch normalization and activation function;

[0019] The SPPF module in the yolov8 network is obtained, the second convolution block in the SPPF module is replaced by the DWCBL unit, and a pooling calculation in the SPPF module is removed to obtain the SPPFD module.

[0020] Optionally, the target detection network includes a backbone network, a neck network and a detection head;

[0021] The step of performing pedestrian attribute recognition on the target detection image according to the target detection network, determining the location area of ​​the pedestrian in the image, and outputting the attributes of the pedestrian in the location area includes:

[0022] Inputting the target detection image into the backbone network of the target detection network in a standardized format, extracting features of the target detection image, and generating a target feature map;

[0023] Based on the neck network, the features in the target feature map are fused to obtain a fused feature map;

[0024] In the detection head of the target detection network, the bounding box coordinates of each pedestrian target are calculated according to the fused feature map, the position area of ​​the pedestrian in the image is determined based on the bounding box coordinates, and the attributes of each position area are classified through the pedestrian structured attribute classification head, and the attributes of the pedestrian in the position area are output.

[0025] Optionally, in the detection head of the target detection network, the bounding box coordinates of each pedestrian target are calculated according to the fused feature map, the position area of ​​the pedestrian in the image is determined based on the bounding box coordinates, and the attribute classification of each position area is performed through the pedestrian structured attribute classification head, and after the step of outputting the attributes of the pedestrian in the position area, the method further includes:

[0026] Calculating the confidence of each of the location areas, and comparing the confidence with a preset confidence threshold;

[0027] When the confidence level is less than the preset confidence threshold, the location area is deleted.

[0028] In addition, to achieve the above-mentioned purpose, the present application also proposes a pedestrian attribute recognition device, the pedestrian attribute recognition device comprising:

[0029] Network construction module, used to build a target detection network based on the yolov8 network;

[0030] An image acquisition module is used to acquire the collected initial detection image and preprocess the initial detection image to obtain a target detection image;

[0031] The attribute recognition module is used to perform pedestrian attribute recognition on the target detection image according to the target detection network, determine the pedestrian area in the image, and output the attribute of each pedestrian in the pedestrian area.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a pedestrian attribute recognition device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the pedestrian attribute recognition method described above.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the pedestrian attribute recognition method described above are implemented.

[0034] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the pedestrian attribute recognition method described above are implemented.

[0035] The present application discloses a method for constructing a target detection network based on the yolov8 network; obtaining an initial detection image collected, and preprocessing the initial detection image to obtain a target detection image; performing pedestrian attribute recognition on the target detection image according to the target detection network, determining the location area of ​​the pedestrian in the image, and outputting the attributes of the pedestrian in the location area. Using yolov8 as the basic network for improvement, the pedestrian detection task is combined with the structured attribute recognition task, and the attributes of each pedestrian target in the image are directly output end-to-end, which improves the recognition effect in complex scenes, thereby significantly improving the robustness and reliability of the pedestrian detection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] Figure 1 This is a flow chart of the first embodiment of the pedestrian attribute recognition method of the present application;

[0039] Figure 2 This is a schematic diagram of the structure of the traditional yolov8 network;

[0040] Figure 3 This is a flow chart of the second embodiment of the pedestrian attribute recognition method of the present application;

[0041] Figure 4 This is the framework diagram of the C2F module in the yolov8 network;

[0042] Figure 5 A schematic diagram of the DC2F module constructed for this application;

[0043] Figure 6 This is the framework diagram of the SPPF module in the yolov8 network;

[0044] Figure 7 A schematic diagram of the SPPFD module constructed for this application;

[0045] Figure 8 This is the framework diagram of the target detection network for this application;

[0046] Fig. 9 This is a flow chart of the third embodiment of the pedestrian attribute recognition method of the present application;

[0047] Fig.10 A schematic diagram of the scene for pedestrian attribute recognition in this application;

[0048] Fig.11 This is a schematic diagram of the module structure of a pedestrian attribute recognition device according to an embodiment of the present application;

[0049] Fig.12 This is a schematic diagram of the device structure of the hardware operating environment involved in the pedestrian attribute recognition method in the embodiment of the present application.

[0050] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0052] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0053] The main solution of the embodiment of the present application is: to build a target detection network based on the yolov8 network; to obtain the collected initial detection image, and to preprocess the initial detection image to obtain a target detection image; to perform pedestrian attribute recognition on the target detection image according to the target detection network, to determine the location area of ​​the pedestrian in the image, and to output the attributes of the pedestrian in the location area.

[0054] Since the recognition of pedestrian structured attributes is an important task in the field of computer vision, it mainly aims to automatically identify the visible attributes of pedestrians from images and represent these attributes in a structured manner. Pedestrian attributes usually include information such as age, gender, height, and posture. These attributes can be used as auxiliary description information for pedestrians and are widely used in scenarios such as intelligent monitoring security, unmanned retail stores, and intelligent traffic management. At present, the conventional pedestrian attribute recognition method is divided into two stages: first, pedestrian detection is performed on the images captured by the monitoring system through target detection technology, and then further structured attribute recognition is performed on the area containing pedestrians, thereby outputting the detection box and attribute value of the pedestrian area. However, this segmented recognition method has certain defects. First, the cropped pedestrian area contains a lot of background noise, which affects the accuracy of attribute recognition; second, in the face of a relatively dense pedestrian scene, the single pedestrian area cropped by detection may include multiple pedestrian information, and the structured attributes are prone to cause recognition conflicts.

[0055] Therefore, this application provides a pedestrian attribute recognition method, improves the detection head, C2F module and SPPF module of yolov8, adds a structured attribute recognition classification head, merges pedestrian detection and pedestrian attribute recognition into one task, reduces the complexity of the pedestrian attribute recognition task and improves the pedestrian detection accuracy.

[0056] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, or an electronic device capable of implementing the above functions, etc. The following takes the target detection network as an example to illustrate this embodiment and the following embodiments.

[0057] Based on this, the present application embodiment provides a method for pedestrian attribute recognition, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the pedestrian attribute recognition method of the present application.

[0058] In this embodiment, the pedestrian attribute recognition method includes:

[0059] Step S10, constructing a target detection network based on the yolov8 network.

[0060] It should be noted that the yolov8 network is a target detection algorithm framework, which is widely used in target detection tasks in the field of computer vision. The yolov8 network structure includes the backbone network (Backbone), the neck network (Neck) and the detection head (Head). The backbone network is responsible for extracting the deep features of the input image, and uses a series of convolutional layers and special modules (such as C2F modules, etc.) to gradually abstract the image information to obtain a representative feature map. The neck network is mainly used to fuse and further process the different levels of features extracted by the backbone network to integrate multi-scale information so that the network can better cope with the detection of targets of different sizes. The detection head performs target positioning (determine the bounding box coordinates of the target) and classification (determine the category to which the target belongs) based on the fused feature map, and finally outputs the category of each target in the image and the corresponding location information.

[0061] It is understandable that in actual scenes, pedestrians have rich postures and body shapes. For example, on the street, pedestrians may walk, run, bend over, lean sideways, and other postures. When extracting features, the convolutional network structure in the traditional yolov8 network has a fixed sampling position of the convolution kernel. It is difficult to accurately extract key features for pedestrians with different postures and body parts that may be occluded or stretched and deformed. At the same time, in environments such as surveillance scenes or urban streets, the size (scale) of pedestrians in the picture will vary depending on the distance from the camera. Therefore, deformable convolution and depth-separable convolution can be used to improve the yolov8 network, thereby effectively improving the feature extraction capabilities of pedestrians with different postures and deformations.

[0062] In one example, reference Figure 2 , Figure 2 Figure 1 is a schematic diagram of the structure of the traditional yolov8 network. It is mainly divided into four parts: input, backbone network, neck network and detection head. The size of the input image in the network is 640×640×3, which means that the width and height of the image are 640 pixels and the number of RGB (Red-Green-Blue) channels is 3. The backbone network consists of multiple convolution blocks (CBS) and C2F modules, where CBS performs convolution operations to preliminarily extract the features of the image. The convolution kernel slides on the image to extract local features at different positions, and the diversity of features is increased by changing the number of channels. In the figure, the input image size of 640×640×3 is changed to 320×320×16 after CBS, which reduces the image size and increases the number of channels, realizing the preliminary abstraction of features. The C2F module can further process the features extracted by CBS, enhance the expressiveness of the features, and enable them to better capture the target information in the image. In the neck network, the low-resolution feature map is enlarged to the same size as other feature maps through upsampling and concatenation operations, and then spliced ​​with the feature maps of other layers according to the channel dimension, so that features of different scales can complement each other in subsequent processing. At the same time, SPPF (Spatial Pyramid Pooling-Fast) is used to fuse features of different scales. At the detection head, the CBS module is used to further process the fused feature map, adjust the number of channels and size of the feature map, and make it suitable for subsequent target positioning and classification operations. The difference between the predicted result and the true label is calculated through different loss functions (such as VFL Loss, BCE Loss, CIoU Loss, etc.), and the network parameters are optimized through the back propagation algorithm, so that the network can accurately detect the location and category of the target.

[0063] Step S20, acquiring the collected initial detection image, and preprocessing the initial detection image to obtain a target detection image.

[0064] It should be noted that the initial detection image is the raw image data directly obtained by the image acquisition device (such as camera, scanner, etc.) without any processing. It may have problems such as noise, color imbalance, and size inconsistency, and cannot be directly used for subsequent processing such as target detection. After obtaining the initial detection image, a series of operations need to be performed on the initial detection image to improve the image quality and make the image more convenient for the target detection algorithm to process.

[0065] It can be understood that the preprocessing of the initial detection image can be to reduce the interference of noise on the real information of the image through filtering and other methods, so that the subsequent feature extraction and target detection are more accurate; it can also be to map the range of image pixel values ​​to a specific interval for normalization processing to avoid training difficulties caused by excessive differences in pixel value ranges; it can also be to change the image size, including enlarging or reducing it, so that the image meets the input requirements of the target detection model.

[0066] Step S30, performing pedestrian attribute recognition on the target detection image according to the target detection network, determining the location area of ​​the pedestrian in the image, and outputting the attributes of the pedestrian in the location area.

[0067] It should be noted that pedestrian attribute recognition is a sub-field of target detection. By analyzing the image, the position of pedestrians in the image is detected, and various attributes of pedestrians are identified, such as age, gender, clothing color, whether they are carrying items, etc. The location area of ​​pedestrians refers to the spatial range occupied by pedestrians in the image, usually represented by a rectangular box (bounding box). The coordinates of the box (such as the coordinates of the upper left corner and the lower right corner) can determine the specific position of the pedestrian in the image. The attributes of pedestrians refer to various characteristic information of pedestrians, including but not limited to appearance characteristics (such as hairstyle, clothing style), identity characteristics (such as age, gender), behavioral characteristics (such as walking, running, carrying items), etc.

[0068] It can be understood that when the target detection network performs pedestrian attribute recognition on the target detection image, the backbone network first uses operations such as convolutional layers and pooling layers to extract deep feature maps from the image. These feature maps contain various information about pedestrians in the image, such as contours and textures. Then the neck network fuses the feature maps of different levels extracted by the backbone network. Since feature maps of different levels have different resolutions and semantic information, the network can make better use of this information through fusion. For example, low-level feature maps have more detailed information and are suitable for accurately locating the position of pedestrians; high-level feature maps have more semantic information and are suitable for identifying the attributes of pedestrians. Finally, in the detection head, the bounding box coordinates of each pedestrian target are calculated based on the fused feature map to determine the location area of ​​the pedestrian in the image. At the same time, the pedestrian structured attribute classification head is used to classify the attributes of each location area. The pedestrian structured attribute classification head can be a subnetwork added on the basis of the detection head specifically for classifying pedestrian attributes, and can judge various attributes of pedestrians based on the information in the feature map.

[0069] In one example, when detecting a surveillance image taken at the entrance of a shopping mall based on a target detection network, it is first necessary to input the preprocessed surveillance image into the backbone network of the target detection network to extract features such as the general outline of pedestrians at the entrance of the shopping mall and the texture of their clothing. In the neck network, the low-level feature map containing the details of the pedestrian's clothing and the high-level feature map containing the overall posture of the pedestrian are fused, so that the target detection network can not only accurately locate the pedestrian, but also identify its attributes well. Finally, the location area of ​​the pedestrian is determined based on the fused feature map. For example, if there is a pedestrian in the middle left position of the image, the network will calculate a rectangular bounding box to indicate the location of the pedestrian. At the same time, the pedestrian structured attribute classification head starts working, and recognizes that the pedestrian's gender is female, the color of the clothing is red, and that he is carrying an item (bag) based on the information in the feature map, and outputs these attributes next to the rectangular bounding box.

[0070] In this embodiment, a target detection network is constructed based on the yolov8 network; the collected initial detection image is obtained, and the initial detection image is preprocessed to obtain a target detection image; pedestrian attributes are recognized on the target detection image according to the target detection network, the location area of ​​the pedestrian in the image is determined, and the attributes of the pedestrian in the location area are output. Using yolov8 as the basic network for improvement, the pedestrian detection task is combined with the structured attribute recognition task, and the attributes of each pedestrian target in the image are directly output end-to-end, which significantly improves the robustness and reliability of the pedestrian detection network.

[0071] Reference Figure 3 , Figure 3This is a flow chart of the second embodiment of the pedestrian attribute recognition method of the present application. Based on the above-mentioned first embodiment, the second embodiment of the pedestrian attribute recognition method of the present application is proposed.

[0072] In the second embodiment, the step S10 includes:

[0073] Step S101, using deformable convolution to build a DC2F module.

[0074] It should be noted that deformable convolution is an improvement on traditional convolution. In traditional convolution, the sampling point position when the convolution kernel slides on the image is fixed. The sampling point position of deformable convolution can be adaptively adjusted according to the input feature map, which can better adapt to the shape change and posture change of the target, so as to extract features more effectively. The DC2F module is a feature extraction module built on deformable convolution. In the network target detection network improved based on yolov8, the DC2F module can be used to replace the original feature extraction module C2F module to enhance the network's feature extraction capability for pedestrians and other targets, thereby improving the accuracy of target detection and attribute recognition.

[0075] It can be understood that building a DC2F module can be to use deformable convolution to replace the traditional convolution Conv2d (2D Convolution) in the C2F module, and then replace the convolution block in the C2F module with a deformable convolution block; or a deformable convolution layer can be built to generate a DC2F module.

[0076] Furthermore, in order to improve the detection accuracy of the model in processing complex scenes such as pedestrian deformation and occlusion, the step S101 may include:

[0077] The structural parameters of the deformable convolution kernel are determined according to the preset attribute recognition requirements, and the deformable convolution block is constructed using the deformable convolution based on the structural parameters; the stacking quantity and order of the deformable convolution blocks are determined based on the preset attribute recognition requirements to form a target bottleneck layer; the first convolution block and the bottleneck layer of the C2F module in the yolov8 network are replaced with the deformable convolution block and the target bottleneck layer to generate a DC2F module.

[0078] It should be noted that the preset attribute recognition requirements are target attribute requirements set according to the actual application scenario and the expected recognition effect before performing the attribute recognition task of a specific target (such as a pedestrian). For example, for pedestrian attribute recognition, the preset attribute recognition requirements include accurate identification of the age range of pedestrians (such as children, youth, middle-aged, elderly, etc.), gender, whether they carry items, clothing color and other specific attribute information. The structural parameters of the deformable convolution kernel are the physical properties of the deformable convolution kernel, including the convolution kernel size, step size, padding, etc.

[0079] It should be understood that before constructing the feature extraction module, it is necessary to analyze the characteristics of the attribute recognition task to determine the structural parameters. For example, if you want to recognize the facial expression details of pedestrians, you need a smaller convolution kernel with a smaller step size to finely capture the subtle texture and muscle changes on the face; if you focus on the overall clothing style characteristics of pedestrians, you can appropriately increase the size of the convolution kernel to obtain a wider range of clothing texture and contour features. Based on the determined structural parameters, the deformable convolution kernel is applied to the input feature map for convolution operations, so that it can adaptively adjust the sampling point position according to the actual situation of the input features, and better focus on the key feature areas of the target. For example, when processing pedestrian images, in the case where body parts are occluded or posture changes cause feature deformation, the deformable convolution can accurately extract the effective features of the unoccluded part by adjusting the sampling points.

[0080] In one example, reference Figure 4 and Figure 5 , Figure 4 This is the framework diagram of the C2F module in the yolov8 network. Figure 5 Framework diagram of the DC2F module constructed for this application. After the input data inside the C2F module is processed by CBS, Split (data splitting) splits the features into two parts by channel, one part is not processed, and the other part passes through multiple consecutive Bottleneck (bottleneck layer), and then the output of each Bottleneck is spliced ​​with the unprocessed part by channel, and the spliced ​​features are finally output through CBS. Among them, the CBS module is used to perform convolution operations on the input data and extract features, including a Conv2d (two-dimensional convolution) module, a BN (batch normalization) module, and finally output through the SiLU (activation function) module; the Split operation distributes the data to multiple Bottleneck modules for parallel processing, and the Bottleneck module is composed of two CBS modules; the Concat operation merges the features after parallel processing, and then further processes them through the CBS module. The DC2F module replaces the Conv2d module in the CBS module with DCN (deformable convolution) to obtain a DCBS (deformable convolution block) module, and uses DCBS to reconstruct the Bottleneck module. The DCBS structure built with deformable convolution replaces the CBS in C2F, and the Bottleneck based on CBS also replaces the original Bottleneck in C2F.

[0081] Step S102, using depthwise separable convolution to build the SPPFD module.

[0082] It is important to understand that depthwise separable convolution is a special convolution operation that decomposes the standard convolution into two parts: depthwise convolution and pointwise convolution. Depthwise convolution performs convolution operations on each input channel separately, while pointwise convolution is used to fuse the channel information output by depthwise convolution. This decomposition method can significantly reduce the amount of calculation and the number of parameters, while maintaining good feature extraction capabilities to a certain extent.

[0083] Furthermore, in order to significantly reduce the computational complexity and parameter amount of the detection network while ensuring the detection accuracy, the step S102 may include:

[0084] A DWCBL unit is constructed based on deep separable convolution, batch normalization and activation function; the SPPF module in the yolov8 network is obtained, the second convolution block in the SPPF module is replaced by the DWCBL unit, and a pooling calculation in the SPPF module is removed to obtain the SPPFD module.

[0085] It should be noted that batch normalization is a technology used in deep neural networks to normalize the input of each layer. Activation functions are used to introduce nonlinear factors to neurons, including Sigmoid, ReLU, Leaky ReLU, etc. The SPPF module is a module used to fuse features of different scales in the yolov8 network. The SPPFD module is a module obtained by transforming the SPPF module in the yolov8 network. The DWCBL unit is used to replace the second convolution block in the SPPF module and a pooling calculation is removed. It aims to optimize the feature processing and fusion capabilities of the module, reduce the amount of calculation, and maintain or improve the performance of target detection and attribute recognition.

[0086] In one example, reference Figure 6 and Figure 7 , Figure 6 This is the framework diagram of the SPPF module in the yolov8 network. Figure 7Framework diagram of the SPPFD module constructed for this application. In the SPPF module, it first passes through a CBS, and then passes through three Maxpool (maximum pooling) operations in sequence. Finally, the results of each pooling are connected through the Concat (connection) operation and output through the subsequent CBS. The SPPFD module replaces the CBS in the SPPF module with DWCBL (depth-separable convolution block), and at the same time reduces one Maxpool operation to reduce the amount of calculation. Among them, DWCBL contains operations such as DWConv2d (Depthwise2-DimensionalConvolution, depth-separable two-dimensional convolution), Conv 1×1 (point-by-point convolution), BN (Batch Normalization) and activation function ReLU. Compared with the SPPF module, the SPPFD module has significant advantages in reducing the amount of calculation and the amount of parameters. It can accelerate the training and reasoning process of the network while maintaining or improving the feature fusion effect, which helps to improve the performance of tasks such as target detection and attribute recognition.

[0087] Step S103, optimizing the C2F module and the SPPF module in the yolov8 network according to the DC2F module and the SPPFD module to obtain an improved yolov8 network model.

[0088] It is understandable that when optimizing the yolov8 network, it is first necessary to find the location of the C2F module and its connection relationship in the network in the yolov8 network, remove the original C2F module, and place the pre-built DC2F module in the same position to ensure that the replaced DC2F module is correctly connected to the input and output of other modules in the network. At the same time, find the SPPF module in the yolov8 network and remove it, and replace it with the SPPFD module. After completing the module replacement, it is also necessary to initialize or adjust the learnable parameters in the new module and retrain.

[0089] It should be understood that Arcface loss (angular margin loss) can be used to calculate the loss of structured attribute classification during model training. Because Arcface takes the geometric structure of the feature space into consideration in its design, it can increase the angular spacing between classes in the feature space, making the feature vectors of the same type more concentrated. Especially for pedestrian structured attributes, which have many categories or small differences between classes, Arcface can improve the model's ability to distinguish pedestrian attributes and provide a more robust feature representation.

[0090] Step S104, connecting the pedestrian structured attribute classification head to the detection head of the improved yolov8 network model to obtain a target detection network.

[0091] It should be noted that the pedestrian structured attribute classification head is a sub-network module used to classify various attributes of pedestrians. It discriminates and classifies different attributes of pedestrians based on the feature graphs extracted and fused from the backbone network and the neck network. At the same time, the decoupled classification head can extract the semantic information of pedestrian features.

[0092] It can be understood that the convolution layer is connected after the previous CBS block of the classification output to obtain the category output of the pedestrian structured attributes, and the number of output channels is the number of structured attribute categories. Compared with the segmented recognition method of cropping first and then classifying, the addition of the pedestrian attribute classification head significantly improves the classification accuracy of the pedestrian structured attributes, especially the recognition effect in complex scenes (such as mutual occlusion between pedestrians).

[0093] In one example, reference Figure 8 , Figure 8 This is the framework diagram of the target detection network of this application. Compared with the yolov8 network, the DCBS structure built with deformable convolution replaces the C2F module in the backbone network (Backbone) and the neck network (Neck), and the SPPF module is replaced by the improved SPPFD module; a pedestrian structured attribute detection head is added to the detection head (Head), pedestrian detection and structured attribute recognition are regarded as the same task, and the convolution layer is connected after the previous CBS block of the classification output to obtain the category output of the pedestrian structured attribute, and the number of output channels is the number of structured attribute categories.

[0094] In this embodiment, a DC2F module is constructed using deformable convolution; a SPPFD module is constructed using deep separable convolution; the C2F module and the SPPFD module in the yolov8 network are optimized according to the DC2F module and the SPPFD module to obtain an improved yolov8 network model; a pedestrian structured attribute classification head is connected to the detection head of the improved yolov8 network model to obtain a target detection network. By improving the yolov8 network and adding the pedestrian attribute classification head, the classification accuracy of the pedestrian structured attributes is significantly improved, especially the recognition effect in complex scenes (such as mutual occlusion between pedestrians).

[0095] Reference Fig. 9 , Fig. 9 This is a flow chart of the third embodiment of the pedestrian attribute recognition method of the present application. Based on the above second embodiment, the third embodiment of the pedestrian attribute recognition method of the present application is proposed.

[0096] In the third embodiment, the step S30 includes:

[0097] Step S301: input the target detection image into the backbone network of the target detection network in a standardized format, extract the features of the target detection image, and generate a target feature map.

[0098] It should be noted that each element in the target feature map represents a certain feature of the target at a specific position and scale, such as texture, shape, etc. The subsequent network part will perform target positioning and classification operations based on these feature maps.

[0099] It can be understood that when extracting the features of the target detection image, feature extraction is first performed through a series of convolutional layers, in which the convolution kernel slides on the image and captures local features through operations such as weighted summation; after the convolution operation, a pooling operation follows to reduce the size of the feature map without changing the number of channels in the feature map, reduce the amount of data and calculation, and retain the main feature information; at the same time, between the convolution layer and the pooling layer, and after the last layer of convolution operation, an activation function is used to increase the nonlinear expression ability of the network.

[0100] Step S302: fusing the features in the target feature map based on the neck network to obtain a fused feature map.

[0101] It is understandable that the backbone network usually produces feature maps of different scales, which contain information at different levels. The feature maps produced by the shallower network layers have high resolution but less semantic information, which is suitable for locating the precise position of the target; the feature maps produced by the deeper network layers have low resolution but contain richer semantic information, which is suitable for identifying the category of the target. The feature fusion of the neck network can effectively integrate these features of different scales and levels, so that the fused feature maps have both accurate location information and rich semantic information.

[0102] Step S303: In the detection head of the target detection network, the bounding box coordinates of each pedestrian target are calculated according to the fused feature map, the position area of ​​the pedestrian in the image is determined based on the bounding box coordinates, and the attributes of each position area are classified through the pedestrian structured attribute classification head, and the attributes of the pedestrian in the position area are output.

[0103] It should be understood that the bounding box coordinates are used to represent the position and range of the target in the image, and are usually embodied in the form of a rectangular box, which may include the coordinates of the upper left corner vertex and the lower right corner vertex.

[0104] It can be understood that for each determined pedestrian location area, the pedestrian structured attribute classification head will extract the feature information of the corresponding area from the fused feature map. After obtaining the pedestrian-related features, the classification head will flatten and transform the features through the fully connected layer, and then use the classifier to map the features to the probability space corresponding to each attribute category, and finally output the classification results of each attribute of the pedestrian.

[0105] In one example, reference Fig.10 , Fig.10 A schematic diagram of the scene for pedestrian attribute recognition in this application. The figure shows an example scene of pedestrian attribute recognition in an indoor storage room. Each person is framed by a box and labeled with the person's attributes. Person attributes include gender, hair length, clothing, posture, etc. The attributes displayed by the pedestrian in the lower left corner of the figure include: Male, Short hair, White upper, Black lower, Greyshoes, Sit.

[0106] Furthermore, in order to reduce false detection and improve the result quality, after step S303, the following may also be included:

[0107] Calculate the confidence of each of the location areas, and compare the confidence with a preset confidence threshold; when the confidence is less than the preset confidence threshold, delete the location area.

[0108] It should be noted that in target detection, the confidence of a location area refers to the network's degree of certainty that a target exists in the area. It is usually a value between 0 and 1. The higher the value, the more certain the network is that a target exists in the area. The preset confidence threshold is a pre-set value used to determine whether the target detection result is reliable. The calculated confidence of each location area is compared with this threshold. If the confidence is lower than this threshold, the target detection result in the location area is considered unreliable.

[0109] It should be understood that the preset confidence threshold requires the network to be determined by learning a large amount of labeled data during the training phase. For pedestrian detection, it should be clear which feature combination corresponds to the high-confidence pedestrian presence area and which feature combination corresponds to the low-confidence area.

[0110] In this embodiment, the target detection image is input into the backbone network of the target detection network in a standardized format, the features of the target detection image are extracted, and a target feature map is generated; the features in the target feature map are fused based on the neck network to obtain a fused feature map; in the detection head of the target detection network, the bounding box coordinates of each pedestrian target are calculated based on the fused feature map, the position area of ​​the pedestrian in the image is determined based on the bounding box coordinates, and the attributes of each position area are classified by the pedestrian structured attribute classification head, and the attributes of the pedestrian in the position area are output. Pedestrian detection is performed on the images captured by the monitoring system through target detection technology, and the pedestrian detection task is combined with the structured attribute recognition task to directly output the attributes of each pedestrian target in the image end-to-end. The detection speed of the target detection network is improved.

[0111] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the pedestrian attribute recognition method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0112] This application also provides a pedestrian attribute recognition device, please refer to Fig.11 , the pedestrian attribute recognition device comprises:

[0113] A network construction module 10 is used to construct a target detection network based on the yolov8 network;

[0114] The image acquisition module 20 is used to acquire the collected initial detection image and pre-process the initial detection image to obtain a target detection image;

[0115] The attribute recognition module 30 is used to perform pedestrian attribute recognition on the target detection image according to the target detection network, determine the pedestrian area in the image, and output the attribute of each pedestrian in the pedestrian area.

[0116] The pedestrian attribute recognition device provided by the present application adopts the pedestrian attribute recognition method in the above embodiment, which can solve the technical problem that the conventional pedestrian attribute recognition method currently faces a relatively dense pedestrian scene, and the single pedestrian area cut out may include multiple pedestrian information, and the structured attributes are prone to cause recognition conflicts. Compared with the prior art, the beneficial effects of the pedestrian attribute recognition device provided by the present application are the same as the beneficial effects of the pedestrian attribute recognition method provided by the above embodiment, and the other technical features of the pedestrian attribute recognition device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0117] The present application provides a pedestrian attribute recognition device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the pedestrian attribute recognition method in the above-mentioned embodiment one.

[0118] Reference below Fig.12 , which shows a schematic diagram of the structure of a pedestrian attribute recognition device suitable for implementing the embodiment of the present application. The pedestrian attribute recognition device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.12 The pedestrian attribute recognition device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0119] like Fig.12As shown, the pedestrian attribute recognition device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the pedestrian attribute recognition device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the pedestrian attribute recognition device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a pedestrian attribute recognition device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0120] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0121] The pedestrian attribute recognition device provided by the present application adopts the pedestrian attribute recognition method in the above embodiment, which can solve the technical problem that the conventional pedestrian attribute recognition method currently faces a relatively dense pedestrian scene, and the single pedestrian area detected and cropped may include multiple pedestrian information, and the structured attributes are prone to cause recognition conflicts. Compared with the prior art, the beneficial effects of the pedestrian attribute recognition device provided by the present application are the same as the beneficial effects of the pedestrian attribute recognition method provided by the above embodiment, and the other technical features of the pedestrian attribute recognition device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0122] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0124] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the pedestrian attribute recognition method in the above-mentioned embodiment.

[0125] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0126] The computer-readable storage medium may be included in the pedestrian attribute recognition device; or may exist independently without being assembled into the pedestrian attribute recognition device.

[0127] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the pedestrian attribute recognition device, the pedestrian attribute recognition device executes the pedestrian attribute recognition method described above.

[0128] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0131] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned pedestrian attribute recognition method, and can solve the technical problem that the current conventional pedestrian attribute recognition method faces a relatively dense pedestrian scene, and the single pedestrian area cut out may include multiple pedestrian information, and the structured attributes are prone to cause recognition conflicts. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the pedestrian attribute recognition method provided by the above-mentioned embodiment, which will not be repeated here.

[0132] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned pedestrian attribute recognition method when executed by a processor.

[0133] The computer program product provided by the present application can solve the technical problem that the conventional pedestrian attribute recognition method currently faces a relatively dense pedestrian scene, the single pedestrian area detected and cropped may include multiple pedestrian information, and the structured attributes are prone to cause recognition conflicts. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the pedestrian attribute recognition method provided by the above embodiment, which will not be repeated here.

[0134] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A pedestrian attribute recognition method, characterized in that: The pedestrian attribute recognition method comprises: Build a target detection network based on the yolov8 network; Acquire the collected initial detection image, and preprocess the initial detection image to obtain a target detection image; The target detection image is subjected to pedestrian attribute recognition according to the target detection network, a location area of ​​the pedestrian in the image is determined, and attributes of the pedestrian in the location area are output.

2. The pedestrian attribute recognition method according to claim 1, characterized in that: The steps of constructing a target detection network based on the yolov8 network include: Use deformable convolution to build DC2F module; Use depthwise separable convolution to build the SPPFD module; According to the DC2F module and the SPPFD module, the C2F module and the SPPF module in the yolov8 network are optimized to obtain an improved yolov8 network model; The pedestrian structured attribute classification head is connected to the detection head of the improved yolov8 network model to obtain the target detection network.

3. The pedestrian attribute recognition method according to claim 2, characterized in that: The step of building a DC2F module using deformable convolution includes: Determine structural parameters of a deformable convolution kernel according to preset attribute recognition requirements, and construct a deformable convolution block using deformable convolution based on the structural parameters; Determine the stacking quantity and order of the deformable convolution blocks based on the preset attribute recognition requirement, and form a target bottleneck layer; The first convolution block and the bottleneck layer of the C2F module in the yolov8 network are replaced with the deformable convolution block and the target bottleneck layer to generate a DC2F module.

4. The pedestrian attribute recognition method according to claim 2, characterized in that: The step of using depth-wise separable convolution to build the SPPFD module includes: Construct a DWCBL unit based on depthwise separable convolution, batch normalization and activation function; The SPPF module in the yolov8 network is obtained, the second convolution block in the SPPF module is replaced by the DWCBL unit, and a pooling calculation in the SPPF module is removed to obtain the SPPFD module.

5. The pedestrian attribute recognition method according to any one of claims 1 to 4, characterized in that: The target detection network includes a backbone network, a neck network and a detection head; The step of performing pedestrian attribute recognition on the target detection image according to the target detection network, determining the location area of ​​the pedestrian in the image, and outputting the attributes of the pedestrian in the location area includes: Inputting the target detection image into the backbone network of the target detection network in a standardized format, extracting features of the target detection image, and generating a target feature map; Based on the neck network, the features in the target feature map are fused to obtain a fused feature map; In the detection head of the target detection network, the bounding box coordinates of each pedestrian target are calculated according to the fused feature map, the position area of ​​the pedestrian in the image is determined based on the bounding box coordinates, and the attributes of each position area are classified through the pedestrian structured attribute classification head, and the attributes of the pedestrian in the position area are output.

6. The pedestrian attribute recognition method according to claim 5, characterized in that: In the detection head of the target detection network, the bounding box coordinates of each pedestrian target are calculated according to the fused feature map, the position area of ​​the pedestrian in the image is determined based on the bounding box coordinates, and the attribute classification of each position area is performed through the pedestrian structured attribute classification head, and the attribute of the pedestrian in the position area is output, and the step of further comprising: Calculating the confidence of each of the location areas, and comparing the confidence with a preset confidence threshold; When the confidence level is less than the preset confidence threshold, the location area is deleted.

7. A pedestrian attribute recognition device, characterized in that: The device comprises: Network construction module, used to build a target detection network based on the yolov8 network; An image acquisition module is used to acquire the collected initial detection image and preprocess the initial detection image to obtain a target detection image; The attribute recognition module is used to perform pedestrian attribute recognition on the target detection image according to the target detection network, determine the pedestrian area in the image, and output the attribute of each pedestrian in the pedestrian area.

8. A pedestrian attribute recognition device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the pedestrian attribute recognition method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the pedestrian attribute recognition method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the pedestrian attribute recognition method according to any one of claims 1 to 6 are implemented.

Images