Human body attribute identification method and device

Abstract

The invention discloses a human body attribute identification method and device. The method comprises the following steps: extracting features of a human body image to obtain an initial human body feature map; according to the initial human body feature map, a category confidence map of each part of the human body is generated, and the category confidence map is used for representing the probability that features in the initial human body feature map belong to human body part categories; fusing the initial human body feature map and the category confidence map to obtain a target human body feature map; and based on the target human body feature map, the attribute of the target human body in the human body image is identified to obtain an attribute identification result of the target human body, and the attribute identification result comprises the identification information of the target human body. According to the invention, the technical problem that the precision is low when a light-weight image classification network is adopted to identify human body attributes in order to reduce the computing power in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for recognizing human attributes. Background Art

[0002] Human attribute recognition is one of the basic functions in the field of security monitoring. For example, for pedestrian attribute recognition, based on the input human body picture, it is possible to identify whether the pedestrian is carrying something, whether he is carrying a backpack, gender, hair length, sleeve length, top and bottom clothing color and category and other attributes. Based on the attribute description, the user can search for pedestrian events and video clips with specific attributes on the front-end network video recorder (Network Video Recorder, referred to as NVR) or mobile device, which is very important for security analysis and event backtracking.

[0003] The most direct way to identify pedestrian attributes is to treat it as a classic image multi-classification task: input the pedestrian image into the backbone network of the image classification network to extract features, obtain a three-dimensional feature map, then reduce the spatial dimension through the pooling layer to obtain a feature vector, and finally input it into the fully connected layer to obtain the confidence of each attribute classification. However, when a lightweight image classification network is used to reduce computing power, there is often a problem of low accuracy.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for identifying human attributes, so as to at least solve the technical problem that in the related art, when a lightweight image classification network is used to identify human attributes in order to reduce computing power, the accuracy is often low.

[0006] According to one aspect of an embodiment of the present invention, a method for recognizing human attributes is provided, comprising: extracting features of a human body image to obtain an initial human body feature map; generating category confidence maps of various human body parts based on the initial human body feature map, wherein the category confidence maps are used to represent the probabilities that the features in the initial human body feature map respectively belong to human body part categories; fusing the initial human body feature map and the category confidence map to obtain a target human body feature map; based on the target human body feature map, recognizing the attributes of a target human body in the human body image to obtain an attribute recognition result of the target human body, wherein the attribute recognition result includes identification information of the target human body.

[0007] Optionally, the fusing the initial human feature map and the category confidence map to obtain a target human feature map includes: fusing the depth features among the three-dimensional features of depth, height and width in the initial human feature map with the category features in the category confidence map to obtain the target human feature map.

[0008] Optionally, based on the target human body feature map, the attributes of the target human body in the human body image are identified to obtain the attribute recognition result of the target human body, including: segmenting the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image; performing attribute recognition on the upper body feature map and the lower body feature map respectively to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map; obtaining the attribute recognition result of the target human body according to the first attribute recognition result and the second attribute recognition result.

[0009] Optionally, segmenting the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image includes: segmenting the target human body feature map according to the height among the three dimensions of depth, height and width in the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image.

[0010] Optionally, the attribute recognition is performed on the upper body feature map and the lower body feature map respectively to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map, including: inputting the upper body feature map into an attribute classifier corresponding to the upper body to obtain the first attribute recognition result of the upper body feature map; inputting the lower body feature map into an attribute classifier corresponding to the lower body to obtain the second attribute recognition result of the lower body feature map.

[0011] Optionally, the attribute recognition result of the target human body is obtained based on the first attribute recognition result and the second attribute recognition result, including: determining a first attribute recognition confidence corresponding to the first attribute recognition result, and determining a second attribute recognition confidence corresponding to the second attribute recognition result; and obtaining the attribute recognition result of the target human body based on the comparison results of the first attribute recognition confidence and the second attribute recognition confidence with corresponding confidence thresholds.

[0012] Optionally, the recognition method recognizes the human body image based on a human body attribute recognition model to obtain an attribute recognition result of the target human body, wherein the human body attribute recognition model is trained using multiple sample images. When performing model training, the three-dimensional features of the sample images and the part signal features of each part of the human body are extracted, and the part signal features are used as supervisory signals to optimize the model training process based on the three-dimensional features to obtain the human body attribute recognition model.

[0013] According to another aspect of an embodiment of the present invention, a device for identifying human attributes is provided, comprising: an extraction module, used to extract features of a human body image to obtain an initial human body feature map; a generation module, used to generate category confidence maps of various human body parts based on the initial human body feature map, wherein the category confidence map is used to represent the probability that the features in the initial human body feature map respectively belong to human body part categories; a fusion module, used to fuse the initial human body feature map and the category confidence map to obtain a target human body feature map; and a recognition module, used to recognize the attributes of a target human body in the human body image based on the target human body feature map to obtain an attribute recognition result of the target human body, wherein the attribute recognition result includes identification information of the target human body.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for identifying human attributes.

[0015] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes any one of the above-mentioned methods for recognizing human attributes when running.

[0016] According to another aspect of the embodiments of the present invention, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0017] In an embodiment of the present invention, an initial human feature map corresponding to a human body image and a category confidence map representing the probability that the features in the initial human feature map belong to a human body part category are fused to obtain a target human feature map, and attribute recognition is performed on the target human body in the human body image based on the target human feature map to obtain an attribute recognition result of the target human body. Since the features in the initial human feature map are combined with the features of the human body part category, when performing attribute recognition on the target human body, the richer part and region features of the target human body are considered, which effectively improves the accuracy of attribute recognition, and can stably and accurately recognize human attributes, especially for the input of missing images, avoiding the model from erroneously outputting the attribute results of the missing part, and can effectively filter out erroneous output results, thereby improving the reliability and accuracy of recognition, thereby solving the technical problem of low accuracy when using a lightweight image classification network to reduce computing power in related technologies to recognize human attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for recognizing human attributes according to an embodiment of the present invention;

[0020] Figure 2 is a flow chart of a method for identifying human attributes according to an embodiment of the present invention;

[0021] Figure 3 is a flow chart of a method for identifying human attributes according to an optional embodiment of the present invention;

[0022] Figure 4 is a flow chart of generating a feature graph G according to an optional embodiment of the present invention;

[0023] Figure 5 is a flow chart of another optional method for recognizing human attributes according to an optional embodiment of the present invention;

[0024] Figure 6 is a schematic diagram of a human attribute recognition result according to an optional implementation manner of the present invention;

[0025] Figure 7 is a schematic diagram of another human attribute recognition result according to an optional embodiment of the present invention;

[0026] Figure 8 is a schematic diagram comparing the effects of a classic attribute recognition model and an optimized attribute recognition model according to an embodiment of the present invention;

[0027] Fig. 9 is a schematic diagram comparing the effects of an optional classic attribute recognition model and an optimized attribute recognition model according to an embodiment of the present invention;

[0028] Fig.10 is a schematic diagram comparing the effects of another optional classic attribute recognition model and an optimized attribute recognition model according to an embodiment of the present invention;

[0029] Fig.11 is a schematic diagram of a device for recognizing human attributes according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] For the convenience of description, some nouns or terms appearing in the embodiments of the present invention are explained below.

[0033] A Network Video Recorder (NVR) is a device specifically used for monitoring cameras, recording and storing video data. It is a network storage device used in surveillance systems. Unlike traditional video recorders (DVRs), NVRs are mainly used for video storage and management of network cameras (IP Cameras), while DVRs are usually connected to analog cameras. The working principle of NVR is to receive digital video streams from network cameras, which are usually compressed and encoded, such as H.264, H.265 and other encoding formats. NVR not only stores videos, but also has video management functions, including video playback, video search, video analysis and remote access. It usually communicates with network cameras and other surveillance devices through a network (such as a local area network or the Internet) without a direct physical connection.

[0034] Front-end NVR refers to the NVR device installed near the surveillance camera, which is used to record and store the video data transmitted by the camera. Through the front-end NVR, users can monitor the images captured by the camera in real time, and can also view historical videos through the recordings stored in the NVR. Front-end NVR can usually save video data through network connection or hard disk storage.

[0035] The RepVGG (Reparameterized Visual Geometry Group) network is a deep learning convolutional neural network (CNN) architecture that aims to maintain the simplicity and ease of use of the VGG network while further improving model performance and computational efficiency through improved design. The core idea of ​​the RepVGG network is to use different network structures in the training and inference stages to achieve better model expression capabilities and computational speed. During the training phase, RepVGG uses a special structure called "RepBlock", which allows the model to simultaneously learn information from multiple convolution kernels of different sizes (such as 3x3, 1x1, and 5x5, etc.), thereby enhancing the model's feature expression capabilities. RepBlock enables the model to flexibly learn features at multiple scales by fusing multiple convolution operations into a single block, which is similar to the residual connection in ResNet, but different because RepBlock is reparameterized to a simpler structure during inference. In the inference phase, RepVGG simplifies the complex structure learned during training into an equivalent, more efficient single structure through a technique called reparameterization. This means that when deployed and applied in practice, RepVGG can run with fewer computing resources and faster speeds without sacrificing the high accuracy achieved during training.

[0036] The characteristics of the RepVGG network include: Simplicity: RepVGG can be expressed as a very concise network structure at inference time, similar to the traditional VGG network, but the model's expressiveness is enhanced through multi-scale learning in the training phase. Computational efficiency: Due to the simpler structure after reparameterization, RepVGG has higher computational efficiency when running on edge devices and mobile devices, suitable for real-time processing and low-resource environments. Performance improvement: Experimental results of RepVGG on multiple benchmark datasets show that it can achieve comparable or even better performance than more complex networks, especially in tasks such as pedestrian attribute recognition, object detection, and image classification. Easy to deploy: Due to its simplified structure at the inference stage, RepVGG has higher flexibility and lower latency in network deployment, suitable for various application scenarios, including security monitoring, autonomous driving, and drone vision. The design of the RepVGG network combines model complexity and efficiency, providing a new balance point for computer vision tasks, allowing it to run efficiently on resource-limited devices while maintaining high accuracy.

[0037] A three-dimensional feature map is a multidimensional data structure output by the network layer in deep learning, especially when a convolutional neural network (CNN) processes image or video data. It is usually represented as a tensor of (C\times H\times W), where (C), (H) and (W) represent the number of channels (Channel), height (Height) and width (Width) of the feature map, respectively. This data structure reflects the network's multi-level and multi-angle feature extraction and understanding of the input data. Among them, the number of channels ((C)): The number of channels represents the depth of the feature map, that is, how many different feature channels there are. Each channel can be regarded as an abstract representation of a specific feature of the input data. For example, in the input layer, for an RGB image, the number of channels is 3, corresponding to the three color channels of red, green and blue. In the deeper layers of the network, the number of channels may increase to dozens or even hundreds, and each channel represents a more complex and abstract feature learned from the input. Height ((H)) and width ((W)): These two dimensions correspond to the vertical and horizontal spatial dimensions of the feature map, respectively. They reflect the network's understanding of the input data in space. As the depth of the network increases, the height and width of the feature map may decrease. This is because the convolutional and pooling layers in the network compress the data spatially, thereby reducing the amount of computation and parameters while retaining the most critical spatial features. Three-dimensional feature maps play a core role in CNN. The network gradually converts the input image into this feature representation through a series of convolution operations, activation functions, and pooling steps. These feature maps are then used for tasks such as classification, detection, and segmentation. By designing different network architectures and adjusting parameters, the expressive power of these feature maps can be optimized to meet the needs of specific computer vision tasks.

[0038] As introduced above, in order to reduce computing power, the related technology often uses lightweight image classification networks to identify human attributes, which often results in low accuracy. For example, the use of image classification networks to solve the problem of pedestrian attribute recognition has low computational complexity and is suitable for deployment on edge devices with low computing power, but the accuracy is low. When only the upper body image is input, the model will output the wrong lower body attribute results that are consistent with the upper body attribute results. In actual application, a large number of erroneous retrieval results will be generated, affecting the use of basic security functions and user experience. In the related technology, the following human attribute recognition scheme is also proposed.

[0039] For example, in addition to the basic attribute recognition network, additional small object target detection networks and target tracking networks are introduced to correct the attribute recognition results. The purpose is to smooth the attribute recognition results of video pedestrians so that the same pedestrian can have a relatively stable output in the video sequence. However, target detection and target tracking usually have high computational complexity and are difficult to deploy to edge devices. At the same time, the recognition performance bottleneck still lies in the attribute recognition network itself, that is, the performance bottleneck still depends on the accuracy of the attribute recognition network.

[0040] For another example, the feature map and heat map of human body blocks are generated by bone key points, and then the attribute recognition results of the corresponding blocks are generated. The model focuses on the regional features of different parts of the human body and can obtain better attribute recognition results. However, the computational complexity is still too high compared to the single attribute recognition network. Therefore, although the introduction of bone key points to generate feature maps and heat maps of human body blocks can also improve the accuracy of attribute recognition, it also introduces additional computing costs and is difficult to deploy to edge devices for real-time computing.

[0041] In addition, it is proposed to use the human body mask results to remove the interference of background information on attribute recognition results. A common perception is that the calculation of human body segmentation masks requires the model to use higher memory to maintain a relatively large resolution feature map, otherwise the low-resolution rough mask cannot filter the background information well, and will introduce certain noise to affect the recognition results. At the same time, simply separating the human body and background information cannot deal with the problems faced in actual implementation, that is, only inputting the upper body human body picture, the model mistakenly outputs the lower body attribute results that are consistent with the upper body attributes. Therefore, the above-mentioned actual false alarm problem still cannot be handled.

[0042] Therefore, for human attribute recognition, the above solution fails to strike a good balance between accuracy and speed, and also fails to properly handle the false alarm problem faced in actual implementation.

[0043] In view of the above-mentioned defects, a method and apparatus for identifying human attributes, a computer-readable storage medium, a processor, and a computer program product are provided in embodiments of the present invention.

[0044] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0045] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device, for example, it can be executed in an edge device with low computing power in a monitoring network system. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for recognizing human attributes according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0046] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the human attribute recognition method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific example of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0047] According to an embodiment of the present invention, a method embodiment of a method for identifying human attributes is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Figure 2 is a flow chart of a method for identifying human attributes according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0049] Step S202, extracting features of the human body image to obtain an initial human body feature map.

[0050] As an optional embodiment, in the embodiment of the present invention, the human body image may be a pedestrian image. For example, the image may be used to collect images of pedestrians passing by at an entrance or exit where security is required, such as the entrance or exit of a supermarket.

[0051] As an optional embodiment, the above-mentioned initial human feature map can be a three-dimensional feature map, which characterizes the human image from three dimensions (C), (H) and (W), wherein (C), (H) and (W) respectively represent the number of channels (Channel), height (Height) and width (Width) of the feature map. When extracting the features of the human image, the backbone network of the neural network model can be used for feature extraction. Among them, in order to improve the efficiency of attribute recognition, the neural network model can adopt some lightweight neural network models. Lightweight neural network models are designed to overcome the computational efficiency and storage requirements of traditional deep learning models in resource-constrained environments such as mobile devices and embedded systems. These models achieve lower computational complexity and smaller model size while maintaining high accuracy by reducing the number of parameters, optimizing the calculation process, and using more efficient operations. The types of lightweight neural network models can also include multiple types, for example, MobileNet series, SqueezeNet, EfficientNet, Lite-HRNet, RepVGG, etc. Optionally, in an embodiment of the present invention, a RepVGG model is selected that uses a multi-scale convolution kernel during training and can be reparameterized to a simpler structure during inference. This makes it highly computationally efficient while maintaining performance, and is particularly suitable for running on edge devices and mobile devices.

[0052] For example, in the scene of pedestrian attribute recognition, after the pedestrian image is processed by the backbone network of the RepVGG model, the generated three-dimensional feature map contains a deep understanding of the pedestrian image, and each channel may represent a certain abstract description of the pedestrian's attributes (such as gender, clothing type, whether to carry items, etc.). These feature maps are important inputs for subsequent classification tasks and are used to predict the specific attributes of pedestrians. In turn, they can effectively extract key information from the image that is helpful for subsequent attribute recognition and reduce background interference. By designing a suitable network structure and training strategy for the RepVGG model, the quality of the feature map can be optimized and the accuracy of attribute recognition can be improved.

[0053] Step S204, generating a category confidence map of each human body part according to the initial human body feature map, wherein the category confidence map is used to represent the probability that the features in the initial human body feature map belong to the human body part category respectively.

[0054] As an optional embodiment, when generating the category confidence map of each part of the human body according to the initial human feature map, the convolution operation in the neural network can be used to complete it. For example, the input three-dimensional feature map (i.e., the initial human feature map) is subjected to feature conversion through a designed convolution layer, and then the output is converted into a probability distribution of category confidence through an activation function (such as Softmax), and finally a category confidence map is formed, whose size and number of channels match the number of categories to be identified. In this way, the network can not only identify various parts of the human body, but also output the confidence that each part belongs to different categories, providing richer information and more accurate guidance for subsequent attribute recognition tasks.

[0055] Among them, the design of the convolution layer mainly includes: the filters of the convolution layer will be trained to recognize different patterns in the feature map and then mapped to each category. In order to generate the category confidence map, the size and number of filters need to be designed according to the task and the size of the input feature map. For example, a 3x3 convolution kernel can be used, which can maintain sensitivity to local areas while also having enough receptive field to capture the contextual relationship between parts. The application of activation function mainly includes: after the convolution operation, an activation function is usually applied, such as ReLU (Rectified Linear Unit) or sigmoid. However, in the case of generating confidence maps, sigmoid or softmax functions may be more appropriate because they can convert the output value into a probability form, indicating the confidence level that each pixel belongs to a specific category.

[0056] The combined training of convolutional layers and activation functions also requires the calculation of the corresponding Softmax function and loss function. Softmax function: The task is multi-classification, that is, each pixel (feature point) in the human image may belong to one of multiple categories, so the Softmax function is generally used in the final output layer. The Softmax function converts the output of each pixel into a probability distribution so that the sum of the output values ​​is 1, so that the output of each pixel represents the probability that it belongs to each category, thereby forming a category confidence map. Loss function calculation: The generated category confidence map is compared with the actual human body part category mask label. The cross-entropy loss function is usually used to calculate the difference between the prediction and the actual label. This is used as a training signal, and the weight parameters of the network are adjusted by backpropagation so that the convolutional layer combined with the activation function can generate the category confidence map more accurately.

[0057] As an optional embodiment, the category confidence map is used to represent the probability that the features in the initial human feature map belong to the human body part category. The features in the initial human feature map may refer to the spatial position in the human body image, for example, a pixel point.

[0058] The class confidence map represents the probability that each pixel belongs to different categories. In computer vision and image processing tasks, especially problems such as semantic segmentation and human body part recognition, the class confidence map is a two-dimensional or three-dimensional matrix in which each pixel (or spatial position) has a vector corresponding to the number of categories, and each value in the vector represents the probability that the pixel belongs to that category. For example, in the context of pedestrian attribute recognition, the class confidence map may include the following categories: background, hair, face, upper body clothing, lower body clothing, legs, etc. For each pixel in the image, the class confidence map will give a vector containing these category probabilities. Each value in the vector ranges from 0 to 1, representing the confidence level of predicting that the pixel belongs to a specific category. In multi-classification tasks, these probability values ​​are processed through the Softmax function to ensure that they add up to 1, thus forming a probability distribution. The category confidence map not only predicts the category of each pixel in the image, but also provides the confidence level of these predictions, which is crucial for improving the robustness of the prediction and subsequent processing tasks (such as confidence-based post-processing or integration of other information). A high confidence value indicates that the prediction result is very sure, while a low confidence value may indicate uncertainty in the prediction result, which is very important when dealing with complex scenes such as occlusion, lighting changes, and perspective changes, and can help the system make more reasonable decisions.

[0059] Step S206, fusing the initial human feature map and the category confidence map to obtain a target human feature map.

[0060] As an optional embodiment, the target human feature map is obtained by fusing the initial human feature map and the category confidence map. The target human feature map not only includes the basic image features identified from the human image, but also fuses the category features in the category confidence map. The category features characterize the probability that each pixel in the initial human feature map (the part of the human image involving the target human body) belongs to each part. Therefore, compared with only identifying the basic image features, the image information is richer and more detailed, so that the obtained target human feature map can better reflect the characteristics of the target human body in the human image.

[0061] As an optional embodiment, when the initial human feature map and the category confidence map are fused to obtain the target human feature map, the depth feature of the three-dimensional features of depth, height and width in the initial human feature map can be fused with the category feature in the category confidence map to obtain the target human feature map. The following is an example.

[0062] First, before fusing the initial human feature map and the category confidence map, the two can be matched in size. For example, the initial human feature map and the category confidence map can be made to have the same size in height (H) and width (W) dimensions. If the sizes do not match, it may be necessary to adjust the size of one of the images through operations such as upsampling or downsampling so that they can be aligned in space.

[0063] Secondly, when performing the channel splicing operation, channel splicing means merging the channel dimensions of the initial human feature map and the category confidence map together to form a larger feature map. Specifically, if the dimension of the initial human feature is (C\t imes H\t imes W) and the dimension of the category confidence map is (C'\t imes H\t imes W), then the dimension of the spliced ​​feature map will become ((C+C')\t imes H\t imes W). Here C is the number of channels of the initial human feature map, and C' is the number of channels of the category confidence map. The increase in the number of channels means that the model can simultaneously utilize the high-level semantic features in the initial human feature map and the human body part position confidence information in the category confidence map. Channel splicing is done by adding each channel of the category confidence map to the back of the channel set of the initial human feature map, which is like expanding in the "depth" direction of the feature map, thereby creating a new feature map with more information. Use the concatenated feature map: Once the initial human feature map and the category confidence map are concatenated to form a new feature map, this feature map can be used as the input of subsequent network layers, such as 1x1 convolutional layers, fully connected layers, or pooling layers. These layers can further process and fuse these features to perform more complex tasks, such as pedestrian attribute recognition.

[0064] For example, in the pedestrian attribute recognition scenario, the size of the initial human feature map is 128x16x8, while the category confidence map size is 6x16x8, representing the confidence of 6 human body parts. By splicing along the channel dimension, a 134x16x8 feature map is formed. The feature map is then compressed to 32 channels through a 1x1 convolution operation. This process can be regarded as a feature fusion and dimensionality reduction, the purpose of which is to enhance the accuracy of the classification task while reducing the computational complexity.

[0065] Step S208, based on the target human feature map, identifying the attributes of the target human body in the human body image, and obtaining an attribute recognition result of the target human body, wherein the attribute recognition result includes identification information of the target human body.

[0066] As an optional embodiment, when the attributes of the target human body in the human body image are identified based on the target human body feature map and the attribute identification result of the target human body is obtained, the attributes of the entire target human body can be directly identified, or the target human body can be divided into regions for identification. For example, when it is not necessary to distinguish which part of the target human body, the attributes of the entire target human body can be directly identified to obtain all the attribute identification results of concern. When it is necessary to divide the target human body into regions for identification, the part of concern or all the parts can be identified.

[0067] For example, based on the target body feature map, the attributes of the target body in the body image are identified to obtain the attribute recognition result of the target body, when the part of interest is the upper body and / or lower body of the target body, the following processing method can be adopted: segment the target body feature map to obtain the upper body feature map and the lower body feature map of the target body in the body image; perform attribute recognition on the upper body feature map and the lower body feature map respectively to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map; obtain the attribute recognition result of the target body based on the first attribute recognition result and the second attribute recognition result.

[0068] Through the above processing, the target human body feature map is segmented into the upper body feature map and the lower body feature map of the target human body in the human body image, which can achieve more accurate human attribute recognition and avoid confusion between the upper body features and the lower body features. Even if the image is incomplete, the attributes of the visible part can be recognized.

[0069] In this embodiment, attribute recognition is performed on the upper body and lower body feature maps separately, which can more accurately capture the characteristics of different parts of the human body, more comprehensively and accurately identify human body characteristics, improve recognition effects, and provide more reliable data support for subsequent applications. By separately identifying, the differences in attribute characteristics that may be different between the upper body and the lower body can be more flexibly handled, providing richer human attribute information.

[0070] Among them, when the target human body feature map is segmented to obtain the upper body feature map and the lower body feature map of the target human body in the human body image, the target human body feature map can be segmented according to the height of the three dimensions of depth, height and width in the target human body feature map to obtain the upper body feature map and the lower body feature map of the target human body in the human body image.

[0071] Splitting the target human feature map into two feature maps along the height dimension is a spatial feature separation strategy in deep learning, which is mainly used to process tasks that require distinguishing the features of the upper and lower parts of the image or different spatial regions. This operation is based on the assumption that different height positions in the feature map may carry specific information of different parts of the image. For example, in the scenario of pedestrian attribute recognition in an embodiment of the present invention, this assumption is particularly reasonable because the features of the upper body (such as face, hair, top) and the lower body (such as legs, bottoms) are visually different. The operation steps are as follows:

[0072] The size of the target human feature map is assumed to be (C\times H\times W), where C is the number of channels, H is the height, and W is the width. The size of the target human feature map in the pedestrian attribute recognition scene is 32x16x8. Split along the height direction: Split the target human feature map into two parts along the height H direction, and the height of each part is 8 (i.e. (16 / 2=8)). In this way, the size of the upper body feature map becomes 32x8x8, and this part of the feature map mainly contains the information of the upper body; the size of the lower body feature map is also 32x8x8, which mainly contains the information of the lower body. The split feature maps can be used to process tasks related to their corresponding areas respectively. In this pedestrian attribute recognition scene, the upper body feature map can be used to identify upper body related attributes such as head features, upper clothing type and color, and lower body feature map is used to identify lower body related attributes such as lower clothing type and color, and leg features. Therefore, by splitting the feature map along the height direction and processing them separately, the model can focus more on the local area features of the image, improve the accuracy of attribute recognition, especially when dealing with occlusion, it can avoid confusion between upper and lower body features and reduce false positives. This method is particularly effective when dealing with human bodies or other objects with obvious upper and lower or regional features.

[0073] It should be noted that in order to further decouple the features in the upper body feature map and the lower body feature map, they can be downsampled separately, for example, using a convolutional layer with a step size of 2 to compress them to a smaller size, such as 32x4x4. This can reduce the amount of calculation, compress the features, and extract a more compact representation, which is convenient for subsequent classifier processing.

[0074] As an optional embodiment, when the upper body feature map and the lower body feature map are respectively subjected to attribute recognition, and the first attribute recognition result of the upper body feature map and the second attribute recognition result of the lower body feature map are obtained, the upper body feature map and the lower body feature map can be respectively input into the corresponding attribute classifiers, and the corresponding attribute recognition results can be obtained based on the processing of the attribute classifiers. For example, the upper body feature map can be input into the attribute classifier corresponding to the upper body to obtain the first attribute recognition result of the upper body feature map; the lower body feature map can be input into the attribute classifier corresponding to the lower body to obtain the second attribute recognition result of the lower body feature map. For example, the upper body feature map and the lower body feature map after the above downsampling are respectively input into the corresponding attribute classifiers, and these attribute classifiers are specially designed to identify the attributes of the upper body or the lower body. The attribute classifier usually includes a pooling layer and a fully connected layer, which are used to generate the final attribute classification result, that is, the attribute recognition result.

[0075] In the scenario of pedestrian attribute recognition, a complete initial human feature map is first extracted through the backbone network, and then the initial human feature map is divided into two parts: an upper body feature map and a lower body feature map. The upper body feature map mainly contains the feature information of the upper body, while the lower body feature map mainly focuses on the feature information of the lower body.

[0076] The upper body feature map is fed into an attribute classifier designed specifically for identifying upper body attributes (such as gender, hair length, backpack, top color and type, etc.); the lower body feature map is fed into an attribute classifier for lower body attributes (such as bottom color, type, leg features, etc.). Each attribute classifier further processes the feature map through neural network layers (such as convolutional layers, pooling layers, and fully connected layers) to extract task-related features and classify them. The attribute classifier corresponding to the upper body and the attribute classifier corresponding to the lower body output corresponding confidences respectively. The attribute recognition confidence output by the attribute classifier is a numerical value that indicates the model's confidence that a certain human attribute belongs to a specific category. Confidence is usually calculated using Sigmoid or Softmax functions, and its value ranges from 0 to 1. A high confidence value means that the classification of the attribute is very certain, while a low confidence value indicates high uncertainty.

[0077] By segmenting the spatial feature maps (upper body feature maps and lower body feature maps) in a targeted manner and feeding them into the attribute classifier of a specific task, the accuracy and specificity in identifying specific attributes can be enhanced. This targeted processing can better focus on the details of the upper or lower body of the human body, reducing the recognition error caused by feature confusion. In addition, such processing can also improve robustness: in the case of image occlusion or partial invisibility, for example, only the upper body is visible, the upper body feature map can still be used to accurately predict the upper body attributes, and the lower body features can obtain a very low lower body attribute confidence because the human body part category confidence feature map is added, so the output lower body attribute confidence will be low, which can avoid the wrong prediction of missing information. Therefore, when the whole body is visible, the upper and lower body feature maps can focus on the information of the upper and lower body respectively, so the confidence obtained is more accurate and reliable, and the recognition accuracy is improved. In addition, such processing can also effectively achieve efficiency and resource optimization: this processing allows the upper and lower parts of the image to be processed in parallel, and can utilize parallel computing resources on multi-core processors or image processors to improve computing efficiency. At the same time, since each attribute classifier only needs to process a specific feature map, computing costs and memory consumption can be optimized, which is especially important for edge computing devices. Furthermore, by processing the feature maps of the upper and lower body separately, the mutual interference between features of different body parts can be reduced, making the recognition results more stable and consistent, especially in complex scenes, such as when there are many backgrounds or interfering objects.

[0078] Therefore, this processing method of targeted segmentation of feature maps and sending them to the attribute classifiers corresponding to the upper and lower body for identification not only improves the accuracy and robustness of pedestrian attribute recognition, but also effectively utilizes computing resources, reduces the calculation of unnecessary features, and improves the applicability and efficiency of this processing method on edge devices.

[0079] As an optional embodiment, when obtaining the attribute recognition result of the target human body according to the first attribute recognition result and the second attribute recognition result, a variety of methods can be used. For example, the following processing method can be used: determining the first attribute recognition confidence corresponding to the first attribute recognition result, and determining the second attribute recognition confidence corresponding to the second attribute recognition result; obtaining the attribute recognition result of the target human body according to the comparison result of the first attribute recognition confidence and the second attribute recognition confidence with the corresponding confidence threshold. Based on the confidence corresponding to the attribute recognition result, it can be selected whether the corresponding attribute recognition result is credible, and the attribute recognition result of the target human body can be determined based on this confidence.

[0080] For example, when an upper body image is input, the confidence of the lower body attribute recognition result is very low and the erroneous results can be filtered out; or when a lower body image is input, the confidence of the upper body attribute recognition result is very low and the erroneous results can be filtered out. Therefore, information that does not appear or is partially missing in the image can be processed more accurately.

[0081] For example, in pedestrian attribute recognition, the attributes of pedestrians are judged by analyzing the features in the input image, such as gender, hair length, whether they are carrying a backpack, and the color and type of clothing. These judgments are visual patterns and features learned from a large amount of training data. When the image is a complete pedestrian image, comprehensive visual information can be extracted from the image for attribute recognition and classification, thereby giving a high-confidence prediction result.

[0082] However, when the input image is only the upper body, the lack of visual information of the lower body poses a challenge to identifying the attributes related to the lower body. In traditional pedestrian attribute recognition models, even if the input is an upper body image, the model may still try to predict the attributes of the lower body. Due to the lack of corresponding visual clues, these predictions are often inaccurate, or even random guesses, and the confidence values ​​may still be high, resulting in misleading results.

[0083] However, in an embodiment of the present invention, the initial human feature map is divided into an upper body feature map and a lower body feature map along the height direction, wherein the upper body feature map is mainly responsible for the upper body attribute recognition, while the lower body feature map focuses on the lower body attribute recognition. When the input image only has the upper body, the upper body feature map will not contain the visual features of the lower body, so when it is sent to the lower body attribute recognition classifier, the information obtained by the attribute classifier from the upper body feature map is not enough to make an accurate judgment. In the deep learning model, because the human body part category confidence feature map is combined, a lower confidence value is usually output in the case of such lower body occlusion. Therefore, by splitting the upper and lower body feature maps, the accuracy of human body attribute recognition is effectively improved when the whole body is visible. This is different from the high confidence error prediction that may be output by the traditional model. When the input image only has the lower body, the lower body feature map will not contain the visual features of the upper body, so when it is sent to the upper body attribute recognition classifier, the information obtained by the attribute classifier from the lower body feature map is not enough to make an accurate judgment. In the deep learning model, because the confidence feature map of the human body part category is combined, a lower confidence value is usually output in the case of lower body occlusion. This is also different from the high confidence error prediction that the traditional model may output.

[0084] Therefore, by setting an appropriate threshold to filter the confidence value, it is possible to avoid using these uncertain lower body attribute recognition results as valid outputs, thereby filtering out erroneous results and improving the overall accuracy and reliability of the system. For example, if the confidence output of the lower body attribute recognition classifier for the input upper body image is lower than a certain set threshold, it can be considered that reliable lower body attribute information cannot be provided, so this part of the recognition result is not included or marked as uncertain, avoiding false alarms caused by missing lower body visual information. This processing method is particularly important for practical application scenarios such as security monitoring, because it ensures that the system will not generate misleading or erroneous alarm signals when information is incomplete, reduces noise in data processing, and improves user experience and system practicality.

[0085] As an optional embodiment, the above-mentioned human body attribute recognition method can recognize human body images based on a human body attribute recognition model to obtain the attribute recognition result of the target human body, wherein the human body attribute recognition model is trained using multiple sample images. When training the model, the three-dimensional features of the sample images and the part signal features of each part of the human body are extracted, and the part signal features are used as supervisory signals to optimize the model training process based on the three-dimensional features to obtain the human body attribute recognition model.

[0086] Using position information features as supervisory signals to optimize the model training process based on three-dimensional features can help the model learn more refined and part-related feature representations during the training process.

[0087] During the training phase of the model, not only human images are input, but also a set of additional supervision information is provided to the model, such as a multi-category mask map of human body parts. This mask map can be pre-generated by an annotation tool and is used to annotate the pixel positions of different human body parts (such as hair, face, top, bottom, legs, etc.) in pedestrian images. Each pixel of the mask map has a value, which represents the probability that the pixel belongs to a certain human body part category. For example, if a pixel belongs to the hair part, then there will be a higher probability value on the channel of the hair category and a lower one on other channels.

[0088] During model training, the mask map is not used as a supervisory signal for the final output, but as a supervisory signal for the intermediate layer. This means that after extracting features, the model will generate a predicted mask map, and then compare this predicted mask map with the real mask map to calculate the loss (such as softmax cross entropy loss) to guide the optimization of the model. The advantage of this approach is that it encourages the model to pay attention to different parts of the human body while learning human attribute recognition, thereby generating a new feature map that integrates part information based on the above initial human feature map.

[0089] By supervising the part mask map generated by the intermediate layer, the model can more finely separate and express the features of different parts, which is very beneficial for attribute recognition tasks. For example, the color and type features of the top and the features of the lower body can be extracted and expressed more independently by the model, so that when performing attribute classification, the interference between the features of different parts can be reduced, and the accuracy and robustness of recognition can be improved.

[0090] Therefore, generating and using multi-category mask images as intermediate supervisory signals can guide the model to learn part-related features, thereby improving the accuracy of pedestrian attribute recognition and its adaptability in actual complex scenes. This method not only considers the computational efficiency of the model, but also pays special attention to how to solve the problem of attribute recognition under partial occlusion.

[0091] It should be noted that the above-mentioned mask image is a special type of image used to represent or highlight a specific area or object in an image. The value of each pixel in the mask image indicates whether the location belongs to the area or object of interest, or the degree to which the location belongs to different objects or areas. In the human attribute recognition scenario, the mask image is a semantic mask: for each pixel in the image, the mask image specifies the category to which it belongs. For example, in pedestrian attribute recognition, the semantic mask can contain different categories such as hair, face, top, bottom, legs, etc., each category has a specific numerical value or color code.

[0092] In the training of human attribute recognition models, the use of mask maps (especially multi-category mask maps) can guide the model to learn the characteristics of different body parts, so that it can perform more accurately in attribute recognition tasks. For example, through the semantic mask on the intermediate feature map, the model can be trained to distinguish and recognize the attributes of parts such as the head, face, tops, and bottoms without being disturbed by the background or the characteristics of other body parts. This training strategy helps to improve the recognition ability and robustness of the model when dealing with partially occluded or incomplete human images.

[0093] Through the above processing, the initial human feature map corresponding to the human body image and the category confidence map representing the probability that the features in the initial human feature map belong to the human body part category are fused to obtain the target human feature map, and the target human body in the human body image of the target human feature map is attributed to obtain the attribute recognition result of the target human body. Since the features in the initial human feature map are combined with the features of the human body part category, the richer part and region features of the target human body are considered when the attributes of the target human body are recognized, which effectively improves the accuracy of attribute recognition, and can stably and accurately recognize human attributes, especially for the input of missing images, avoiding the model from erroneously outputting the attribute results of the missing part, and can effectively filter out the erroneous output results, thereby improving the reliability and accuracy of recognition, thereby solving the technical problem of low accuracy when using lightweight image classification networks to reduce computing power in related technologies to recognize human attributes.

[0094] Based on the above processing, feature extraction is performed based on the human body image to generate a three-dimensional feature map and a category confidence map, and then splicing is performed to obtain a target human body feature map. Through the three-dimensional feature map, richer human body spatial information can be obtained to provide a more accurate feature description for subsequent attribute recognition. The splicing operation retains the spatial information in the three-dimensional feature map and the probability information in the category confidence map, so that the final target human body feature map contains more comprehensive data and can better reflect the key attributes of the human body. After the initial human body feature map is divided into an upper body feature map and a lower body feature map, the attributes of the target human body are obtained according to the corresponding first attribute recognition results and second attribute recognition results. After the features of the upper and lower body of the human body image are respectively recognized, these features are combined to determine the specific attributes of the target human body.

[0095] Based on the above embodiments and optional embodiments, an optional implementation manner is provided.

[0096] Figure 3 is a flow chart of a method for identifying human attributes according to an optional embodiment of the present invention, such as Figure 3 As shown, the method includes the following processing:

[0097] Step 1. Extract features using the backbone network. To ensure the computational efficiency of the network, a lightweight RepVGG network structure is selected. The size of the extracted three-dimensional feature map M (i.e., the initial human feature map) is 128x16x8. After that, an intermediate feature map G (i.e., the target human feature map) is generated. Based on the feature map M, a category confidence map F is generated. M and F are concatenated to form the feature map G.

[0098] It should be noted that the above network structure can be replaced by other lightweight networks.

[0099] In the above embodiment of the present invention, feature extraction is performed on a human body image to obtain a three-dimensional feature map corresponding to the human body image, including: inputting the human body image into a human body attribute recognition model, and utilizing the feature extraction module (i.e., the above-mentioned backbone network) in the human body attribute recognition model to perform feature extraction on the human body image to obtain a three-dimensional feature map.

[0100] In this embodiment, a human body image is input into a human body attribute recognition model, and feature extraction is performed using a feature extraction module in the model. Through the feature extraction module, a richer and more abstract feature representation than the original image can be obtained to obtain a three-dimensional feature map, which increases the depth dimension. This helps the model to better understand the spatial relationship and three-dimensional structure in the image, and provide richer information for subsequent attribute recognition tasks.

[0101] For the scene of pedestrian attribute recognition, such as Figure 3 As shown in the figure, a pedestrian picture is input into the backbone network to extract a three-dimensional feature map M of size 128x16x8.

[0102] In the above embodiment of the present invention, a category confidence map of a target human body is generated according to a three-dimensional feature map, including: inputting the three-dimensional feature map into a human attribute recognition model to perform convolution processing on the three-dimensional feature map to obtain a category confidence map.

[0103] In this embodiment, the three-dimensional feature map is input into the human body attribute recognition model to perform convolution processing on the three-dimensional feature map, which can extract richer features of human body parts from the three-dimensional feature map and enhance the expressiveness of the features. The obtained category confidence map reflects the probability that each pixel belongs to a certain part of the target human body, which helps the model to more accurately identify the attributes of different parts of the human body.

[0104] Figure 4 is a flow chart of generating a feature graph G according to an optional embodiment of the present invention, such as Figure 4 As shown in the figure, for the feature map M, a convolution with a kernel size of 3 is performed to output the category confidence map F of the six parts of the target human body on the image, with a size of 6x16x8. The category confidence map F is combined with the corresponding human body part category mask label, and the loss value L1 is calculated through the softmax cross entropy loss function.

[0105] Optionally, the above category confidence map F may include six categories: background, hair, face, upper body clothing, lower body clothing and legs.

[0106] It should be noted that each pixel on the above category confidence map F has 6 values, and the size of the value represents the probability value of the model judging that the pixel belongs to a certain category; the above softmax cross entropy loss function is one of the commonly used loss functions in neural networks, which combines the Softmax function and cross entropy loss and is suitable for multi-classification problems.

[0107] In the above embodiment of the present invention, the three-dimensional feature map and the category confidence map are spliced ​​to obtain a human feature map, including: splicing the three-dimensional feature map and the category confidence map to obtain a human splicing map; and using a human attribute recognition model to aggregate information on the human splicing map to obtain a human feature map. In a convolutional neural network, the output of each convolutional layer is usually a three-dimensional feature map, which contains multiple channels, each channel corresponding to a filter or feature detector in the layer, which is used to capture different features.

[0108] In this embodiment, a human body splicing map is obtained by splicing the three-dimensional feature map and the category confidence map along the channel dimension. The spliced ​​human body splicing map contains richer information. Then, the human body attribute recognition model is used to aggregate information of the human body splicing map, which can effectively fuse and refine the key features of the human body image, thereby generating a human body feature map.

[0109] like Figure 4 As shown in the figure, M (i.e., three-dimensional feature map) and F (i.e., category confidence map) are concatenated along the channel dimension to form a new feature map, and then the information is aggregated through convolution with a kernel size of 1. Finally, a feature map G (i.e., human feature map) of size 32x16x8 is obtained, which contains semantic information of human attributes and human body parts, and can enhance the accuracy of subsequent classification tasks.

[0110] It should be noted that, unlike the binary mask, the above-mentioned category confidence map F does not need to maintain a high-resolution feature map for calculation, and does not consume additional memory. The mask is a multi-category mask of 6 parts of the human body. As a supervisory signal of the intermediate feature, it guides the model to model the features of different regions of the human body separately; the above-mentioned use of the human body part mask map to supervise and enhance the model's feature expression ability does not belong to the unsupervised category. The training process will introduce mixed training of data with 6-part segmentation of the human body (without attribute annotation) and data with attribute annotation, which itself still belongs to the scope of supervised learning.

[0111] In addition, the above-mentioned splicing is to use the human body part confidence map F related to human body segmentation regressed from the intermediate features, which will be spliced ​​with the feature map M extracted by the backbone network to generate the feature map G. In this way, the feature of each pixel contains the information of which part of the human body the pixel belongs to, which explicitly enhances the distinction of the features extracted from different positions of the upper and lower body, and can achieve a very low attribute confidence output of the half-body human body so that it can be filtered.

[0112] According to the above embodiment of the present invention, the human body feature map is segmented to obtain the upper body feature map and the lower body feature map of the target human body in the human body image, including: using the human body attribute recognition model to segment the human body feature map along a high dimension to obtain the upper body feature map and the lower body feature map.

[0113] In this embodiment, the human body feature map is segmented along a high dimension using a human body attribute recognition model. After the segmentation, the feature maps of the upper body and the lower body can be processed in parallel to improve the efficiency of attribute recognition. The obtained upper body and lower body feature maps are more focused on the human body itself and reduce the interference of background information.

[0114] Figure 5 is a flowchart of another optional method for identifying human attributes according to an optional embodiment of the present invention, such as Figure 5 As shown in the figure, in order to guide the model to learn the spatial features of the upper and lower body of the human body, the feature map G is split along the height dimension into two upper and lower feature maps T and B, with a size of 32x8x8.

[0115] It should be noted that the above-mentioned feature map T (i.e., the upper body feature map) contains feature information of the upper body of the human body, and is suitable for attribute recognition tasks related to the upper body of the human body, such as gender, hair length, whether to carry a backpack, and color type of the top; the above-mentioned feature map B (i.e., the lower body feature map) contains feature information of the lower body of the human body, and is suitable for attribute recognition tasks related to the lower body of the human body, such as color type of the lower clothes.

[0116] According to the above embodiment of the present invention, attribute recognition is performed on the upper body feature map and the lower body feature map respectively to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map, including: downsampling the upper body feature map and the lower body feature map using a human body attribute recognition model to obtain the upper body feature map and the lower body feature map after the downsampling; length compression is performed on the upper body feature map and the lower body feature map after the downsampling using a pooling layer of the human body attribute recognition model to obtain a first feature vector corresponding to the upper body feature map and a second feature vector corresponding to the lower body feature map; the first feature vector and the second feature vector are transmitted to the corresponding classifier of the human body attribute recognition model to perform attribute classification processing on the first feature vector and the second feature vector using the corresponding classifier to obtain a first attribute recognition result and a second attribute recognition result.

[0117] In this embodiment, by downsampling the upper body feature map and the lower body feature map and performing subsequent pooling layer compression processing, the downsampling process helps to reduce the consumption of computing resources and improve the processing speed, while retaining the key information of human body features as much as possible, and can effectively extract the key feature vectors of the two parts. Through the pooling operation, a more abstract feature representation, namely, a feature vector, can be extracted from the downsampled feature map, which captures the key attribute information of the upper and lower body of the human body.

[0118] like Figure 5As shown in the figure, in order to further decouple the correlation between feature maps T and B, T and B are respectively passed through a convolutional layer with a kernel size of 3 and a stride of 2, downsampled to a 32x4x4 feature map, and then compressed into a feature vector of length 32 by a pooling layer, which are respectively sent to the classifiers of the upper and lower body related attributes, and the attribute recognition confidence is output.

[0119] According to the above embodiment of the present invention, the attributes of the target human body are obtained according to the first attribute recognition result and the second attribute recognition result, including: determining the first attribute recognition confidence corresponding to the first attribute recognition result, and determining the second attribute recognition confidence corresponding to the second attribute recognition result; and obtaining the attributes of the target human body according to the comparison results of the first attribute recognition confidence and the second attribute recognition confidence with the confidence threshold respectively.

[0120] In this embodiment, the first attribute recognition confidence and the second attribute recognition confidence corresponding to the first attribute recognition result and the second attribute recognition result are determined, and the result of the attribute recognition can be quantitatively evaluated. By comparing the confidence and the threshold, the recognition result can be distinguished into a high-confidence result and a low-confidence result. When the confidence exceeds the threshold, the recognition result is used as the final attribute, which can avoid using a low-confidence erroneous recognition result as the final result, thereby improving the overall recognition accuracy.

[0121] It should be noted that the above attributes use sigmoid cross entropy loss to calculate the loss value L2, and pass back the gradient to update the weight parameters of the model. The final total loss function is as follows: L = L1*0.5+L2, where the balance factor of the L1 loss function is 0.5, and the balance factor of the L2 loss function is 1; the loss function (Loss Function) is a function that measures the gap between the model prediction value and the true value, and is used to guide parameter optimization during model training. The attribute uses sigmoid cross entropy loss to calculate the loss value L2, which means that the cross entropy loss function is used for binary classification problems (such as the presence or absence of attributes) in the model; the above loss function L consists of two parts, each of which is multiplied by a balance factor. The purpose is to balance the contribution of different losses during training to ensure that the model gives appropriate attention to all types of errors.

[0122] Figure 6 is a schematic diagram of a human attribute recognition result according to an optional embodiment of the present invention, Figure 7 is a schematic diagram of another human attribute recognition result according to an optional embodiment of the present invention, such as Figure 6 ,7, different colors represent the five categories of human hair, face, upper body clothes, lower body clothes and legs, and the background is uniformly represented by black. Figure 6 Figure 7 It can be seen that the intermediate features of the model have correctly learned the knowledge of human body parts.

[0123] Figure 8 is a schematic diagram comparing the effects of the classic attribute recognition model and the optimized attribute recognition model according to an embodiment of the present invention. Figure 8 As shown in the figure, the recognition result of the classic attribute recognition model is that the gender is male, there is a backpack, no handcarry, the upper body clothes are white long sleeves, and the lower body clothes are blue trousers. The optimized attribute recognition result is that the gender is male, there is no backpack, no handcarry, the upper body clothes are brown long sleeves, and the lower body clothes are blue trousers. The optimized attribute recognition result is more accurate.

[0124] Fig. 9 is a schematic diagram comparing the effects of the optional classic attribute recognition model and the optimized attribute recognition model according to an embodiment of the present invention. Fig. 9 As shown in the figure, the recognition result of the classic attribute recognition model is that the gender is female, there is no backpack, there are hand-held objects, the upper body clothes are black short-sleeved, and the lower body clothes are black skirts. The optimized attribute recognition result is that the gender is female, there is a backpack, there are no hand-held objects, the upper body clothes are yellow short-sleeved, and the lower body clothes are black skirts. The optimized attribute recognition result is more accurate.

[0125] Fig.10 is a schematic diagram comparing the effects of another optional classical attribute recognition model and the optimized attribute recognition model according to an embodiment of the present invention. Fig.10 As shown, the recognition result of the classic attribute recognition model is that the gender is female, there is no backpack, no handcarry, the upper body clothes are orange long sleeves, and the lower body clothes are orange trousers. The optimized attribute recognition result is that the gender is male, there is no backpack, no handcarry, the upper body clothes are yellow short sleeves, and the lower body clothes are uncertain. The optimized attribute recognition result is more accurate.

[0126] Combination Figure 8 Fig. 9 Fig.10 It can be seen that the optimized model can represent the missing attributes of the input half-body image as uncertain, and the color category is more stable (from left to right are the recognition results of the classic model and the optimized model).

[0127] The technical solution provided by the above-mentioned embodiments of the present invention solves the practical implementation problem of pedestrian attribute recognition technology, thereby achieving the following beneficial effects: 1) The intermediate features are supervised by the mask map of human body parts, and the mask map calculated by the model does not need to be very fine, thus saving the consumption of resources such as memory and CPU; 2) The mask type includes 6 parts of the human body, which has clearer feature distinction than simply separating the human body and the background. The features learned by the model are related to the parts of the human body and are more suitable for the complex task of human attribute recognition; 3) The original feature map is separated into upper and lower feature maps. According to the spatial correlation characteristics of the attribute recognition task, feature maps in different regions are used to perform attribute recognition for different tasks, and the results are more accurate. When the upper body human body map is input, the confidence of the lower body attribute recognition result is very low, and the wrong results can be filtered out.

[0128] From the above, it can be seen that in an embodiment of the present invention, the human body part mask map is used to supervise and enhance the model feature expression ability, and the model structure is designed according to the difficulty of the attribute recognition task. The focus is on implementing the pedestrian attribute recognition function on low-resource edge devices. Therefore, there are very high requirements for the accuracy, resource consumption and real-time performance of the model. The feature map extracted by the backbone network is split up and down without additional overhead. At this time, the features already contain global information, which is simple and effective.

[0129] That is, the technical solution provided by the embodiment of the present invention comprehensively considers various factors and solves the problem of being able to accurately identify human attributes on edge devices with low computing power. When only an upper body image is input (this type of occlusion is common in security monitoring scenarios), the confidence level of the lower body attribute results output by the model is extremely low or not output.

[0130] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0132] According to an embodiment of the present invention, a human attribute recognition device for implementing the human attribute recognition method is also provided. Fig.11 is a schematic diagram of a human attribute recognition device according to an embodiment of the present invention. Fig.11 As shown, the device includes: an extraction module 1101, a generation module 1103, a fusion module 1105 and a recognition module 1107. The device for recognizing human attributes is described below.

[0133] An extraction module 1101 is used to extract features of a human body image and obtain an initial human body feature map; a generation module 1103 is connected to the extraction module 1101 and is used to generate category confidence maps of various parts of the human body based on the initial human body feature map, wherein the category confidence map is used to represent the probability that the features in the initial human body feature map belong to the categories of human body parts; a fusion module 1105 is connected to the generation module 1103 and is used to fuse the initial human body feature map and the category confidence map to obtain a target human body feature map; an identification module 1107 is connected to the fusion module 1105 and is used to identify the attributes of a target human body in the human image based on the target human body feature map to obtain an attribute recognition result of the target human body, wherein the attribute recognition result includes identification information of the target human body.

[0134] It should be noted here that the above-mentioned extraction module 1101, generation module 1103, fusion module 1105 and recognition module 1107 correspond to steps S202 to S208 in the above-mentioned embodiments, and the four modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0135] As can be seen from the above, in the scheme recorded in the above-mentioned embodiment of the present invention, the initial human feature map corresponding to the human body image and the category confidence map representing the probability that the features in the initial human feature map belong to the human body part category are fused to obtain the target human feature map, and the target human body in the human body image of the target human feature map is attributed to obtain the attribute recognition result of the target human body. Since the features of the human body part category are combined with the features in the initial human feature map, when the attributes of the target human body are recognized, the more abundant part and area features of the target human body are considered, which effectively improves the accuracy of attribute recognition, and can stably and accurately recognize human attributes, especially for the input of missing images, it avoids the model from erroneously outputting the attribute results of the missing part, and can effectively filter out the erroneous output results, thereby improving the reliability and accuracy of recognition, thereby solving the technical problem of low accuracy when using a lightweight image classification network to reduce computing power in related technologies to recognize human attributes.

[0136] In an optional embodiment, the fusion module 1105 is further used to fuse the depth feature of the three-dimensional features of depth, height and width in the initial human feature map with the category feature in the category confidence map to obtain a target human feature map.

[0137] In an optional embodiment, the recognition module includes: a segmentation unit, a recognition unit and a processing unit, wherein the segmentation unit is used to segment the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image; the recognition unit is connected to the above-mentioned segmentation unit, and is used to perform attribute recognition on the upper body feature map and the lower body feature map respectively to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map; the processing unit is connected to the above-mentioned recognition unit, and is used to obtain an attribute recognition result of the target human body based on the first attribute recognition result and the second attribute recognition result.

[0138] In an optional embodiment, the segmentation unit is used to segment the target human feature map according to the height of the three dimensions of depth, height and width in the target human feature map to obtain the upper body feature map and the lower body feature map of the target human body in the human body image.

[0139] In an optional embodiment, the above-mentioned recognition unit is used to input the upper body feature map into the attribute classifier corresponding to the upper body to obtain a first attribute recognition result of the upper body feature map; and input the lower body feature map into the attribute classifier corresponding to the lower body to obtain a second attribute recognition result of the lower body feature map.

[0140] In an optional embodiment, the above-mentioned processing unit is used to determine the first attribute recognition confidence corresponding to the first attribute recognition result, and determine the second attribute recognition confidence corresponding to the second attribute recognition result; and obtain the attribute recognition result of the target human body based on the comparison results of the first attribute recognition confidence and the second attribute recognition confidence with the corresponding confidence thresholds.

[0141] In an optional embodiment, the above-mentioned device is also used to identify human body images based on a human body attribute recognition model to obtain attribute recognition results of the target human body, wherein the human body attribute recognition model is trained using multiple sample images. When training the model, the three-dimensional features of the sample images and the part signal features of each part of the human body are extracted, and the part signal features are used as supervisory signals to optimize the model training process based on the three-dimensional features to obtain the human body attribute recognition model.

[0142] According to another aspect of an embodiment of the present invention, a computer program product is provided, including computer instructions, and when the computer instructions are executed by a processor, any one of the above-mentioned methods for recognizing human attributes is executed.

[0143] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, including a memory storing an executable program; and a processor for running the program, wherein when the program is run, any one of the above-mentioned methods for recognizing human attributes is executed.

[0144] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for identifying human attributes.

[0145] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the communication devices in a communication device group.

[0146] Optionally, in this embodiment, a computer-readable storage medium is configured to store program codes for executing the following steps: extracting features of a human body image to obtain an initial human body feature map; generating category confidence maps for various parts of the human body based on the initial human body feature map, wherein the category confidence map is used to represent the probability that the features in the initial human body feature map belong to the categories of human body parts respectively; fusing the initial human body feature map and the category confidence map to obtain a target human body feature map; based on the target human body feature map, identifying the attributes of a target human body in the human body image to obtain an attribute recognition result of the target human body, wherein the attribute recognition result includes identification information of the target human body.

[0147] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: fusing the initial human feature map and the category confidence map to obtain a target human feature map, including: fusing the depth features among the three dimensional features of depth, height and width in the initial human feature map with the category features in the category confidence map to obtain the target human feature map.

[0148] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: based on the target human feature map, identifying the attributes of the target human body in the human body image to obtain an attribute recognition result of the target human body, including: segmenting the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image; performing attribute recognition on the upper body feature map and the lower body feature map respectively to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map; obtaining an attribute recognition result of the target human body based on the first attribute recognition result and the second attribute recognition result.

[0149] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: segmenting the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image, including: segmenting the target human body feature map according to the height of the three dimensions of depth, height and width in the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image.

[0150] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: performing attribute identification on the upper body feature map and the lower body feature map respectively to obtain a first attribute identification result of the upper body feature map and a second attribute identification result of the lower body feature map, including: inputting the upper body feature map into an attribute classifier corresponding to the upper body to obtain a first attribute identification result of the upper body feature map; inputting the lower body feature map into an attribute classifier corresponding to the lower body to obtain a second attribute identification result of the lower body feature map.

[0151] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining an attribute recognition result of the target human body based on the first attribute recognition result and the second attribute recognition result, including: determining a first attribute recognition confidence corresponding to the first attribute recognition result, and determining a second attribute recognition confidence corresponding to the second attribute recognition result; obtaining the attribute recognition result of the target human body based on the comparison results of the first attribute recognition confidence and the second attribute recognition confidence with the corresponding confidence thresholds.

[0152] Optionally, in this embodiment, a computer-readable storage medium is configured to store program codes for executing the following steps: the recognition method recognizes a human body image based on a human body attribute recognition model to obtain an attribute recognition result of the target human body, wherein the human body attribute recognition model is trained using multiple sample images, and when performing model training, three-dimensional features of the sample image and part signal features of various parts of the human body are extracted, and the part signal features are used as supervisory signals to optimize the model training process based on the three-dimensional features to obtain a human body attribute recognition model.

[0153] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0154] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0157] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0159] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying human attributes, characterized in that: include: Extract the features of the human body image to obtain an initial human body feature map; Generating a category confidence map of each human body part according to the initial human body feature map, wherein the category confidence map is used to represent the probability that the features in the initial human body feature map belong to the category of human body parts respectively; Fusing the initial human feature map and the category confidence map to obtain a target human feature map; Based on the target human feature map, the attributes of the target human body in the human body image are identified to obtain an attribute recognition result of the target human body, wherein the attribute recognition result includes identification information of the target human body.

2. The method for identifying human attributes according to claim 1, characterized in that: The initial human feature map and the category confidence map are fused to obtain a target human feature map, including: The depth feature of the three-dimensional features of depth, height and width in the initial human feature map is fused with the category feature in the category confidence map to obtain the target human feature map.

3. The method for identifying human attributes according to claim 1, characterized in that: The identifying the attributes of the target human body in the human body image based on the target human body feature map to obtain the attribute identification result of the target human body includes: Segmenting the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image; Respectively performing attribute recognition on the upper body feature map and the lower body feature map to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map; An attribute recognition result of the target human body is obtained according to the first attribute recognition result and the second attribute recognition result.

4. The method for identifying human attributes according to claim 3, characterized in that: The step of segmenting the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image includes: The target human body feature map is segmented according to the height of the three dimensions of depth, height and width in the target human body feature map to obtain an upper body feature map and a lower body feature map of the target human body in the human body image.

5. The method for identifying human attributes according to claim 3, characterized in that: The performing attribute recognition on the upper body feature map and the lower body feature map respectively to obtain a first attribute recognition result of the upper body feature map and a second attribute recognition result of the lower body feature map includes: Inputting the upper body feature map into an attribute classifier corresponding to the upper body to obtain the first attribute recognition result of the upper body feature map; The lower body feature map is input into an attribute classifier corresponding to the lower body to obtain the second attribute recognition result of the lower body feature map.

6. The method for identifying human attributes according to claim 3, characterized in that: The step of obtaining the attribute recognition result of the target human body according to the first attribute recognition result and the second attribute recognition result includes: Determining a first attribute recognition confidence level corresponding to the first attribute recognition result, and determining a second attribute recognition confidence level corresponding to the second attribute recognition result; The attribute recognition result of the target human body is obtained according to the comparison results of the first attribute recognition confidence and the second attribute recognition confidence with the corresponding confidence thresholds respectively.

7. The method for recognizing human attributes according to any one of claims 1 to 6, characterized in that: The recognition method recognizes the human body image based on a human body attribute recognition model to obtain the attribute recognition result of the target human body, wherein the human body attribute recognition model is trained using multiple sample images. When performing model training, the three-dimensional features of the sample images and the part signal features of each part of the human body are extracted, and the part signal features are used as supervisory signals to optimize the model training process based on the three-dimensional features to obtain the human body attribute recognition model.

8. A device for identifying human attributes, characterized in that: include: An extraction module is used to extract features of a human body image and obtain an initial human body feature map; A generating module, used for generating a category confidence map of each human body part according to the initial human body feature map, wherein the category confidence map is used for indicating the probability that the features in the initial human body feature map respectively belong to the category of human body parts; A fusion module, used for fusing the initial human feature map and the category confidence map to obtain a target human feature map; The recognition module is used to recognize the attributes of the target human body in the human body image based on the target human body feature map to obtain the attribute recognition result of the target human body, wherein the attribute recognition result includes the identification information of the target human body.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the human attribute recognition method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program, when running, executes the method for recognizing human attributes as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images