Pedestrian attribute identification method and device, electronic equipment and computer program product

Abstract

The invention provides a pedestrian attribute recognition method and device, electronic equipment and a computer program product, and belongs to the field of computer vision. The method comprises the following steps: respectively inputting pedestrian attribute features into a position consistency classifier, a plurality of color classifiers and a color attribution classifier of a pedestrian attribute identification model to obtain a position consistency identification result, a plurality of color identification results and a color attribution identification result; determining color attribute information in the to-be-recognized image according to the position consistency recognition result, the multiple color recognition results and the color attribution recognition result; inputting the pedestrian attribute features into a non-color attribute recognition branch of a pedestrian attribute recognition model to obtain non-color attribute information in the to-be-recognized image; and determining pedestrian attribute information according to the color attribute information and the non-color attribute information. According to the method, the use of classifiers is reduced, so that the number of dichotomy is reduced, and the problem of resource waste caused by constructing a large number of dichotomy for attribute recognition in a traditional method is solved.

Description

Technical Field

[0001] The present application belongs to the field of computer vision technology, and in particular, relates to a pedestrian attribute recognition method, device, electronic equipment and computer program product. Background Art

[0002] With the updating and iteration of science and technology, the functions and performance of hardware image acquisition equipment are becoming more and more perfect, and the cost of equipment is also decreasing. As a result, image acquisition equipment is becoming more and more popular in daily life. For example, monitoring equipment in scenes such as businesses and communities can be seen everywhere. The large amount of image data generated provides a rich data source for the development of pedestrian attribute extraction technology. Among them, the extraction of pedestrian attributes in images collected by image acquisition equipment is an important task in the field of computer vision, which aims to extract various attribute information from pedestrian images, such as color, gender, age, clothing and other attribute information.

[0003] When traditional methods are used to identify and extract pedestrian attributes in images, a large number of binary classifications need to be constructed to identify attribute information such as color, age, gender, glasses, hats, etc., which leads to a waste of resources. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and computer program product for pedestrian attribute recognition, which can solve the technical problem of resource waste caused by constructing a large number of binary classifications for attribute recognition in traditional methods.

[0005] The present application provides a method for pedestrian attribute recognition, comprising:

[0006] Acquire an image to be identified, wherein the image to be identified includes a pedestrian;

[0007] Inputting the image to be recognized into a feature extraction network of a pedestrian attribute recognition model to obtain pedestrian attribute features;

[0008] Inputting the pedestrian attribute features into the position consistency classifier, multiple color classifiers and color attribution classifier of the pedestrian attribute recognition model respectively, to obtain a position consistency recognition result, multiple color recognition results and color attribution recognition result;

[0009] Determining color attribute information in the image to be identified according to the position consistency identification result, the multiple color identification results, and the color attribution identification result;

[0010] Inputting the pedestrian attribute feature into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain non-color attribute information in the image to be recognized;

[0011] Pedestrian attribute information is determined according to the color attribute information and the non-color attribute information.

[0012] In one embodiment, the step of determining the color attribute information in the image to be identified based on the position consistency identification result, the multiple color identification results and the color attribution identification result includes:

[0013] If the position consistency recognition result is inconsistent, the top color and the bottom color in the color attribute information are determined according to the color attribution recognition result and the multiple color recognition results.

[0014] In one embodiment, the step of determining the color attribute information in the image to be identified based on the position consistency identification result, the multiple color identification results and the color attribution identification result includes:

[0015] If the position consistency recognition result is consistent, the color attribute information is determined according to the multiple color recognition results.

[0016] In one embodiment, the step of inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized includes:

[0017] Inputting the pedestrian attribute features into the first age classifier, the second age classifier, the age certainty classifier and the age position classifier of the non-color attribute recognition branch respectively, to obtain the first age recognition result, the second age recognition result, the age certainty recognition result and the age position recognition result;

[0018] If the age certainty recognition result is certain, then the age attribute information is determined from the first age recognition result and the second age recognition result according to the age position recognition result.

[0019] In one embodiment, the step of inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized includes:

[0020] Inputting the pedestrian attribute features into the first classifier, the second classifier, and the first deterministic classifier of the non-color attribute recognition branch respectively to obtain a first recognition result, a second recognition result, and a first deterministic recognition result;

[0021] If the first deterministic recognition result is positive, determining first attribute information according to the first recognition result; wherein the first attribute information is gender attribute information or glasses attribute information or top type attribute information;

[0022] Alternatively, if the first deterministic recognition result is a top, the second attribute information is determined based on the second recognition result; wherein the second attribute information is reflective clothing attribute information.

[0023] In one embodiment, the step of inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized includes:

[0024] Inputting the pedestrian attribute features into the third classifier, the fourth classifier and the second deterministic classifier of the non-color attribute recognition branch respectively to obtain a third recognition result, a fourth recognition result and a second deterministic recognition result;

[0025] If the second deterministic recognition result is certain, determining third attribute information according to the third recognition result and the fourth recognition result;

[0026] The third attribute information is hat attribute information or bottoms attribute information.

[0027] In one embodiment, before the step of acquiring the image to be identified, the method further includes:

[0028] Get the pedestrian attribute dataset;

[0029] Inputting the pedestrian attribute data set into a feature extraction network of an initial pedestrian attribute recognition model to obtain initial pedestrian attribute features;

[0030] Inputting the initial pedestrian attribute features into the color attribute recognition branch of the initial pedestrian attribute recognition model to obtain a probability distribution of color attributes; wherein the color attribute recognition branch includes the position consistency classifier, the multiple color classifiers and the color attribution classifier;

[0031] Inputting the initial pedestrian attribute features into the non-color attribute recognition branch of the initial pedestrian attribute recognition model to obtain a probability distribution of the non-color attribute;

[0032] Constructing a first loss function according to the probability distribution of the color attribute and the true color label in the pedestrian attribute data set;

[0033] Constructing a second loss function according to the probability distribution of the non-color attribute and the non-color true label in the pedestrian attribute data set;

[0034] According to the first loss function and the second loss function, the initial pedestrian attribute recognition model is trained, evaluated, and optimized to obtain the pedestrian attribute recognition model.

[0035] The present application provides a pedestrian attribute recognition device, comprising:

[0036] An image acquisition module, used to acquire an image to be identified, wherein the image to be identified includes a pedestrian;

[0037] A feature extraction module is used to extract features from the image to be identified to obtain pedestrian attribute features;

[0038] A color attribute recognition module is used to classify the pedestrian attribute features into clothing consistency, multiple color classifications, and clothing position classifications to obtain a position consistency recognition result, multiple color recognition results, and a color attribution recognition result;

[0039] A color attribute determination module, configured to determine color attribute information in the image to be identified based on the position consistency identification result, the multiple color identification results, and the color attribution identification result;

[0040] A non-color attribute recognition module, used to perform non-color attribute recognition on the pedestrian attribute features to obtain non-color attribute information in the image to be recognized;

[0041] The pedestrian attribute determination module is used to determine the pedestrian attribute information according to the color attribute information and the non-color attribute information.

[0042] The present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements a method as described in any one of the above embodiments.

[0043] The present application provides a computer program product, including a computer program. When the computer program is executed, the method described in any one of the above embodiments is executed.

[0044] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0045] The position consistency classifier of the pedestrian attribute recognition model can be used to identify whether the labels of two clothes are consistent. The colors of two clothes labels can be identified through multiple color classifiers. The color attribution classifier can be used to identify the attribution position of the color. Furthermore, the clothes labels are distinguished for consistency through the position consistency classifier and the color attribution classifier, and the colors of the clothes labels identified by the multiple color classifiers are distinguished for attribution based on the position consistency recognition results. The color recognition process of the top and the bottom color recognition process share the same color classifier for recognition, which reduces the use of color classifiers. Furthermore, based on the position consistency recognition results, the multiple color recognition results and the color attribution recognition results, the color attribute information in the image to be recognized can be determined.

[0046] Therefore, through the position consistency classifier, multiple color classifiers and color attribution classifier of the pedestrian attribute recognition model in the pedestrian attribute recognition method provided by the present application, when the color recognition of the pedestrian's top and bottom is performed, the common color classifier can recognize the colors of the two clothing labels, and there is no need to construct 12 classifiers for the color of the top and 12 classifiers for the color of the bottom. Compared with at least 24 classifiers constructed when the traditional method recognizes the color of the pedestrian's clothes in the image, the pedestrian attribute recognition method provided by the present application only requires a small number of color classifiers to realize the recognition of the color of the pedestrian's clothes. Thus, the number of color classifiers used is reduced, so that the number of classifiers of the overall pedestrian attribute recognition model is also reduced, and the required computing resources and storage resources are further reduced, thereby improving the reasoning speed and recognition efficiency. Therefore, through the pedestrian attribute recognition method provided by the present application, the use of classifiers can be reduced, so that the number of binary classifications is reduced, thereby avoiding resource waste, and solving the problem of resource waste caused by constructing a large number of binary classifications for attribute recognition in the traditional method. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 It is a schematic diagram of the step flow of the pedestrian attribute recognition method provided in this application.

[0049] Figure 2 It is a structural schematic diagram of the pedestrian attribute recognition device provided in this application.

[0050] Figure 3 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0051] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0052] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0053] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0054] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0055] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0057] When traditional methods are used to identify and extract pedestrian attributes in images, a large number of binary classification branches need to be constructed to realize the recognition of pedestrian attribute information such as top color, bottom color, age, gender, glasses, hats, tops, bottoms, reflective clothing, backpacks, etc. Among them, at least 12 categories need to be constructed for the top color attribute, including black, white, gray, red, yellow, blue, green, purple, pink, brown, color, and unknown. At least 12 categories also need to be constructed for the bottom color attribute, including black, white, gray, red, yellow, blue, green, purple, pink, brown, color, and unknown. It can be seen that at least 24 categories are required for the color attribute in the pedestrian attribute information, resulting in a waste of computing resources and storage resources.

[0058] At least 5 categories need to be constructed for age attributes. At least 3 categories need to be constructed for gender attributes. At least 3 categories need to be constructed for glasses attributes. At least 4 categories need to be constructed for hat attributes. At least 3 categories need to be constructed for top category attributes. At least 4 categories need to be constructed for bottom category attributes. At least 3 categories need to be constructed for reflective clothing attributes. At least 3 categories need to be constructed for backpack attributes. Therefore, when using traditional methods to identify and extract pedestrian attributes in images, at least 52 categories need to be constructed, resulting in a waste of computing and storage resources. Moreover, too many categories constructed using traditional methods will lead to a decrease in recognition accuracy, and it is difficult to balance the losses of each branch. In response to the above problems, the present application provides a method for pedestrian attribute recognition, and the specific steps can be found in the descriptions of the following embodiments.

[0059] See also Figure 1 , the present application provides a pedestrian attribute recognition method, comprising:

[0060] Step S10, obtaining an image to be identified, wherein the image to be identified includes a pedestrian;

[0061] Step S20, inputting the image to be recognized into the feature extraction network of the pedestrian attribute recognition model to obtain pedestrian attribute features;

[0062] Step S30, inputting the pedestrian attribute features into the position consistency classifier, multiple color classifiers and color attribution classifier of the pedestrian attribute recognition model respectively, to obtain a position consistency recognition result, multiple color recognition results and color attribution recognition result;

[0063] Step S40, determining color attribute information in the image to be identified based on the position consistency identification result, the multiple color identification results and the color attribution identification result;

[0064] Step S50, inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized;

[0065] Step S60: determining pedestrian attribute information according to the color attribute information and the non-color attribute information.

[0066] In this embodiment, in step S10, the image to be identified contains pedestrian attribute information, which can be acquired through image acquisition equipment, and specifically can be images of scenes such as shopping malls, communities, parks, etc. in daily life. In step S20, the feature extraction network of the pedestrian attribute recognition model can be a convolutional neural network, a recurrent neural network, a long short-term memory network, or a multimodal feature fusion network. Through the feature extraction network, features with pedestrian attributes can be extracted from the image to be identified to obtain pedestrian attribute features. In one embodiment, the pedestrian attribute recognition model can be a model architecture such as a deep residual network (Deep Residual Network, ResNet), a mobile network (MobileNet), or an efficient network (EfficientNet).

[0067] In step S30, the position consistency classifier is used to identify whether the position of the pedestrian's clothes is consistent, which can also be understood as whether the position of the pedestrian's clothes is the same label. The position of the pedestrian's clothes can be understood as the pedestrian's top and bottom. Multiple color classifiers are used to identify the color of the clothes, which can be at least 12 classifiers such as black, white, gray, red, yellow, blue, green, purple, pink, brown, color, and unknown. Multiple color classifiers can identify the color of the top and the color of the bottom. The color attribution classifier is used to identify the attribution position of the color, which can also be understood as identifying which color is the color of the top and which color is the color of the bottom.

[0068] The position consistency recognition result is the prediction result obtained after passing the position consistency classifier. The multiple color recognition result is the prediction result obtained after passing the multiple color classifiers. The color attribution recognition result is the prediction result obtained after passing the color attribution classifier.

[0069] In step S40, based on the position consistency recognition result, it can be known whether the labels of the two clothes in the prediction result are the same label, for example, whether they are both tops, or both bottoms, or one is a top and the other is a bottom. Based on multiple color recognition results, the colors of the two clothing labels can be known, for example: one is blue and the other is pink; or one is blue and the other is also blue, etc. Based on the color attribution recognition result, the attribution positions of the two colors can be known, which can also be understood as which color corresponds to the top and which color corresponds to the bottom.

[0070] The position consistency classifier of the pedestrian attribute recognition model can be used to identify whether the labels of two clothes are consistent. The colors of two clothing labels can be identified through multiple color classifiers. The color attribution classifier can be used to identify the attribution position of the color. Furthermore, the colors of clothing labels identified by multiple color classifiers are distinguished through the position consistency classifier and the color attribution classifier, and the color recognition process of the top and the bottom share the same color classifier for recognition, reducing the use of color classifiers.

[0071] In step S50, non-color attribute recognition includes age, gender, glasses, hats, tops, bottoms, reflective clothing, backpacks and other attributes not related to color. Through the non-color attribute recognition branch, the recognition of non-color attributes in the image to be recognized is realized. Furthermore, based on the color attribute information and the non-color attribute information, all attribute information of the pedestrian can be obtained, and the process of pedestrian attribute recognition is completed.

[0072] Through the position consistency classifier, multiple color classifiers and color attribution classifier of the pedestrian attribute recognition model in the pedestrian attribute recognition method provided by the present application, when the color recognition of the upper and lower clothes of the pedestrian is performed, multiple color classifiers can be shared to identify the colors of the two clothing labels, without the need to construct 12 classifiers for the color of the upper clothes and 12 classifiers for the color of the lower clothes. Compared with at least 24 classifiers constructed when the traditional method recognizes the color of the pedestrian's clothes in the image, the pedestrian attribute recognition method provided by the present application only requires a small number of color classifiers to realize the recognition of the color of the pedestrian's clothes. Thus, the number of color classifiers used is reduced, so that the number of classifiers of the overall pedestrian attribute recognition model is also reduced, and the required computing resources and storage resources are further reduced, thereby improving the reasoning speed and recognition efficiency. Therefore, through the pedestrian attribute recognition method provided by the present application, the use of classifiers can be reduced, so that the number of binary classifications is reduced, thereby avoiding resource waste, and solving the problem of resource waste caused by constructing a large number of binary classifications for attribute recognition in the traditional method.

[0073] In one embodiment, the present application provides a pedestrian attribute recognition method, which can be applied to scenarios such as pedestrian monitoring and intelligent security to provide accurate pedestrian attribute recognition information.

[0074] In one embodiment, step S40, the step of determining color attribute information in the image to be identified based on the position consistency identification result, the multiple color identification results, and the color attribution identification result, includes:

[0075] Step S410: If the position consistency recognition result is inconsistent, the top color and the bottom color in the color attribute information are determined according to the color attribution recognition result and the multiple color recognition results.

[0076] In this embodiment, the position consistency recognition result is distinguished by 0 and 1. In one embodiment, if the position consistency recognition result is consistent, it is 0. If the position consistency recognition result is inconsistent, it is 1. If the position consistency recognition result is inconsistent, it indicates that there are colors corresponding to two clothing tags. There are two predicted values ​​of 1 in multiple color recognition results, and it is necessary to further distinguish which clothing tag each predicted value belongs to. The color recognition results are distinguished by 0 and 1. In one embodiment, if the color recognition result is 1, the corresponding color is the predicted color. If the color recognition result is 0, the corresponding color is not the predicted color.

[0077] The color attribution recognition result is distinguished by 0 and 1. In one embodiment, if the color attribution recognition result is 0, it indicates that the color corresponding to the first prediction value of 1 in the multiple color recognition results belongs to the color of the top, and the color corresponding to the second prediction value of 1 in the multiple color recognition results belongs to the color of the bottom. In one embodiment, if the color attribution recognition result is 1, the color corresponding to the first prediction value of 1 in the multiple color recognition results belongs to the color of the bottom, and the color corresponding to the second prediction value of 1 in the multiple color recognition results belongs to the color of the top.

[0078] If the position consistency recognition result is inconsistent, the color attribution recognition result obtained by the color attribution classifier in step S410 of this embodiment and the multiple color recognition results obtained by multiple color classifiers can make it possible to share multiple color classifiers to identify the colors of the two clothing labels when performing color recognition on the pedestrian's top and bottom, thereby reducing the number of classifiers and avoiding waste of resources.

[0079] In one embodiment, step S40, the step of determining color attribute information in the image to be identified based on the position consistency identification result, the multiple color identification results, and the color attribution identification result, includes:

[0080] Step S420: If the position consistency recognition result is consistent, the color attribute information is determined according to the multiple color recognition results.

[0081] In this embodiment, if the position consistency recognition result is consistent, it indicates that the two labels correspond to both tops or both bottoms, and there is a prediction value with a color recognition result of 1. Furthermore, if the position consistency recognition result is consistent, the corresponding color with a prediction value of 1 is selected from the multiple color recognition results, that is, the color of the top or the color of the bottom.

[0082] In one embodiment, step S50, inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized, includes:

[0083] Step S510, inputting pedestrian attribute features into the first age classifier, the second age classifier, the age certainty classifier and the age position classifier of the non-color attribute recognition branch respectively, and obtaining the first age recognition result, the second age recognition result, the age certainty recognition result and the age position recognition result;

[0084] Step S520: If the age certainty recognition result is certain, then determine the age attribute information from the first age recognition result and the second age recognition result according to the age position recognition result.

[0085] In this embodiment, in step S510, the first age classifier, the second age classifier, the age certainty classifier, and the age position classifier form four categories regarding age attributes. The first age classifier is used to identify whether the pedestrian is in the first age interval or the second age interval. The second age classifier is used to identify whether the pedestrian is in the third age interval or the fourth age interval. The age certainty classifier is used to identify whether the age is certain, and is divided into two categories: uncertain age and certain age. The age position classifier is used to identify which classifier's result is valid between the first age classifier and the second age classifier.

[0086] In one embodiment, the first age recognition result is 0, indicating that the person is less than 16 years old. The first age recognition result is 1, indicating that the person is greater than 16 years old and less than 45 years old. The second age recognition result is 0, indicating that the person is greater than 45 years old and less than 60 years old. The second age recognition result is 1, indicating that the person is greater than 60 years old. The age certainty recognition result is 0, indicating that the age is uncertain. The age certainty recognition result is 1, indicating that the age is certain. The age position recognition result is 0, indicating that the first age recognition result is valid. The age position recognition result is 1, indicating that the second age recognition result is valid.

[0087] In step S520, if the age certainty recognition result is certain, it indicates that the age attribute recognition result is highly reliable. If the age certainty recognition result is certain, the source location of the data is selected according to the age location recognition result, and one of the first age recognition result and the second age recognition result is selected as the age attribute information.

[0088] Through step S510 and step S520 in this embodiment, the first age classifier, the second age classifier, the age certainty classifier and the age position classifier are used to determine the age attribute information of the pedestrian. Compared with the traditional method that requires the construction of at least 5 classifiers to determine the age attribute information, through step S510 and step S520 in this embodiment, 4 classifiers can be used to determine the age attribute information, which reduces the use of classifiers, so that the number of classifiers in the overall pedestrian attribute recognition model is also reduced, and the required computing resources and storage resources are further reduced, thereby improving the reasoning speed and recognition efficiency.

[0089] In one embodiment, step S50, inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized, includes:

[0090] Step S530, inputting the pedestrian attribute features into the first classifier, the second classifier, and the first deterministic classifier of the non-color attribute recognition branch respectively, to obtain a first recognition result, a second recognition result, and a first deterministic recognition result;

[0091] Step S540: if the first deterministic recognition result is positive, determine first attribute information according to the first recognition result; wherein the first attribute information is gender attribute information or glasses attribute information or top type attribute information;

[0092] Alternatively, in step S550, if the first deterministic recognition result is that the garment is a top, the second attribute information is determined according to the second recognition result; wherein the second attribute information is the attribute information of the reflective garment.

[0093] In this embodiment, in step S530, the first classifier is used to identify gender classification, or glasses classification, or top classification, etc. The second classifier is used to identify whether reflective clothing is worn. The first deterministic classifier is used to identify whether the gender is determined, or whether the glasses are determined, or whether the top is determined. In one embodiment, the first recognition result is male or female, which can be represented by 0 and 1, 0 for male and 1 for female.

[0094] In one embodiment, the first recognition result is not wearing glasses or wearing glasses, which can be distinguished by 0 and 1, 0 for not wearing glasses and 1 for wearing glasses. In one embodiment, the first recognition result is long sleeves or short sleeves, which can be distinguished by 0 and 1, 0 for long sleeves and 1 for short sleeves. In one embodiment, the second recognition result is not wearing reflective clothing or wearing reflective clothing.

[0095] In one embodiment, the first deterministic recognition result is whether the gender is determined. In one embodiment, the first deterministic recognition result is whether the glasses are determined. In one embodiment, the first deterministic recognition result is whether the top is determined. Reflective clothing is also a type of top, and whether the top is determined also determines whether the reflective clothing is determined. If the top is determined, the reflective clothing is determined. If the top is uncertain, the reflective clothing is also uncertain.

[0096] In step S540, the first deterministic recognition result is determined, indicating that the recognition result is highly reliable. When the first deterministic recognition result is determined, gender attribute information, glasses attribute information, or top attribute information, etc. can be determined based on the first recognition result.

[0097] In step S550, if the first deterministic recognition result is that the top is confirmed, it indicates that the reflective clothing is also confirmed, and then according to the second recognition result, it is determined whether the reflective clothing is worn, thereby determining the second attribute information, that is, the reflective clothing attribute information.

[0098] Through step S530, step S540 and step S550 in this embodiment, the first classifier, the second classifier and the first deterministic classifier are used to determine the gender attribute information or the glasses attribute information or the top type attribute information or the reflective clothing attribute information of the pedestrian. Compared with the traditional method, which requires at least 3 classifications to realize gender attribute information, at least 3 classifications to realize glasses attribute information, at least 3 classifications to realize top type attribute information, and at least 3 classifications of reflective clothing attribute information, at least 12 classifications in total, through step S530, step S540 and step S550 in this embodiment, 7 classifiers are used to realize the determination of gender attribute information, glasses attribute information, top type attribute information and reflective clothing attribute information. Therefore, through step S530, step S540 and step S550 in this embodiment, the use of classifiers is reduced, so that the number of classifiers of the overall pedestrian attribute recognition model is also reduced, and the required computing resources and storage resources are further reduced, thereby improving the reasoning speed and recognition efficiency.

[0099] In one embodiment, step S50, inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized, includes:

[0100] Step S560, inputting the pedestrian attribute features into the third classifier, the fourth classifier and the second deterministic classifier of the non-color attribute recognition branch respectively, to obtain a third recognition result, a fourth recognition result and a second deterministic recognition result;

[0101] Step S570, if the second deterministic recognition result is certain, determine the third attribute information according to the third recognition result and the fourth recognition result;

[0102] The third attribute information is hat attribute information or bottoms attribute information.

[0103] In this embodiment, in step S560, the third classifier is used to identify whether a hat is worn, or the classification of lower garments and pants. The fourth classifier is used to identify the classification of hats, or whether a skirt is worn. The second deterministic classifier identifies whether the hat is certain, or whether the lower garment is certain. In one embodiment, the third recognition result is no hat or hat, which can be distinguished by 0 and 1, 0 for no hat, and 1 for wearing a hat. In one embodiment, the third recognition result is trousers or shorts, which can be distinguished by 0 and 1, 0 for trousers, and 1 for shorts.

[0104] In one embodiment, the fourth recognition result is wearing an ordinary hat or wearing a helmet, which can be distinguished by 0 and 1, 0 for wearing an ordinary hat, and 1 for wearing a helmet. In one embodiment, the fourth recognition result is not wearing a skirt or wearing a skirt, which can be distinguished by 0 and 1, 0 for not wearing a skirt, and 1 for wearing a skirt. In one embodiment, the second deterministic recognition result is hat confirmed or hat uncertain. In one embodiment, the second deterministic recognition result is lower garment confirmed or lower garment uncertain.

[0105] In step S570, if the second deterministic recognition result is determined, it indicates that the recognition result is highly reliable. In the case where the second deterministic recognition result is determined, the hat attribute information or the bottoms attribute information can be determined based on the third recognition result and the fourth recognition result.

[0106] Through step S560 and step S570 in this embodiment, the third classifier, the fourth classifier and the second deterministic classifier are used to determine the hat attribute information or the lower clothing attribute information of the pedestrian. Compared with the traditional method that requires the construction of at least 4 classifications to realize the hat attribute information and the construction of at least 4 classifications to realize the lower clothing attribute information, at least 8 classifications in total, through step S560 and step S570 in this embodiment, 6 classifiers can be used to determine the hat attribute information and the lower clothing attribute information. Therefore, through step S560 and step S570 in this embodiment, the use of classifiers is reduced, so that the number of classifiers of the overall pedestrian attribute recognition model is also reduced, and the required computing resources and storage resources are further reduced, and the reasoning speed and recognition efficiency are improved.

[0107] In one embodiment, step S50, inputting the pedestrian attribute features into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized, includes:

[0108] Step S580, inputting the pedestrian attribute features into the backpack classifier of the non-color attribute recognition branch respectively, obtaining the backpack recognition result, and determining the backpack attribute information according to the backpack recognition result.

[0109] In this embodiment, a backpack classifier is used to identify whether a backpack is worn. The backpack recognition result is whether a backpack is worn or not, which can be distinguished by 0 and 1, 0 indicating no backpack and 1 indicating wearing a backpack. Through step S580 in this embodiment, a backpack classifier is used to determine the backpack attribute information of the pedestrian. Compared with the traditional method that requires the construction of at least 3 classifications to realize backpack attribute information, through step S580 in this embodiment, the backpack attribute information can be determined by using 1 classifier. Therefore, through step S580 in this embodiment, the use of classifiers is reduced, so that the overall number of classifiers in the pedestrian attribute recognition model is also reduced, which further reduces the required computing resources and storage resources, and improves the reasoning speed and recognition efficiency.

[0110] In one embodiment, through the various classifiers of the pedestrian attribute recognition model in the pedestrian attribute recognition method provided by the present application, 32 classifications can be achieved. Compared with the traditional method, the number of classifications is reduced, so that the overall number of classifications of the pedestrian attribute recognition model is also reduced, and the required computing resources and storage resources are further reduced, thereby improving the reasoning speed and recognition efficiency.

[0111] In one embodiment, based on the recognition results of each classifier of the pedestrian attribute recognition model, the attribute information of the pedestrian is determined as 16 to 45 years old, male, glasses confirmed, wearing glasses, no hat, top confirmed, short sleeves, bottoms confirmed, shorts, no reflective clothing, no backpack, top and bottom colors inconsistent, top is red, bottoms are blue.

[0112] In one embodiment, before step S10, the step of obtaining the image to be identified, the method further includes:

[0113] Step S01, obtaining a pedestrian attribute data set;

[0114] Step S02, inputting the pedestrian attribute data set into the feature extraction network of the initial pedestrian attribute recognition model to obtain initial pedestrian attribute features;

[0115] Step S03, inputting the initial pedestrian attribute features into the color attribute recognition branch of the initial pedestrian attribute recognition model to obtain the probability distribution of the color attribute; wherein the color attribute recognition branch includes a position consistency classifier, a plurality of color classifiers and a color attribution classifier;

[0116] Step S04, inputting the initial pedestrian attribute features into the non-color attribute recognition branch of the initial pedestrian attribute recognition model to obtain the probability distribution of the non-color attributes;

[0117] Step S05, constructing a first loss function according to the probability distribution of the color attribute and the true color label in the pedestrian attribute data set;

[0118] Step S06, constructing a second loss function according to the probability distribution of the non-color attributes and the non-color true labels in the pedestrian attribute data set;

[0119] Step S07: training, evaluating, and optimizing the parameters of the initial pedestrian attribute recognition model according to the first loss function and the second loss function to obtain a pedestrian attribute recognition model.

[0120] In this embodiment, in step S01, the pedestrian attribute dataset includes pedestrian images and corresponding attribute labels. Attribute labels can also be understood as real labels. In one embodiment, data loading and data enhancement can be performed on the pedestrian attribute dataset. Data enhancement of the pedestrian attribute dataset can improve the generalization ability of the model. Data enhancement includes random cropping, rotation, flipping, etc. In step S02, the initial pedestrian attribute recognition model can select a model architecture such as ResNet, MobileNet, EfficientNet, or a multi-task learning model architecture can be used.

[0121] In step S03, the color attribute recognition branch includes a position consistency classifier, multiple color classifiers, and a color attribution classifier. The initial pedestrian attribute features are recognized by the color attribute recognition branch to obtain the probability distribution of the color attribute. In step S04, the initial pedestrian attribute features are recognized by the non-color attribute recognition branch to obtain the probability distribution of the non-color attribute. In steps S05 and S06, the first loss function and the second loss function are constructed respectively by the probability distribution of the color attribute and the probability distribution of the non-color attribute. Calculating the two loss functions separately is more flexible, can balance different goals, and improve the convergence speed.

[0122] Furthermore, the initial pedestrian attribute recognition model is trained, evaluated, and optimized according to the first loss function and the second loss function to obtain a pedestrian attribute recognition model.

[0123] In one embodiment, the size of the head part of the pedestrian attribute recognition model provided by the present application is 1*32*1*1, and the model head part of the traditional method is classified and improved to improve the accuracy of pedestrian attribute extraction. 1 represents batchsize, c represents the number of channels, and h and w represent height and width. Assuming that the true label is labels and the predicted recognition result is predict, then:

[0124] p = Sigmoid (predict), the Sigmoid function is applied to the predicted recognition result predict for activation. The Sigmoid function maps any real number to the interval (0, 1) and converts it into a probability value p.

[0125] pt=labels*p+(1-labels)*(1-p), pt is an intermediate variable, which combines the true label labels and the probability value p after Sigmoid activation. If the true label labels is 1, then pt=p. If the true label labels is 0, then pt=1-p.

[0126] FL = focal_loss(pt,gamma)s, call the focal_loss function, set pt and parameter gamma. focal_loss is used to solve the problem of class imbalance. By assigning different weights to samples of different difficulty, the model pays more attention to samples that are difficult to classify.

[0127] The loss function Loss = FL + pow (1-pt, gamma + 1), where the second part is to perform a polynomial transformation on 1-pt, gamma is equal to 2, and 1-pt is squared before further polynomial transformation.

[0128] According to Loss = FL + pow (1-pt, gamma + 1), the first loss function and the second loss function are constructed respectively. The first loss function and the second loss function are added together to jointly train, evaluate and optimize the initial pedestrian attribute recognition model to obtain the pedestrian attribute recognition model.

[0129] The present application provides a pedestrian attribute recognition device 100. The pedestrian attribute recognition device 100 includes an image acquisition module 10, a feature extraction module 20, a color attribute recognition module 30, a color attribute determination module 40, a non-color attribute recognition module 50, and a pedestrian attribute determination module 60. The image acquisition module 10 is used to acquire an image to be recognized, which includes a pedestrian. The feature extraction module 20 is used to extract features from the image to be recognized to obtain pedestrian attribute features. The color attribute recognition module 30 is used to perform clothing consistency classification, multiple color classifications, and clothing position classification on the pedestrian attribute features to obtain position consistency recognition results, multiple color recognition results, and color attribution recognition results.

[0130] The color attribute determination module 40 is used to determine the color attribute information in the image to be identified based on the position consistency recognition result, the multiple color recognition results and the color attribution recognition result. The non-color attribute recognition module 50 is used to perform non-color attribute recognition on the attribute features of pedestrians to obtain the non-color attribute information in the image to be identified. The pedestrian attribute determination module 60 is used to determine the pedestrian attribute information based on the color attribute information and the non-color attribute information.

[0131] In this embodiment, the relevant description of the image acquisition module 10 can refer to the relevant description of step S10 in the above embodiment. The relevant description of the feature extraction module 20 can refer to the relevant description of step S20 in the above embodiment. The relevant description of the color attribute recognition module 30 can refer to the relevant description of step S30 in the above embodiment. The relevant description of the color attribute determination module 40 can refer to the relevant description of step S40 in the above embodiment. The relevant description of the non-color attribute recognition module 50 can refer to the relevant description of step S50 in the above embodiment. The relevant description of the pedestrian attribute determination module 60 can refer to the relevant description of step S60 in the above embodiment.

[0132] The present application provides an electronic device 200, including a memory 210, a processor 230, and a computer program 220 stored in the memory 210 and executable on the processor 230. When the processor 230 executes the computer program 220, the electronic device 200 implements a pedestrian attribute recognition method as described in any one of the above embodiments.

[0133] In this embodiment, the pedestrian attribute recognition method provided in the embodiment of the present application can be applied to electronic devices 200 such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of the electronic device 200.

[0134] For example, the electronic device 200 may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a TV set top box (STB), customer premise equipment (CPE) and / or other devices for communicating on a wireless system and a next-generation communication system, such as a mobile terminal in a 5G network or a mobile terminal in a future evolved Public Land Mobile Network (PLMN) network, etc.

[0135] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0136] In some embodiments, the memory may be an internal storage unit of the electronic device 200, such as a hard disk or memory of the electronic device 200. In other embodiments, the memory may also be an external storage device of the electronic device 200, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 200. Further, the memory may also include both an internal storage unit of the electronic device 200 and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program. The memory may also be used to temporarily store data that has been output or is to be output.

[0137] The present application provides a computer program product, including a computer program. When the computer program is run, the pedestrian attribute recognition method described in any of the above embodiments is executed. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of each of the above method embodiments can be implemented. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device 200, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk. In some jurisdictions, under legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunications signals.

[0138] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0139] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0140] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0142] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0143] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A pedestrian attribute recognition method, characterized in that: include: Obtain an image to be recognized; Inputting the image to be recognized into a feature extraction network of a pedestrian attribute recognition model to obtain pedestrian attribute features; Inputting the pedestrian attribute features into the position consistency classifier, multiple color classifiers and color attribution classifier of the pedestrian attribute recognition model respectively, to obtain a position consistency recognition result, multiple color recognition results and color attribution recognition result; Determining color attribute information in the image to be identified according to the position consistency identification result, the multiple color identification results, and the color attribution identification result; Inputting the pedestrian attribute feature into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain non-color attribute information in the image to be recognized; Pedestrian attribute information is determined according to the color attribute information and the non-color attribute information.

2. The pedestrian attribute recognition method according to claim 1, characterized in that: The step of determining the color attribute information in the image to be identified according to the position consistency identification result, the multiple color identification results and the color attribution identification result comprises: If the position consistency recognition result is inconsistent, the top color and the bottom color in the color attribute information are determined according to the color attribution recognition result and the multiple color recognition results.

3. The pedestrian attribute recognition method according to claim 1, characterized in that: The step of determining the color attribute information in the image to be identified according to the position consistency identification result, the multiple color identification results and the color attribution identification result comprises: If the position consistency recognition result is consistent, the color attribute information is determined according to the multiple color recognition results.

4. The pedestrian attribute recognition method according to claim 1, characterized in that: The step of inputting the pedestrian attribute feature into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized includes: Inputting the pedestrian attribute features into the first age classifier, the second age classifier, the age certainty classifier and the age position classifier of the non-color attribute recognition branch respectively, to obtain the first age recognition result, the second age recognition result, the age certainty recognition result and the age position recognition result; If the age certainty recognition result is certain, then the age attribute information is determined from the first age recognition result and the second age recognition result according to the age position recognition result.

5. The pedestrian attribute recognition method according to claim 1, characterized in that: The step of inputting the pedestrian attribute feature into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized includes: Inputting the pedestrian attribute features into the first classifier, the second classifier, and the first deterministic classifier of the non-color attribute recognition branch respectively to obtain a first recognition result, a second recognition result, and a first deterministic recognition result; If the first deterministic recognition result is positive, determining first attribute information according to the first recognition result; wherein the first attribute information is gender attribute information or glasses attribute information or top type attribute information; Alternatively, if the first deterministic recognition result is a top, the second attribute information is determined based on the second recognition result; wherein the second attribute information is reflective clothing attribute information.

6. The pedestrian attribute recognition method according to claim 1, characterized in that: The step of inputting the pedestrian attribute feature into the non-color attribute recognition branch of the pedestrian attribute recognition model to obtain the non-color attribute information in the image to be recognized includes: Inputting the pedestrian attribute features into the third classifier, the fourth classifier and the second deterministic classifier of the non-color attribute recognition branch respectively to obtain a third recognition result, a fourth recognition result and a second deterministic recognition result; If the second deterministic recognition result is certain, determining third attribute information according to the third recognition result and the fourth recognition result; The third attribute information is hat attribute information or bottoms attribute information.

7. The pedestrian attribute recognition method according to any one of claims 1 to 6, characterized in that: Before the step of acquiring the image to be identified, the method further includes: Get the pedestrian attribute dataset; Inputting the pedestrian attribute data set into a feature extraction network of an initial pedestrian attribute recognition model to obtain initial pedestrian attribute features; Inputting the initial pedestrian attribute features into the color attribute recognition branch of the initial pedestrian attribute recognition model to obtain a probability distribution of color attributes; wherein the color attribute recognition branch includes the position consistency classifier, the multiple color classifiers and the color attribution classifier; Inputting the initial pedestrian attribute features into the non-color attribute recognition branch of the initial pedestrian attribute recognition model to obtain a probability distribution of the non-color attribute; Constructing a first loss function according to the probability distribution of the color attribute and the true color label in the pedestrian attribute data set; Constructing a second loss function according to the probability distribution of the non-color attribute and the non-color true label in the pedestrian attribute data set; According to the first loss function and the second loss function, the initial pedestrian attribute recognition model is trained, evaluated, and optimized to obtain the pedestrian attribute recognition model.

8. A pedestrian attribute recognition device, characterized in that: include: An image acquisition module, used to acquire an image to be identified, wherein the image to be identified includes a pedestrian; A feature extraction module is used to extract features from the image to be identified to obtain pedestrian attribute features; A color attribute recognition module is used to classify the pedestrian attribute features into clothing consistency, multiple color classifications, and clothing position classifications to obtain a position consistency recognition result, multiple color recognition results, and a color attribution recognition result; A color attribute determination module, configured to determine color attribute information in the image to be identified based on the position consistency identification result, the multiple color identification results, and the color attribution identification result; A non-color attribute recognition module, used to perform non-color attribute recognition on the pedestrian attribute features to obtain non-color attribute information in the image to be recognized; The pedestrian attribute determination module is used to determine the pedestrian attribute information according to the color attribute information and the non-color attribute information.

9. An electronic device, characterized in that: The electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed, causes the method according to any one of claims 1 to 7 to be performed.

Images