Pedestrian attribute analysis method and device, storage medium and electronic equipment

Abstract

The application relates to the technical field of artificial intelligence, and provides a pedestrian attribute analysis method and device, a storage medium and an electronic device. The pedestrian attribute analysis method comprises the following steps: obtaining semantic context information of a pedestrian attribute in a pedestrian image; the semantic context information is obtained by analyzing the pedestrian image or associated information thereof, and comprises at least one of individual attribute association, group attribute information and space-time constraint information; and performing a pedestrian attribute analysis task by using the semantic context information, wherein the pedestrian attribute analysis task comprises at least one of a model training task, a model inference task and a post-processing task. When analyzing the pedestrian attribute, the semantic context information of the pedestrian attribute, such as the individual attribute association, the group attribute information and the space-time constraint information, is used in the training, inference and post-processing stages, so that the accuracy of the pedestrian attribute analysis can be significantly improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a pedestrian attribute analysis method and apparatus, storage medium and electronic device. Background Technology

[0002] Pedestrian attribute analysis uses computer vision technology to identify various attributes of pedestrians in surveillance videos. Identifiable pedestrian attributes include gender, age, body orientation, type of clothing, color of clothing, type of clothing, color of clothing, type of shoes, whether wearing a coat, hat, glasses, or mask, hair length, whether carrying shopping bags, tote bags, shoulder bags, backpacks, suitcases, trolleys, mobile phones, umbrellas, umbrella color, occupation, etc.

[0003] Currently, the mainstream method for pedestrian attribute analysis is based on deep learning. However, the diverse and complex acquisition scenarios of surveillance videos result in low accuracy of existing methods for pedestrian attribute analysis. Summary of the Invention

[0004] The purpose of this application is to provide a pedestrian attribute analysis method, apparatus, storage medium, and electronic device to improve the above-mentioned technical problems.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] In a first aspect, embodiments of this application provide a pedestrian attribute analysis method, comprising: acquiring semantic context information of pedestrian attributes in a pedestrian image; and performing a pedestrian attribute analysis task using the semantic context information;

[0007] The semantic context information is obtained by analyzing the pedestrian image or its associated information, and includes at least one of the following: individual attribute association, including the statistical laws governing the association between multiple pedestrian attributes of an individual pedestrian, or a specific set of rules that they conform to; group attribute information, including the statistical laws governing at least one pedestrian attribute of a group of pedestrians, or a specific set of rules that they conform to; and spatiotemporal constraint information, including the statistical laws governing the association between the spatiotemporal information corresponding to the pedestrian image and at least one pedestrian attribute, or a specific set of rules that they conform to, wherein the spatiotemporal information includes at least one piece of information in time, space, and scene.

[0008] The pedestrian attribute analysis task includes at least one of the following tasks: Model training task: Input the pedestrian image and pedestrian attribute labels, and train a neural network model for inferring pedestrian attributes by combining the semantic context information of the pedestrian image; Model inference task: Input the pedestrian image, and infer pedestrian attributes using the neural network model by combining the semantic context information of the pedestrian image; Post-processing task: Input the inference results of the neural network model for pedestrian attributes in the pedestrian image, and correct the inference results by combining the semantic context information of the pedestrian image.

[0009] The above method utilizes semantic contextual information of pedestrian attributes, such as individual attribute association, group attribute information, and spatiotemporal constraint information, in stages such as model training, model inference, and post-processing of model inference results when analyzing pedestrian attributes, thereby significantly improving the accuracy of pedestrian attribute analysis.

[0010] The reason is that although pedestrian images may have diversity and complexity in the acquisition scene, this diversity and complexity generally only affects the image features extracted by the model, and will not have a significant impact on the semantic context information of pedestrian attributes in the pedestrian image. That is, semantic context information has a certain stability in a statistical sense. Therefore, by using semantic context information to constrain the analysis process of pedestrian attributes, better results can be obtained in pedestrian attribute analysis.

[0011] Understandably, this method can well meet the needs of pedestrian attribute analysis in video surveillance, and of course, it can also be applied to other occasions besides video surveillance.

[0012] In one implementation of the first aspect, the statistical regularities in the individual attribute association include: an attribute association heatmap among multiple pedestrian attributes of a pedestrian individual, the attribute association heatmap including numerical values ​​representing the degree of correlation between each pair of pedestrian attributes; or, a correlation coefficient among multiple pedestrian attributes of a pedestrian individual; or, a conditional probability distribution among multiple pedestrian attributes of a pedestrian individual; or, a joint distribution among multiple pedestrian attributes of a pedestrian individual; the specific rule set in the individual attribute association includes: at least one rule describing the multiple pedestrian attributes being constrained by the correlation between attributes.

[0013] In one implementation of the first aspect, the statistical regularity in the group attribute information includes: the statistical distribution followed by at least one pedestrian attribute of the pedestrian group; or, the statistical quantity possessed by at least one pedestrian attribute of the pedestrian group, the statistical quantity including at least one of: mean, variance, covariance, maximum value, minimum value, higher-order moments, higher-order cumulants, and higher-order cumulant spectrum; the specific rule set in the group attribute information includes: at least one rule describing the group constraint of the pedestrian attribute on its own attribute.

[0014] In one implementation of the first aspect, the statistical regularity in the spatiotemporal constraint information includes: a statistical distribution that at least one pedestrian attribute follows, with the spatiotemporal information corresponding to the pedestrian image as a priori constraint; or, a statistical quantity possessed by at least one pedestrian attribute, the statistical quantity including at least one of: mean, variance, covariance, maximum value, minimum value, higher-order moments, higher-order cumulants, and higher-order cumulant spectrum; the specific rule set in the spatiotemporal constraint information includes: at least one rule describing the spatiotemporal constraint on pedestrian attributes.

[0015] Of the three implementation methods described above, typical forms of individual attribute association, group attribute information, and spatiotemporal constraint information are presented respectively. In summary, semantic context information can be either statistically derived results or fixed rules (such as hard-coded rules), and its setting is very flexible.

[0016] In one implementation of the first aspect, the pedestrian image is a training sample in the training set. The model training task is performed using the semantic context information, including: extracting image features from the pedestrian image using an attribute analysis network, and classifying and predicting pedestrian attributes in the pedestrian image based on the image features to obtain attribute prediction results; wherein the attribute analysis network is a neural network model to be trained; calculating a classification loss based on the difference between the attribute prediction results and the pedestrian attribute labels of the pedestrian image, and calculating a semantic loss based on the semantic context information; calculating a fusion loss based on the classification loss and the semantic loss, and updating the parameters of the attribute analysis network based on the fusion loss.

[0017] The above implementation method calculates not only the traditional classification loss based on image features, but also the semantic loss based on semantic context information when training the attribute analysis network. This allows the trained model to consider not only the influence of image features on attribute values ​​at the data level, but also the influence of semantic context information on attribute values ​​at the semantic level when classifying and predicting pedestrian attributes. Therefore, it can significantly improve the accuracy of the model in pedestrian attribute analysis.

[0018] In one implementation of the first aspect, the attribute analysis network includes a feature extraction network and an attribute classification network connected to the feature extraction network. The step of extracting image features from the pedestrian image using the attribute analysis network and classifying and predicting pedestrian attributes in the pedestrian image based on the image features to obtain attribute prediction results includes: extracting image features from the pedestrian image using the feature extraction network; classifying and predicting pedestrian attributes in the pedestrian image based on the image features using the attribute classification network to obtain the attribute prediction results; the semantic context information includes individual attribute associations, and the attribute analysis network further includes an attribute association extraction network connected to the feature extraction network. The step of obtaining semantic context information of pedestrian attributes in the pedestrian image includes: extracting individual attribute associations of pedestrian attributes in the pedestrian image based on the image features using the attribute association extraction network.

[0019] In the above implementation, the attribute analysis network comprises two parts: a feature extraction network and an attribute classification network. The feature extraction network extracts common image features, while the attribute classification network can be a multi-branch network, with each branch classifying and predicting a specific pedestrian attribute. Each branch can also extract specific image features for its corresponding attribute. Individual attribute associations are also extracted using the attribute association extraction network based on the common image features, essentially sharing the feature extraction network with the attribute classification prediction part. This approach saves computation and allows the associations between attributes to constrain the attribute prediction results. This constraint is reflected in updating the parameters of the shared feature extraction network by calculating semantic loss.

[0020] In one implementation of the first aspect, the semantic context information includes individual attribute associations, and the step of calculating semantic loss based on the semantic context information includes: calculating the average individual attribute association of all pedestrian images in the training batch to which the pedestrian image belongs; and calculating a first semantic loss in the semantic loss based on the difference between the individual attribute associations and the average individual attribute associations.

[0021] The associations between attributes should be statistically stable, so the associations of individual attributes in a single pedestrian image should tend to be consistent with the statistical associations of individual attributes. This is the starting point for setting the first semantic loss.

[0022] In one implementation of the first aspect, the semantic context information includes group attribute information, and the step of obtaining the semantic context information of pedestrian attributes in pedestrian images includes: obtaining the group attribute information of pedestrian attributes in pedestrian images by statistically analyzing at least one pedestrian attribute in some or all pedestrian images in the training set.

[0023] According to the definition of group attribute information, it includes the statistical laws (such as statistical distributions or statistics) that at least one pedestrian attribute of a pedestrian group follows, or the specific set of rules that it conforms to. Therefore, by performing statistics on pedestrian images in the training set, the required statistical laws can be obtained, or the required set of rules can be formed based on the statistical results.

[0024] In one implementation of the first aspect, the semantic context information includes group attribute information, and the step of calculating semantic loss based on the semantic context information includes: calculating a group attribute prediction result, wherein the group attribute prediction result represents the pedestrian attributes in the pedestrian image predicted based on the group attribute information; and calculating a second semantic loss in the semantic loss based on the difference between the attribute prediction result and the group attribute prediction result.

[0025] Individual attributes and group attributes should be similar to some extent, so the attribute prediction results obtained based on a single pedestrian image should tend to be consistent with the group attribute prediction results obtained based on group attribute information. This is the starting point for setting the second semantic loss.

[0026] In one implementation of the first aspect, the semantic context information includes spatiotemporal constraint information, and the step of obtaining the semantic context information of pedestrian attributes in the pedestrian image includes: obtaining the spatiotemporal information corresponding to the pedestrian image from a camera or monitoring system; using the spatiotemporal information as a priori constraint, obtaining the spatiotemporal constraint information of pedestrian attributes in the pedestrian image by statistically analyzing at least one pedestrian attribute in some or all pedestrian images in the training set.

[0027] According to the definition of spatiotemporal constraint information, it includes the statistical laws (such as statistical distributions or statistics) governing the relationship between the spatiotemporal information corresponding to a pedestrian image and at least one pedestrian attribute, or the specific set of rules that it conforms to. Therefore, after obtaining the spatiotemporal information corresponding to a pedestrian image from a camera or surveillance system, statistical analysis of the pedestrian images in the training set using this spatiotemporal information as a priori constraint can yield the desired statistical laws, or the desired set of rules can be formed based on the statistical results.

[0028] In one implementation of the first aspect, the semantic context information includes spatiotemporal information, and the step of calculating semantic loss based on the semantic context information includes: calculating a spatiotemporal attribute prediction result, wherein the spatiotemporal attribute prediction result represents the pedestrian attributes in the pedestrian image predicted based on the spatiotemporal constraint information; and calculating a third semantic loss in the semantic loss based on the difference between the attribute prediction result and the spatiotemporal attribute prediction result.

[0029] When performing pedestrian attribute analysis based on pedestrian images, we should try our best to extract information from the images themselves so that the obtained attribute prediction results are as consistent as possible with the spatiotemporal attribute prediction results obtained based on spatiotemporal constraint information. This is the starting point for setting the third semantic loss.

[0030] In one implementation of the first aspect, the model inference task is performed using the semantic context information, including: extracting preliminary image features of the pedestrian image using an attribute analysis network; wherein the attribute analysis network is a trained neural network model, and the preliminary image features are used to classify and predict pedestrian attributes in the pedestrian image; performing calculations on the preliminary image features based on the semantic context information to obtain final image features; and classifying and predicting pedestrian attributes in the pedestrian image based on the final image features to obtain attribute prediction results.

[0031] When inferring pedestrian attributes, the above method first uses an attribute analysis network to extract preliminary image features of pedestrian images, which takes into account the influence of image features on attribute values ​​at the data level. Then, it further uses semantic context information to perform calculations on the preliminary attribute image features to obtain the final image features. Finally, it obtains the attribute prediction results based on the final image features, which takes into account the influence of semantic context information on attribute values ​​at the semantic level. Since it considers both data-level and semantic-level factors, the above method has high accuracy in predicting pedestrian attributes.

[0032] Furthermore, since the diversity and complexity of pedestrian images in the acquisition scenarios generally only affect the image features extracted by the model, without significantly affecting the semantic context information of pedestrian attributes in the pedestrian images, the above method is very suitable for pedestrian attribute analysis in the field of video surveillance, but is not limited to this field.

[0033] In one implementation of the first aspect, the semantic context information includes group attribute information. Obtaining the semantic context information of pedestrian attributes in pedestrian images includes: obtaining group attribute information of pedestrian attributes in pedestrian images by statistically analyzing some or all pedestrian images in the training set, and / or some or all inferred pedestrian images; or, obtaining group attribute information of pedestrian attributes in pedestrian images by calculating previously obtained group attribute information; wherein, the inferred pedestrian images refer to pedestrian images for which pedestrian attribute prediction has been performed, and the training set refers to the set of pedestrian images used to train the attribute analysis network.

[0034] In the model inference stage, there are multiple ways to obtain group attribute information. It can be obtained by statistically analyzing pedestrian images in the training set and / or inferred pedestrian images, or by calculating it based on existing group attribute information, or by combining both methods. For example, an initial group attribute information can be calculated based on existing group attribute information, and then this initial group attribute information can be continuously updated using statistical results of pedestrian attributes in the inferred pedestrian images. This ensures that the group attribute information conforms to the objective changing patterns of group attributes, thereby improving the accuracy of pedestrian attribute analysis. In short, its calculation method is very flexible.

[0035] In one implementation of the first aspect, the post-processing task is performed using the semantic context information, including: obtaining attribute prediction results for pedestrian attributes in the pedestrian image output by an attribute analysis network; wherein the attribute analysis network is a trained neural network model; and correcting the attribute prediction results according to the semantic context information to obtain corrected attribute prediction results.

[0036] When post-processing pedestrian attributes, the above method first uses an attribute analysis network to obtain preliminary attribute prediction results, that is, it considers the influence of image features on attribute values ​​at the data level. Then, it further uses semantic context information to correct the attribute prediction results to obtain the final attribute prediction results, that is, it considers the influence of semantic context information on attribute values ​​at the semantic level. Since it considers both data-level and semantic-level factors, the above method has high accuracy in predicting pedestrian attributes.

[0037] Furthermore, since the diversity and complexity of pedestrian images in the acquisition scenarios generally only affect the image features extracted by the model, without significantly affecting the semantic context information of the pedestrian images, the above method is very suitable for pedestrian attribute analysis in the field of video surveillance.

[0038] Secondly, embodiments of this application provide a pedestrian attribute analysis device, comprising: an information acquisition module for acquiring semantic context information of pedestrian attributes in a pedestrian image; and an attribute analysis module for performing a pedestrian attribute analysis task using the semantic context information.

[0039] The semantic context information is obtained by analyzing the pedestrian image or its associated information, and includes at least one of the following: individual attribute association, including the statistical laws governing the association between multiple pedestrian attributes of an individual pedestrian, or a specific set of rules that they conform to; group attribute information, including the statistical laws governing at least one pedestrian attribute of a group of pedestrians, or a specific set of rules that they conform to; and spatiotemporal constraint information, including the statistical laws governing the association between the spatiotemporal information corresponding to the pedestrian image and at least one pedestrian attribute, or a specific set of rules that they conform to, wherein the spatiotemporal information includes at least one piece of information in time, space, and scene.

[0040] The pedestrian attribute analysis task includes at least one of the following tasks: Model training task: Input the pedestrian image and pedestrian attribute labels, and train a neural network model for inferring pedestrian attributes by combining the semantic context information of the pedestrian image; Model inference task: Input the pedestrian image, and infer pedestrian attributes using the neural network model by combining the semantic context information of the pedestrian image; Post-processing task: Input the inference results of the neural network model for pedestrian attributes in the pedestrian image, and correct the inference results by combining the semantic context information of the pedestrian image.

[0041] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the method provided in the first aspect or any possible implementation thereof.

[0042] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores computer program instructions, and the computer program instructions are read and executed by the processor to perform the method provided in the first aspect or any possible implementation of the first aspect. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 The flowchart of the first pedestrian attribute analysis method provided in the embodiments of this application is shown;

[0045] Figure 2 This application illustrates a structure of an attribute association heatmap provided in an embodiment.

[0046] Figure 3 The flowchart of the second pedestrian attribute analysis method provided in the embodiments of this application is shown;

[0047] Figure 4 It shows Figure 3 The working principle of pedestrian attribute analysis methods in China;

[0048] Figure 5 The flowchart of the third pedestrian attribute analysis method provided in the embodiments of this application is shown;

[0049] Figure 6 The flowchart of the fourth pedestrian attribute analysis method provided in the embodiments of this application is shown;

[0050] Figure 7 It shows Figure 6 The working principle of pedestrian attribute analysis methods in China;

[0051] Figure 8 The flowchart of the fifth pedestrian attribute analysis method provided in the embodiments of this application is shown;

[0052] Figure 9 It shows Figure 8 The working principle of pedestrian attribute analysis methods in China;

[0053] Figure 10 The structure of a pedestrian attribute analysis device provided in an embodiment of this application is shown;

[0054] Figure 11 The structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0055] In the field of video surveillance, the diverse and complex acquisition scenarios of surveillance videos result in low accuracy for pedestrian attribute analysis using existing methods. The inventors' long-term research has revealed the following main problems with existing methods:

[0056] (1) It is susceptible to environmental factors, which significantly reduces the accuracy of pedestrian attribute prediction. Influencing factors include various lighting changes, occlusion, complex backgrounds, various resolutions, and various image qualities.

[0057] (2) Existing technologies analyze pedestrians based on a single attribute without considering the correlation between multiple attributes, resulting in low accuracy of pedestrian attribute analysis. For example, "men" rarely wear "long skirts". If a "man" is predicted to wear a "long skirt", the result is likely to be incorrect in reality. Existing technologies cannot handle this problem.

[0058] (3) Existing technologies analyze the attributes of individual pedestrians without considering the influence of group attributes on individual pedestrian attributes, resulting in low accuracy of pedestrian attribute analysis. For example, if there are many pedestrians wearing "short sleeves" in the scene, then predicting that a certain pedestrian is wearing "long sleeves" is obviously different from the group attributes, so it is very likely to be wrong. Existing technologies cannot handle this problem.

[0059] (4) Existing technologies only perform attribute analysis based on pedestrian images themselves, without considering the influence of time and space factors on individual pedestrian attributes, resulting in low accuracy of pedestrian attribute analysis. For example, if the time is summer and the location is Chongqing, the probability of wearing a "long skirt" is low, while if the time is winter and the location is Northeast China, the probability of wearing a "coat" is high; another example is that if the scene is a supermarket entrance, the probability of "holding a shopping cart" is high, while if the scene is "school," the probability of "holding a shopping cart" is low. Existing technologies cannot handle the problem of predicted attribute values ​​deviating from the time, location, or scene (for example, predicting that a pedestrian in Chongqing in summer will wear a "long skirt").

[0060] To address the aforementioned problems in the existing technology, embodiments of this application provide a pedestrian attribute analysis method, apparatus, storage medium, and electronic device, which incorporates the semantic context information of pedestrian attributes in pedestrian images into the model training, model inference, and post-processing stages of pedestrian attribute analysis, thereby improving the accuracy of pedestrian attribute analysis and enhancing the effectiveness of video surveillance.

[0061] It should be noted that, apart from the new technical solutions proposed in this application, the inventor's discovery of the above problems (1)-(4) and the reasons for the above problems (1)-(4) analyzed by the inventor should be regarded as the inventor's contribution to this application, and should not be regarded as content that already exists in the prior art.

[0062] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. It should be noted that similar reference numerals and letters in the following drawings indicate similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0063] Figure 1 The flowchart of the first pedestrian attribute analysis method provided in the embodiments of this application is shown. Figure 1 The methods in the text can be, but are not limited to, by Figure 10 The electronic device shown performs the operation; see the following section for details. Figure 10 The explanation. (Refer to...) Figure 1 The method includes:

[0064] Step S01: Obtain the semantic context information of pedestrian attributes in the pedestrian image.

[0065] Pedestrian images refer to images that may contain pedestrians, such as images captured by surveillance cameras installed on roads or at access control points. Of course, some pedestrian images may not contain pedestrians, but the following discussion will mainly focus on pedestrian images that do contain pedestrians.

[0066] The solution in this application does not limit which pedestrian attributes in the pedestrian image should be analyzed. For example, it may include one or more of the following attributes:

[0067] Gender (male, female), age (20, 25, 30, etc.), body orientation (facing forward, sideways, back, etc.), type of clothing (long sleeves, short sleeves, etc.), color of clothing (red, yellow, black, etc.), type of clothing (long pants, shorts, long skirts, short skirts, etc.), color of clothing (red, yellow, black, etc.), type of shoes (sneakers, leather shoes, etc.), color of shoes (red, yellow, black, etc.), whether wearing a coat, whether wearing a hat, whether wearing glasses, whether wearing a mask, color of mask (white, pink, black, etc.), hair length (bald, short hair, long hair, etc.), hair color (black, white, brown, etc.), whether carrying a shopping bag, whether carrying a tote bag, whether carrying a shoulder bag, whether carrying a backpack, color of bag (green, blue, white, etc.), whether carrying a suitcase, whether carrying a trolley, whether carrying a mobile phone, whether carrying an umbrella, color of umbrella, occupation (courier, sanitation worker, delivery person, security guard, etc.), and items held (infant, pet, etc.).

[0068] The semantic context information of pedestrian attributes can be defined as information that is semantically related to the pedestrian attributes, and can influence the values ​​of pedestrian attributes at the semantic level. In contrast, although image features can also affect the values ​​of pedestrian attributes, this influence arises at the data level and is not directly related to the semantics of pedestrian attributes.

[0069] The following are three types of semantic contextual information:

[0070] A. Individual attribute association

[0071] Individual attribute associations refer to the statistical laws or specific set of rules governing the relationships between various pedestrian attributes of an individual.

[0072] The statistical regularity in the association of individual attributes can be the statistical distribution or statistical measure that governs the association between various pedestrian attributes of an individual. Information representing the statistical distribution can include, but is not limited to, the type of statistical distribution (e.g., normal distribution, binomial distribution) and parameters (e.g., mean, variance). For example, this statistical distribution could be a joint distribution of multiple pedestrian attributes, and the statistical measure could be conditional probability, correlation coefficient, etc.

[0073] In some implementations, the statistical distribution of the associations between various pedestrian attributes of an individual can be realized as: an attribute association heatmap of various pedestrian attributes of an individual. The attribute association heatmap includes numerical values ​​representing the degree of correlation between each pair of pedestrian attributes, that is, the association of individual attributes is quantitatively represented, which facilitates related calculations.

[0074] Figure 2 This illustrates one structure of an attribute association heatmap, with reference to Figure 2 An attribute correlation heatmap can be viewed as a table. The rows and columns represent pedestrian attributes to be analyzed for correlation, such as "gender," "age," "hairstyle," "clothing type," and "clothing color." Each cell corresponds to a value indicating the correlation between the two pedestrian attributes in that row and column. For example, the correlation between "gender" and "age" is 0.1, and the correlation between "gender" and itself is 1, and so on. Understandably, an attribute correlation heatmap can include all pedestrian attributes to be analyzed in the pedestrian image, or it can include only some of the attributes of interest, such as those attributes that significantly influence each other's values.

[0075] In addition to attribute association heatmaps, the statistical distribution of the associations between multiple pedestrian attributes of an individual can also be realized as: the conditional probability distribution of multiple pedestrian attributes of an individual, the joint distribution of multiple pedestrian attributes of an individual, etc.

[0076] The specific rule set for individual attribute association includes at least one rule describing the correlation constraints between multiple pedestrian attributes. These rules can be hard-coded into the pedestrian attribute analysis program. For example, if the pedestrian attribute analysis results in "male" gender and "long hair," the hairstyle can be automatically corrected to "short hair" according to the rule. Similarly, if the pedestrian attribute analysis results in "doctor," the clothing color can be automatically predicted as "white" according to the rule. The rules in the specific rule set can be freely formulated; for example, they can be determined based on expert experience or summarized from statistical patterns.

[0077] B. Group attribute information

[0078] Group attribute information includes the statistical regularity or specific set of rules that at least one pedestrian attribute of a pedestrian group follows.

[0079] The statistical regularity in group attribute information can be a statistical distribution or a statistical measure that a group of pedestrians follows for at least one pedestrian attribute. Information representing the statistical distribution can include, but is not limited to, the type of statistical distribution (e.g., normal distribution, binomial distribution) and parameters (e.g., mean, variance). For example, the statistical distribution can be a distribution following a certain pedestrian attribute, or a joint distribution following multiple pedestrian attributes. The statistical measure can be a low-order statistical measure such as mean, maximum, minimum, expected value, variance, covariance, or correlation coefficient, or a high-order statistical measure such as higher-order moments, higher-order cumulants, or higher-order cumulant spectra.

[0080] For example, based on prior knowledge, if the "age" attribute follows a normal distribution, then the corresponding group attribute information can include the mean and variance of this normal distribution. With the mean and variance, the probability density function of the normal distribution can be obtained.

[0081] The specific rule set in the group attribute information includes at least one rule describing the group-based constraint of pedestrian attributes. These rules can be hard-coded into the pedestrian attribute analysis program. For example, if the pedestrian attribute analysis results indicate that more than 95% of the pedestrians in the image are wearing "short-sleeved" clothing, then the attribute of pedestrians wearing "long-sleeved" clothing can be automatically corrected to "short-sleeved" according to the rule. The rules in the specific rule set can be freely formulated; for example, they can be determined based on expert experience or summarized based on statistical regularities.

[0082] C. Spatiotemporal constraint information

[0083] Spatiotemporal constraint information includes the statistical laws governing the relationship between the spatiotemporal information corresponding to a pedestrian image and at least one pedestrian attribute, or the specific set of rules that the relationship conforms to.

[0084] The spatiotemporal information corresponding to pedestrian images includes at least one of the following: time, space, and scene. The time information can be the season, day of the week, day of the week, or whether the pedestrian image was captured. The space information can be the specific location information of the pedestrian image, such as the latitude and longitude, region (e.g., province, city, South China, North China), or abstract location information, such as roadway or sidewalk. The scene information can be a specific environment of the pedestrian image, such as a supermarket, hospital, school, indoor, or outdoor.

[0085] The statistical regularity in the spatiotemporal constraint information can be a statistical distribution or statistical quantity that at least one pedestrian attribute follows, with the spatiotemporal information corresponding to the pedestrian image as a priori constraint. The information representing the statistical distribution can include, but is not limited to, the type of statistical distribution (e.g., normal distribution, binomial distribution) and parameters (e.g., mean, variance). For example, the statistical distribution can be a conditional distribution or joint distribution that at least one pedestrian attribute follows, and the statistical quantity can be a low-order statistical quantity such as mean, maximum, minimum, expectation, variance, covariance, or correlation coefficient, or a high-order statistical quantity such as higher-order moments, higher-order cumulants, or higher-order cumulant spectra.

[0086] For example, the "age" attribute follows a distribution P(θ), where θ is the distribution parameter. When considering spatiotemporal constraints, the "age" attribute can be considered to follow a conditional distribution P(θ|s) or a joint distribution P(θ,s), where s represents the distribution parameter corresponding to the spatiotemporal information.

[0087] The specific rule set in the spatiotemporal constraint information includes at least one rule describing how pedestrian attributes are constrained by spatiotemporal information. These rules can be hard-coded into the pedestrian attribute analysis program. For example, if the pedestrian attribute analysis result indicates the type of clothing as "long skirt," and the time in the spatiotemporal information is "summer" and the location is "Chongqing," then the clothing type can be automatically corrected to "short skirt" according to the rule. As another example, if the pedestrian attribute analysis result indicates the pedestrian is "carrying a handcart," and the scene in the spatiotemporal information is "school," then the attribute of whether or not the pedestrian is carrying a handcart can be automatically predicted as "no" according to the rule. The rules in the specific rule set can be freely formulated; for example, they can be determined based on expert experience, or they can be summarized based on statistical patterns.

[0088] Understandably, semantic context information is not limited to the three types of information mentioned above, and it may contain one or more of these three types simultaneously. For example, in... Figure 4 The semantic context information includes individual attribute associations, group attribute information, and spatiotemporal constraint information.

[0089] Semantic context information of pedestrian attributes can be obtained by analyzing pedestrian images or their associated information. For example, it can be obtained by analyzing image features or by statistically analyzing pedestrian attributes in pedestrian images. When performing statistical analysis, the associated information of pedestrian images (such as the spatiotemporal information mentioned above) can also be considered. Specific methods for obtaining semantic context information will be given later.

[0090] Step S02: Perform pedestrian attribute analysis using semantic context information.

[0091] The pedestrian attribute analysis task includes at least one of the following tasks:

[0092] Model training task: Input pedestrian images and pedestrian attribute labels, and combine the semantic context information of pedestrian images to train a neural network model for inferring pedestrian attributes.

[0093] Model inference task: Input pedestrian images and combine them with semantic context information to infer pedestrian attributes using a neural network model.

[0094] Post-processing task: Input the inference results of the neural network model on the pedestrian attributes in the pedestrian image, and combine the semantic context information of the pedestrian image to correct the inference results.

[0095] Specific examples of performing these pedestrian attribute analysis tasks will be given later, and will not be elaborated here.

[0096] In summary, the pedestrian attribute analysis method provided in this application uses semantic context information of pedestrian attributes, such as individual attribute association, group attribute information, and spatiotemporal constraint information, in stages such as model training, model inference, and post-processing of model inference results, which can significantly improve the accuracy of pedestrian attribute analysis.

[0097] The reason is that although pedestrian images may have diversity and complexity in the acquisition scene, this diversity and complexity generally only affect the image features extracted by the model, and will not have a significant impact on the semantic context information of the attributes in the pedestrian images. This is because semantic context information is either a certain statistical law or a fixed rule, that is, it has a certain stability in a statistical sense. Therefore, by using semantic context information to constrain the analysis process of pedestrian attributes, better results can be obtained in pedestrian attribute analysis.

[0098] Understandably, this method can well meet the needs of pedestrian attribute analysis in video surveillance, and of course, it can also be applied to other occasions besides video surveillance.

[0099] The following is a brief analysis of how pedestrian attribute analysis improves the problems (1)-(4) mentioned above. It is understandable that if the semantic context information includes individual attribute associations, then using it for pedestrian attribute analysis can improve problem (2); if the semantic context information includes group attribute information, then using it for pedestrian attribute analysis can improve problem (3); and if the semantic context information includes spatiotemporal constraint information, then using it for pedestrian attribute analysis can improve problem (4). As for problem (1), semantic context information also has an improvement effect on it, because the changes in lighting, occlusion, complex background, various resolutions, and various image qualities mentioned in problem (1) directly affect the image of pedestrians, but have little impact on semantic context information. For example, if the upper body of a pedestrian is obscured by other pedestrians, it is difficult to identify whether he is wearing a coat directly from the pedestrian image. However, if the neural network model can identify that the surrounding pedestrians are all "wearing coats" and that the pedestrian image was collected in "winter", then it can predict with a relatively high confidence that the pedestrian is also "wearing a coat", rather than "guessing" an attribute value without any basis. Understandably, in practice, the judgment logic of a neural network model can be reflected in the model parameters obtained through training. The above analysis is just an explanation of the model's predictive behavior from an easily understandable perspective. Figure 3 The flowchart of the second pedestrian attribute analysis method provided in the embodiment of this application is shown. This flowchart can be regarded as an execution flow of the model training task mentioned in step S02. Figure 4 This illustrates the detailed working principle of the method. Figure 3 The methods in the text can be, but are not limited to, by Figure 10 The electronic device shown performs the operation; see the following section for details. Figure 10 The explanation. (Refer to...) Figure 3 The method includes:

[0100] Step S110: Extract image features of pedestrian images using an attribute analysis network, and classify and predict pedestrian attributes in pedestrian images based on image features to obtain attribute prediction results.

[0101] The pedestrian images in step S110 are pedestrian images from the training set, accompanied by pedestrian attribute labels, i.e., pre-labeled true attribute values ​​of pedestrian attributes, used for supervised model training. The attribute analysis network in step S110 is the neural network model to be trained, and its specific structure is not limited. The attribute analysis network treats the prediction of each attribute as a multi-label classification problem. For example, for the gender attribute, there are two possible attribute values: "male" and "female," so it can be regarded as a binary classification problem. The classification labels include "male" and "female." The prediction result of the gender attribute can be the confidence score of each attribute value, for example, "male"—0.9, "female"—0.1.

[0102] Step S120: Calculate the classification loss based on the difference between the attribute prediction results and the pedestrian attribute labels in the pedestrian image.

[0103] For a specific pedestrian in a pedestrian image, its attribute label is fixed; for example, the label for the gender attribute is "male". Therefore, by substituting the predicted gender attribute and its label into a pre-defined loss function (the form of which is not limited), the loss generated by the attribute analysis network in predicting the gender attribute can be calculated. This loss, to some extent, characterizes the difference between the predicted gender attribute and the gender attribute label. The classification loss in step S120 can be considered as the sum of the losses generated by the attribute analysis network in predicting each pedestrian attribute.

[0104] Reference Figure 4 In some implementations, the attribute analysis network further includes a feature extraction network and an attribute classification network connected to the feature extraction network.

[0105] The feature extraction network extracts image features from pedestrian images. For example, it can be a convolutional neural network such as VGG or ResNet. The attribute classification network classifies and predicts pedestrian attributes based on the extracted image features and outputs the prediction results. For instance, it can be a multi-branch network, where each branch classifies and predicts one attribute (or a single-branch network if only one pedestrian attribute is predicted). In this case, the image features output by the feature extraction network are common to all branches. In some implementations, each branch can include a fully connected layer and a classifier (or a 1×1 convolution can be used instead). Furthermore, depending on the requirements, several convolutional layers can be added before the fully connected layers to extract specific image features of the pedestrian attribute corresponding to that branch based on the common image features.

[0106] Step S130: Obtain the semantic context information of pedestrian attributes in the pedestrian image, and calculate the semantic loss based on the semantic context information.

[0107] The definition of semantic context information has been explained above. Below, we list and illustrate three methods for obtaining semantic context information, and explain the corresponding methods for calculating semantic loss.

[0108] A. Individual attribute association

[0109] In some implementations, if the attribute analysis network includes a feature extraction network and an attribute classification network, an attribute association extraction network can be set up to obtain individual attribute associations. This attribute association extraction network is part of the attribute analysis network and is structurally connected to the feature extraction network. The attribute association extraction network takes the image features output by the feature extraction network as input and outputs the individual attribute associations of the pedestrian image; its specific network structure is not limited. Figure 4 The attribute association extraction network is not directly specified, but the position of the box "Individual Attribute Association" can be understood as the attribute association extraction network used to obtain individual attribute associations.

[0110] In these implementations, the attribute analysis network can be viewed as two parts: the first part includes a feature extraction network and an attribute classification network, used to classify and predict pedestrian attributes in pedestrian images; the second part includes a feature extraction network and an attribute association extraction network, used to obtain individual attribute associations. It's easy to see that these two parts share the feature extraction network. This network architecture is beneficial for saving computation and allows the associations between attributes to constrain the attribute prediction results: the second part updates the parameters of the feature extraction network by calculating semantic loss, while the first part also uses the image features extracted by the feature extraction network for pedestrian attribute prediction.

[0111] Understandably, in some other implementations, the attribute association extraction network can also directly obtain individual attribute associations based on pedestrian images, without being connected to the feature extraction network.

[0112] Obviously, there are other ways to obtain the association of individual attributes, such as statistical analysis of pedestrian images in the training set or directly reading the specific set of rules set for the association of individual attributes.

[0113] In some implementations, considering that the associations between attributes should be statistically stable, the associations of individual attributes in a single pedestrian image should tend to be consistent with the statistical associations of individual attributes. Therefore, the first semantic loss corresponding to the associations of individual attributes can be calculated as follows:

[0114] First, calculate the average individual attribute association among all pedestrian images in the training batch (e.g., a mini-batch). The average individual attribute association can be the mean of the individual attribute associations of all pedestrian images in that training batch (of course, the individual attribute associations should be quantized to allow for mean calculation; for example, the attribute association heatmap mentioned earlier is one quantization method). Then, calculate the first semantic loss based on the difference between the individual attribute associations and the average individual attribute associations; the specific loss function used is not limited. Based on the meaning of the loss, after training the model using the first semantic loss, the individual attribute associations will gradually approach the average individual attribute associations.

[0115] B. Group attribute information

[0116] Group attribute information can be obtained by statistically analyzing at least one pedestrian attribute from some or all pedestrian images in the training set, although other calculation methods are also possible. For example, for the "age" attribute, its distribution type can be determined as a normal distribution based on prior knowledge, and then the mean and variance of the normal distribution can be determined based on the statistical results of all pedestrian images in the training set. Alternatively, for the "age" attribute, its distribution type can be determined without prior specification, by statistically analyzing all pedestrian images in the training set, thus establishing a distribution and its corresponding parameters (or some methods may estimate a distribution without parameters).

[0117] Obviously, there are other ways to obtain group attribute information, such as directly reading the specific set of rules set for group attribute information.

[0118] In some implementations, considering that individual attributes and group attributes should be similar to some extent, the attribute prediction results obtained based on a single pedestrian image should tend to be consistent with the group attribute prediction results obtained based on group attribute information. Therefore, the second semantic loss corresponding to the group attribute information can be calculated as follows:

[0119] First, the attribute prediction results can be represented by quantification. For example, if each attribute value is represented by a numerical value, the attribute prediction results can be represented as a vector.

[0120] Then, the predicted group attributes are calculated and quantified, representing the pedestrian attributes in the pedestrian image predicted based on the group attribute information. For example, if a pedestrian attribute follows a distribution P(θ), then P(θ) can be considered as group attribute information. Sampling P(θ) yields a specific attribute value, which represents a pedestrian attribute predicted based on the group attribute information. Similarly, after sampling the distribution followed by each pedestrian attribute, all pedestrian attributes predicted based on the group attribute information can be obtained, and these pedestrian attributes can then be quantized into a vector.

[0121] Finally, a second semantic loss is calculated based on the difference between the quantized attribute prediction results and the group attribute prediction results. The specific calculation method is not limited; for example, the L2 distance between two vectors can be calculated. According to the meaning of the loss, after training the model using the second semantic loss, the attribute prediction results will gradually approach the group attribute prediction results.

[0122] C. Spatiotemporal constraint information

[0123] The spatiotemporal information corresponding to pedestrian images can come directly from the cameras that capture these images, or from platforms or systems with video capture or storage capabilities. For example, some surveillance systems annotate the videos captured by cameras, and these annotations contain spatiotemporal information. By accessing these surveillance systems, the required spatiotemporal information can be obtained. After obtaining the spatiotemporal information corresponding to the pedestrian images, this spatiotemporal information can be used as a priori constraints to statistically analyze at least one pedestrian attribute in some or all pedestrian images in the training set. This yields the spatiotemporal constraint information of pedestrian attributes in the pedestrian images. The statistical method can refer to the statistical analysis of group attribute information.

[0124] Obviously, there are other ways to obtain spatiotemporal constraint information, such as directly reading the specific set of rules set for spatiotemporal constraint information.

[0125] When performing pedestrian attribute analysis based on pedestrian images, we should extract as much information as possible from the images themselves, so that the obtained attribute prediction results are as consistent as possible with the spatiotemporal attribute prediction results obtained based on spatiotemporal constraint information. Based on this motivation, the third semantic loss corresponding to spatiotemporal constraint information can be calculated as follows:

[0126] First, the attribute prediction results can be represented by quantification. For example, if each attribute value is represented by a numerical value, the attribute prediction results can be represented as a vector.

[0127] Then, the spatiotemporal attribute prediction results are calculated and quantified, representing the pedestrian attributes in the pedestrian image predicted based on spatiotemporal constraint information. For example, a hard-coding method can be used; for instance, if the time is "winter," the location is "northeast," and the scene is "outdoor," the attribute of whether or not the person is wearing a coat can be directly set to "wearing a coat." Alternatively, if a pedestrian attribute follows a distribution P(θ), when considering spatiotemporal constraint information, it can be assumed that the pedestrian attribute follows a conditional distribution P(θ|s) or a joint distribution P(θ,s), where s represents the distribution parameter corresponding to the spatiotemporal information. Sampling P(θ|s) or P(θ,s) yields a specific attribute value, which represents a pedestrian attribute predicted based on spatiotemporal constraint information. Similarly, after sampling the distributions followed by each pedestrian attribute, all pedestrian attributes predicted based on group attribute information can be obtained, and these pedestrian attributes can then be quantized into a vector.

[0128] Finally, a third semantic loss is calculated based on the difference between the quantized attribute prediction results and the spatiotemporal attribute prediction results. The specific calculation method is not limited; for example, the L2 distance between two vectors can be calculated. According to the meaning of the loss, after training the model using the third semantic loss, the attribute prediction results will gradually approach the spatiotemporal attribute prediction results.

[0129] Understandably, one or more of the above three semantic losses can be calculated; for example, due to... Figure 4 The semantic context information includes individual attribute associations, group attribute information, and spatiotemporal constraint information. In this case, the semantic loss can be the sum of the first semantic loss, the second semantic loss, and the third semantic loss (it can be directly summed or weighted summed).

[0130] In addition, it should be pointed out that during the model training process, some attribute semantic information does not need to be calculated repeatedly. For example, once the group attribute information is calculated, it only needs to be obtained and used each time the second semantic loss is calculated, without having to be calculated again.

[0131] Step S140: Calculate the fusion loss based on the classification loss and semantic loss, and update the parameters of the attribute analysis network based on the fusion loss.

[0132] exist Figure 2 In the illustrated implementation, the sum of the classification loss and the semantic loss (which can be directly summed or weighted summed) is used as the fusion loss. After obtaining the fusion loss, the parameters of the attribute analysis network can be updated using the backpropagation algorithm based on the fusion loss; the specific implementation can refer to existing technologies. It is also possible that in some implementations, the semantic loss is directly used as the fusion loss, in which case the step of calculating the classification loss in step S120 may not be necessary.

[0133] In actual training, batch training (e.g., mini-batch) is often used, where the sum of the fusion loss of all pedestrian images in a batch is calculated each time and the model parameters are updated.

[0134] In summary, Figure 3 The pedestrian attribute analysis method in this paper calculates not only the traditional classification loss based on image features, but also the semantic loss based on semantic context information when training the attribute analysis network. This allows the trained model to consider not only the influence of image features on attribute values ​​at the data level, but also the influence of semantic context information on attribute values ​​at the semantic level when classifying and predicting pedestrian attributes. Therefore, it can significantly improve the accuracy of the model in pedestrian attribute analysis.

[0135] Understandably, this method can well meet the needs of pedestrian attribute analysis in video surveillance, and of course, it can also be applied to other occasions besides video surveillance.

[0136] Figure 5 This application illustrates a third method for pedestrian attribute analysis provided in its embodiments. Figure 5 The methods in the text can be, but are not limited to, by Figure 10 The electronic device shown performs the operation; see the following section for details. Figure 10 The explanation. (Refer to...) Figure 5 The method includes:

[0137] Step S210: Obtain the pedestrian image to be analyzed.

[0138] Step S220: Utilize pedestrian attribute analysis methods during model training (i.e. Figure 3 The method and its various possible implementations) train an attribute analysis network to extract image features from pedestrian images, and classify and predict pedestrian attributes in pedestrian images based on image features to obtain attribute prediction results.

[0139] Figure 5 The steps of the method are similar to those of step S110 and will not be repeated here. The attribute prediction result in step S220 may refer to the confidence level of each attribute value, for example, "male" - 0.9, "female" - 0.1, or it may refer to the attribute prediction result finally output to the user, for example, "male" (the attribute value with the highest confidence level). This application does not strictly limit this.

[0140] Figure 5 The pedestrian attribute analysis method in [the text], when classifying and predicting pedestrian attributes, uses [the text] because [it] Figure 3 The attribute analysis network trained using the Chinese method has high prediction accuracy and is very suitable for pedestrian attribute analysis in the field of video surveillance.

[0141] Figure 6 The flowchart of the fourth pedestrian attribute analysis method provided in this application embodiment is shown. This flowchart can be regarded as an execution flow of the model inference task mentioned in step S02. Figure 7 This illustrates the detailed working principle of the method. Figure 6 The methods in the text can be, but are not limited to, by Figure 10 The electronic device shown performs the operation; see the following section for details. Figure 10 The explanation. (Refer to...) Figure 6 The method includes:

[0142] Step S310: Use an attribute analysis network to extract preliminary image features of pedestrian images.

[0143] The attribute analysis network is a pre-trained neural network model, and its training method can be... Figure 3 The training method shown (or its possible implementation) can also employ existing training methods for this type of network. Figure 7 For example, the attribute analysis network in step S310 may include a feature extraction network and an attribute classification network, whose functions and Figure 4 Similar to the network with the same name in the text, but please note that... Figure 4 The attribute classification network in the middle outputs the attribute prediction results, while Figure 7 The attribute classification network in the image outputs preliminary image features; this can be understood as follows. Figure 7 The attribute classification network in the middle is just Figure 4 The main part of the attribute classification network in the model, for example, will Figure 4 After removing the fully connected layers, classifiers, and other structures at the end of the attribute classification network, we can obtain... Figure 7 Attribute classification network in Figure 7 The preliminary image features output by the attribute classification network in the image can also be directly used to classify and predict pedestrian attributes in pedestrian images (e.g., directly input into fully connected layers, classifiers, etc.), but... Figure 6 In this method, the preliminary image features will be further processed (step S320). If Figure 7 If the attribute classification network in the image includes multiple branch networks, then the initial image features can also include multiple features.

[0144] Step S320: Obtain the semantic context information of pedestrian attributes in the pedestrian image, and perform calculations on the preliminary image features based on the semantic context information to obtain the final image features.

[0145] Regarding semantic context information, in the introduction Figure 1 As described in the previous section, the methods for obtaining semantic context information may differ somewhat between the inference and training phases.

[0146] For example, Figure 7 The semantic context information includes individual attribute associations, group attribute information, and spatiotemporal constraint information. Specifically, the individual attribute associations used in the inference phase do not necessarily need to be obtained through an attribute association extraction network. For example, a fixed individual attribute association can be determined based on the individual attribute associations obtained during the training phase (e.g., by calculating the mean), and this fixed individual attribute association can be directly used in the inference phase. This approach also reflects the statistical stability of the associations between attributes. Figure 7 In the diagram, there is no connection between the boxes "Individual Attribute Association" and "Feature Extraction Network," indicating that obtaining individual attribute associations does not require image features. Alternatively, fixed rules can be formed based on the individual attribute associations obtained during the training phase, i.e., individual attribute associations can be provided in a hard-coded form.

[0147] There are several methods for obtaining the group attribute information used in the inference stage: for example, it can be obtained by statistically analyzing some or all pedestrian images in the training set; it can also be obtained by statistically analyzing some or all pedestrian images that have already been inferred (i.e., whose attribute prediction results have been obtained); it can also be obtained by statistically analyzing some or all pedestrian images that have already been inferred plus some or all pedestrian images in the training set; or it can be obtained by calculating the previously obtained group attribute information, and so on. Of course, if the group attribute information is a specific rule set, it can be read directly. Since inference is a continuous process, inferred pedestrian images will be continuously generated. In some implementations, after obtaining a new attribute prediction result each time (i.e., after executing step S330), the new prediction result can be added to the statistics of the group attribute information to update the group attribute information. This makes the obtained group attribute information conform to the objective change law of group attributes, thereby helping to improve the accuracy of pedestrian attribute analysis.

[0148] Specifically, at the beginning of inference, since there are no inferred pedestrian images yet, it is impossible to statistically analyze group attribute information based solely on the inferred pedestrian images. Therefore, it is necessary to obtain initial group attribute information. For example, the group attribute information used in the training phase can be directly used as the initial group attribute information for the inference phase. Alternatively, the initial group attribute information for the inference phase can be calculated based on group attribute information obtained from other sources. For instance, if the pedestrian images to be analyzed in the inference phase are from city X, and the group attribute information for cities Y and Z was statistically analyzed during the inference of pedestrian images from cities Y and Z respectively, then the initial group attribute information for city X can be obtained by averaging the group attribute information from cities Y and Z.

[0149] For the spatiotemporal constraint information used in the inference stage, the spatiotemporal information corresponding to the pedestrian images can first be obtained from the cameras, platforms, or systems that collect pedestrian images. Then, using this spatiotemporal information as a priori constraint, the pedestrian attributes can be statistically analyzed. The statistical method can refer to the statistical method for group attribute information mentioned above. Of course, if the spatiotemporal constraint information is a specific rule set, it can be read directly.

[0150] After obtaining the semantic context information, the preliminary image features can be processed based on this information. The purpose is to integrate the semantic context information into the preliminary image features. The specific processing method is not limited; for example, it can be calculating the dot product, cross product, weighted operations, calculating L2 distance, etc. After the processing in step S320 is completed, the resulting image features are called the final image features.

[0151] Step S330: Classify and predict pedestrian attributes in the pedestrian image based on the final image features to obtain attribute prediction results.

[0152] The final image features can be directly input into fully connected layers, classifiers, and other structures to obtain attribute prediction results. Alternatively, pedestrian attribute prediction can be performed after further feature extraction of the final image features. This application does not limit the comparison.

[0153] In summary, Figure 6 The pedestrian attribute analysis method in this paper first extracts preliminary image features of pedestrian images using an attribute analysis network, which considers the influence of image features on attribute values ​​at the data level. Then, it further uses semantic context information to correct the preliminary attribute image features to obtain the final image features. Finally, it obtains the attribute prediction result based on the final image features, which considers the influence of semantic context information on attribute values ​​at the semantic level. Because it considers both data-level and semantic-level factors, the above method has high accuracy in predicting pedestrian attributes.

[0154] Furthermore, since the diversity and complexity of pedestrian images in the acquisition scenarios generally only affect the image features extracted by the model, without significantly affecting the semantic context information of pedestrian attributes in the pedestrian images, the above method is very suitable for pedestrian attribute analysis in the field of video surveillance, but is not limited to this field.

[0155] Figure 8 The flowchart of the fifth pedestrian attribute analysis method provided in this application embodiment is shown. This flowchart can be regarded as an execution flow of the post-processing task mentioned in step S02. Figure 9 This illustrates the detailed working principle of the method. Figure 8 The methods in the text can be, but are not limited to, by Figure 10 The electronic device shown performs the operation; see the following section for details. Figure 10 The explanation. (Refer to...) Figure 8 The method includes:

[0156] Step S410: Obtain the attribute prediction results of pedestrian attributes in the pedestrian image output by the attribute analysis network.

[0157] Among them, the attribute analysis network is a pre-trained neural network, and its training method can be adopted as follows: Figure 3 The training method shown (or its possible implementation) can also employ existing training methods for this type of network. Figure 9 For example, the attribute analysis network in step S410 may include a feature extraction network and an attribute classification network, whose functions and Figure 4 Similar to the network of the same name in the text, it will not be elaborated upon again.

[0158] Step S410 includes at least two cases: one is to directly obtain the already obtained attribute prediction results (i.e. Figure 8 The method described in step S110 only handles post-processing and does not calculate the attribute prediction results. Another method calculates the attribute prediction results in a similar manner to step S110, which will not be repeated here. The following mainly uses the latter case as an example. It should be noted that step S410 only obtains preliminary attribute prediction results, not the final attribute prediction results.

[0159] Step S420: Obtain the semantic context information of the pedestrian image, and correct the attribute prediction results based on the semantic context information to obtain the corrected attribute prediction results.

[0160] Regarding semantic context information, in the introduction Figure 1 The method described in the previous section has already been explained. The method for obtaining it can be referred to the description in step S320, and will not be repeated here.

[0161] After obtaining the semantic context information, the attribute prediction results can be corrected based on this information. The purpose is to adjust the values ​​of the attribute prediction results according to the semantic context. The specific correction method is not limited; for example, it can be performed using methods such as calculating the dot product, cross product, weighted operations, or calculating L2 distance. The corrected attribute prediction result obtained after the correction in step S420 can also be called the final attribute prediction result. Since the attribute prediction result has already been obtained in step S410, Figure 8 The method described can be viewed as a method used in the post-processing stage (the stage after the results are obtained in the inference stage).

[0162] In addition, each time a new final attribute prediction result is obtained (i.e. after executing step S420), the new final attribute prediction result can be added to the statistics of the group attribute information in order to update the group attribute information.

[0163] The following example illustrates the correction process for attribute prediction results, using the case where semantic context information includes associations of individual attributes:

[0164] For example, in a simpler implementation, the individual attribute associations in the post-processing stage can be hard-coded into rules. For instance, if the gender predicted in the attribute prediction result is "male" and the hairstyle is "long hair", then the hairstyle will be automatically corrected to "short hair" according to the hard-coded rules.

[0165] For example, in a slightly more complex implementation, the attribute prediction results can be corrected using dot product operations. Assume the attribute prediction results are:

[0166] Gender: "Male" - 0.9 "Female" - 0.1

[0167] Hairstyle: "Long hair" - 0.6 "Short hair" - 0.4

[0168] If we directly rely on the attribute prediction results, the pedestrian attributes given would be "male" and "long hair". Now, let's consider the correlation between individual attributes. Assuming that the correlation between gender and hairstyle in the attribute correlation heatmap is 0.8, we can specify this by setting the correlation between "male" and "short hair" and "female" and "long hair" to 0.8 respectively.

[0169] Therefore, the dot product operation can be performed on the attribute prediction results based on the association of individual attributes, as follows:

[0170]

[0171]

[0172] In this formula, 2 in the denominator represents the normalization factor. It's clear from the above formula that, considering the confidence level of 0.9 for "male" and the correlation between "male" and "short hair" is 0.8, the probability of predicting "long hair" decreases, while the probability of predicting "short hair" increases. The corrected attribute prediction result is:

[0173] Hairstyle: "Long hair" - 0.24 "Short hair" - 0.76

[0174] According to the corrected attribute prediction results, the pedestrian's attributes can be given as "male" and "short hair". Although it is possible for a "male" to have "long hair" in reality, it is more likely due to inaccurate model prediction. Overall, the corrected attribute prediction results are more accurate.

[0175] In summary, Figure 8The pedestrian attribute analysis method in the text first uses an attribute analysis network to obtain preliminary attribute prediction results when post-processing pedestrian attributes. This means that the influence of image features on attribute values ​​at the data level is considered. Then, semantic context information is used to further correct the preliminary attribute prediction results to obtain the final attribute prediction results. This means that the influence of semantic context information on attribute values ​​at the semantic level is considered. Because both data-level and semantic-level factors are considered, the above method has high accuracy in predicting pedestrian attributes.

[0176] Furthermore, since the diversity and complexity of pedestrian images in the acquisition scenarios generally only affect the image features extracted by the model, without significantly affecting the semantic context information of the pedestrian images, the above method is very suitable for pedestrian attribute analysis in the field of video surveillance, but is not limited to this field.

[0177] about Figure 3 , Figure 5 , Figure 6 , Figure 8 The improvement effect of the pedestrian attribute analysis method on the problems (1)-(4) mentioned above can be seen in the previous section on the improvement effect of the pedestrian attribute analysis method on the problems (1)-(4) mentioned above. Figure 1 The analysis of the Bank of China's attribute analysis method will not be repeated here.

[0178] In summary, the key to this application's solution lies in using semantic context information for pedestrian attribute analysis. Depending on the stage at which semantic context information is used, this application's solution has at least the following five typical forms:

[0179] (1) Use it during the training phase, but not during the inference phase or the post-processing phase;

[0180] (2) Use it during the training phase, but not during the inference phase, and use it during the post-processing phase;

[0181] (3) Used in the inference phase, but not in the training and post-processing phases;

[0182] (4) Used in both the training and inference phases, but not in the post-processing phase;

[0183] (5) It is not used in the training and inference phases, but is used in the post-processing phase.

[0184] The above five forms are combined Figure 3 , Figure 5 , Figure 6 , Figure 8 The method can be obtained, and will not be elaborated further.

[0185] Figure 10This illustration shows a structure of a pedestrian attribute analysis device 500 provided in an embodiment of this application. (Refer to...) Figure 10 The pedestrian attribute analysis device 500 includes:

[0186] The information acquisition module 510 is used to acquire semantic context information of pedestrian attributes in pedestrian images;

[0187] The attribute analysis module 520 is used to perform pedestrian attribute analysis tasks using the semantic context information;

[0188] The semantic context information is obtained by analyzing the pedestrian image or its associated information, and includes at least one of the following: individual attribute association, including the statistical laws governing the association between multiple pedestrian attributes of an individual pedestrian, or a specific set of rules that they conform to; group attribute information, including the statistical laws governing at least one pedestrian attribute of a group of pedestrians, or a specific set of rules that they conform to; and spatiotemporal constraint information, including the statistical laws governing the association between the spatiotemporal information corresponding to the pedestrian image and at least one pedestrian attribute, or a specific set of rules that they conform to, wherein the spatiotemporal information includes at least one piece of information in time, space, and scene.

[0189] The pedestrian attribute analysis task includes at least one of the following tasks: Model training task: Input the pedestrian image and pedestrian attribute labels, and train a neural network model for inferring pedestrian attributes by combining the semantic context information of the pedestrian image; Model inference task: Input the pedestrian image, and infer pedestrian attributes using the neural network model by combining the semantic context information of the pedestrian image; Post-processing task: Input the inference results of the neural network model for pedestrian attributes in the pedestrian image, and correct the inference results by combining the semantic context information of the pedestrian image.

[0190] In one implementation of the pedestrian attribute analysis device 500, the statistical regularities in the individual attribute association include: an attribute association heatmap among multiple pedestrian attributes of a pedestrian individual, the attribute association heatmap including numerical values ​​representing the degree of correlation between each pair of pedestrian attributes; or, a correlation coefficient among multiple pedestrian attributes of a pedestrian individual; or, a conditional probability distribution that multiple pedestrian attributes of a pedestrian individual follow; or, a joint distribution that multiple pedestrian attributes of a pedestrian individual follow; the specific rule set in the individual attribute association includes: at least one rule describing the multiple pedestrian attributes being constrained by the correlation between attributes.

[0191] In one implementation of the pedestrian attribute analysis device 500, the statistical regularities in the group attribute information include: the statistical distribution followed by at least one pedestrian attribute of the pedestrian group; or, the statistical quantity possessed by at least one pedestrian attribute of the pedestrian group, wherein the statistical quantity includes at least one of the following: mean, variance, covariance, maximum value, minimum value, higher-order moments, higher-order cumulants, and higher-order cumulant spectrum; the specific rule set in the group attribute information includes: at least one rule describing the group constraint of the pedestrian attribute itself.

[0192] In one implementation of the pedestrian attribute analysis device 500, the statistical regularities in the spatiotemporal constraint information include: a statistical distribution that at least one pedestrian attribute follows, with the spatiotemporal information corresponding to the pedestrian image as a priori constraint; or, a statistical quantity possessed by at least one pedestrian attribute, the statistical quantity including at least one of the following: mean, variance, covariance, maximum value, minimum value, higher-order moments, higher-order cumulants, and higher-order cumulant spectrum; the specific rule set in the spatiotemporal constraint information includes: at least one rule describing the spatiotemporal constraints on pedestrian attributes.

[0193] In one implementation of the pedestrian attribute analysis device 500, the pedestrian image is a training sample in the training set. The attribute analysis module 520 uses the semantic context information to perform the model training task, including: extracting image features of the pedestrian image using an attribute analysis network, and classifying and predicting pedestrian attributes in the pedestrian image based on the image features to obtain attribute prediction results; wherein, the attribute analysis network is a neural network model to be trained; calculating a classification loss based on the difference between the attribute prediction results and the pedestrian attribute labels of the pedestrian image, and calculating a semantic loss based on the semantic context information; calculating a fusion loss based on the classification loss and the semantic loss, and updating the parameters of the attribute analysis network based on the fusion loss.

[0194] In one implementation of the pedestrian attribute analysis device 500, the attribute analysis network includes a feature extraction network and an attribute classification network connected to the feature extraction network. The attribute analysis module 520 uses the attribute analysis network to extract image features of the pedestrian image and performs classification prediction on the pedestrian attributes in the pedestrian image based on the image features to obtain attribute prediction results. This includes: extracting image features of the pedestrian image using the feature extraction network; and performing classification prediction on the pedestrian attributes in the pedestrian image based on the image features using the attribute classification network to obtain the attribute prediction results. The semantic context information includes individual attribute associations. The attribute analysis network also includes an attribute association extraction network connected to the feature extraction network. The information acquisition module 510 acquires the semantic context information of the pedestrian attributes in the pedestrian image, including: extracting individual attribute associations of the pedestrian attributes in the pedestrian image based on the image features using the attribute association extraction network.

[0195] In one implementation of the pedestrian attribute analysis device 500, the semantic context information includes individual attribute associations. The attribute analysis module 520 calculates semantic loss based on the semantic context information, including: calculating the average individual attribute association of all pedestrian images in the training batch to which the pedestrian image belongs; and calculating a first semantic loss in the semantic loss based on the difference between the individual attribute associations and the average individual attribute associations.

[0196] In one implementation of the pedestrian attribute analysis device 500, the semantic context information includes group attribute information. The information acquisition module 510 acquires the semantic context information of pedestrian attributes in pedestrian images, including: obtaining group attribute information of pedestrian attributes in pedestrian images by statistically analyzing at least one pedestrian attribute in some or all pedestrian images in the training set.

[0197] In one implementation of the pedestrian attribute analysis device 500, the semantic context information includes group attribute information. The attribute analysis module 520 calculates semantic loss based on the semantic context information, including: calculating a group attribute prediction result, where the group attribute prediction result represents the pedestrian attributes in the pedestrian image predicted based on the group attribute information; and calculating a second semantic loss in the semantic loss based on the difference between the attribute prediction result and the group attribute prediction result.

[0198] In one implementation of the pedestrian attribute analysis device 500, the semantic context information includes spatiotemporal constraint information. The information acquisition module 510 acquires the semantic context information of pedestrian attributes in the pedestrian image, including: acquiring the spatiotemporal information corresponding to the pedestrian image from a camera or monitoring system; using the spatiotemporal information as a priori constraint, and obtaining the spatiotemporal constraint information of pedestrian attributes in the pedestrian image by statistically analyzing at least one pedestrian attribute in some or all pedestrian images in the training set.

[0199] In one implementation of the pedestrian attribute analysis device 500, the semantic context information includes spatiotemporal information. The attribute analysis module 520 calculates semantic loss based on the semantic context information, including: calculating a spatiotemporal attribute prediction result, whereby the spatiotemporal attribute prediction result represents the pedestrian attributes in the pedestrian image predicted based on the spatiotemporal constraint information; and calculating a third semantic loss in the semantic loss based on the difference between the attribute prediction result and the spatiotemporal attribute prediction result.

[0200] In one implementation of the pedestrian attribute analysis device 500, the attribute analysis module 520 performs the model inference task using the semantic context information, including: extracting preliminary image features of the pedestrian image using an attribute analysis network; wherein the attribute analysis network is a trained neural network model, and the preliminary image features are used to classify and predict pedestrian attributes in the pedestrian image; performing calculations on the preliminary image features according to the semantic context information to obtain final image features; and classifying and predicting pedestrian attributes in the pedestrian image according to the final image features to obtain attribute prediction results.

[0201] In one implementation of the pedestrian attribute analysis device 500, the semantic context information includes group attribute information. The information acquisition module 510 acquires the semantic context information of pedestrian attributes in pedestrian images, including: obtaining group attribute information of pedestrian attributes in pedestrian images by statistically analyzing some or all pedestrian images in the training set, and / or some or all inferred pedestrian images; or, obtaining group attribute information of pedestrian attributes in pedestrian images by calculating previously obtained existing group attribute information; wherein, the inferred pedestrian images refer to pedestrian images that have already undergone pedestrian attribute prediction, and the training set refers to the set of pedestrian images used to train the attribute analysis network.

[0202] In one implementation of the pedestrian attribute analysis device 500, the attribute analysis module 520 performs the post-processing task using the semantic context information, including: obtaining the attribute prediction result of the attribute analysis network for the pedestrian attributes in the pedestrian image; wherein, the attribute analysis network is a trained neural network model; and correcting the attribute prediction result according to the semantic context information to obtain the corrected attribute prediction result.

[0203] The pedestrian attribute analysis device 500 provided in this application embodiment has been described in the foregoing method embodiment in terms of its implementation principle and technical effects. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the method embodiment.

[0204] Figure 11 This illustration shows a possible structure of the electronic device 600 provided in an embodiment of this application. (Refer to...) Figure 11 The electronic device 600 includes a processor 610, a memory 620, and a communication interface 630. These components are interconnected and communicate with each other via a communication bus 640 and / or other forms of connection mechanism (not shown).

[0205] The memory 620 includes one or more (only one is shown in the figure), which may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 610 and other possible components may access the memory 620 to read and / or write data therein.

[0206] Processor 610 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 610 can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Network Processor (NP), or other conventional processors; it can also be a special-purpose processor, including a Graphics Processing Unit (GPU), a Neural-network Processing Unit (NPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Furthermore, when there are multiple processors 610, some can be general-purpose processors and others can be special-purpose processors.

[0207] Communication interface 630 includes one or more (only one is shown in the figure) that can be used to communicate directly or indirectly with other devices for data exchange. Communication interface 630 may include interfaces for wired and / or wireless communication.

[0208] One or more computer program instructions may be stored in the memory 620. The processor 610 may read and run these computer program instructions to implement the model training method and / or pedestrian attribute analysis method provided in the embodiments of this application.

[0209] Understandable. Figure 11 The structure shown is for illustrative purposes only; the electronic device 600 may also include more than [other components]. Figure 11 The more or fewer components shown, or having the same Figure 11 The different configurations shown. Figure 11 The components shown can be implemented using hardware, software, or a combination thereof. Electronic device 600 may be a physical device, such as a PC, laptop, tablet, mobile phone, server, or embedded device, or a virtual device, such as a virtual machine or virtualization container. Furthermore, electronic device 600 is not limited to a single device; it can also be a combination of multiple devices or a cluster of numerous devices.

[0210] This application also provides a computer-readable storage medium storing computer program instructions. These instructions are read and executed by a computer's processor to perform the model training method and / or pedestrian attribute analysis method provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 11 The memory 620 in the electronic device 600.

[0211] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for pedestrian attribute analysis, characterized in that, include: Obtain semantic context information of pedestrian attributes in pedestrian images; The semantic context information is used to perform pedestrian attribute analysis tasks; The semantic context information is obtained by analyzing the pedestrian image or its associated information, and includes at least one of the following: Individual attribute association includes the statistical laws governing the associations between various pedestrian attributes of an individual, or the specific set of rules in the individual attribute association that they conform to; wherein, the specific set of rules in the individual attribute association includes: at least one rule describing the constraints of the correlation between various pedestrian attributes; Group attribute information includes statistical laws governing at least one pedestrian attribute of a pedestrian group, or a specific set of rules conforming to the group attribute information; wherein, the specific set of rules in the group attribute information includes: at least one rule describing the group-based constraint of the pedestrian attribute on itself; Spatiotemporal constraint information includes statistical laws governing the association between spatiotemporal information corresponding to a pedestrian image and at least one pedestrian attribute, or a specific set of rules conforming to the spatiotemporal constraint information, wherein the spatiotemporal information includes at least one piece of information in time, space, and scene; wherein the specific set of rules in the spatiotemporal constraint information includes at least one rule describing the spatiotemporal constraint on pedestrian attributes. The pedestrian attribute analysis task includes at least one of the following tasks: Model training task: Input the pedestrian image and pedestrian attribute labels, and combine the semantic context information of the pedestrian image to train a neural network model for inferring pedestrian attributes; Model inference task: Input the pedestrian image and, in conjunction with the semantic context information of the pedestrian image, use a neural network model to infer the pedestrian attributes; Post-processing task: Input the inference results of the neural network model on the pedestrian attributes in the pedestrian image, and combine the semantic context information of the pedestrian image to correct the inference results.

2. The pedestrian attribute analysis method according to claim 1, characterized in that, The statistical regularities in the association of individual attributes include: A heatmap showing the correlation between various pedestrian attributes of an individual pedestrian, wherein the heatmap includes numerical values ​​representing the degree of correlation between each pair of pedestrian attributes; Alternatively, the correlation coefficients between various pedestrian attributes of an individual pedestrian; Alternatively, the conditional probability distribution that a pedestrian's various attributes follow; Alternatively, the joint distribution that a pedestrian's various pedestrian attributes follow.

3. The pedestrian attribute analysis method according to claim 1, characterized in that, The statistical regularities in the group attribute information include: The statistical distribution of at least one pedestrian attribute of a pedestrian group; Alternatively, a statistical measure of at least one pedestrian attribute of a pedestrian group, the statistical measure including at least one of the following: mean, variance, covariance, maximum value, minimum value, higher-order moments, higher-order cumulants, and higher-order cumulant spectrum.

4. The pedestrian attribute analysis method according to claim 1, characterized in that, The statistical regularities in the spatiotemporal constraint information include: At least one pedestrian attribute follows a statistical distribution constrained by the spatiotemporal information corresponding to the pedestrian image; Alternatively, at least one statistic possessed by a pedestrian attribute, said statistic including at least one of the following: mean, variance, covariance, maximum value, minimum value, higher-order moments, higher-order cumulants, and higher-order cumulant spectrum.

5. The pedestrian attribute analysis method according to claim 1, characterized in that, The pedestrian images are training samples in the training set. The model training task is performed using the semantic context information, including: The image features of the pedestrian image are extracted using an attribute analysis network, and the attributes of the pedestrians in the pedestrian image are classified and predicted based on the image features to obtain the attribute prediction results; wherein, the attribute analysis network is a neural network model to be trained; The classification loss is calculated based on the difference between the attribute prediction result and the pedestrian attribute label of the pedestrian image, and the semantic loss is calculated based on the semantic context information. The fusion loss is calculated based on the classification loss and the semantic loss, and the parameters of the attribute analysis network are updated based on the fusion loss.

6. The pedestrian attribute analysis method according to claim 5, characterized in that, The attribute analysis network includes a feature extraction network and an attribute classification network connected to the feature extraction network. The step of using the attribute analysis network to extract image features from the pedestrian image and classifying and predicting pedestrian attributes in the pedestrian image based on the image features to obtain attribute prediction results includes: The feature extraction network is used to extract image features from the pedestrian image; Based on the image features, the attribute classification network is used to classify and predict the pedestrian attributes in the pedestrian image to obtain the attribute prediction results. The semantic context information includes individual attribute associations, and the attribute analysis network further includes an attribute association extraction network connected to the feature extraction network. Obtaining the semantic context information of pedestrian attributes in the pedestrian image includes: Based on the image features, the attribute association extraction network is used to extract individual attribute associations of pedestrian attributes in the pedestrian image.

7. The pedestrian attribute analysis method according to claim 5 or 6, characterized in that, The semantic context information includes individual attribute associations, and the calculation of semantic loss based on the semantic context information includes: Calculate the average individual attribute association of all pedestrian images in the training batch to which the pedestrian image belongs; The first semantic loss in the semantic loss is calculated based on the difference between the individual attribute association and the average individual attribute association.

8. The pedestrian attribute analysis method according to claim 5, characterized in that, The semantic context information includes group attribute information, and the step of obtaining the semantic context information of pedestrian attributes in the pedestrian image includes: By statistically analyzing at least one pedestrian attribute in some or all pedestrian images in the training set, group attribute information of pedestrian attributes in pedestrian images can be obtained.

9. The pedestrian attribute analysis method according to claim 5 or 8, characterized in that, The semantic context information includes group attribute information, and the step of calculating semantic loss based on the semantic context information includes: Calculate the group attribute prediction result, wherein the group attribute prediction result represents the pedestrian attributes in the pedestrian image predicted based on the group attribute information; The second semantic loss in the semantic loss is calculated based on the difference between the attribute prediction result and the group attribute prediction result.

10. The pedestrian attribute analysis method according to claim 5, characterized in that, The semantic context information includes spatiotemporal constraint information, and the step of obtaining the semantic context information of pedestrian attributes in the pedestrian image includes: Obtain the spatiotemporal information corresponding to the pedestrian images from cameras or surveillance systems; Using the spatiotemporal information as a priori constraints, spatiotemporal constraint information of pedestrian attributes in pedestrian images is obtained by statistically analyzing at least one pedestrian attribute in some or all pedestrian images in the training set.

11. The pedestrian attribute analysis method according to claim 5 or 10, characterized in that, The semantic context information includes spatiotemporal information, and the step of calculating semantic loss based on the semantic context information includes: Calculate the spatiotemporal attribute prediction result, which represents the pedestrian attributes in the pedestrian image predicted based on the spatiotemporal constraint information; The third semantic loss in the semantic loss is calculated based on the difference between the attribute prediction result and the spatiotemporal attribute prediction result.

12. The pedestrian attribute analysis method according to claim 1, characterized in that, Utilizing the semantic context information to perform the model inference task includes: Preliminary image features of the pedestrian image are extracted using an attribute analysis network; wherein, the attribute analysis network is a trained neural network model, and the preliminary image features are used to classify and predict pedestrian attributes in the pedestrian image; The initial image features are processed based on the semantic context information to obtain the final image features; Based on the final image features, the pedestrian attributes in the pedestrian image are classified and predicted to obtain the attribute prediction results.

13. The pedestrian attribute analysis method according to claim 12, characterized in that, The semantic context information includes group attribute information, and the step of obtaining the semantic context information of pedestrian attributes in the pedestrian image includes: By statistically analyzing some or all pedestrian images in the training set, and / or some or all inferred pedestrian images, group attribute information of pedestrian attributes in pedestrian images can be obtained; or, By calculating the previously obtained group attribute information, the group attribute information of pedestrian attributes in the pedestrian image is obtained; Here, the inferred pedestrian image refers to a pedestrian image that has already undergone pedestrian attribute prediction, and the training set refers to the set of pedestrian images used to train the attribute analysis network.

14. The pedestrian attribute analysis method according to claim 1, characterized in that, Utilizing the semantic context information to perform the post-processing task includes: Obtain the attribute prediction results of the pedestrian attributes in the pedestrian image output by the attribute analysis network; wherein, the attribute analysis network is a trained neural network model; The attribute prediction result is corrected based on the semantic context information to obtain the corrected attribute prediction result.

15. A pedestrian attribute analysis device, characterized in that, include: The information acquisition module is used to acquire semantic context information of pedestrian attributes in pedestrian images; The attribute analysis module is used to perform pedestrian attribute analysis tasks using the semantic context information; The semantic context information is obtained by analyzing the pedestrian image or its associated information, and includes at least one of the following: Individual attribute association includes the statistical laws governing the associations between various pedestrian attributes of an individual, or the specific set of rules in the individual attribute association that they conform to; wherein, the specific set of rules in the individual attribute association includes: at least one rule describing the constraints of the correlation between various pedestrian attributes; Group attribute information includes statistical laws governing at least one pedestrian attribute of a pedestrian group, or a specific set of rules conforming to the group attribute information; wherein, the specific set of rules in the group attribute information includes: at least one rule describing the group-based constraint of the pedestrian attribute on itself; Spatiotemporal constraint information includes statistical laws governing the association between spatiotemporal information corresponding to a pedestrian image and at least one pedestrian attribute, or a specific set of rules conforming to the spatiotemporal constraint information, wherein the spatiotemporal information includes at least one piece of information in time, space, and scene; wherein the specific set of rules in the spatiotemporal constraint information includes at least one rule describing the spatiotemporal constraint on pedestrian attributes. The pedestrian attribute analysis task includes at least one of the following tasks: Model training task: Input the pedestrian image and pedestrian attribute labels, and combine the semantic context information of the pedestrian image to train a neural network model for inferring pedestrian attributes; Model inference task: Input the pedestrian image and, in conjunction with the semantic context information of the pedestrian image, use a neural network model to infer the pedestrian attributes; Post-processing task: Input the inference results of the neural network model on the pedestrian attributes in the pedestrian image, and combine the semantic context information of the pedestrian image to correct the inference results.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-14.

17. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores computer program instructions, which are read and executed by the processor to perform the method of any one of claims 1-14.

Images