Pedestrian attribute recognition method, electronic device, and storage medium

Abstract

The application discloses a pedestrian attribute recognition method, an electronic device and a storage medium. The method comprises the following steps: acquiring a pedestrian image; recognizing the pedestrian image by using an attribute recognition model to obtain a confidence result of the pedestrian image, wherein the confidence result comprises confidence degrees of the pedestrian image belonging to each category under the same attribute; acquiring reference probabilities of each category; adjusting the confidence degree of the corresponding category based on the reference probability of each category; and determining the category of the pedestrian image based on the adjusted confidence degree. In the foregoing manner, the accuracy of the category recognition of the pedestrian image can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a pedestrian attribute recognition method, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, in order to meet the increasingly high safety needs of people, monitoring systems are deployed in streets and alleys to obtain video data through the monitoring systems and analyze the video data to determine whether there is illegal behavior. The increase in monitoring systems means that the amount of video data obtained increases exponentially, increasing the difficulty of analyzing the video data.

[0003] Among them, pedestrians are one of the targets that need to be analyzed in video data. The existing method is to analyze pedestrians in video data through a pedestrian attribute recognition method to obtain the category of pedestrians, so as to reduce the burden of manual observation and improve efficiency. However, the accuracy of the existing pedestrian attribute recognition method is not high. SUMMARY

[0004] The present application provides a pedestrian attribute recognition method, an electronic device and a storage medium, which can solve the problem of low accuracy of the category obtained by the existing pedestrian attribute recognition method.

[0005] To solve the above technical problems, one technical solution adopted by the present application is to provide a pedestrian attribute recognition method. The method comprises: obtaining a pedestrian image; identifying the pedestrian image using an attribute recognition model to obtain a confidence result of the pedestrian image, wherein the confidence result comprises the confidence of each category of the pedestrian image belonging to the same attribute; obtaining a reference probability of each category; adjusting the confidence of the corresponding category based on the reference probability of each category; and determining the category of the pedestrian image based on the adjusted confidence.

[0006] To solve the above technical problems, another technical solution adopted by the present application is to provide an electronic device, which comprises a processor and a memory connected to the processor, wherein the memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the above method.

[0007] To solve the above technical problems, still another technical solution adopted by the present application is to provide a storage medium storing program instructions, which can implement the above method when executed.

[0008] Through the above method and the implementation of this embodiment, the confidence result of the pedestrian image is obtained through the attribute recognition model in this application. However, instead of directly using the category of the obtained confidence result as the category of the pedestrian image, the confidence of each category included in the confidence result is adjusted using reference probability, and the category of the adjusted confidence result is used as the category of the pedestrian image, which can improve the accuracy of the category recognition of the pedestrian image. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating an embodiment of the pedestrian attribute recognition method of this application;

[0010] Figure 2 This is a flowchart illustrating Embodiment 2 of the pedestrian attribute recognition method of this application;

[0011] Figure 3 This is a flowchart illustrating Embodiment 3 of the pedestrian attribute recognition method of this application;

[0012] Figure 4 yes Figure 3 A detailed flowchart of the S32 process;

[0013] Figure 5 yes Figure 4 A detailed flowchart of the S322 process;

[0014] Figure 6 This is a flowchart illustrating Embodiment 4 of the pedestrian attribute recognition method of this application;

[0015] Figure 7 yes Figure 6 A detailed flowchart of the S42 process;

[0016] Figure 8 yes Figure 7 A detailed flowchart of the S421 process;

[0017] Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application;

[0018] Figure 10 This is a schematic diagram of the structure of an embodiment of the storage medium of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] The terms "first", "second", "third", etc., in the present application are only for descriptive purpose and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Thus, the features defined with "first", "second", "third" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "plurality" is at least two, for example, two, three, etc., unless otherwise explicitly and specifically limited.

[0021] Reference to "embodiments" in this document means that a particular feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of the application. The occurrence of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily independent or alternative embodiments to other embodiments. It is explicitly and implicitly understood by those skilled in the art that an embodiment described herein can be combined with another embodiment unless there is a conflict.

[0022] Figure 1 is a flowchart of an embodiment of the pedestrian attribute recognition method of the present application. It should be noted that the present embodiment is not limited to the order of the flowchart shown in Figure 1 . As shown in Figure 1 , the present embodiment can include:

[0023] S11: Obtain a pedestrian image.

[0024] The pedestrian image can be an image containing a pedestrian, which can be obtained by a camera device arranged in a monitoring area.

[0025] S12: Use an attribute recognition model to recognize the pedestrian image to obtain a confidence result of the pedestrian image.

[0026] The confidence result includes the confidence of each category of the pedestrian image belonging to the same attribute.

[0027] The attributes of the pedestrian can include color, gender, age, etc. Each attribute can include multiple categories. For example, color can include three categories of red, green and yellow, gender can include two categories of male and female, and age can include multiple categories of 1-18 years old, 19-30 years old, etc.

[0028] The attribute recognition model can be used to recognize the pedestrian image to obtain the confidence result of each attribute of the pedestrian image. The attribute recognition model can be a trained neural network model, which can recognize multiple attributes of the pedestrian image or only recognize a single attribute of the pedestrian image. In other words, the attribute recognition models used to recognize different attributes of the pedestrian image in this step can be the same or different. The following of the present application is described for one attribute of the pedestrian image.

[0029] The category corresponding to the highest confidence score in the confidence result is the category obtained through the attribute recognition model. The confidence result can be a confidence vector composed of the confidence scores for each category. For example, the confidence vector for attribute k. in For category 1 confidence level, For category 2 confidence level, ..., Let be the confidence level for category c. The confidence level is the highest, therefore the category of this confidence level result is 1.

[0030] However, considering that the confidence scores obtained through the attribute recognition model may be inaccurate, the categories of the confidence scores obtained through the attribute recognition model may also be inaccurate. Therefore, this application does not directly use the categories of the confidence scores obtained through the attribute recognition model as the categories of pedestrian images, but instead first adjusts the confidence scores obtained through the attribute recognition model as follows.

[0031] S13: Obtain the reference probability for each category.

[0032] For each category under the same attribute, there is a corresponding reference probability. Please refer to the following examples for details on how to obtain this probability.

[0033] S14: Adjust the confidence level of the corresponding category based on the reference probability of each category.

[0034] Please refer to the following examples for a detailed description of this step.

[0035] S15: Determine the category of pedestrian images based on adjusted confidence levels.

[0036] After adjusting the confidence scores for each category, the category corresponding to the highest adjusted confidence score can be determined as the category of the pedestrian image. For example, in the confidence score results obtained by the attribute recognition model, the highest confidence score is... In the adjusted confidence results, the highest confidence level is Therefore, category 1 is determined as the category for pedestrian images.

[0037] Understandably, after adjusting the confidence level of the corresponding category based on the reference probability of each category, that is, after adjusting the confidence level of each category included in the confidence level result, if the category of the confidence level result does not change, it means that the confidence level result obtained by the attribute recognition model is reliable; if the category corresponding to the maximum confidence level in the confidence level result changes, it means that the confidence level result obtained by the attribute recognition model is unreliable.

[0038] Through the implementation of this embodiment, the confidence result of pedestrian image is obtained through attribute recognition model. However, instead of directly using the category of the obtained confidence result as the category of pedestrian image, the confidence of each category included in the confidence result is adjusted using reference probability. The category of the adjusted confidence result is then used as the category of pedestrian image, which can improve the accuracy of pedestrian image category recognition.

[0039] Figure 2 This is a flowchart illustrating Embodiment Two of the pedestrian attribute recognition method of this application. It should be noted that if substantially the same result is obtained, this embodiment is not necessarily identical. Figure 2 The illustrated process sequence is limited. This embodiment is a further extension of Embodiment 1. Figure 2 As shown, this embodiment may include:

[0040] S21: Obtain pedestrian images.

[0041] S22: Use an attribute recognition model to identify pedestrian images and obtain the confidence results of the pedestrian images.

[0042] The confidence results include the confidence that pedestrian images belong to different categories under the same attribute.

[0043] S23: Determine whether the confidence level of each category is less than the confidence threshold.

[0044] If so, then execute S24.

[0045] Understandably, if the confidence scores of all categories of pedestrian images obtained through the attribute recognition model are less than the confidence threshold, it means that the attribute recognition model has a weak ability to distinguish the pedestrian image and is more likely to make mistakes. The confidence scores of the pedestrian image obtained are not accurate enough. Only in this case should the confidence scores be adjusted to be more effective.

[0046] S24: Obtain the reference probability for each category.

[0047] S25: Adjust the confidence level of the corresponding category based on the reference probability of each category.

[0048] S26: Determine the category of pedestrian images based on adjusted confidence levels.

[0049] For further detailed descriptions of this step, please refer to Example 1, which will not be repeated here.

[0050] Figure 3 This is a flowchart illustrating Embodiment 3 of the pedestrian attribute recognition method of this application. It should be noted that if substantially the same result is obtained, this embodiment is not necessarily identical. Figure 3 The illustrated process sequence is limited. This embodiment is a further extension of S13 / S24, such as... Figure 3As shown, this embodiment may include:

[0051] S31: Obtain the difference between the confidence results of the pedestrian images and the confidence results of at least one reference image.

[0052] Understandably, a reference image set is pre-established prior to step S31, and this set may include multiple reference images. When adjusting the confidence level of the preset images, the confidence level results of the reference images can be used as a reference. The confidence level results of the reference images are also obtained by the attribute recognition model.

[0053] Specifically, the confidence scores of some or all reference images in the reference image set can be used as a reference to adjust the confidence scores of the pedestrian images. More specifically, the feature similarity between the pedestrian images and the reference images can be measured based on the difference between their confidence scores. This difference can be represented as the distance between the confidence vectors of the pedestrian images and the reference images; a larger distance indicates a greater difference.

[0054] Therefore, S31 may include: obtaining the distance between the confidence vector of the pedestrian image and the confidence vector of at least one reference image.

[0055] For example, the confidence vector of attribute k of a pedestrian image is p = [p1, p2, ..., p c The confidence vector of the reference image is T = [T1, T2, ..., T]. c ], where T1 is the confidence level of category 1, T2 is the confidence level of category 2, ..., T c The confidence score for category c. The distance between T and p.

[0056] S32: Take the confidence results of a preset number of reference images whose differences meet the preset difference requirements as the reference confidence results, and obtain the reference probabilities of each category based on the reference confidence results.

[0057] The preset difference requirement can be the first preset number of reference images with the smallest difference. Therefore, the confidence results of the preset number of reference images whose differences meet the preset difference requirement are used as the reference confidence results, that is, the confidence results of the preset number of reference images with the smallest confidence vector distance to the pedestrian image are used as the reference confidence results.

[0058] The preset number can be K. In this case, the method described above for obtaining K reference confidence results can be called the K nearest neighbor method.

[0059] See also Figure 4In step S32, obtaining the reference probabilities for each category based on the reference confidence results may include the following sub-steps:

[0060] S321: Determine the category of the reference confidence result.

[0061] The category of the reference confidence result can be the category corresponding to the highest confidence level in the reference confidence result.

[0062] S322: Based on the relationship between the maximum confidence score in the reference confidence score results of the same category and the confidence scores of the corresponding category in all reference confidence score results, obtain the reference probability of the corresponding category.

[0063] See also Figure 5 S322 may include the following sub-steps:

[0064] S3221: Obtain the first sum of the maximum confidence scores among the reference confidence scores of the same category, and obtain the second sum of the confidence scores of the corresponding categories among all reference confidence scores.

[0065] S3222: Obtain the first ratio between the first sum and the second sum, as a reference probability for the corresponding category.

[0066] The following explanation uses category a as an example to illustrate S3221-S3222:

[0067] There are K reference confidence results, and K reference confidence results for category a. a One. Get and in The maximum confidence score (confidence score of category a) is the m-th reference confidence score result for category a. Let $\frac{n}{n}$ be the confidence score for category $a$ in the nth reference confidence score result out of all reference confidence scores. The reference probability for category $a$ is $\frac{n}{n}$.

[0068] Figure 6 This is a flowchart illustrating Embodiment 4 of the pedestrian attribute recognition method of this application. It should be noted that if substantially the same result is obtained, this embodiment is not necessarily identical. Figure 6 The illustrated process sequence is limited. This embodiment is a further extension of S14 / S25, such as... Figure 6 As shown, this embodiment may include:

[0069] S41: Obtain the product between the confidence level of each category and the reference probability of the corresponding category.

[0070] S42: Adjust the confidence level of each category based on the product corresponding to each category.

[0071] See also Figure 7S42 may include the following sub-steps:

[0072] S421: Normalize the product corresponding to each category to obtain the normalized result for each category.

[0073] See also Figure 8 S421 may include the following sub-steps:

[0074] S4211: Get the sum of the products corresponding to all categories.

[0075] S4212: For each category, obtain the second ratio of its corresponding product to the sum of products, as the result of the normalization process.

[0076] S422: Replace the confidence scores of each category with the corresponding normalization results.

[0077] The following description will still use category a as an example to illustrate this embodiment:

[0078] The confidence level of category a is p a The reference probability for category a is T(a). The product of the confidence level and the reference probability for category a is p. a T(a), the sum of the products of the confidence level and the reference probability for each category is The second ratio is The confidence level p of category a a Replace with the second ratio p′ a .

[0079] Before using the trained attribute recognition model in the above embodiments, the attribute recognition model can be validated. Specifically, this can be done as follows:

[0080] Obtain a validation image set, which may include multiple validation images, each with a true class label. The validation image set is independent and identically distributed with the training image set used to train the attribute recognition model. Use the attribute recognition model to identify each validation image and obtain the confidence score result for each validation image. Evaluate the performance of the attribute recognition model based on the category of the confidence score result. If the confidence scores of all categories included in the confidence score result of the validation image are less than the confidence score threshold (hereinafter referred to as hard samples), it means that the model has a weak ability to distinguish that type of validation image.

[0081] Before the pedestrian attribute recognition method provided in the above embodiments is actually put into use, it can be tested to evaluate its effectiveness. Specifically, it can be as follows:

[0082] A test image set is obtained, which may include multiple test images, each with a true class label. The test image set and the validation image set are independent and identically distributed, so the hard samples in the test image set and the hard samples in the validation image set have similar characteristics. The pedestrian attribute recognition method mentioned above is used to process each test image to obtain the category of each test image. The category of each test image is compared with the true category, and the effectiveness of the pedestrian attribute recognition method is evaluated based on the comparison results.

[0083] Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. Figure 9 As shown, the electronic device includes a processor 51 and a memory 52 coupled to the processor 51.

[0084] The memory 52 stores program instructions for implementing the methods of any of the above embodiments; the processor 51 executes the program instructions stored in the memory 52 to implement the steps of the above method embodiments. The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor may be a microprocessor or any conventional processor.

[0085] Figure 10 This is a schematic diagram of the structure of an embodiment of the storage medium of this application. Figure 10 As shown, the computer-readable storage medium 60 of this application embodiment stores program instructions 61, which, when executed, implement the methods provided in the above embodiments of this application. The program instructions 61 can form a program file and be stored in the computer-readable storage medium 60 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned computer-readable storage medium 60 includes various media capable of storing program code, such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0087] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for pedestrian attribute recognition, characterized in that, include: Acquire pedestrian images; The pedestrian image is identified using an attribute recognition model to obtain the confidence result of the pedestrian image, wherein the confidence result includes the confidence of the pedestrian image belonging to each category under the same attribute; Determine whether the confidence scores for each category are all less than the confidence threshold; If so, the difference between the confidence result of the pedestrian image and the confidence result of at least one reference image is obtained, wherein the confidence result of the reference image is obtained by the attribute recognition model; The confidence results of the preset number of reference images with the smallest difference are used as the reference confidence results; The category of the reference confidence result is determined, wherein the category of the reference confidence result is the category corresponding to the highest confidence level among the reference confidence results; Obtain the first sum of the maximum confidence scores among the reference confidence scores for the same category, and obtain the second sum of the confidence scores for the corresponding category among all the reference confidence scores; Obtain a first ratio between the first sum and the second sum, as a reference probability for the corresponding category; The confidence level corresponding to each category is adjusted based on the reference probability of each category; The category of the pedestrian image is determined based on the adjusted confidence level.

2. The method according to claim 1, characterized in that, The confidence result is a confidence vector composed of the confidence scores of each category; obtaining the difference between the confidence results of the pedestrian image and the confidence results of at least one of the reference images includes: The distance between the confidence vector of the pedestrian image and the confidence vector of at least one of the reference images is obtained respectively.

3. The method according to claim 1, characterized in that, The adjustment of the confidence level corresponding to the category based on the reference probability of the categories includes: Obtain the product between the confidence level of each category and the reference probability of the corresponding category; The confidence level of each category is adjusted based on the product corresponding to each category.

4. The method according to claim 3, characterized in that, The step of adjusting the confidence level of each category based on the product corresponding to each category includes: The products corresponding to each category are normalized to obtain the normalization results corresponding to each category; Replace the confidence scores of each category with the corresponding normalization results.

5. The method according to claim 4, characterized in that, The normalization process for the products corresponding to each category includes: Obtain the sum of the products corresponding to all the categories; For each category, the second ratio of its corresponding product to the sum of the products is obtained as the result of the normalization process.

6. The method according to claim 1, characterized in that, Determining the category of the pedestrian image based on the adjusted confidence level includes: The category corresponding to the maximum confidence level obtained after adjustment is determined as the category of the pedestrian image.

7. An electronic device, characterized in that, Includes a processor and a memory connected to the processor, wherein, The memory stores program instructions; The processor is used to execute the program instructions stored in the memory to implement the method of any one of claims 1-6.

8. A storage medium, characterized in that, The storage medium stores program instructions, which, when executed, implement the method as described in any one of claims 1-6.

Images