Human body attribute recognition method, device, electronic device and storage medium
By detecting and identifying the video frame images, extracting the detection frame area for attribute recognition, and combining face and clothing recognition, the problem of insufficient accuracy of human attribute recognition in the video is solved, and more efficient recognition results are achieved.
Patent Information
- Application Number
- CN202210828064.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-14
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The accuracy of human body attribute recognition (such as age, gender, etc.) in the prior art is insufficient in videos, making it difficult to effectively improve the accuracy of identification results.
By performing human detection on video frame images, the detection frame information is determined, and the image area is extracted from the detection frame position for human attribute recognition, combining face and clothing recognition, and recognition results are generated.
It improves the accuracy and efficiency of human attribute recognition, and can accurately identify the gender, age, clothing type and other information of the characters in the video.
Smart Images

Figure CN115131825B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, specifically the field of deep learning, image processing, and computer vision technology, and can be applied to scenarios such as human body recognition, and specifically relates to a method, device, electronic device, and storage medium for recognizing human attributes. Background Art
[0002] In practical applications, for many scenarios, it is necessary to identify the body attributes of people in the collected videos to determine the person's age, gender and other attribute information, so as to facilitate subsequent analysis based on the recognition results, and the accuracy of the recognition results is crucial. Summary of the invention
[0003] The present application provides a method, device, electronic device and storage medium for identifying human attributes. The specific scheme is as follows:
[0004] According to one aspect of the present application, a method for identifying human attributes is provided, comprising:
[0005] Obtain at least two frames of images from the video to be processed;
[0006] Determining detection frame information of a human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position;
[0007] Extracting a first image area corresponding to the position of the first detection frame from the image;
[0008] Human attributes are recognized on the first image area to generate a first recognition result.
[0009] According to another aspect of the present application, a device for identifying human attributes is provided, comprising:
[0010] An acquisition module, used for acquiring at least two frames of images from the video to be processed;
[0011] A first determination module is used to determine detection frame information of a human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position;
[0012] An extraction module, used for extracting a first image area corresponding to the position of the first detection frame from the image;
[0013] The recognition module is used to perform human attribute recognition on the first image area to generate a first recognition result.
[0014] According to another aspect of the present application, there is provided an electronic device, including:
[0015] at least one processor; and
[0016] a memory communicatively connected to the at least one processor; wherein,
[0017] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the above embodiment.
[0018] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above embodiment.
[0019] According to another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the steps of the method described in the above embodiment when executed by a processor.
[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.
[0022] Figure 1 A schematic diagram of a flow chart of a method for identifying human attributes provided in an embodiment of the present application;
[0023] Figure 2 A schematic diagram of a flow chart of a method for identifying human attributes provided by another embodiment of the present application;
[0024] Figure 3 A schematic diagram of a flow chart of a method for identifying human attributes provided by another embodiment of the present application;
[0025] Figure 4 A schematic diagram of a flow chart of a method for identifying human attributes provided by another embodiment of the present application;
[0026] Figure 5 A schematic diagram of a flow chart of a method for identifying human attributes provided by another embodiment of the present application;
[0027] Figure 6 A schematic diagram of a process for human attribute recognition provided in an embodiment of the present application;
[0028] Figure 7 A schematic diagram of the structure of a human attribute recognition device provided in one embodiment of the present application;
[0029] Figure 8It is a block diagram of an electronic device used to implement the human body attribute recognition method of the embodiment of the present application. DETAILED DESCRIPTION
[0030] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0031] Artificial intelligence is a discipline that studies the use of computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technical fields and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies include computer vision technology, speech recognition technology, natural language processing technology, as well as deep learning, big data processing technology, knowledge graph technology, and other major directions.
[0032] Deep learning is a new research direction in the field of machine learning. Deep learning is the study of the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds.
[0033] Computer vision is a science that studies how to make machines "see". It refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing so that the computer processes the images into images that are more suitable for human eye observation or transmission to instruments for detection.
[0034] The following describes a method, device, electronic device, and storage medium for recognizing human attributes according to an embodiment of the present application with reference to the accompanying drawings.
[0035] Figure 1 A schematic diagram of a flow chart of a method for identifying human attributes provided in one embodiment of the present application.
[0036] The method for recognizing human attributes of the embodiment of the present application can be executed by the device for recognizing human attributes of the embodiment of the present application. The device can be configured in an electronic device. By first performing human body detection on each frame of the image, and then extracting the first image area corresponding to the human body detection frame from each frame of the image, human body attributes are recognized on the first image area, thereby improving the accuracy of the recognition result.
[0037] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.
[0038] like Figure 1 As shown, the human body attribute recognition method includes:
[0039] Step 101: Obtain at least two frames of images from the video to be processed.
[0040] In the present application, the video to be processed can be framed according to a preset frame frequency, and at least two frames of images can be obtained from the video to be processed. The video to be processed can be collected from a shopping mall, a commercial street, or other scenes, and the present application does not limit this.
[0041] Step 102, determining detection frame information of the human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position.
[0042] In the present application, human body detection can be performed frame by frame on at least two acquired frames to determine the detection frame information of the human body in each frame. The detection frame information can include the position of the human body detection frame, that is, the first detection frame position, which refers to the position of the first detection frame in the image, and the position of the first detection frame can be represented by the coordinates of the four vertices of the first detection frame.
[0043] When performing human body detection on each frame of image, each frame of image may be input into a pre-trained human body detection model for human body detection to obtain the first detection frame position of the human body in each frame of image output by the human body detection model.
[0044] In the present application, the number of first detection frames of the human body in each frame of image may be one or more.
[0045] Step 103: extracting a first image region corresponding to the position of the first detection frame from the image.
[0046] In the present application, the area of the human body in the image can be determined according to the position of the first detection frame. Then, according to the position of the first detection frame, the image of the area surrounded by the first detection frame can be extracted from the image, that is, the first image area corresponding to the first detection frame position is obtained.
[0047] Step 104: Perform human attribute recognition on the first image area to generate a first recognition result.
[0048] Among them, human attribute recognition can refer to the recognition of a person's gender, age, clothing type, clothing color, etc.
[0049] In the present application, a first image area corresponding to a first detection frame position of a human body may be input into a pre-trained human attribute recognition model for recognition to obtain a first recognition result output by the human attribute recognition model.
[0050] It can be understood that if there are multiple first detection frames of human bodies in a frame image, the first image areas corresponding to the first detection frame positions of the multiple human bodies can be extracted, and then the human attribute recognition can be performed on the first image area corresponding to the first detection frame position of each human body to obtain the recognition result corresponding to the detection frame of each human body in the frame image.
[0051] In an embodiment of the present application, at least two frames of images are obtained from the video to be processed, human body detection is first performed on each frame of the image to obtain detection frame information of the human body in the image, and a first image area corresponding to the first detection frame position is extracted from the image, and then human body attribute recognition is performed on the first image area to obtain a recognition result. Thus, human body detection is first performed on the image, and then human body attribute recognition is performed on the human body area image extracted based on the human body detection result, thereby improving the accuracy of the recognition result.
[0052] Figure 2 A schematic diagram of a flow chart of a method for identifying human attributes provided in another embodiment of the present application.
[0053] like Figure 2 As shown, the human body attribute recognition method includes:
[0054] Step 201: Acquire at least two frames of images from the video to be processed.
[0055] In the present application, step 201 is similar to that described in the above embodiment, so it will not be repeated here.
[0056] Step 202 , determining detection frame information of the human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position and a first detection frame identifier.
[0057] In the present application, human body detection and human body tracking can be performed on at least two frames of images acquired as mentioned above, so that the identification of the same person in different frames of images can be determined, and information such as the first detection frame identification and the first detection frame position of the human body detection frame in each frame of image can be obtained.
[0058] For example, if the first detection frame identifier in the detection frame information of a human body in image A in at least two frames of images is the same as the first detection frame identifier in the detection frame information of a human body in image B, it means that the first detection frame identifier in the two images corresponds to the same person, that is, the person appears in both image A and image B.
[0059] Step 203: According to the first detection frame identifier, it is detected whether the recognition result set corresponding to the video to be processed includes the target recognition result corresponding to the first detection frame identifier.
[0060] The recognition result set corresponding to the video to be processed may include the human attribute recognition results of the people appearing in the video to be processed. Since different people in the video to be processed can be distinguished by the first detection frame identifier in the human body detection frame information, the recognition result set may include the recognition results corresponding to the detection frame identifier of the human body detected in the video to be processed.
[0061] In the present application, when detecting and identifying each frame of image, it is possible to detect whether the recognition result set corresponding to the video to be processed already contains the target recognition result corresponding to the first detection frame identifier of the human body in each frame of image.
[0062] Step 204 , in response to not detecting the target recognition result corresponding to the first detection frame identifier in the recognition result set, extracting a first image region from the image according to the position of the first detection frame.
[0063] For each frame image, if a target recognition result corresponding to a first detection frame identifier in the frame image is not detected in the recognition result set, it means that there is no human attribute recognition result of the person identified by the first detection frame identifier in the recognition result set. Then, based on the position of the first detection frame in the frame image, the first image area corresponding to the first detection frame position can be extracted from the frame image.
[0064] If the target recognition result corresponding to the first detection frame identifier in the frame image is detected in the recognition result set, it means that the human attribute recognition result of the person identified by the first detection frame is already in the recognition result set, and the recognition result set can be queried based on other first detection frame identifiers in the frame image. When all the first detection frame identifiers in the frame image have been detected and processed, the next frame image of the image can be processed.
[0065] Step 205: Perform human attribute recognition on the first image area to generate a first recognition result.
[0066] In the present application, step 205 is similar to that described in the above embodiment, so it will not be repeated here.
[0067] After performing human attribute recognition on the first image area to generate a recognition result, the target recognition result corresponding to the first detection frame identifier can be added to the recognition result set, so that the recognition results corresponding to each first detection frame identifier can be concentrated together for subsequent processing.
[0068] In an embodiment of the present application, the detection frame information may further include a first detection frame identifier. When extracting the first image area corresponding to the first detection frame position, the recognition result set corresponding to the video to be processed may be detected based on the first detection frame identifier. When the recognition result corresponding to the first detection frame identifier is not detected in the recognition result set, the human body area image is extracted based on the first detection frame position, and human attributes are recognized on the extracted human body area image, thereby reducing the amount of calculation and improving recognition efficiency.
[0069] Figure 3 A schematic diagram of a flow chart of a method for identifying human attributes provided in another embodiment of the present application.
[0070] like Figure 3 As shown, the human body attribute recognition method includes:
[0071] Step 301: Obtain at least two frames of images from the video to be processed.
[0072] Step 302, determining detection frame information of the human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position.
[0073] Step 303: extracting a first image region corresponding to the position of the first detection frame from the image.
[0074] In the present application, steps 301 to 303 are similar to those described in the above embodiment, and thus will not be described in detail here.
[0075] Step 304: determine the position of a second detection frame of the face detection frame in the first image area by performing face detection on the first image area.
[0076] In the present application, human attribute recognition may include face recognition, clothing recognition, etc. The first image area may be input into a pre-trained face detection model, and face detection may be performed using the face detection model to determine the second detection frame position of the face detection frame in the first image.
[0077] The position of the second detection frame can be represented by the coordinates of the four vertices of the face detection frame in the first image area.
[0078] Step 305: extracting a second image area corresponding to the position of the second detection frame from the first image area.
[0079] In the present application, based on the second detection frame position of the face detection frame, the area of the face detection frame in the first image area can be determined. Then, based on the second detection frame position, the image of the area surrounded by the face detection frame can be extracted from the first image area, that is, the second image area corresponding to the second detection frame position is obtained.
[0080] Step 306: Obtain a face recognition result by performing face recognition on the second image area.
[0081] In the present application, the second image area corresponding to the face detection frame can be input into a pre-trained face recognition model. The face recognition model can extract facial features and decode the facial features to obtain the face recognition result corresponding to the face detection frame.
[0082] Among them, the face recognition results may include information such as gender and age.
[0083] Therefore, by performing face detection on the first image area corresponding to the detection frame of the human body, the second detection frame position is obtained, and based on the second detection frame position of the face detection frame, the second image area corresponding to the face detection frame is referred to. By performing face recognition on the second image area, the accuracy of the face recognition result can be improved.
[0084] Step 307, obtaining a clothing recognition result by performing clothing recognition on the first image area.
[0085] In the present application, the second image region may be input into a pre-trained clothing recognition model to obtain a clothing recognition result output by the clothing recognition model, wherein the clothing recognition result may include the type and color of the clothing.
[0086] It should be noted that in the present application, human attribute recognition may include one of face recognition and clothing recognition, or may include both face recognition and clothing recognition. If human attribute recognition includes face recognition and clothing recognition, face recognition may be performed first and then clothing recognition in the above order, or clothing recognition may be performed first and then face recognition, or face recognition and clothing recognition may be performed simultaneously. The present application does not limit this.
[0087] Step 308, generating a first recognition result according to at least one of the face recognition result and the clothing recognition result.
[0088] In the present application, if human attribute recognition includes face recognition, then a first recognition result can be generated based on the face recognition result, and the first recognition result can include information such as gender and age; if human attribute recognition includes clothing recognition, then a first recognition result can be generated based on clothing recognition, and the first recognition result can include information such as the color of the clothing and the type of clothing; if human attribute recognition includes face recognition and clothing recognition, then a first recognition result can be generated based on the face recognition result and the clothing recognition result, and the first recognition result can include information such as gender, age, type of clothing, and color of clothing.
[0089] In an embodiment of the present application, when human attribute recognition is performed on the first image area to generate a first recognition result, face detection can be performed on the first image area, that is, the human body area image, to determine the second detection frame position of the face detection frame in the human body area image area, and the second image area, that is, the face image, corresponding to the second detection frame position is extracted from the human body area image area, and face recognition is performed on the face image to obtain a face recognition result, and clothing recognition is performed on the human body area image to obtain a clothing recognition result, thereby improving the accuracy of the face recognition result and the clothing recognition result, and generating the first recognition result based on at least one of the face recognition result and the clothing recognition result, thereby improving the accuracy of the first recognition result.
[0090] Figure 4 A schematic diagram of a flow chart of a method for identifying human attributes provided in another embodiment of the present application.
[0091] like Figure 4 As shown, the human body attribute recognition method includes:
[0092] Step 401: Acquire at least two frames of images from the video to be processed.
[0093] Step 402 , determining detection frame information of the human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position and a second detection frame identifier.
[0094] In the present application, the second detection frame identifier is the same as the first detection frame identifier described above.
[0095] Step 403: extracting a first image region corresponding to the position of the first detection frame from the image.
[0096] Step 404: determine the position of a second detection frame of the face detection frame in the first image region by performing face detection on the first image region.
[0097] Step 405: extracting a second image area corresponding to the position of the second detection frame from the first image area.
[0098] In the present application, steps 401 to 405 are similar to those described in the above embodiment, and thus will not be described in detail here.
[0099] Step 406: According to the second detection frame identifier, detect whether the recognition result set corresponding to the video to be processed includes the target recognition result corresponding to the second detection frame identifier.
[0100] In the present application, after extracting the second image area, it is possible to detect whether the recognition result set corresponding to the video to be processed contains the target recognition result corresponding to the second detection frame identifier based on the second detection frame identifier, which is similar to the above step 203 and will not be repeated here.
[0101] Step 407 : In response to not detecting the target recognition result corresponding to the second detection frame identifier in the recognition result set, determining whether the second image region meets the image quality condition.
[0102] In practical applications, when capturing video, the quality of the captured image may be affected by light, camera performance, etc. Based on this, in this application, if the target recognition result corresponding to the second detection frame identification is not detected in the recognition result set, it means that there is no human attribute recognition result of the person identified by the second detection frame in the recognition result set, then it can be determined whether the second image area meets the preset image quality condition.
[0103] The image quality conditions may include one or more of the following: the resolution is greater than a resolution threshold, the face is not blocked, the brightness meets the requirements, etc.
[0104] If the recognition result corresponding to the second detection box identifier in the frame image is detected in the recognition result set, it means that the recognition result of the human attribute recognition result of the person identified by the second detection box is already in the recognition result set, and the recognition result set can be queried based on other second detection box identifiers in the frame image. When all the second detection box identifiers in the frame image have been queried and processed, the next frame image of the image can be processed.
[0105] Step 408 , when it is determined that the second image region meets the preset image quality condition, a face recognition result is obtained by performing face recognition on the second image region.
[0106] If it is determined that the second image area meets the preset image quality conditions, face recognition can be performed on the second image area, and the second image area can be input into the face recognition model, that is, the face image is input into the face recognition model for face recognition to obtain the face recognition result corresponding to the human body detection frame.
[0107] Step 409: Acquire clothing recognition results by performing clothing recognition on the first image area.
[0108] Step 410, generating a first recognition result based on at least one of the face recognition result and the clothing recognition result.
[0109] In the present application, steps 409 and 410 are similar to those described in the above embodiment, and thus will not be described in detail here.
[0110] In an embodiment of the present application, when performing face recognition on the second image area, it is possible to first detect, based on the second detection frame identifier, whether the recognition result set corresponding to the video to be processed contains the target recognition result corresponding to the second detection frame identifier. If the target recognition result corresponding to the second detection frame identifier is not detected in the recognition result set, it is possible to determine whether the face image meets the image quality condition. If the face image meets the image quality condition, face recognition can be performed on the face image, which can reduce the amount of calculation, improve recognition efficiency, and improve the accuracy of the face recognition result.
[0111] Figure 5 A schematic diagram of a flow chart of a method for identifying human attributes provided in another embodiment of the present application.
[0112] like Figure 5 As shown, the human body attribute recognition method includes:
[0113] Step 501: Acquire at least two frames of images from the video to be processed.
[0114] Step 502, determining detection frame information of the human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position.
[0115] Step 503: extracting a first image region corresponding to the position of the first detection frame from the image.
[0116] Step 504: Perform human attribute recognition on the first image area to generate a first recognition result.
[0117] In the present application, steps 501 to 504 are similar to those described in the above embodiment, and thus will not be described in detail here.
[0118] Step 505: classify the first recognition result according to a preset classification rule to obtain a recognition result category.
[0119] The preset classification rules may include classification according to clothing category, or classification according to gender and clothing category, etc.
[0120] In the present application, the first recognition result can be classified according to preset classification rules. For example, the classification rule is to classify according to clothing categories, and determine the number of people corresponding to each clothing category in the first recognition result. The clothing category with the highest number of people can be used as the recognition result category. Here, the clothing categories can include, for example, casual, fashion, formal wear, etc.
[0121] For example, the classification rule is to classify according to gender and clothing category. The number of people of each gender, the clothing category and number corresponding to each gender in the first recognition result can be determined, and the number of people of which gender is higher and the number of people of which clothing category is the largest can be determined. In this way, the category of the recognition result can be determined. For example, the number of women is higher and the number of women wearing formal clothes is the highest, so the women's formal wear category can be used as the recognition result category.
[0122] It should be noted that the preset classification rules can be set according to actual needs, and this application does not limit this.
[0123] Step 506: Determine the video category of the video to be processed according to the recognition result category.
[0124] In the present disclosure, after determining the category of the recognition result, the video category of the recognition video to be processed can be determined according to the category of the recognition result. For example, the classification rule is to classify according to the clothing category. If the recognition result category is fashion, since there are many people wearing fashion clothes in the video to be processed, the video category of the video to be processed can be determined as a fashion video. If the recognition result category is formal wear, the video to be processed can be determined as a formal wear video. For another example, the classification rule is to classify according to gender and clothing category. If the recognition result category is women's formal wear, the video category of the video to be processed can be determined as a women's formal wear video.
[0125] In an embodiment of the present application, after human attributes are recognized in the first image area to generate a first recognition result, the first recognition result can be classified according to a preset classification rule to obtain a recognition result category, and then the video category of the video to be processed can be determined according to the recognition result category. In this way, the category of the video to be processed can be determined, which facilitates the classification of the video.
[0126] In one embodiment of the present application, the detection frame information may further include a third detection frame identifier, and the first recognition result may be further processed according to the third detection frame identifier, wherein the third detection frame identifier is the same as the above-mentioned first detection frame identifier and the second detection frame identifier.
[0127] Since the same person may appear in multiple frames of images, in the present application, the same third detection frame identifier may correspond to multiple recognition results. In the first recognition result, the recognition result corresponding to the same third detection frame identifier in each frame of image can be determined to be called the second recognition result, and based on the second recognition result, the final recognition result corresponding to the third detection frame identifier can be determined to be called the third recognition result.
[0128] For example, the recognition result with the largest number among the second recognition results corresponding to the same third detection frame identifier can be used as the third recognition result. For example, if the gender attribute in the second recognition results corresponding to a third detection frame identifier in two frames of images is female, and the gender attribute in the second recognition result corresponding to the third detection frame identifier in one frame of image is male, then the gender attribute in the third recognition result corresponding to the third detection frame identifier is female.
[0129] After determining the third recognition result corresponding to the third detection frame identifier, the fourth detection frame identifier corresponding to the target gender can be determined based on the gender attribute in the third recognition result, and the age group corresponding to the fourth detection frame identifier can be determined based on the age attribute in the third recognition result corresponding to the fourth detection frame identifier. Based on the age group corresponding to the fourth detection frame identifier and the clothing recognition result in the third recognition result corresponding to the fourth detection frame identifier, the clothing information corresponding to each age group of the target gender can be determined. For example, if the target gender is female, the clothing type corresponding to each age group of females can be determined.
[0130] Thus, the first recognition result can be regularized according to the age groups of the target gender, and the clothing information corresponding to each age group of the target gender can be determined, so that the clothing types, colors, etc. that each age group of the target gender likes can be determined.
[0131] After determining the clothing information corresponding to each age group of the target gender, the clothing promotion information corresponding to each age group of the target gender can also be determined based on the clothing information corresponding to each age group of the target gender, thereby improving the accuracy of the clothing promotion information.
[0132] For example, the favorite clothing type of women between the ages of 25 and 28 is jeans, so the promotion information of jeans can be recommended to female users in this age group.
[0133] In an embodiment of the present application, after human attributes are recognized in the first image area to generate a first recognition result, the recognition results corresponding to the human detection frames in multiple frames of images can be processed according to preset classification rules, so that the recognition results can be regularized.
[0134] In addition, according to the human attribute recognition method of the embodiment of the present application, multiple videos can be processed to obtain recognition results corresponding to each video, and then classified by year, quarter, region, etc. The obtained classification results can assist in judging future clothing trends.
[0135] In order to facilitate understanding of the above embodiments, the following takes human attribute recognition including face recognition and clothing recognition as an example. Figure 6 To explain, Figure 6 A schematic diagram of the process of human attribute recognition provided in one embodiment of the present application.
[0136] like Figure 6 As shown, human body detection and human body tracking are performed on the video frame, and the human body identifier (i.e., the first detection frame identifier described in the above embodiment) and the position of the human body detection frame (i.e., the first detection frame position described in the above embodiment) in the video frame can be determined, and then it is determined whether the human body identifier has a corresponding recognition result. If the human body identifier has a corresponding recognition result, human body tracking is performed; if the human body identifier does not have a corresponding recognition result, face detection is performed to obtain the position of the face detection frame (i.e., the second detection frame position described in the above embodiment), and it is determined whether the image quality of the enclosed area of the human body detection frame meets the conditions, such as whether the human body is blocked in the human body area image. If the image quality of the enclosed area of the human body detection frame meets the conditions, the human body is cut out according to the position of the human body detection frame to obtain the human body area image (i.e., the first image area corresponding to the first detection frame position described in the above embodiment), and the face is cut out according to the position of the face detection frame to obtain the face image (i.e., the second image area corresponding to the second detection frame position described in the above embodiment).
[0137] After obtaining the human body region image and the face image, it is determined whether the human body marker has a corresponding recognition result. If the human body marker does not have a corresponding recognition result, it is determined whether the face image meets the image quality requirements, for example, whether the resolution of the face image is greater than the resolution threshold, whether it is not blocked, etc. If the face image meets the image quality requirements, the face key points are extracted from the human body image, and then the face attributes, face features, etc. are extracted based on the face key points, where the face attributes may include gender, age, etc. If the human body marker has a corresponding recognition result, or the face image does not meet the image quality requirements, it is determined whether other human body markers in the video frame have corresponding recognition results. If not, face detection is performed.
[0138] In order to count the clothing information of the same people, a face search can be performed in the face feature library according to the face features, and it can be determined whether the face features match the face features in the feature library. If the similarity between the face feature and any face feature in the feature library is greater than the preset similarity threshold, it can be considered that the face feature matches the face feature in the feature library, and then clothing recognition is performed on the human image area to obtain a clothing recognition result, and finally the recognition result corresponding to the human body identifier is output, and the recognition result includes the face features, face attributes and clothing recognition results corresponding to the human body identifier. Among them, the face feature library can be obtained when the video collected in a certain area is recognized for the first time. If the face features do not match the face features in the feature library, it is determined whether other human body identifiers in the video frame have corresponding recognition results, and if not, face detection is performed.
[0139] In order to implement the above embodiment, the embodiment of the present application also proposes a human attribute recognition device. Figure 7 A schematic diagram of the structure of a device for recognizing human attributes provided in one embodiment of the present application.
[0140] like Figure 7 As shown, the human attribute recognition device 700 includes:
[0141] An acquisition module 710 is used to acquire at least two frames of images from the video to be processed;
[0142] A first determination module 720 is used to determine the detection frame information of the human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes the first detection frame position;
[0143] An extraction module 730 is used to extract a first image area corresponding to a first detection frame position from the image;
[0144] The recognition module 740 is used to perform human attribute recognition on the first image area to generate a first recognition result.
[0145] In a possible implementation manner of the embodiment of the present application, the detection frame information further includes a first detection frame identifier; and the first determination module 720 is configured to:
[0146] According to the first detection frame identifier, detecting whether the recognition result set corresponding to the video to be processed includes the target recognition result corresponding to the first detection frame identifier;
[0147] In response to not detecting a target recognition result corresponding to the first detection box identifier in the recognition result set, extracting a first image region from the image according to each first detection box position.
[0148] In a possible implementation of the embodiment of the present application, the device may further include:
[0149] The adding module is used to add the target recognition result to the recognition result set.
[0150] In a possible implementation of the embodiment of the present application, the identification module 740 is used to:
[0151] By performing face detection on the first image area, a second detection frame position of the face detection frame in the first image area is determined;
[0152] Extracting a second image area corresponding to the position of the first detection frame from the first image area;
[0153] Obtaining a face recognition result by performing face recognition on the second image area;
[0154] Acquire clothing recognition results by performing clothing recognition on the first image area;
[0155] A first recognition result is generated according to at least one of the face recognition result and the clothing recognition result.
[0156] In a possible implementation manner of the embodiment of the present application, the detection frame information further includes a second detection frame identifier; and the identification module 740 is configured to:
[0157] According to the second detection frame identifier, detecting whether the recognition result set corresponding to the video to be processed includes a target recognition result corresponding to the second detection frame identifier;
[0158] In response to not detecting a recognition result corresponding to the second detection frame identifier in the recognition result set, determining whether the second image region meets an image quality condition;
[0159] When it is determined that the second image region meets the preset image quality condition, a face recognition result is obtained by performing face recognition on the second image region.
[0160] In a possible implementation of the embodiment of the present application, the device may further include:
[0161] The second determination module is used to classify the recognition results corresponding to the human body detection frames in the multiple frames of images according to a preset classification rule to obtain the recognition result category;
[0162] The third determination module is used to determine the video category of the video to be processed according to the recognition result category.
[0163] In a possible implementation manner of the embodiment of the present application, the human body detection frame information also includes a third detection frame identifier; the device may also include:
[0164] A fourth determination module is used to determine, in the first recognition results, a second recognition result corresponding to the same third detection frame identifier in each frame of the image;
[0165] A fifth determination module, configured to determine a third recognition result corresponding to the third detection frame identifier according to the second recognition result;
[0166] A sixth determination module, configured to determine a fourth detection frame marker corresponding to the target gender according to the gender attribute in the third recognition result;
[0167] a seventh determination module, configured to determine an age group corresponding to the fourth detection frame mark according to the age attribute in the third recognition result;
[0168] The eighth determination module is used to determine the clothing information corresponding to each age group of the target gender according to the age group and the clothing recognition result in the third recognition result.
[0169] In a possible implementation of the embodiment of the present application, the device may further include:
[0170] The ninth determination module is used to determine the clothing promotion information corresponding to each age group of the target gender according to the clothing information corresponding to each age group of the target gender.
[0171] It should be noted that the explanation of the above-mentioned human attribute recognition method embodiment is also applicable to the human attribute recognition device of this embodiment, so it will not be repeated here.
[0172] In an embodiment of the present application, at least two frames of images are obtained from the video to be processed, human body detection is first performed on each frame of the image to obtain detection frame information of the human body in the image, and a first image area corresponding to the first detection frame position is extracted from the image, and then human body attribute recognition is performed on the first image area to obtain a recognition result. Thus, human body detection is first performed on the image, and then human body attribute recognition is performed on the human body area image extracted based on the human body detection result, thereby improving the accuracy of the recognition result.
[0173] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.
[0174] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0175] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 802 or a computer program loaded from a storage unit 808 to a RAM (Random Access Memory) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An I / O (Input / Output) interface 805 is also connected to the bus 804.
[0176] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0177] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the method for identifying human attributes. For example, in some embodiments, the method for identifying human attributes may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the method for identifying human attributes described above may be executed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the human body attribute recognition method in any other appropriate manner (eg, by means of firmware).
[0178] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0179] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0180] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0181] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0182] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.
[0183] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0184] According to an embodiment of the present application, the present application further provides a computer program product, which, when an instruction processor in the computer program product executes, executes the human body attribute recognition method proposed in the above embodiment of the present application.
[0185] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0186] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. A method for identifying human attributes, comprising: Obtain at least two frames of images from the video to be processed; Determining detection frame information of a human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position; Extracting a first image area corresponding to the position of the first detection frame from the image; Performing human attribute recognition on the first image area to generate a first recognition result; The human body detection frame information further includes a third detection frame identifier; and the method further includes: Determine, in the first recognition results, a second recognition result corresponding to the same third detection frame identifier in each frame of the image; Determine, according to the second recognition result, a third recognition result corresponding to the third detection frame identifier; Determining a fourth detection frame identifier corresponding to the target gender according to the gender attribute in the third recognition result; Determining, according to the age attribute in the third recognition result, an age group corresponding to the fourth detection frame identifier; Determining clothing information corresponding to each age group of the target gender according to the age group and the clothing recognition result in the third recognition result; The determining, based on the second recognition result, a third recognition result corresponding to the third detection frame identifier includes: taking the recognition result with the largest number among the second recognition results corresponding to the third detection frame identifier as the third recognition result.
2. The method according to claim 1, wherein: The detection frame information also includes a first detection frame identifier; The extracting a first image area corresponding to the position of the first detection frame from the image includes: According to the first detection frame identifier, detecting whether the recognition result set corresponding to the to-be-processed video includes a target recognition result corresponding to the first detection frame identifier; In response to not detecting the target recognition result corresponding to the first detection box identifier in the recognition result set, extracting the first image area from the image according to the first detection box position.
3. The method of claim 2, wherein: The method further comprises: The target recognition result is added to the recognition result set.
4. The method according to claim 1, wherein: The performing human attribute recognition on the first image area to generate a first recognition result includes: Determine a second detection frame position of a face detection frame in the first image region by performing face detection on the first image region; Extracting a second image area corresponding to the position of the second detection frame from the first image area; Obtaining a face recognition result by performing face recognition on the second image area; Acquire a clothing recognition result by performing clothing recognition on the first image area; The first recognition result is generated according to at least one of the face recognition result and the clothing recognition result.
5. The method of claim 4, wherein: The detection frame information also includes a second detection frame identifier; The performing face recognition on the second image area to obtain the face recognition result includes: According to the second detection frame identifier, detecting whether the recognition result set corresponding to the to-be-processed video includes a target recognition result corresponding to the second detection frame identifier; In response to not detecting the target recognition result corresponding to the second detection frame identifier in the recognition result set, determining whether the second image area meets the image quality condition; When it is determined that the second image area meets a preset image quality condition, the face recognition result is obtained by performing face recognition on the second image area.
6. The method of claim 1, wherein: The method further comprises: Classifying the first recognition result according to a preset classification rule to obtain a recognition result category; The video category of the video to be processed is determined according to the recognition result category.
7. The method of claim 6, wherein: After determining the clothing information corresponding to each age group of the target gender, the method further includes: According to the clothing information corresponding to each age group of the target gender, the clothing promotion information corresponding to each age group of the target gender is determined.
8. A device for identifying human attributes, comprising: An acquisition module, used for acquiring at least two frames of images from the video to be processed; A first determination module is used to determine detection frame information of a human body in the image by performing human body detection on each frame of the image, wherein the detection frame information includes a first detection frame position; An extraction module, used for extracting a first image area corresponding to the position of the first detection frame from the image; A recognition module, configured to perform human attribute recognition on the first image area to generate a first recognition result; The human body detection frame information also includes a third detection frame identifier; and the device also includes: A fourth determination module is used to determine, in the first recognition results, a second recognition result corresponding to the same third detection frame identifier in each frame of the image; a fifth determining module, configured to determine, according to the second recognition result, a third recognition result corresponding to the third detection frame identifier; A sixth determination module, configured to determine a fourth detection frame identifier corresponding to the target gender according to the gender attribute in the third recognition result; a seventh determination module, configured to determine the age group corresponding to the fourth detection frame mark according to the age attribute in the third recognition result; an eighth determination module, configured to determine clothing information corresponding to each age group of the target gender according to the age group and the clothing recognition result in the third recognition result; The fifth determination module is further configured to use the recognition result with the largest number among the second recognition results corresponding to the third detection frame identifier as the third recognition result.
9. The device of claim 8, wherein: The detection frame information also includes a first detection frame identifier; and the first determination module is used to: According to the first detection frame identifier, detecting whether the recognition result set corresponding to the video to be processed contains the target recognition result corresponding to the first detection frame identifier; In response to not detecting the target recognition result corresponding to the first detection box identifier in the recognition result set, extracting the first image area from the image according to the first detection box position.
10. The apparatus of claim 9, further comprising: An adding module is used to add the target recognition result to the recognition result set.
11. The device of claim 8, wherein: The identification module is used to: Determine a second detection frame position of a face detection frame in the first image region by performing face detection on the first image region; Extracting a second image area corresponding to the position of the second detection frame from the first image area; Obtaining a face recognition result by performing face recognition on the second image area; Acquire a clothing recognition result by performing clothing recognition on the first image area; The first recognition result is generated according to at least one of the face recognition result and the clothing recognition result.
12. The device of claim 11, wherein: The detection frame information further includes a second detection frame identifier, and the identification module is used to: According to the second detection frame identifier, detecting whether the recognition result set corresponding to the to-be-processed video includes a target recognition result corresponding to the second detection frame identifier; In response to not detecting a recognition result corresponding to the second detection frame identifier in the recognition result set, determining whether the second image area satisfies the image quality condition; When it is determined that the second image area meets a preset image quality condition, the face recognition result is obtained by performing face recognition on the second image area.
13. The apparatus of claim 8, further comprising: A second determination module, used to classify the first recognition result according to a preset classification rule to obtain a recognition result category; The third determination module is used to determine the video category of the to-be-processed video according to the recognition result category.
14. The apparatus of claim 8, further comprising: The ninth determining module is used to determine the clothing promotion information corresponding to each age group of the target gender according to the clothing information corresponding to each age group of the target gender.
15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.
16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.
17. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Pedestrian gender identification method based on regional blocks
CN105160317A
Face detection method, device and equipment, and storage medium
CN108416265A
Method and apparatus for generating information
CN109117758A
Human-body attribute recognition method, device, storage medium and electronic device
CN109522790A
Method and device for determining attribute information of a to-be-identified object
CN109800679A
