A method and device for determining user attributes

Abstract

The present disclosure provides a method and apparatus for determining user attributes. This method extracts attribute feature information for human body part images of different human body parts, that is, it can specifically extract the attribute feature information of different human body parts, so as to focus on the characteristics or key attribute feature information of each human body part, making the determination of the attribute feature information of each human body part more effective and accurate; moreover, since the attribute feature information between different human body parts may be related and need to be combined with each other to more accurately identify user attributes, therefore, this method fuses the attribute feature information corresponding to each of multiple human body part images to obtain target enhanced feature information, so that the related attributes of different human body parts are fused, and thus the user attributes of the image to be recognized can be determined by combining the fused target enhanced feature information, making the determined user attributes more accurate.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a method and apparatus for determining user attributes. Background Art

[0002] User attribute recognition is a technical means for high-level semantic mining of pedestrians, which can summarize rules from a large amount of pedestrian data and is of great significance to various fields. Specifically, pedestrian attribute recognition first detects a target pedestrian from an image, and then based on the image region of the target pedestrian, identifies attribute features such as the age, gender, hair, and clothes of the target pedestrian.

[0003] Currently, the existing technologies adopted need to identify various attribute features from the overall image region of the target user. However, if each attribute focuses on all the human body information in the overall image region, it will not only increase computing resources, but also due to excessive focus on the whole body feature points, the more important features will be insufficiently concerned, resulting in poor accuracy of the recognition results of the attribute features. Therefore, there is an urgent need for a user attribute determination scheme with high recognition accuracy and high computing and processing efficiency. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a method, apparatus, computer device, and computer-readable storage medium for determining user attributes, so as to solve the problems of low recognition accuracy and low computing and processing efficiency existing in the existing user attribute determination methods.

[0005] In the first aspect of the embodiments of the present disclosure, a method for determining user attributes is provided. The method includes:

[0006] Obtain multiple human body part images of a user image to be recognized; wherein, each human body part image corresponds to a different human body part;

[0007] For each human body part image, determine the attribute feature information corresponding to the human body part image;

[0008] Fuse the attribute feature information respectively corresponding to the multiple human body part images to obtain target enhanced feature information;

[0009] Determine the user attributes of the image to be recognized according to the target enhanced feature information and the attribute feature information respectively corresponding to the multiple human body part images.

[0010] In the second aspect of the embodiments of the present disclosure, a device for determining user attributes is provided. The device includes:

[0011] An image acquisition module, configured to obtain multiple human body part images of a user image to be recognized; wherein, each human body part image corresponds to a different human body part;

[0012] A feature determination module, configured to determine, for each human body part image, the attribute feature information corresponding to the human body part image;

[0013] A feature enhancement module, configured to obtain target enhanced feature information according to the attribute feature information respectively corresponding to the multiple human body part images;

[0014] An attribute determination module, configured to determine the user attribute of the image to be recognized according to the target enhanced feature information and the attribute feature information respectively corresponding to the multiple human body part images.

[0015] In a third aspect of the embodiments of the present disclosure, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0016] In a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0017] The beneficial effects of the embodiments of the present disclosure compared with the prior art are as follows: In the embodiments of the present disclosure, multiple human body part images of a user image to be recognized can be obtained first, where each human body part image corresponds to a different human body part; then, for each human body part image, the attribute feature information corresponding to the human body part image can be determined; next, the attribute feature information respectively corresponding to the multiple human body part images can be fused to obtain target enhanced feature information; finally, based on the target enhanced feature information and the attribute feature information respectively corresponding to the multiple human body part images, the user attribute of the image to be recognized can be determined. Since the method provided in this embodiment can extract attribute feature information for human body part images of different human body parts, that is, can specifically extract the attribute feature information of different human body parts, so as to focus on the characteristics or key attribute feature information of each human body part, making the determination of the attribute feature information of each human body part more effective and accurate; moreover, since the attribute feature information between different human body parts may be related and need to be combined with each other to more accurately recognize the user attribute. For example, the attribute "dress" needs to combine the attribute feature information of the upper and lower body parts to be recognized. Therefore, the method provided in this embodiment fuses the attribute feature information respectively corresponding to the multiple human body part images to obtain target enhanced feature information, so that the related attributes of different human body parts can be fused, and thus the user attribute of the image to be recognized can be determined by combining the fused target enhanced feature information, making the determined user attribute more accurate; in summary, the method provided in this embodiment can improve the accuracy of the recognition result of the user attribute. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 is a schematic diagram of the application scenario of the embodiments of the present disclosure;

[0020] Figure 2 is a flowchart of the method for determining the user attribute provided by the embodiments of the present disclosure;

[0021] Figure 3 is a schematic diagram of the network architecture of the user attribute determination model provided by the embodiments of the present disclosure;

[0022] Figure 4 is a block diagram of the user attribute determination device provided by the embodiments of the present disclosure;

[0023] Figure 5 It is a schematic diagram of a computer device provided by an embodiment of the present disclosure. Detailed implementation manners

[0024] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present disclosure. However, those skilled in the art should clearly understand that the present disclosure can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present disclosure.

[0025] A method and apparatus for determining user attributes according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0026] In the prior art, since the existing technology currently adopted needs to identify various attribute features from the overall image area of the target user, but if each attribute focuses on all the human body information in the overall image area, it will not only increase computing resources, but also due to excessive focus on the whole body feature points, the more important features will be insufficiently concerned, which easily leads to poor accuracy of the recognition results of the attribute features. Therefore, there is an urgent need for a user attribute determination scheme with high recognition accuracy and high computing and processing efficiency.

[0027] To solve the above problems, the present invention provides a method for determining user attributes. In this method, since the method provided by this embodiment can extract attribute feature information for the human body part images of different human body parts, that is, it can specifically extract the attribute feature information of different human body parts, so as to focus on the characteristics or key attribute feature information of each human body part, making it more effective and accurate to determine the attribute feature information of each human body part; and, since the attribute feature information between different human body parts may be related and need to be combined with each other to more accurately identify the user attributes. For example, the attribute "dress" needs to combine the attribute feature information of the upper and lower body parts to be recognized. Therefore, the method provided by this embodiment fuses the attribute feature information respectively corresponding to the multiple human body part images to obtain target enhanced feature information, so that the related attributes of different human body parts are fused, and thus the user attributes of the image to be recognized can be determined by combining the fused target enhanced feature information, making the determined user attributes more accurate; in summary, the method provided by this embodiment can improve the accuracy of the recognition results of user attributes.

[0028] For example, the embodiments of the present invention can be applied to an application scenario as Figure 1 shown. In this scenario, it may include a terminal device 1 and a server 2.

[0029] The terminal device 1 can be either hardware or software. When the terminal device 1 is hardware, it can be various electronic devices with functions of collecting images, storing images and supporting communication with the server 2, including but not limited to smart phones, tablet computers, laptop computers, digital cameras, monitors, video recorders, desktop computers, etc.; when the terminal device 1 is software, it can be installed in the above-mentioned electronic devices. The terminal device 1 can be implemented as multiple software or software modules, or can be implemented as a single software or software module, and the embodiments of the present disclosure do not limit this. Further, various applications can be installed on the terminal device 1, such as image acquisition applications, image storage applications, instant messaging applications, etc.

[0030] The server 2 can be a server that provides various services. For example, it can be a background server that receives requests sent by the terminal device establishing a communication connection with it. This background server can receive and analyze requests sent by the terminal device and generate processing results. The server 2 can be a single server, or can be a server cluster composed of several servers, or can also be a cloud computing service center, and the embodiments of the present disclosure do not limit this.

[0031] It should be noted that the server 2 can be either hardware or software. When the server 2 is hardware, it can be various electronic devices that provide various services for the terminal device 1. When the server 2 is software, it can be multiple software or software modules that provide various services for the terminal device 1, or can be a single software or software module that provides various services for the terminal device 1, and the embodiments of the present disclosure do not limit this.

[0032] The terminal device 1 and the server 2 can be communicatively connected through a network. The network can be a wired network connected by coaxial cables, twisted pairs and optical fibers, or can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc., and the embodiments of the present disclosure do not limit this.

[0033] Specifically, the user can determine the user image to be recognized through the terminal device 1 and send the user image to be recognized to the server 2. After receiving the user image to be recognized, the server 2 can obtain multiple human body part images of the user image to be recognized. Then, for each human body part image, the server 2 can determine the attribute feature information corresponding to the human body part image. Next, the server 2 can fuse the attribute feature information corresponding to each of the multiple human body part images to obtain the target enhanced feature information. Immediately afterwards, the server 2 can determine the user attribute of the image to be recognized based on the target enhanced feature information and the attribute feature information corresponding to each of the multiple human body part images. Finally, the server 2 can send the user attribute of the image to be recognized to the terminal device 1 so that the terminal device 1 can display the user attribute of the image to be recognized to the user. In this way, since the method provided in this embodiment can extract attribute feature information for human body part images of different human body parts, that is, it can specifically extract the attribute feature information of different human body parts, so as to focus on the characteristics or key attribute feature information of each human body part, making the determination of the attribute feature information of each human body part more effective and accurate; and, since the attribute feature information between different human body parts may be related and need to be combined with each other to more accurately identify the user attribute, therefore, the method provided in this embodiment fuses the attribute feature information corresponding to each of the multiple human body part images to obtain the target enhanced feature information, so that the related attributes of different human body parts can be fused, and thus the user attribute of the image to be recognized can be determined by combining the fused target enhanced feature information, making the determined user attribute more accurate; in summary, the method provided in this embodiment can improve the accuracy of the recognition result of the user attribute.

[0034] It should be noted that the specific types, quantities, and combinations of the terminal device 1, the server 2, and the network can be adjusted according to the actual requirements of the application scenario, and the embodiments of the present disclosure do not limit this.

[0035] It should be noted that the above application scenarios are only shown for the convenience of understanding the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0036] Figure 2 It is a flowchart of a method for determining a user attribute provided by an embodiment of the present disclosure. Figure 2 A method for determining a user attribute can be executed by Figure 1 the terminal device or the server of. As Figure 2 shown, the method for determining the user attribute includes:

[0037] S201: Obtain multiple human body part images of the user image to be recognized.

[0038] In this embodiment, the image for which user attributes need to be recognized can be referred to as the user image to be recognized. Among them, user attributes can be understood as tags abstracted from the specific information of the user. For example, user attributes can include gender, age, hairstyle, clothing style, clothing color, whether wearing a hat, wearing a mask, etc. It can be understood that the user image to be recognized can include photos of the user's face and other various human body parts. In one implementation, the user image to be recognized can be an image collected by an image acquisition module (such as a camera). For example, the image of a customer entering a store or a user passing by a building or a street can be collected by using a surveillance camera, or it can also be a pre-stored user image.

[0039] Since there are many human body whole attributes, and the characteristic attributes of different human body parts vary greatly, and the human body parts that need to be concerned about for each characteristic attribute are also different. For example, the age attribute of the user only needs to be recognized according to the face characteristics, and the clothing attribute of the user only needs to be recognized according to the human body characteristics (such as upper body characteristics, lower body characteristics, or upper body characteristics and lower body characteristics). If the recognition process of each attribute needs to pay attention to all human body information, it will not only increase the computing resources, but also due to over-focusing on the whole body feature points, the more important features will be insufficiently concerned. Therefore, in this embodiment, for different user attributes, different human body part images can be used for recognition. Specifically, in this embodiment, after obtaining the user image to be recognized, multiple human body part images of the user to be recognized can be determined according to the user image to be recognized. Among them, the human body parts corresponding to each human body part image are different.

[0040] As one implementation, after obtaining the user image to be recognized, the human body parts of the user image to be recognized can be recognized to obtain the recognition result of the human body parts corresponding to the user image to be recognized. Then, according to a variety of preset human body part types, multiple human body part images of the user image to be recognized can be determined. For example, the human body parts corresponding to the preset human body part types can be marked in the user image to be recognized by using a marking frame, or the user image to be recognized can be cropped to obtain multiple human body part images of the user image to be recognized. As an example, assuming that the variety of preset human body part types includes the head, upper body part, lower body part, and feet, then the multiple human body part images of the user image to be recognized determined according to the variety of preset human body part types can include a head image, an upper body part image, a lower body part image, and a feet image. Among them, the human body part type corresponding to the head image is the head, the human body part type corresponding to the upper body part image is the upper body part, the human body part type corresponding to the lower body part image is the lower body part, and the human body part type corresponding to the feet image is the feet.

[0041] S202: For each human body part image, determine the attribute feature information corresponding to the human body part image.

[0042] Since different user attributes focus on different human body parts, after obtaining multiple human body part images of the user image to be recognized, the attribute feature information can be extracted for each human body part image. Among them, the attribute feature information corresponding to the human body part image can be understood as the feature map in the human body part image that can reflect the user attribute. For example, the attribute feature information corresponding to the human body part image can include feature information such as the color feature, texture feature, and local feature information in the human body part image. In this way, the user attribute corresponding to the human body part image can be determined by using the attribute feature information corresponding to the human body part image.

[0043] As an example, the attribute feature information corresponding to each human body part image can be extracted by using an image processing algorithm or a neural network. It can be understood that since the types of human body parts corresponding to different human body part images are different, for different human body part images, the methods or neural networks for extracting attribute feature information will also be different. Specifically, different image processing algorithms or different attribute attention modules can be used for different human body part images to determine the attribute feature information corresponding to the human body part image. For example, after obtaining multiple human body part images of the user image to be recognized, for each human body part image, first determine the attribute attention module corresponding to the human body part image, and then input the human body part image into the attribute attention module corresponding to the human body part image to obtain the attribute feature information corresponding to the human body part image output by the attribute attention module.

[0044] S203: Fuse the attribute feature information corresponding to each of the multiple human body part images to obtain target enhanced feature information.

[0045] Since some user attributes can only be determined based on the attribute feature information corresponding to multiple human body part images, that is, there is a correlation between the attribute feature information corresponding to each human body part image. For example, if the user attribute is a dress, this user attribute needs to combine the attribute feature information of the upper body part image and the lower body part image to determine the user attribute "dress". Therefore, in this embodiment, after obtaining the attribute feature information corresponding to each of the multiple human body part images, the attribute feature information corresponding to each of the multiple human body part images can be fused to obtain target enhanced feature information. Among them, the target enhanced feature information can be understood as the feature information obtained based on multiple relevant attribute feature information, and the target enhanced feature information can reflect at least one user attribute. That is to say, the user attribute reflected by the target enhanced feature information is determined by combining multiple relevant attribute feature information. As an example, relevant attribute feature information can be first determined from the attribute feature information corresponding to each of the multiple human body part images. Then, the target enhanced feature information can be determined based on the relevant attribute feature information. For example, the relevant attribute feature information can be fused to determine the target enhanced feature information. In this way, the relevant attributes of different human body parts can be fused, so that the user attribute of the image to be recognized can be determined by combining the fused target enhanced feature information, and the determined user attribute can be made more accurate.

[0046] S204: Determine the user attribute of the image to be recognized according to the target enhanced feature information and the attribute feature information corresponding to each of the multiple human body part images.

[0047] After obtaining the target enhanced feature information and the attribute feature information corresponding to each of the multiple human body part images, since both the target enhanced feature information and the attribute feature information corresponding to each of the multiple human body part images can reflect user attributes. Therefore, the user attributes corresponding to each of the human body part images can be determined respectively according to the target enhanced feature information of the image to be recognized and the attribute feature information corresponding to each of the multiple human body part images. For example, the user attributes corresponding to the head image can be determined according to the target enhanced feature information and the head image (such as the hairstyle, the style of glasses and hats), the user attributes corresponding to the upper body part image can be determined according to the target enhanced feature information and the upper body part image (such as the type of upper body clothes (coat, short-sleeved) and color), the user attributes corresponding to the lower body part image can be determined according to the target enhanced feature information and the lower body part image (such as the type of lower body clothes (shorts, trousers) and color), and the user attributes corresponding to the feet can be determined according to the target enhanced feature information and the foot image (such as the style of shoes (e.g., sports, casual)). Finally, the user attributes corresponding to all the human body part images can be used as the user attributes of the image to be recognized. For example, the user attributes corresponding to the head image, the user attributes corresponding to the upper body part image, the user attributes corresponding to the lower body part image, and the user attributes corresponding to the foot image can be used as the user attributes of the image to be recognized.

[0048] It can be seen that the beneficial effects of the embodiments of the present disclosure compared with the prior art are as follows: In the embodiments of the present disclosure, multiple human body part images of the user image to be recognized can be obtained first, where each human body part image corresponds to a different human body part; then, for each human body part image, the attribute feature information corresponding to the human body part image can be determined; next, the attribute feature information corresponding to each of the multiple human body part images can be fused to obtain target enhanced feature information; finally, based on the target enhanced feature information and the attribute feature information corresponding to each of the multiple human body part images, the user attribute of the image to be recognized can be determined. Since the method provided in this embodiment can extract attribute feature information for human body part images of different human body parts, that is, it can specifically extract the attribute feature information of different human body parts, so as to focus on the characteristics or key attribute feature information of each human body part, making the determination of the attribute feature information of each human body part more effective and accurate; and, since the attribute feature information between different human body parts may be related and need to be combined with each other to more accurately identify the user attribute. For example, the attribute "dress" needs to combine the attribute feature information of the upper and lower body parts to be recognized. Therefore, the method provided in this embodiment fuses the attribute feature information corresponding to each of the multiple human body part images to obtain target enhanced feature information, so that the related attributes of different human body parts can be fused, and then the user attribute of the image to be recognized can be determined by combining the fused target enhanced feature information, making the determined user attribute more accurate; in summary, the method provided in this embodiment can improve the accuracy of the recognition result of the user attribute.

[0049] Next, an implementation manner of "determining multiple human body part images of the user image to be recognized according to a preset variety of human body part types", that is, how to determine multiple human body part images of the user image to be recognized, will be introduced. In this embodiment, "determining multiple human body part images of the user image to be recognized according to a preset variety of human body part types" may include the following steps:

[0050] Input the user image to be recognized into a trained body region classification model to obtain the head image, upper body part image, lower body part image, and foot image of the user image to be recognized.

[0051] Specifically, the trained body region classification model may include a feature extraction network and a human body region extraction network. First, the user image to be recognized may be input into the feature extraction network to obtain the features of the user image to be recognized. Then, the features of the user image to be recognized may be input into the human body region extraction network to obtain the head image region, upper body part image region, lower body part image region, and foot image region of the user image to be recognized. Next, according to the head image region, upper body part image region, lower body part image region, and foot image region of the user image to be recognized, the head image, upper body part image, lower body part image, and foot image of the user image to be recognized may be generated.

[0052] Next, an implementation manner of S202 "for each human body part image, determine the attribute feature information corresponding to the human body part image" will be introduced, that is, how to determine the attribute feature information corresponding to the human body part image. In this embodiment, S202 "for each human body part image, determine the attribute feature information corresponding to the human body part image" may include the following steps:

[0053] S202a: For each human body part image, extract the image features of the human body part image.

[0054] After obtaining the human body part image, the image features of each human body part image may be extracted first. As an example, for each human body part image, the human body part image may be input into the average pooling layer first to obtain the image features of the human body part image. It can be understood that the average pooling operation performed by the average pooling layer on the human body part image is used to extract the features of a specific region (i.e., the image features of the human body part image), so as to more easily predict user attributes using the spatial neighborhood of the relevant group. For example, hairstyle, glasses, and hat are all in the head region, and they are in the same group, and these attributes are predicted simultaneously. In this way, the image features having a semantic relationship with the entire human body region can be divided into several groups to predict user attributes using the semantic relationship between them.

[0055] S202b: According to the image features of the human body part image, determine the preset attribute weight value and attribute feature vector corresponding to the human body part image;

[0056] After obtaining the image features of each human body part image, the attribute feature information corresponding to each human body part image may be determined. Specifically, as Figure 3As shown, for each human body part image, the corresponding attribute attention module can be determined first. It can be understood that the network architectures of the attribute attention modules corresponding to different human body part images are the same, but the network model parameters are different. For example, the attribute attention module corresponding to the head image can be determined according to the head image, the attribute attention module corresponding to the upper body part image can be determined according to the upper body part image, the attribute attention module corresponding to the lower body part image can be determined according to the lower body part image, and the attribute attention module corresponding to the foot image can be determined according to the foot image.

[0057] The image features of the human body part image can be input into the visual attention module, spatial normalization module, and ReLu module in the attribute attention module corresponding to the human body part image to obtain the attribute feature vector corresponding to the human body part image (i.e., the attribute feature vector corresponding to each user attribute). Among them, the attention mechanism in the visual attention module can map the image feature representation of the human body part image to channels of the same size to obtain spatial features; then, the spatial normalization module can perform normalization of the spatial features, enabling the model to specifically learn the image information on the image region most relevant to the human body part image; the visual attention module can include four stacked convolutional layers. It can be understood that the visual attention module, spatial normalization module, and ReLu module are mainly used to map the attention of the attribute feature vector to z, c, h, w, and then perform spatial feature normalization to c, h, where z represents the attribute feature vector, c represents the user attribute corresponding to the attribute feature vector, and h, w represent the height and width of the attribute feature vector.

[0058] The image features of the human body part image can be input into the confidence weight assignment module in the attribute attention module corresponding to the human body part image to obtain the preset attribute weight value corresponding to the human body part image. The preset attribute weight value can reflect the importance of the attribute feature. For example, the larger the preset attribute weight value, the higher the importance of the attribute feature; conversely, the smaller the preset attribute weight value, the lower the importance of the attribute feature. The confidence weight assignment module can include a convolutional layer with the same number of user attributes corresponding to the image features of the human body part image and containing feature channels, as well as a sigmoid function. It can be understood that the role of the confidence weight assignment module is to assign weights to the attention mapping according to the confidence of the attribute feature, avoiding learning features from the attention mask in the absence of preset attributes. Among them, the preset attribute weight value can not only reflect the attribute features at different spatial positions but also reflect the confidence of the attribute features. In this way, this branch helps the attention mechanism better learn features.

[0059] Among them, the confidence weight allocation module can use the spatial softmax operation to obtain the preset attribute weight value att corresponding to the human body part image. The specific formula is as follows:

[0060]

[0061] Among them, z represents the attribute feature vector, c represents the user attribute corresponding to the attribute feature vector, and h and w represent the height and width of the attribute feature vector. The spatial softmax operation is used to generate an attention mask with characteristics for each user attribute C and make the network focus on the most relevant regions in the image. Among them, the attention mechanism containing four convolutional layers can be called Att (i.e., the visual attention module). For each spatial resolution i, first obtain the unnormalized attention Z i (x) = Att(k i (x)), and then the spatial normalization module uses to perform spatial normalization on it to obtain the normalized attention mask Att i (x).

[0062] S202c: Determine the attention mask corresponding to the human body part image according to the preset attribute weight value and the attribute feature vector corresponding to the human body part image.

[0063] In this embodiment, after obtaining the preset attribute weight value and the attribute feature vector corresponding to the human body part image, the preset attribute weight value and the attribute feature vector corresponding to the human body part image can be fused to obtain the attention mask corresponding to the human body part image. For example, each attribute feature vector is multiplied by its corresponding preset attribute weight value, and then the sum of all products is used as the attention mask corresponding to the human body part image. Among them, the attention mask is weighted by the attribute feature vector, which can give a greater weight to the region corresponding to the attribute feature vector. As an example, as Figure 3 shown, a fully connected module is also connected after each attribute attention module. After inputting the preset attribute weight value and the attribute feature vector corresponding to the human body part image into the fully connected layer, the fully connected layer can extract the fused channel information to obtain the attention mask corresponding to the human body part image.

[0064] S202d: Determine the attribute feature information corresponding to the human body part image according to the image feature and the attention mask of the human body part image.

[0065] In this embodiment, as Figure 3 shown, the image feature and the attention mask of the human body part image can be fused to obtain the attribute feature information corresponding to the human body part image.

[0066] Next, an implementation manner of S203 "fusing the attribute feature information respectively corresponding to the multiple human body part images to obtain target enhanced feature information" will be introduced, that is, how to determine the target enhanced feature information. In this embodiment, S203 "fusing the attribute feature information respectively corresponding to the multiple human body part images to obtain target enhanced feature information" may include the following steps:

[0067] S203a: For every two human body part images, determine the associated attribute feature information corresponding to the two human body part images in the attribute feature information respectively corresponding to the two human body part images; generate the attribute enhanced feature information corresponding to the two human body part images according to the associated attribute feature information corresponding to the two human body part images.

[0068] In this embodiment, the associated attribute feature information with relevance can be first determined from the attribute feature information respectively corresponding to each two human body part images. In this embodiment, the attribute feature information with relevance can be referred to as the associated attribute feature information. Then, the attribute enhanced feature information corresponding to the two human body part images can be generated according to the associated attribute feature information corresponding to the two human body part images.

[0069] As an example, the associated attribute feature extraction module corresponding to each human body part image can be first used to extract the associated attribute feature information between each human body part image and other human body part images. For example, the attribute feature information corresponding to the head image can be input into the associated attribute feature extraction module corresponding to the head image to obtain the associated attribute feature information between the head image and the upper body part image, the lower body part image, and the foot image respectively; the attribute feature information corresponding to the upper body part image can be input into the associated attribute feature extraction module corresponding to the upper body part image to obtain the associated attribute feature information between the upper body part image and the head image, the lower body part image, and the foot image respectively; the attribute feature information corresponding to the lower body part image can be input into the associated attribute feature extraction module corresponding to the lower body part image to obtain the associated attribute feature information between the lower body part image and the head image, the upper body part image, and the foot image respectively; the attribute feature information corresponding to the foot image can be input into the associated attribute feature extraction module corresponding to the foot image to obtain the associated attribute feature information between the foot image and the head image, the upper body part image, and the lower body part image respectively.

[0070] Next, for every two human body part images, the attribute enhanced feature information corresponding to the two human body part images can be generated according to the associated attribute feature information corresponding to the two human body part images; as an example, Figure 3The splicing module in [[]] generates attribute enhancement feature information corresponding to the two human body part images according to the associated attribute feature information corresponding to the two human body part images. For example, the attribute enhancement feature information corresponding to the head image and the upper body part image can be generated according to the associated attribute feature information corresponding to the head image and the upper body part image; the attribute enhancement feature information corresponding to the head image and the lower body part image can be generated according to the associated attribute feature information corresponding to the head image and the lower body part image; the attribute enhancement feature information corresponding to the head image and the foot image can be generated according to the associated attribute feature information corresponding to the head image and the foot image; the attribute enhancement feature information corresponding to the upper body part image and the lower body part image can be generated according to the associated attribute feature information corresponding to the upper body part image and the lower body part image; the attribute enhancement feature information corresponding to the upper body part image and the foot image can be generated according to the associated attribute feature information corresponding to the upper body part image and the foot image; the attribute enhancement feature information corresponding to the lower body part image and the foot image can be generated according to the associated attribute feature information corresponding to the lower body part image and the foot image. It should be noted that, in one implementation, the splicing module may further include a spatial attention module to improve the enhancement effect of the attribute enhancement feature information.

[0071] S203b: Determine the target enhancement feature information according to all the attribute enhancement feature information.

[0072] In this embodiment, it is possible to use Figure 3 the splicing module in [[]] to determine the target enhancement feature information according to all the attribute enhancement feature information. As an example, for each channel dimension, first, the feature vectors of each attribute enhancement feature information in the channel dimension can be spliced to obtain the spliced feature vector of the channel dimension; then, the target enhancement feature information can be determined according to the spliced feature vectors of each channel dimension. That is, Figure 3 the splicing module in [[]] can splice the feature vectors of each attribute enhancement feature information in the same channel dimension to obtain the spliced feature vectors of each channel dimension; then, the target enhancement feature information can be determined according to the spliced feature vectors of each channel dimension.

[0073] Suppose there are four attribute enhancement feature information, Feature2, Feature3, Feature3, Feature4. The following formula can be used to determine the target enhancement feature information Feature concat :

[0074] Feature concat= concat(Feature1 + Feature2 + Feature3 + Feature4); where concat() is a channel concatenation and fusion function.

[0075] It should be noted that the model processing branches of the upper body part image and the lower body part image can be merged into one branch so that the upper and lower body part images of the body can be jointly recognized.

[0076] Next, an implementation manner of S204 "determine the user attribute of the image to be recognized according to the target enhanced feature information and the attribute feature information respectively corresponding to each of the multiple human body part images", that is, how to determine the user attribute of the image to be recognized will be introduced. In this embodiment, S204 "determine the user attribute of the image to be recognized according to the target enhanced feature information and the attribute feature information respectively corresponding to each of the multiple human body part images" may include the following steps:

[0077] S204a: Determine the distinct attribute features respectively corresponding to each human body part image according to the target enhanced feature information.

[0078] Among them, the distinct attribute features can be understood as the important channel dimensions in each human body part image, that is, the features that are different from other part images. As an example, the distinct attribute features respectively corresponding to each human body part image can be extracted by using the distinct attribute feature extraction module corresponding to each human body part image in Figure 4 As an example, in order to fully extract the integrated information, for each branch, a component needs to be extracted from the target enhanced feature information Feature

[0079] This component is the distinct attribute feature. feature concat characterizes the distinct features of four networks (that is, the network branches respectively corresponding to each human body part image), and can be calculated by a convolutional layer with a 1×1 kernel based on Feature extract : concat feature

[0080] = conv extract 1×1 (Feature concat )

[0081] S204b: For each human body part image, determine the predicted user attribute feature corresponding to the human body part image according to the attribute feature information and the distinct attribute feature corresponding to the human body part image.

[0082] ​In this embodiment, for each human body part image, the attribute feature information and the distinguishing attribute features corresponding to the human body part image can be fused to obtain the predicted user attribute features corresponding to the human body part image. Since the attribute feature information corresponding to the human body part image is the feature information corresponding to the user attribute carried by the human body part image, and the distinguishing attribute features corresponding to the human body part image are the attribute features of the user attribute that is more important and prominent than other human body part images, the predicted user attribute features corresponding to the human body part image obtained by fusion of the two can more accurately and comprehensively reflect the user attribute corresponding to the human body part image.

[0083] S204c: Determine the user attribute of the image to be identified according to the predicted user attribute features corresponding to each of the human body part images.

[0084] In this embodiment, the predicted user attribute features corresponding to each human body part image can be input into a preset fully connected module to obtain superimposed predicted user attribute features, wherein the dimension of the superimposed predicted user attribute features is the same as the number of attributes in the relevant grouping (such as head, upper body, lower body, feet).

[0085] Then, the superimposed predicted user attribute features can be input into the batch normalization layer, and the batch normalization layer can output the user attributes of the image to be recognized. Among them, the batch normalization layer can normalize the superimposed predicted user attribute features into a vector with zero mean and unit variance, and then scale it and add a bias. It can be understood that the batch normalization layer is used to balance the positive output and negative output of the network. The output of the batch normalization layer can be used to calculate the weighted cross entropy loss so that the loss can be continuously minimized during the model learning process to learn and train. It can be seen that this embodiment provides such Figure 3 An end-to-end user attribute determination model is shown.

[0086] It should be noted that, in an implementation of this embodiment, the user image to be identified is a user image of a user passing by a target location. Figure 2 The method shown may also include: determining a tracing label of the target location according to a user attribute of the image to be identified.

[0087] Among them, the description label of the target location can reflect the corresponding characteristics of the target location. For example, when the target location is a shop, the description label of the target location can include labels such as the age group of shop customers, the clothing style of shop customers, and the number of male and female shop customers corresponding to the shop. For another example, when the target location is a street, the description label of the target location can include labels such as the age group and identity category of street pedestrians.

[0088] As an example, after obtaining the user attributes of the user images of multiple users collected based on the target location, the user attributes of the multiple users can be analyzed and counted to determine the tracing label of the target location. For example, based on multiple user images of a target location being a shop, it is determined that the user attributes are all customers wearing glasses and the age of the customers is 10-18 years old, then it can be determined that the tracing label of the shop is that the customers corresponding to the shop are all wearing glasses and are teenagers. In this way, after determining the user attributes of each user at the target location, analysis and statistics can be performed based on the obtained user attributes of each user, so as to achieve the purpose of determining the tracing label of the target location.

[0089] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0090] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0091] Figure 4 Schematic diagram of a device for determining user attributes provided by an embodiment of the present disclosure. Figure 4 As shown, the user attribute determination device includes:

[0092] The image acquisition module 401 is used to acquire multiple human body part images of the user image to be identified; wherein each human body part image corresponds to a different human body part;

[0093] A feature determination module 402 is used to determine attribute feature information corresponding to each human body part image;

[0094] The feature enhancement module 403 is used to obtain target enhancement feature information according to the attribute feature information corresponding to each of the plurality of human body part images;

[0095] The attribute determination module 404 is used to determine the user attribute of the image to be identified according to the target enhancement feature information and the attribute feature information respectively corresponding to the plurality of human body part images.

[0096] In some embodiments, the image acquisition module 401 is used to:

[0097] Obtaining the image of the user to be identified;

[0098] According to the preset multiple types of human body parts, multiple human body part images of the user image to be identified are determined.

[0099] In some embodiments, the multiple types of human body parts include the head, upper body part, lower body part, and feet, and the multiple human body part images include a head image, an upper body part image, a lower body part image, and a feet image; the image acquisition module 401 is configured to:

[0100] Input the user image to be recognized into a trained body area classification model to obtain the head image, upper body part image, lower body part image, and feet image of the user image to be recognized.

[0101] In some embodiments, the feature determination module 402 is configured to:

[0102] For each human body part image, extract the image features of the human body part image;

[0103] According to the image features of the human body part image, determine the preset attribute weight value and attribute feature vector corresponding to the human body part image;

[0104] According to the preset attribute weight value and attribute feature vector corresponding to the human body part image, determine the attention mask corresponding to the human body part image;

[0105] According to the image features and attention mask of the human body part image, determine the attribute feature information corresponding to the human body part image.

[0106] In some embodiments, the feature enhancement module 403 is configured to:

[0107] For every two human body part images, determine the associated attribute feature information corresponding to the two human body part images from the attribute feature information respectively corresponding to the two human body part images; generate the attribute enhancement feature information corresponding to the two human body part images according to the associated attribute feature information corresponding to the two human body part images;

[0108] Determine the target enhancement feature information according to all the attribute enhancement feature information.

[0109] In some embodiments, the feature enhancement module 403 is configured to:

[0110] For each channel dimension, splice the feature vectors of each attribute enhancement feature information in the channel dimension to obtain the spliced feature vector of the channel dimension;

[0111] Determine the target enhancement feature information according to the spliced feature vectors of each channel dimension.

[0112] In some embodiments, the attribute determination module 404 is configured to:

[0113] Determining the distinguishing attribute features corresponding to each of the human body part images according to the target enhanced feature information;

[0114] For each human body part image, determining a predicted user attribute feature corresponding to the human body part image according to the attribute feature information and the distinguishing attribute feature corresponding to the human body part image;

[0115] The user attribute of the image to be identified is determined according to the predicted user attribute features respectively corresponding to each of the human body part images.

[0116] In some embodiments, the user image to be identified is a user image of a user passing through a target location; the method further includes:

[0117] The tracing label of the target location is determined according to the user attribute of the image to be identified.

[0118] According to the technical solution provided by the embodiment of the present disclosure, the device for determining user attributes includes: an image acquisition module, used to acquire multiple human body part images of the user image to be identified; wherein the human body part corresponding to each human body part image is different; a feature determination module, used to determine the attribute feature information corresponding to the human body part image for each human body part image; a feature enhancement module, used to obtain target enhanced feature information according to the attribute feature information respectively corresponding to the multiple human body part images; and an attribute determination module, used to determine the user attribute of the image to be identified according to the target enhanced feature information and the attribute feature information respectively corresponding to the multiple human body part images. Since the device provided in the present embodiment can extract attribute feature information from human body part images of different human body parts, that is, it can extract attribute feature information of different human body parts in a targeted manner, so as to focus on the characteristics or key attribute characteristic information of each human body part, so as to make it more effective and accurate to determine the attribute feature information of each human body part; and, since the attribute feature information between different human body parts may be correlated, they need to be combined with each other to more accurately identify user attributes, for example, the attribute "dress" needs to be combined with the attribute feature information of the upper and lower body parts to be identified, therefore, the method provided in the present embodiment fuses the attribute feature information corresponding to each of the multiple human body part images to obtain target enhanced feature information, so that the correlated attributes of different human body parts can be fused, so that the user attributes of the image to be identified can be determined in combination with the fused target enhanced feature information, and the determined user attributes can be more accurate; in summary, the method provided in the present embodiment can improve the accuracy of the recognition results of user attributes.

[0119] It should be understood that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present disclosure.

[0120] Figure 5 is a schematic diagram of the computer device 5 provided by the embodiments of the present disclosure. As Figure 5 shown, the computer device 5 of this embodiment includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, the steps in the above various method embodiments are implemented. Alternatively, when the processor 501 executes the computer program 503, the functions of each module / unit in the above various device embodiments are implemented.

[0121] Exemplarily, the computer program 503 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 502 and executed by the processor 501 to complete the present disclosure. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 503 in the computer device 5.

[0122] The computer device 5 can be a desktop computer, a notebook, a palm computer, a cloud server, or other computer devices. The computer device 5 may include, but is not limited to, the processor 501 and the memory 502. Those skilled in the art can understand that Figure 5 merely an example of the computer device 5, which does not constitute a limitation to the computer device 5. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, the computer device may further include input / output devices, network access devices, a bus, etc.

[0123] The processor 501 may be a central processing module (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc.

[0124] The memory 502 may be an internal storage module of the computer device 5. For example, it can be the hard disk or memory of the computer device 5. The memory 502 may also be an external storage device of the computer device 5. For example, it can be a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the computer device 5. Further, the memory 502 may also include both the internal storage module and the external storage device of the computer device 5. The memory 502 is used to store computer programs and other programs and data required by the computer device. The memory 502 may also be used to temporarily store the data that has been output or will be output.

[0125] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned functional modules and module divisions are used as examples. In actual applications, the above-mentioned functions can be allocated to different functional modules or modules according to needs, that is, the internal structure of the device can be divided into different functional modules or modules to complete all or part of the functions described above. Each functional module and module in the embodiments can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. In addition, the specific names of each functional module and module are only for the convenience of mutual distinction and do not limit the protection scope of the present disclosure. The specific working processes of the modules and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0126] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] Those of ordinary skill in the art can realize that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0128] In the embodiments provided by the present disclosure, it should be understood that the disclosed device / computer equipment and method can be implemented in other ways. For example, the device / computer equipment embodiments described above are merely illustrative. For example, the division of modules or components is only a logical function division. In actual implementation, there may be other division methods. Multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or module can be in electrical, mechanical or other forms.

[0129] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, in each embodiment of the present disclosure, the functional modules can be integrated into a processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0131] If the integrated module / module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of the present disclosure, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. The computer program can include computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0132] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit it; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the present disclosure's various embodiments, and should all be included within the protection scope of the present disclosure.

Claims

1. A method for determining user attributes, characterized in that, The method includes: Obtaining multiple human body part images of a user image to be recognized; wherein, each human body part image corresponds to a different human body part; For each human body part image, determining the attribute feature information corresponding to the human body part image; Fusing the attribute feature information respectively corresponding to the multiple human body part images to obtain target enhanced feature information; Determining the user attribute of the user image to be recognized according to the target enhanced feature information and the attribute feature information respectively corresponding to the multiple human body part images; The fusing the attribute feature information respectively corresponding to the multiple human body part images to obtain target enhanced feature information includes: For every two human body part images, determining the associated attribute feature information corresponding to the two human body part images from the attribute feature information respectively corresponding to the two human body part images; generating the attribute enhanced feature information corresponding to the two human body part images according to the associated attribute feature information corresponding to the two human body part images; For each channel dimension, splicing the feature vectors of the respective attribute enhanced feature information in the channel dimension to obtain the spliced feature vector of the channel dimension; Determining the target enhanced feature information according to the spliced feature vectors of the respective channel dimensions; The determining the user attribute of the user image to be recognized according to the target enhanced feature information and the attribute feature information respectively corresponding to the multiple human body part images includes: Determining the distinguishing attribute features respectively corresponding to each human body part image according to the target enhanced feature information; For each human body part image, determining the predicted user attribute feature corresponding to the human body part image according to the attribute feature information and the distinguishing attribute feature corresponding to the human body part image; Determining the user attribute of the user image to be recognized according to the predicted user attribute features respectively corresponding to each human body part image.

2. The method according to claim 1, wherein The obtaining multiple human body part images of a user image to be recognized includes: Obtaining the user image to be recognized; Determining multiple human body part images of the user image to be recognized according to a preset variety of human body part types.

3. The method according to claim 2, wherein The variety of human body part types includes the head, upper body part, lower body part, and feet, and the multiple human body part images include a head image, an upper body part image, a lower body part image, and a feet image; the determining multiple human body part images of the user image to be recognized according to a preset variety of human body part types includes: Inputting the user image to be recognized into a trained body region classification model to obtain the head image, upper body part image, lower body part image, and feet image of the user image to be recognized.

4. The method according to claim 1, wherein The for each human body part image, determining the attribute feature information corresponding to the human body part image includes: For each human body part image, extracting the image features of the human body part image; According to the image features of the human body part image, determining the preset attribute weight value and the attribute feature vector corresponding to the human body part image; Determining an attention mask corresponding to the human body part image according to a preset attribute weight value and an attribute feature vector corresponding to the human body part image; According to the image features of the human body part image and the attention mask, attribute feature information corresponding to the human body part image is determined.

5. The method according to claim 1, characterized in that, The user image to be identified is a user image of a user passing through a target location; the method further includes: The description label of the target location is determined according to the user attribute of the user image to be identified.

6. An apparatus for determining user attributes, characterized in that The device comprises: An image acquisition module, used to acquire multiple human body part images of the user image to be identified; wherein each human body part image corresponds to a different human body part; A feature determination module, used to determine attribute feature information corresponding to each human body part image; A feature enhancement module, used to obtain target enhancement feature information according to the attribute feature information respectively corresponding to the plurality of human body part images; An attribute determination module, used to determine the user attribute of the to-be-identified user image according to the target enhancement feature information and the attribute feature information respectively corresponding to the plurality of human body part images; The feature enhancement module is specifically used to: for each two human body part images, determine the associated attribute feature information corresponding to the two human body part images in the attribute feature information respectively corresponding to the two human body part images; generate the attribute enhancement feature information corresponding to the two human body part images according to the associated attribute feature information corresponding to the two human body part images; for each channel dimension, splice the feature vectors of each attribute enhancement feature information in the channel dimension to obtain the spliced ​​feature vector of the channel dimension; determine the target enhancement feature information according to the spliced ​​feature vectors of each channel dimension; The attribute determination module is specifically used to: determine the distinguishing attribute features corresponding to each human body part image according to the target enhancement feature information; for each human body part image, determine the predicted user attribute features corresponding to the human body part image according to the attribute feature information and the distinguishing attribute features corresponding to the human body part image; and determine the user attributes of the user image to be identified according to the predicted user attribute features corresponding to each human body part image.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images