Attribute identification method, electronic device and computer-readable storage medium

Abstract

The present application discloses an attribute recognition method, an electronic device, and a computer-readable storage medium, the method comprising: obtaining a target image containing a target object; performing feature extraction on the target image to obtain a first target image feature; extracting a second target image feature related to at least one object attribute from the first target image feature; for each object attribute, performing object attribute recognition on the target image based on the second target image feature related to the object attribute to obtain an attribute recognition result corresponding to the object attribute. By the above method, the present application can improve the accuracy of attribute recognition.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an attribute recognition method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Target object (e.g., pedestrian) attribute recognition is to identify the relevant attributes of the target object from the video or image, such as whether the person is wearing a helmet, whether the person is wearing a helmet, whether the person is wearing clothes of the corresponding style, etc.

[0003] Currently, when performing attribute recognition on a target object, it is impossible to highlight the features related to the attributes, resulting in poor attribute recognition results. Summary of the invention

[0004] The main technical problem solved by the present application is to provide an attribute recognition method, an electronic device and a computer-readable storage medium, which can improve the accuracy of attribute recognition.

[0005] To solve the above technical problems, a technical solution adopted in the present application is: to provide an attribute recognition method, the method comprising: obtaining a target image containing a target object; performing feature extraction on the target image to obtain a first target image feature; extracting a second target image feature related to at least one object attribute from the first target image feature; for each object attribute, performing object attribute recognition on the target image based on the second target image feature related to the object attribute to obtain an attribute recognition result corresponding to the object attribute.

[0006] Wherein, before extracting the second target image feature related to at least one object attribute from the first target image feature, the attribute recognition method also includes: for each object attribute, determining the target object area associated with the object attribute in the target image; extracting the second target image feature related to at least one object attribute from the first target image feature, including: for each object attribute, extracting the third target image feature belonging to the target object area associated with the object attribute from the first target image feature as the second target image feature.

[0007] There are at least two target object areas associated with the object attributes; for each object attribute, a third target image feature belonging to the target object area associated with the object attribute is extracted from the first target image feature, including: for each object attribute, the target area attention position maps of each target object area associated with the object attribute are fused to obtain a fused attention position map; wherein each target element in the target area attention position map of the target object area represents the correlation between the corresponding target pixel point in the target image and the target object area; based on the fused attention position map, the third target image feature belonging to the target object area associated with the object attribute is extracted from the first target image feature.

[0008] Among them, before extracting, for each object attribute, a third target image feature belonging to a target object area associated with the object attribute from the first target image feature as the second target image feature, the attribute recognition method also includes: for each object attribute, determining the correlation between each target object area corresponding to the object attribute and the object attribute; for each object attribute, fusing the target area attention position maps of each target object area associated with the object attribute to obtain a fused attention position map, including: for each object attribute, using the correlation of the target object area corresponding to the object attribute as the weight of the target object area; based on the weight of each target object area, weightedly fusing the target area attention position maps of each target object area to obtain a fused attention position map.

[0009] Among them, the step of determining the target area attention position map of each target object area includes: using different sub-networks of the attention generation network to process the first target image features respectively to obtain the target area attention position map corresponding to each sub-network; wherein different sub-networks generate target area attention position maps for different target object areas.

[0010] Among them, the training steps of the attention generation network include: obtaining a sample image containing a sample object; and, using a region segmentation model to segment the sample image to obtain a region segmentation result of each sample object region in the sample image; extracting features from the sample image to obtain a first sample image feature; using different subnetworks of the attention generation network to process the first sample image feature respectively to obtain a sample region attention position map corresponding to each subnetwork; wherein different subnetworks generate sample region attention position maps for different sample object regions, and each sample element in the sample region attention position map of the sample object region represents the correlation between the corresponding sample pixel point in the sample image and the sample object region; based on the region segmentation results of each sample object region and the sample region attention position map corresponding to each subnetwork, the network parameters of each subnetwork are adjusted.

[0011] Among them, based on the regional segmentation results of each sample object area and the sample area attention position map corresponding to each sub-network, the network parameters of each sub-network are adjusted, including: for each sub-network, based on the sample area attention position map of the sub-network and the corresponding regional segmentation results, the sub-loss of the sub-network is determined; based on the sum of the sub-losses of each sub-network, the network parameters of each sub-network are adjusted.

[0012] Among them, for each sub-network, based on the sample area attention position map of the sub-network and the corresponding area segmentation result, the sub-loss of the sub-network is determined, including: for each sub-network, extracting the area segmentation contour of the corresponding area segmentation result; in response to the area segmentation contour being within the corresponding sample area attention position map, determining the sub-loss of the sub-network to be a first preset value; in response to at least part of the area segmentation contour being outside the corresponding sample area attention position map, determining the sub-loss of the sub-network to be the target distance between the area segmentation contour and the corresponding sample area attention position map.

[0013] Among them, the target distance is the shortest distance.

[0014] Among them, the first target image feature includes at least two levels of sub-features; based on the fused attention position map, the third target image feature belonging to the target object area associated with the object attribute is extracted from the first target image feature, including: based on the fused attention position map, respectively extracting the features of the target object area associated with the object attribute from the sub-features of different levels as the different levels of features corresponding to the object attribute; fusing the different levels of features corresponding to the object attribute to obtain the third target image feature.

[0015] Wherein, before extracting the second target image feature related to at least one object attribute from the first target image feature, the attribute recognition method also includes: for each object attribute, determining the target object area associated with the object attribute and the correlation degree of the object attribute; determining the target object area corresponding to each object attribute and the correlation degree corresponding to the target object area, including: using several gated sub-networks in the gated network to process the first target image feature respectively to obtain the processing results of each gated sub-network; wherein the gated sub-network is used to determine the correlation between an object attribute and a target object area, and at least one of the object attributes or target object areas corresponding to different gated sub-networks is different; in response to the processing result of the gated sub-network being a second preset value, determining that there is no correlation between the object attribute corresponding to the gated sub-network and the target object area; in response to the processing result of the gated sub-network being a third preset value, determining that there is a correlation between the object attribute corresponding to the gated sub-network and the target object area, and the correlation degree is the third preset value.

[0016] Among them, based on the second target image features related to the object attributes, object attributes of the target image are recognized to obtain attribute recognition results corresponding to the object attributes, including: pooling the second target image features to obtain pooling results; linearly processing the pooling results to obtain attribute recognition results corresponding to the object attributes; and / or the target object includes pedestrians.

[0017] In order to solve the above technical problems, another technical solution adopted by the present application is: to provide an electronic device, the electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the above attribute recognition method.

[0018] In order to solve the above technical problem, another technical solution adopted by the present application is: providing a computer-readable storage medium, wherein the computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the above attribute identification method.

[0019] In the above technical solution, the second target image feature related to the object attribute is an extracted image feature that is more relevant to the object attribute; therefore, object attributes of the target image are recognized based on the second target image feature related to the object attribute, that is, object attributes of the target image are recognized based on the image feature that is more relevant to the object attribute, which can improve the accuracy of attribute recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of an embodiment of an attribute identification method provided by the present application;

[0021] Figure 2 is a flow chart of an embodiment of a method for extracting a third target image feature belonging to a target object area associated with an object attribute provided by the present application;

[0022] Figure 3 yes Figure 2 The flowchart of step S21 of an embodiment is shown;

[0023] Figure 4 yes Figure 2 The flowchart of step S22 of an embodiment is shown;

[0024] Figure 5 It is a flowchart of an embodiment of a training method for an attention generation network provided in the present application;

[0025] Figure 6 yes Figure 5 The flowchart of step S55 is shown as an embodiment;

[0026] Figure 7 It is a structural diagram of an embodiment of an attribute identification device provided by the present application;

[0027] Figure 8 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application;

[0028] Fig. 9 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0029] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0030] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0031] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] See also Figure 1 , Figure 1 is a flow chart of an embodiment of the attribute identification method provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0033] Step S11: Acquire a target image containing a target object.

[0034] In this embodiment, a target image including a target object is acquired.

[0035] In one embodiment, the target image containing the target object can be obtained from local storage or cloud storage. Of course, in other embodiments, an image acquisition device can be used to acquire images of the target object in real time to obtain the target image containing the target object.

[0036] In one embodiment, the target object may be a pedestrian. Of course, in other embodiments, the target object may also be an animal, an object, etc., which is not limited here.

[0037] Step S12: extracting features from the target image to obtain first target image features.

[0038] In this implementation, feature extraction is performed on the target image to obtain a first target image feature.

[0039] In one embodiment, the first target image feature includes at least two levels of sub-features. For example, at least two levels of sub-features include low-level features, middle-level features, and high-level features. Low-level features are the most basic image features, usually calculated directly from pixel values, including color features, texture features, edge features, shape features, etc. Middle-level features are further combinations and abstractions of low-level features, usually involving some more complex image processing techniques and machine learning algorithms, used to capture local structures and patterns in images. High-level features are further abstractions and combinations of middle-level features, usually involving deep learning models, used to capture global semantic information and contextual relationships in images.

[0040] In a specific embodiment, the first target image feature includes at least two levels of sub-features, and the at least two levels of sub-features include low-level features, middle-level features and high-level features; at this time, feature extraction is performed on the target image to obtain the first target image feature, specifically: feature extraction is performed on the target image to obtain low-level features; feature extraction is performed on the low-level features to obtain middle-level features; feature extraction is performed on the middle-level features to obtain high-level features.

[0041] Step S13: extracting second target image features related to at least one object attribute from the first target image features.

[0042] In this embodiment, a second target image feature related to at least one object attribute is extracted from the first target image feature. The second target image feature related to at least one object attribute is extracted from the first target image feature, so that the extracted image feature is more relevant to the object attribute, and the object attribute recognition is subsequently performed on the target image based on the image feature that is more relevant to the object attribute, which can improve the accuracy of attribute recognition.

[0043] In one embodiment, for each object attribute, an image feature of a target object component associated with the object attribute may be extracted from the first target image feature as an image feature related to the object attribute. For example, taking a target object as a pedestrian, the target object component may be a pedestrian's head, a pedestrian's upper body, a pedestrian's lower body, etc.

[0044] In other embodiments, before extracting the second target image feature related to at least one object attribute from the first target image feature, for each object attribute, a target object region associated with the object attribute in the target image is determined; at this time, extracting the second target image feature related to at least one object attribute is specifically: for each object attribute, extracting the third target image feature belonging to the target object region associated with the object attribute from the first target image feature as the second target image feature. That is, for each object attribute, an image feature belonging to the target object region associated with the object attribute can be extracted from the first target image feature as the image feature related to the object attribute.

[0045] For example, if the target object is a pedestrian, the target object region may be the pedestrian's left arm, the pedestrian's right arm, the pedestrian's left foot, the pedestrian's right foot, etc. Compared with the target object component, the target object region is smaller and more detailed, so that the extracted image features are more relevant to the object attributes. Subsequently, the object attributes of the target image are recognized based on the image features that are more relevant to the object attributes, which can improve the accuracy of attribute recognition.

[0046] In a specific implementation, there is one target object region associated with the object attribute, and a third target image feature belonging to the target object region associated with the object attribute is extracted from the first target image feature as the second target image feature. For example, taking the target object as a pedestrian and the object attribute recognition as whether the behavior is wearing a helmet as an example: the helmet is worn on the human head, so the target object region associated with the object attribute is only the head; then, the third target image feature belonging to the pedestrian's head is extracted from the first target image feature as the second target image feature.

[0047] In other specific embodiments, there are at least two target object regions associated with the object attribute, and a third target image feature belonging to each target object region associated with the object attribute is extracted from the first target image feature as the second target image feature. For example, take the target object as a pedestrian and the object attribute identification as identifying whether the pedestrian is wearing work clothes as an example: work clothes are worn on the upper body of the human body except the left arm and the right arm and on the lower body of the human body, so the target object region associated with the object attribute includes region A—the lower body of the human body and region B—the upper body of the human body except the left arm and the right arm; then, from the first target image feature, the third target image feature belonging to region A and region B is extracted as the second target image frame.

[0048] Step S14: for each object attribute, based on the second target image feature related to the object attribute, perform object attribute recognition on the target image to obtain an attribute recognition result corresponding to the object attribute.

[0049] In this embodiment, for each object attribute, the object attribute recognition is performed on the target image based on the second target image feature related to the object attribute to obtain an attribute recognition result corresponding to the object attribute. The second target image feature related to the object attribute is an extracted image feature that is more relevant to the object attribute; therefore, the object attribute recognition is performed on the target image based on the second target image feature related to the object attribute, that is, the object attribute recognition is performed on the target image based on the image feature that is more relevant to the object attribute, which can improve the accuracy of attribute recognition.

[0050] In one embodiment, based on the second target image features related to the object attributes, the target image is subjected to object attribute recognition to obtain an attribute recognition result corresponding to the object attribute, specifically: the second target image features are subjected to pooling processing to obtain a pooling result; the pooling result is subjected to linear processing to obtain an attribute recognition result corresponding to the object attribute. In other words, the second target image features related to the object attributes are subjected to the pooling layer and the linear layer in sequence to obtain an attribute recognition result.

[0051] See also Figure 2 , Figure 2 1 is a flow chart of an embodiment of a method for extracting a third target image feature belonging to a target object region associated with an object attribute provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not necessarily use Figure 2 The process sequence shown is limited. Figure 2 As shown, there are at least two target object areas associated with the object attribute. This embodiment includes:

[0052] Step S21: For each object attribute, the target area attention position maps of each target object area associated with the object attribute are fused to obtain a fused attention position map.

[0053] In this embodiment, for each object attribute, the target area attention position maps of each target object area associated with the object attribute are fused to obtain a fused attention position map; wherein each target element in the target area attention position map of the target object area represents the correlation between the corresponding target pixel point in the target image and the target object area.

[0054] Each target element in a target area attention position map of a target object area represents the correlation between the corresponding target pixel point in the target image and the target object area. Therefore, through the target area attention position map of the target object area, the position of the target object area associated with the object attribute in the target image can be determined.

[0055] Therefore, through the target area attention position map of different target object areas associated with the object attributes, the positions of different target object areas associated with the object attributes in the target image can be determined.

[0056] Furthermore, the target area attention position maps of each target object area associated with the object attributes are fused to obtain a fused attention position map. By fusing the attention position map, the position of each target object area associated with the object attributes in the target image can be determined.

[0057] In addition, for each object attribute, the target area attention position maps of different target object areas associated with the object attribute are fused to solve the attribute cross-region problem.

[0058] In one implementation, for each object attribute, the target area attention position maps of each target object area associated with the attribute may be directly fused to obtain a fused attention position map.

[0059] In other implementations, for each object attribute, the target area attention position maps of each target object area associated with the attribute may also be weighted fused to obtain a fused attention position map.

[0060] Specifically, before extracting the third target image feature belonging to the target object area associated with the object attribute from the first target image feature as the second target image feature for each object attribute, the association degree between each target object area corresponding to the object attribute and the object attribute is determined for each object attribute. Figure 3 As shown, Figure 3 yes Figure 2 The flowchart of step S21 of an embodiment shown in FIG. 1 is a flowchart of a method for fusing target area attention position maps of target object areas associated with each object attribute to obtain a fused attention position map, specifically including the following sub-steps:

[0061] Step S31: For each object attribute, the relevance of the target object region corresponding to the object attribute is used as the weight of the target object region.

[0062] In this embodiment, for each object attribute, the relevance of the target object area corresponding to the object attribute is used as the weight of the target object area. In other words, for each object attribute, the relevance of the target object area corresponding to the object attribute and the object attribute is used as the weight of the target object area corresponding to the object attribute.

[0063] Step S32: Based on the weight of each target object area, the target area attention position map of each target object area is weightedly fused to obtain a fused attention position map.

[0064] In this embodiment, based on the weight of each target object area, the target attention position map of each target object area is weighted fused to obtain a fused attention position map. Therefore, by fusion of the attention position map, the position of each target object area associated with the object attribute in the target image can be determined. In addition, the fusion of the attention position map is based on the correlation between each target object area and the object attribute, so the target object area strongly related to the object attribute can also be determined by fusion of the attention position map. When attribute recognition is performed later, it can guide more attention to the image features of the target object area strongly related to the object attribute, thereby improving the accuracy of attribute recognition.

[0065] Step S22: Based on the fused attention position map, extract the third target image feature belonging to the target object area associated with the object attribute from the first target image feature.

[0066] In this embodiment, based on the fused attention position map, the third target image feature belonging to the target object area associated with the object attribute is extracted from the first target image feature. Since the fused attention position map reflects the position of each target object area associated with the object attribute in the target image; therefore, based on the fused attention position map, the third target image feature belonging to the target object area associated with the object attribute can be extracted from the first target image feature, so that the extracted third target image feature has a higher regional correlation with the object attribute. Subsequently, the object attribute recognition of the target image is performed based on the third target image feature that has a higher regional correlation with the object attribute, which can improve the accuracy of attribute recognition.

[0067] In one embodiment, if Figure 4 As shown, Figure 4 yes Figure 2 The flowchart of step S22 of an embodiment is shown, the first target image feature includes at least two levels of sub-features, and based on the fused attention position map, a third target image feature belonging to the target object area associated with the object attribute is extracted from the first target image feature, specifically including the following sub-steps:

[0068] Step S41: Based on the fused attention position map, extract features belonging to the target object area associated with the object attributes from sub-features at different levels as features at different levels corresponding to the object attributes.

[0069] In this embodiment, based on the fused attention position map, the features of the target object area associated with the object attribute are extracted from the sub-features of different levels as the features of different levels corresponding to the object attribute. Since the fused attention position map reflects the positions of the target object areas associated with the object attribute in the target image, based on the fused attention position map, the features of the target object area associated with the object attribute can be extracted from the sub-features of different levels as the features of different levels corresponding to the object attribute, so that the extracted features of different levels have more regional correlation with the object attribute. Subsequently, the object attribute recognition of the target image is performed based on the features of different levels that have more regional correlation with the object attribute, which can improve the accuracy of attribute recognition.

[0070] In one embodiment, the first target image features include low-level features, middle-level features, and high-level features. At this time, based on the fused attention position map, features belonging to the target object area associated with the object attributes are extracted from the low-level features, middle-level features, and high-level features respectively as the low-level features, middle-level features, and high-level features corresponding to the object attributes.

[0071] In a specific embodiment, the first target image features include high-level features, low-level features and middle-level features, and the target area attention position maps of each target object area associated with the object attributes are fused to obtain a fused attention position map. The target area attention position maps of each target object area associated with the object attributes are determined based on the high-level features. Before extracting the features belonging to the target object area associated with the object attributes from the low-level features and the middle-level features respectively, the low-level features and the middle-level features are downsampled first so that the low-level features and the middle-level features are aligned with the size of the fused attention position map.

[0072] Step S42: Fusing different levels of features corresponding to the object attributes to obtain a third target image feature.

[0073] In this embodiment, the features at different levels corresponding to the object attributes are fused to obtain the third target image feature. The third target image feature is obtained by fusing the features at different levels corresponding to the object attributes; therefore, the subsequent object attribute recognition of the target image based on the third target image feature can make full use of the information of the features at different levels, capture more details and context information, and thus improve the accuracy of attribute recognition. In addition, the subsequent attribute recognition based on the third target image feature obtained by fusing the features at different levels can meet the requirements of different attributes for different types of features at different levels.

[0074] In one embodiment, the target area attention position map of each target object area is determined by: using different sub-networks of the attention generation network to process the first target image features respectively, to obtain the target area attention position map corresponding to each sub-network, wherein different sub-networks generate target area attention position maps for different target object areas.

[0075] For example, the attention generation network includes sub-network 1, sub-network 2, sub-network 3 and sub-network 4; sub-network 1 is used to generate a target area attention position map of target object area A, sub-network 2 is used to generate a target area attention position map of target object area B, sub-network 3 is used to generate a target area attention position map of target object area C, and sub-network 4 is used to generate a target area attention position map of target object area D. Feature extraction is performed on a target image containing a target object to obtain a first target image feature; then, the first target image feature is input into sub-network 1, sub-network 2, sub-network 3 and sub-network 4 respectively, and sub-network 1 outputs a target area attention position map of target object area A, sub-network 2 outputs a target area attention position map of target object area B, sub-network 3 outputs a target area attention position map of target object area C, and sub-network 4 outputs a target area attention position map of target object area D.

[0076] In a specific embodiment, the first target image feature includes high-level features, and different sub-networks of the attention generation network are used to process the high-level features respectively to obtain the target area attention position map corresponding to each sub-network.

[0077] In a specific embodiment, Figure 5 As shown, Figure 5 1 is a flow chart of an embodiment of a training method for an attention generation network provided in the present application. The training method for an attention generation network specifically includes the following sub-steps:

[0078] Step S51: Acquire a sample image containing a sample object.

[0079] In this embodiment, a sample image including a sample object is acquired.

[0080] In one embodiment, a sample image containing the sample object may be obtained from local storage or cloud storage. Of course, in other embodiments, an image acquisition device may be used to acquire images of the sample object in real time to obtain a sample image containing the sample object.

[0081] In one embodiment, the sample object may be a pedestrian. Of course, in other embodiments, the sample object may also be an animal, an object, etc., which is not limited here.

[0082] Step S52: Segment the sample image using the region segmentation model to obtain region segmentation results for each sample object region in the sample image.

[0083] In this embodiment, the sample image is segmented using the region segmentation model to obtain the region segmentation result of each sample object region in the sample image.

[0084] Step S53: extracting features from the sample image to obtain first sample image features.

[0085] In this implementation, feature extraction is performed on the sample image to obtain a first sample image feature.

[0086] Step S54: Use different sub-networks of the attention generation network to process the first sample image features respectively to obtain the sample area attention position map corresponding to each sub-network.

[0087] In this embodiment, different sub-networks of the attention generation network are used to process the first sample image features respectively to obtain the sample area attention position map corresponding to each sub-network.

[0088] In one embodiment, the first sample image features include high-level features, and different sub-networks of the attention generation network are used to process the high-level features respectively to obtain the sample area attention position map corresponding to each sub-network.

[0089] Step S55: Based on the region segmentation results of each sample object region and the sample region attention position map corresponding to each sub-network, adjust the network parameters of each sub-network.

[0090] In this embodiment, the network parameters of each sub-network are adjusted based on the regional segmentation results of each sample object area and the sample area attention position map corresponding to each sub-network. The regional segmentation results of the sample image are used by the regional segmentation model to provide additional supervision signals for each sub-network, so that the generated sample area attention position map of each sub-network is more accurate, thereby improving the subsequent attribute recognition effect. In addition, the regional segmentation model is used for training, does not require additional annotation, and is only used in the training stage of the attention generation network, and does not occupy additional performance overhead when deployed.

[0091] In one embodiment, if Figure 6 As shown, Figure 6 yes Figure 5 The flowchart of step S55 of an embodiment shown in FIG. 1 is to adjust the network parameters of each sub-network based on the region segmentation results of each sample object region and the sample region attention position map corresponding to each sub-network, and specifically includes the following sub-steps:

[0092] Step S61: For each sub-network, the sub-loss of the sub-network is determined based on the sample area attention position map of the sub-network and the corresponding area segmentation result.

[0093] In this implementation, for each sub-network, the sub-loss of the sub-network is determined based on the sample region attention position map of the sub-network and the corresponding region segmentation result.

[0094] In one embodiment, for each sub-network, the sub-loss of the sub-network is determined based on the sample region attention position map of the sub-network and the corresponding region segmentation result, specifically: for each sub-network, the region segmentation contour of the corresponding region segmentation result is extracted; in response to the region segmentation contour being within the corresponding sample region attention position map, the sub-loss of the sub-network is determined to be a first preset value; in response to at least a portion of the region segmentation contour being outside the corresponding sample region attention position map, the sub-loss of the sub-network is determined to be the target distance between the region segmentation contour and the corresponding sample region attention position map.

[0095] That is to say, for each sub-network, when the region segmentation contour of the region segmentation result corresponding to the sub-network is located within the sample region attention position map corresponding to the sub-network, the sub-loss of the sub-network is a first preset value; when the region segmentation contour of the region segmentation result corresponding to the sub-network is at least partially located outside the sample region attention position map corresponding to the sub-network, the sub-loss of the sub-network is the target distance between the region segmentation contour and the sample region attention position map.

[0096] In a specific embodiment, the target distance is the shortest distance. That is, when the region segmentation contour of the region segmentation result corresponding to the sub-network is at least partially located outside the sample region attention position map corresponding to the sub-network, the sub-loss of the sub-network is the shortest distance between the region segmentation contour and the sample region attention position map.

[0097] In a specific implementation, the first preset value is 0.

[0098] Step S62: adjusting the network parameters of each sub-network based on the sum of the sub-losses of each sub-network.

[0099] In this embodiment, the network parameters of each sub-network are adjusted based on the sum of the sub-losses of each sub-network.

[0100] In one embodiment, before extracting a second target image feature related to at least one object attribute from the first target image feature, for each object attribute, a target object region in the target image associated with the object attribute and a correlation between the target object region and the object attribute are determined. At this time, the target object region corresponding to each object attribute and the correlation between the target object region and the object attribute are determined, specifically: using a plurality of gated subnetworks in the gated network to process the first target image feature respectively, and obtaining the processing results of each gated subnetwork; wherein the gated subnetwork is used to determine the correlation between an object attribute and a target object region, and at least one of the object attributes or target object regions corresponding to different gated subnetworks is different; in response to the processing result of the gated subnetwork being a second preset value, it is determined that there is no correlation between the object attribute corresponding to the gated subnetwork and the target object region; in response to the processing result of the gated subnetwork being a third preset value, it is determined that there is a correlation between the object attribute corresponding to the gated subnetwork and the target object region, and the correlation degree is the third preset value.

[0101] For example, the gated network includes gated subnetwork 1, gated subnetwork 2, gated subnetwork 3 and gated subnetwork 4; gated subnetwork 1 is used to determine the association between object attribute A and target object region α, gated subnetwork 2 is used to determine the association between object attribute A and target object region β, gated subnetwork 3 is used to determine the association between object attribute B and target object region α, and gated subnetwork 4 is used to determine the association between object attribute B and target object region β. Feature extraction is performed on a target image containing a target object to obtain a first target image feature; then, the first target image feature is input into gated subnetwork 1, gated subnetwork 2, gated subnetwork 3 and gated subnetwork 4 respectively; since object attribute A is related to target object region α, and object attribute B is related to target object region β, the processing results output by gated subnetwork 1 and gated subnetwork 4 are the third preset value, and the processing results output by gated subnetwork 2 and gated subnetwork 3 are the second preset value.

[0102] For example, the gated network includes gated subnetwork 1, gated subnetwork 2, gated subnetwork 3 and gated subnetwork 4; gated subnetwork 1 is used to determine the association between object attribute A and target object region α, gated subnetwork 2 is used to determine the association between object attribute A and target object region β, gated subnetwork 3 is used to determine the association between object attribute A and target object region γ, and gated subnetwork 4 is used to determine the association between object attribute A and target object region δ. Feature extraction is performed on a target image containing a target object to obtain a first target image feature; then, the first target image feature is input into gated subnetwork 1, gated subnetwork 2, gated subnetwork 3 and gated subnetwork 4 respectively; since object attribute A is related to target object region α and target object region β, the processing results output by gated subnetwork 1 and gated subnetwork 2 are the third preset value, and the processing results output by gated subnetwork 3 and gated subnetwork 4 are the second preset value.

[0103] In a specific implementation, the second preset value is 0. That is, when the output of the gated sub-network is 0, it indicates that there is no correlation between the object attribute determined by the gated sub-network and the target object region.

[0104] In a specific implementation, the third preset value is a non-zero positive value. That is, when the gated sub-network output is a non-zero positive value, it indicates that there is a correlation between the object attribute determined by the gated sub-network and the target object region, and the correlation degree is the third preset value.

[0105] In a specific implementation, the first target image feature includes high-level features, and the high-level features are processed separately using a plurality of gated sub-networks in the gated network to obtain processing results of each gated sub-network.

[0106] In one embodiment, the attribute recognition method provided by the present application is performed using an attribute recognition network, and there are three losses in the training of the attribute recognition network. The first loss is the attribute recognition result loss L1, which is calculated by the attribute recognition result and the true value label. The attribute recognition network includes an attention generation network, which includes several sub-networks, and the second loss is the sum of the sub-losses of each sub-network L2. The attribute recognition network includes a gating network, which includes several gating sub-networks. In order to improve the accuracy of the association between object attribute regions and reduce the influence of irrelevant regional features, the processing result output by the gating sub-network is used to calculate the loss L3, L3=L1-norm loss, to obtain a relatively sparse association vector. Therefore, the total loss TotalLoss corresponding to the attribute recognition network = a1*L1 + a2*L2 + a3*L3, where a1, a2 and a3 are hyperparameters that control the degree of influence of different losses. Then, the total loss corresponding to the attribute recognition network is back-propagated through the gradient to update the network parameters of the attribute recognition network. When the set network training stop condition is reached, the final attribute recognition network is obtained.

[0107] See also Figure 7 , Figure 7 : is a schematic diagram of the structure of an embodiment of an attribute recognition device provided by the present application. The attribute recognition device 70 includes an acquisition module 71, a first extraction module 72, a second extraction module 73 and a recognition module 74; the acquisition module 71 is used to acquire a target image containing a target object; the first extraction module 72 is used to perform feature extraction on the target image to obtain a first target image feature; the second extraction module 73 is used to extract a second target image feature related to at least one object attribute from the first target image feature; the recognition module 74 is used to perform object attribute recognition on the target image based on the second target image feature related to the object attribute for each object attribute, and obtain an attribute recognition result corresponding to the object attribute.

[0108] Among them, the attribute recognition device 70 also includes a determination module 75, which is used to determine, for each object attribute, a target object area associated with the object attribute in the target image before extracting a second target image feature related to at least one object attribute from the first target image feature; the second extraction module 73 is used to extract a second target image feature related to at least one object attribute from the first target image feature, including: for each object attribute, extracting a third target image feature belonging to the target object area associated with the object attribute from the first target image feature as the second target image feature.

[0109] Among them, there are at least two target object areas associated with the object attributes; the second extraction module 73 is used to extract, for each object attribute, from the first target image features, a third target image feature belonging to the target object area associated with the object attribute, including: for each object attribute, fusing the target area attention position maps of each target object area associated with the object attribute to obtain a fused attention position map; wherein each target element in the target area attention position map of the target object area represents the correlation between the corresponding target pixel point in the target image and the target object area; based on the fused attention position map, extracting the third target image feature belonging to the target object area associated with the object attribute from the first target image features.

[0110] Among them, the determination module 75 is used to extract, for each object attribute, the third target image feature belonging to the target object area associated with the object attribute from the first target image feature, and before using it as the second target image feature, for each object attribute, determine the correlation between each target object area corresponding to the object attribute and the object attribute; the second extraction module 73 is used to fuse the target area attention position maps of each target object area associated with the object attribute for each object attribute to obtain a fused attention position map, including: for each object attribute, using the correlation degree of the target object area corresponding to the object attribute as the weight of the target object area; based on the weight of each target object area, weightedly fused the target area attention position maps of each target object area to obtain a fused attention position map.

[0111] The step of determining the target area attention position map of each target object area includes:

[0112] Different subnetworks of the attention generation network are used to process the first target image features respectively to obtain the target area attention position map corresponding to each subnetwork; wherein different subnetworks generate target area attention position maps corresponding to different target object areas.

[0113] Among them, the attribute recognition device 70 also includes a training module 76, and the training module 76 is used for training the attention generation network, including: obtaining a sample image containing a sample object; and, using a region segmentation model to segment the sample image to obtain a region segmentation result of each sample object region in the sample image; extracting features from the sample image to obtain a first sample image feature; using different subnetworks of the attention generation network to process the first sample image feature respectively to obtain a sample region attention position map corresponding to each subnetwork; wherein different subnetworks generate sample region attention position maps for different sample object regions, and each sample element in the sample region attention position map of the sample object region represents the correlation between the corresponding sample pixel point in the sample image and the sample object region; based on the region segmentation results of each sample object region and the sample region attention position map corresponding to each subnetwork, the network parameters of each subnetwork are adjusted.

[0114] Among them, the training module 76 is used to adjust the network parameters of each sub-network based on the regional segmentation results of each sample object area and the sample area attention position map corresponding to each sub-network, including: for each sub-network, based on the sample area attention position map of the sub-network and the corresponding regional segmentation results, determine the sub-loss of the sub-network; based on the sum of the sub-losses of each sub-network, adjust the network parameters of each sub-network.

[0115] Among them, the training module 76 is used to determine the sub-loss of the sub-network for each sub-network based on the sample area attention position map of the sub-network and the corresponding area segmentation result, including: for each sub-network, extracting the area segmentation contour of the corresponding area segmentation result; in response to the area segmentation contour being within the corresponding sample area attention position map, determining the sub-loss of the sub-network to be a first preset value; in response to at least a portion of the area segmentation contour being outside the corresponding sample area attention position map, determining the sub-loss of the sub-network to be the target distance between the area segmentation contour and the corresponding sample area attention position map.

[0116] Among them, the above target distance is the shortest distance.

[0117] Among them, the above-mentioned first target image feature includes at least two levels of sub-features; the second extraction module 73 is used to extract the second target image feature related to at least one object attribute from the first target image feature, including: extracting features related to each object attribute from sub-features of different levels, respectively, as different levels of features corresponding to each object attribute; for each object attribute, fusing the different levels of features corresponding to the object attribute to obtain the second target image feature related to the object attribute.

[0118] Among them, the determination module 75 is used to determine, for each object attribute, the target object area associated with the object attribute and the correlation degree of the object attribute before extracting the second target image feature related to at least one object attribute from the first target image feature; the determination module 75 is used to determine the target object area corresponding to each object attribute and the correlation degree corresponding to the target object area, including: using several gated sub-networks in the gated network to process the first target image feature respectively to obtain the processing results of each gated sub-network; wherein the gated sub-network is used to determine the correlation between an object attribute and a target object area, and at least one of the object attributes or target object areas corresponding to different gated sub-networks is different; in response to the processing result of the gated sub-network being the second preset value, it is determined that there is no correlation between the object attribute corresponding to the gated sub-network and the target object area; in response to the processing result of the gated sub-network being the third preset value, it is determined that there is a correlation between the object attribute corresponding to the gated sub-network and the target object area, and the correlation degree is the third preset value.

[0119] Among them, the recognition module 74 is used to perform object attribute recognition on the target image based on the second target image features related to the object attributes, and obtain attribute recognition results corresponding to the object attributes, including: pooling the second target image features to obtain pooling results; linearly processing the pooling results to obtain attribute recognition results corresponding to the object attributes; and / or the above-mentioned target objects include pedestrians.

[0120] See also Figure 8 , Figure 8 80 is a schematic diagram of an embodiment of an electronic device provided by the present application. The electronic device 80 includes a memory 81 and a processor 82 coupled to each other, and the processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-mentioned attribute identification method embodiments. In a specific implementation scenario, the electronic device 80 may include, but is not limited to: a microcomputer, a server, and in addition, the electronic device 80 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0121] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps of any of the above-mentioned attribute identification method embodiments. The processor 82 can also be called a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 82 can be implemented by an integrated circuit chip.

[0122] See also Fig. 9 , Fig. 9 It is a structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 90 of the embodiment of the present application stores a program instruction 91, and when the program instruction 91 is executed, it implements the method provided by any embodiment of the attribute identification method of the present application and any non-conflicting combination. Among them, the program instruction 91 can form a program file and be stored in the above-mentioned computer-readable storage medium 90 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of the methods of each implementation method of the present application. The aforementioned computer-readable storage medium 90 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0123] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0124] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for attribute recognition, characterized in that: The method comprises: Acquire a target image containing a target object; Extracting features from the target image to obtain first target image features; determining at least two target object regions in the target image that are associated with at least one object attribute; For each of the object attributes, the target area attention position maps of each target object area associated with the object attribute are fused to obtain a fused attention position map; wherein each target element in the target area attention position map of the target object area represents the correlation between the corresponding target pixel point in the target image and the target object area; extracting, from the first target image features, a third target image feature belonging to the target object area associated with the object attribute as a second target image feature based on the fused attention position map; For each of the object attributes, object attribute recognition is performed on the target image based on a second target image feature related to the object attribute to obtain an attribute recognition result corresponding to the object attribute.

2. The method according to claim 1, characterized in that Before fusing the target area attention position maps of each target object area associated with each object attribute to obtain a fused attention position map, the method further includes: For each of the object attributes, determining a correlation between each of the target object regions corresponding to the object attribute and the object attribute; For each of the object attributes, the target area attention position maps of each target object area associated with the object attribute are fused to obtain a fused attention position map, including: For each of the object attributes, taking the relevance of the target object area corresponding to the object attribute as the weight of the target object area; Based on the weight of each of the target object regions, the target area attention position maps of each of the target object regions are weighted fused to obtain the fused attention position map.

3. The method according to claim 1, characterized in that: The step of determining the target area attention position map of each target object area comprises: The first target image features are processed respectively using different subnetworks of the attention generation network to obtain target area attention position maps corresponding to each subnetwork; wherein different subnetworks generate target area attention position maps corresponding to different target object areas.

4. The method according to claim 3, characterized in that The training steps of the attention generation network include: Get a sample image containing a sample object; And, segmenting the sample image using a region segmentation model to obtain a region segmentation result of each sample object region in the sample image; Extracting features from the sample image to obtain first sample image features; Using different subnetworks of the attention generation network to process the first sample image features respectively, and obtain sample area attention position maps corresponding to each of the subnetworks; wherein different subnetworks generate sample area attention position maps of different sample object areas correspondingly, and each sample element in the sample area attention position map of the sample object area represents the correlation between the corresponding sample pixel point in the sample image and the sample object area; Based on the region segmentation results of each of the sample object regions and the sample region attention position maps corresponding to each of the sub-networks, the network parameters of each of the sub-networks are adjusted.

5. The method according to claim 4, characterized in that The adjusting the network parameters of each sub-network based on the region segmentation results of each sample object region and the sample region attention position map corresponding to each sub-network includes: For each of the sub-networks, determining a sub-loss of the sub-network based on a sample region attention position map of the sub-network and a corresponding region segmentation result; Based on the sum of the sub-losses of the sub-networks, the network parameters of the sub-networks are adjusted.

6. The method according to claim 5, characterized in that For each of the sub-networks, based on the sample area attention position map of the sub-network and the corresponding area segmentation result, determining the sub-loss of the sub-network includes: For each of the sub-networks, extracting a region segmentation contour of the corresponding region segmentation result; In response to the region segmentation contour being located within the corresponding sample region attention position map, determining the sub-loss of the sub-network to be a first preset value; In response to at least a portion of the region segmentation contour being located outside the corresponding sample region attention position map, a sub-loss of the sub-network is determined as a target distance between the region segmentation contour and the corresponding sample region attention position map.

7. The method according to claim 6, characterized in that The target distance is the shortest distance.

8. The method according to claim 1, characterized in that: The first target image feature includes at least two levels of sub-features; and extracting a third target image feature belonging to the target object area associated with the object attribute from the first target image feature based on the fused attention position map includes: Based on the fused attention position map, extracting features belonging to the target object area associated with the object attribute from sub-features at different levels as features at different levels corresponding to the object attribute; The different levels of features corresponding to the object attributes are integrated to obtain the third target image feature.

9. The method according to claim 1, characterized in that: Before fusing the target area attention position maps of each target object area associated with each object attribute to obtain a fused attention position map, the method further includes: For each of the object attributes, determining a degree of association between a target object region associated with the object attribute and the object attribute; Determining the target object area corresponding to each of the object attributes and the correlation degree corresponding to the target object area includes: Using several gated sub-networks in the gated network to process the first target image features respectively, and obtain processing results of each gated sub-network; wherein the gated sub-network is used to determine the association between an object attribute and a target object region, and at least one of the object attributes or target object regions corresponding to different gated sub-networks is different; In response to the processing result of the gated sub-network being a second preset value, determining that there is no association between the object attribute corresponding to the gated sub-network and the target object region; In response to the processing result of the gated sub-network being a third preset value, it is determined that there is a correlation between the object attribute corresponding to the gated sub-network and the target object region, and the correlation degree is the third preset value.

10. The method according to claim 1, characterized in that The performing object attribute recognition on the target image based on the second target image feature related to the object attribute to obtain an attribute recognition result corresponding to the object attribute includes: Performing pooling processing on the second target image features to obtain a pooling result; Performing linear processing on the pooling result to obtain an attribute recognition result corresponding to the object attribute; And / or, the target object includes a pedestrian.

11. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores program instructions, and the processor is configured to execute the program instructions to implement the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the method according to any one of claims 1 to 10.

Images