Image-based target attribute detection method and device

Abstract

The application discloses an image-based target attribute detection method, which comprises the following steps: acquiring first image data, performing target detection based on the first image data to obtain a detected target, acquiring a target visibility evaluation result of the detected target based on second image data of the detected target, wherein the target visibility evaluation result is used to represent the degree of occlusion of the detected target, matching a corresponding attribute recognition model according to the target visibility evaluation result, wherein the attribute recognition model is different with the degree of overall occlusion of the detected target, and performing attribute recognition on the second image data according to the matched attribute recognition model to obtain an attribute recognition result. The application improves the performance of target attribute recognition under different degrees of occlusion while not reducing the attribute recognition effect of unoccluded targets, and has the characteristics of high recognition accuracy, strong practicability and strong generalization.

Description

Technical Field

[0001] This invention relates to the field of image detection, and in particular, to a method for detecting target attributes based on images. Background Technology

[0002] With the development of image detection technology, image-based target detection has been widely used. Target detection not only identifies the target, but also further identifies the target's attributes. For example, in face detection, attributes such as the target's gender and age group can also be identified.

[0003] In some application scenarios, there are situations where parts of the image of the target to be detected are occluded or covered. For example, due to wearing sunglasses or masks, there may be occluded areas in the facial image. Due to insufficient image information, the target attribute detection performance is unsatisfactory. Summary of the Invention

[0004] This invention provides an image-based target attribute detection method to improve target attribute recognition performance in situations where image information is insufficient.

[0005] This invention provides an image-based target attribute detection method, the method comprising:

[0006] Acquire the first image data.

[0007] Based on the first image data, target detection is performed to obtain the second image data of the detected target.

[0008] Based on the second image data of the detected target, the target visibility assessment result is obtained, whereby the target visibility assessment result is used to characterize the degree to which the detected target is occluded.

[0009] Based on the target visibility assessment results, a corresponding attribute recognition model is matched. The attribute recognition model varies depending on the degree of occlusion of the overall detected target.

[0010] According to the matched attribute recognition model, attribute recognition is performed on the second image data to obtain the attribute recognition result.

[0011] Preferably, the step of obtaining the target visibility assessment result of the detected target based on the second image data of the detected target includes:

[0012] The second image data is segmented to obtain at least one segmented region. Based on the segmented regions, the visibility of the segmented regions is obtained, wherein the visibility of the segmented regions is used to characterize the degree to which the segmented regions are occluded.

[0013] Based on the visibility of all the segmented regions, the target visibility assessment result of the detected target is determined.

[0014] Preferably, the step of performing image segmentation on the second image data to obtain at least one segmented region includes:

[0015] Using a semantic segmentation model, the second image data is semantically segmented to obtain at least one segmented region.

[0016] The step of obtaining the visibility of the segmented region in the detected target based on the segmented region includes:

[0017] For each segmented region, determine the visibility of each segmented region to obtain the segmented region visibility.

[0018] The visibility of the segmented region is defined as the ratio of the area of ​​the unobstructed region within the segmented region to the total area of ​​the segmented region.

[0019] Preferably, the second image data is image data of a T-shaped region formed by the eyes, nose, eyebrows, and mouth;

[0020] The step of using a semantic segmentation model to perform semantic segmentation on the second image data to obtain at least one segmented region includes:

[0021] Using a semantic segmentation model, the T-shaped region image data is segmented into a T-shaped region consisting of the eyes, nose, eyebrows, and mouth.

[0022] For each segmented region, determining the visibility of each segmented region, and obtaining the segmented region visibility of each segmented region includes:

[0023] The visibility of the T-shaped region is obtained by determining the ratio of the area of ​​the unobstructed area to the total area of ​​the T-shaped region.

[0024] Preferably, determining the target visibility assessment result of the detected target based on the visibility of all the segmented regions includes:

[0025] Using a graph convolutional neural network model, the visibility weights of each segmented region are obtained, whereby the weights characterize the degree of occlusion of the segmented region on the detected target.

[0026] For the visibility of each segmented region, a weighted average is calculated using the visibility weights of that segmented region.

[0027] The target visibility assessment result is obtained by summing up the visibility of all weighted segmented regions.

[0028] Preferably, the step of matching the corresponding attribute recognition model based on the target visibility assessment result includes:

[0029] If the target visibility assessment result is greater than the set first threshold, then the first attribute recognition model is matched. The first attribute recognition model is used for attribute recognition of unobstructed targets.

[0030] If the target visibility assessment result is between the first threshold and the second threshold, then a second attribute recognition model is matched. This second attribute recognition model is used for attribute recognition of partially occluded targets; wherein, the second threshold is less than the first threshold.

[0031] If the target visibility assessment result is less than the second threshold, then it is determined whether the detected target is completely occluded, or whether the target itself and the set area outside the target are both occluded.

[0032] If the target itself is completely occluded, then the third attribute recognition model is matched. The third attribute recognition model is used for attribute recognition when the target itself is completely occluded.

[0033] If the target itself and the set range outside the target are both occluded, then the fourth attribute recognition model is matched. The third attribute recognition model is used for attribute recognition when the target itself and the set range outside the target are both occluded.

[0034] Preferably, the step of matching the corresponding attribute recognition model based on the target visibility assessment result includes:

[0035] The target visibility assessment result is classified into three categories: no occlusion, partially occluded, completely occluded, and occluded both the target itself and a set range outside the target itself, and the matched attribute recognition model is output after classifying the target visibility assessment result into three categories: no occlusion, partially occluded, completely occluded, and occluded both the target itself and the target itself.

[0036] The matching model is trained together with the matched attribute recognition model, and the matching parameters of the matching model and the model parameters of the matched attribute recognition model are learned through the loss function.

[0037] Preferably, the attribute recognition model is a neural network model.

[0038] The step of performing attribute recognition on the second image data according to the matched attribute recognition model to obtain the attribute recognition result includes:

[0039] In the unobstructed scenario, the second image data is used to perform attribute recognition on the first attribute recognition model for attribute recognition of unobstructed targets, and the attribute recognition result is obtained.

[0040] In cases of partial occlusion, a second attribute recognition model, designed for attribute identification of partially occluded targets, is used to perform attribute identification on the second image data, yielding the attribute identification results.

[0041] When the target itself is completely occluded, a third attribute recognition model, designed for attribute recognition when the target is completely occluded, is used to perform attribute recognition on the second image data and the supplementary image data, yielding the attribute recognition results.

[0042] When the target itself and the set range outside the target are both occluded, additional image data associated with the detected target are acquired. The second image data and the additional image data are then subjected to attribute recognition by a fourth attribute recognition model used for attribute recognition when the target itself and the set range outside the target are both occluded, and the attribute recognition result is obtained.

[0043] The additional image data consists of feature image data associated with the detected target.

[0044] Preferably, the target attribute detection is facial attribute detection.

[0045] The segmented region includes at least one of the following: an eye region, a nose region, an eyebrow region, a mouth region, and a skin region, wherein the skin region is the area of ​​the face excluding the T-shaped area formed by the eyes, nose, eyebrows, and mouth.

[0046] The step of matching the corresponding attribute recognition model based on the target visibility assessment result includes:

[0047] The target visibility assessment result is classified into four categories: unobstructed face, partially obstructed face, completely obstructed face, and completely obstructed head, and the matched attribute recognition model is output after classifying the matching model for the matching attribute recognition model.

[0048] The step of performing attribute recognition on the second image data according to the matched attribute recognition model to obtain the attribute recognition result includes:

[0049] In cases where the face is unobstructed, the second image data is used to perform attribute recognition on the second image data using a first attribute recognition model for unobstructed facial attribute recognition, thus obtaining the attribute recognition result.

[0050] When a part of the face is obscured, a second attribute recognition model for partially obscured facial attributes is used to perform attribute recognition on the second image data, yielding the attribute recognition result.

[0051] When the face is completely occluded, a third attribute recognition model, used for attribute recognition in cases of complete facial occlusion, is used to perform attribute recognition on the second image data and the attribute feature image data, yielding the attribute recognition result.

[0052] In the case where the head is completely occluded, the fourth attribute recognition model used for attribute recognition when the head is completely occluded is used to perform attribute recognition on the second image data and attribute feature image data to obtain the attribute recognition result.

[0053] The attribute recognition results include age group and gender, and the attribute image data is feature image data associated with age group and gender.

[0054] The present invention also provides an image-based target attribute detection device, the device comprising:

[0055] The image acquisition module is used to acquire the first image data.

[0056] The first detection module is used to perform target detection based on the first image data to obtain the detected target.

[0057] The second detection module is used to obtain the target visibility assessment result of the detected target based on the second image data of the detected target. The target visibility assessment result is used to characterize the degree to which the detected target is occluded.

[0058] Based on the target visibility assessment results, a corresponding attribute recognition model is matched. The attribute recognition model varies depending on the degree of occlusion of the overall detected target.

[0059] According to the matched attribute recognition model, attribute recognition is performed on the second image data to obtain the attribute recognition result.

[0060] The present invention further provides a facial recognition system, the system including an image acquisition device, the device including a memory and a processor and an image module for acquiring image data, the memory storing a computer program, the computer program being executed by the processor to implement the steps of any of the image-based target attribute detection methods.

[0061] Preferably, the system further includes an access control device for access control based on attribute recognition results.

[0062] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the image-based target attribute detection methods.

[0063] The image-based target attribute detection method provided in this invention evaluates the degree of target occlusion and matches the corresponding attribute recognition model for detection. This improves the performance of target attribute recognition under different degrees of occlusion without reducing the attribute recognition effect of unoccluded targets. It features high recognition accuracy, practicality and strong generalization, which is beneficial for the application of target attribute detection in complex scenes. Attached Figure Description

[0064] Figure 1 This is a schematic flowchart of an image-based target attribute detection method according to an embodiment of this application;

[0065] Figure 2 This is a schematic diagram of a target attribute detection method according to an embodiment of this application.

[0066] Figure 3 This is a schematic flowchart of a face-based attribute detection method according to an embodiment of this application.

[0067] Figure 4 This is a schematic diagram illustrating how to obtain face visibility scores using semantic segmentation network models and GCN models.

[0068] Figure 5 This is a schematic diagram of a test image.

[0069] Figure 6 This is a schematic diagram of an image-based target attribute detection device.

[0070] Figure 7 This is another schematic diagram of an image-based target attribute detection device. Detailed Implementation

[0071] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0072] This application quantifies the degree of target occlusion and uses different attribute recognition models to identify target attributes according to the degree of target occlusion, thereby improving the target attribute recognition performance.

[0073] See Figure 1 As shown, Figure 1 This is a schematic flowchart of an image-based target attribute detection method according to an embodiment of this application. The method includes:

[0074] Step 101, acquire the first image data.

[0075] Step 102: Based on the first image data, perform target detection to obtain the second image data of the detected target.

[0076] Step 103: Based on the second image data, obtain the target visibility assessment result of the detected target, wherein the target visibility assessment result is used to characterize the degree to which the detected target is occluded overall.

[0077] Step 104: Based on the target visibility assessment results, match the corresponding attribute recognition model. The attribute recognition model varies depending on the degree of occlusion of the detected target as a whole.

[0078] Step 105: Perform attribute recognition on the second image data according to the matched attribute recognition model to obtain the attribute recognition result.

[0079] To facilitate understanding of the embodiments of this application, specific details are provided herein.

[0080] See Figure 2 As shown, Figure 2 This is a flowchart illustrating a target attribute detection method according to an embodiment of this application. The method includes:

[0081] Step 201: Obtain the first image data of the target to be detected.

[0082] Step 202: Based on the first image data, perform target detection to obtain the detected target.

[0083] As an example, target detection is performed on the first image data to identify targets in the image, and these targets can be marked with bounding boxes. The image data within the bounding boxes is then identified as the second image data.

[0084] Step 203: Based on the second image data of the detected target, obtain the target visibility assessment result of the detected target, wherein the target visibility assessment result is used to characterize the degree to which the detected target is occluded.

[0085] As an example, a semantic segmentation model is used to perform semantic segmentation on the second image data, resulting in at least one segmented region. For each segmented region, the visibility of each segmented region is determined, thus obtaining the visibility of each segmented region.

[0086] The visibility of the segmented region is defined as the ratio of the area of ​​the unobstructed region to the area of ​​the segmented region itself, which represents the degree to which the segmented region is obstructed.

[0087] Using a graph convolutional neural network model, the visibility weights of each segmented region are obtained, whereby the weights characterize the degree of occlusion of the segmented region on the detected target.

[0088] For the visibility of each segmented region, a weighted average is calculated using the visibility weights of that segmented region.

[0089] The target visibility assessment result is obtained by summing up the visibility of all weighted segmented regions.

[0090] Step 204: Based on the target visibility assessment results, match the corresponding attribute recognition model. The attribute recognition model varies depending on the degree of occlusion of the overall detected target.

[0091] As an example:

[0092] If the target visibility assessment result is greater than the set first threshold, then the first attribute recognition model is matched. The first attribute recognition model is used for attribute recognition of unobstructed targets.

[0093] If the target visibility assessment result is between the first threshold and the second threshold, then a second attribute recognition model is matched. This second attribute recognition model is used for attribute recognition of partially occluded targets; wherein, the second threshold is less than the first threshold.

[0094] If the target visibility assessment result is less than the second threshold, then it is determined whether the detected target is completely occluded, or whether the target itself and the set area outside the target are both occluded.

[0095] If the target itself is completely occluded, then the third attribute recognition model is matched. The third attribute recognition model is used for attribute recognition when the target itself is completely occluded.

[0096] If the target itself and the set range outside the target are both occluded, then the fourth attribute recognition model is matched. The third attribute recognition model is used for attribute recognition when the target itself and the set range outside the target are both occluded.

[0097] As another example:

[0098] The matching model used to match the attribute recognition model classifies the target visibility assessment results into three categories: no occlusion, partially occluded, the target itself is completely occluded, and the target itself and the set range outside the target are occluded. The matching attribute recognition model is then output.

[0099] The matching model is trained together with the matching attribute recognition model. The matching parameters of the matching model and the model parameters of the matching attribute recognition model are learned through the loss function. This ensures unobstructed attribute recognition while improving the accuracy of various occluded target attribute recognition.

[0100] The above attribute recognition models are implemented through various attribute recognition models, which are neural network models.

[0101] Step 205: Perform attribute recognition on the second image data according to the matched attribute recognition model to obtain the attribute recognition result.

[0102] As an example:

[0103] In the unobstructed scenario, the second image data is used to perform attribute recognition on the first attribute recognition model for attribute recognition of unobstructed targets, and the attribute recognition result is obtained.

[0104] In cases of partial occlusion, a second attribute recognition model, designed for attribute identification of partially occluded targets, is used to perform attribute identification on the second image data, yielding the attribute identification results.

[0105] When the target itself is completely occluded, a third attribute recognition model, designed for attribute recognition when the target is completely occluded, is used to perform attribute recognition on the second image data and the supplementary image data, yielding the attribute recognition results.

[0106] When the target itself and the set range outside the target are both occluded, additional image data associated with the detected target are acquired. The second image data and the additional image data are then subjected to attribute recognition by a fourth attribute recognition model used for attribute recognition when the target itself and the set range outside the target are both occluded, and the attribute recognition result is obtained.

[0107] The additional image data consists of feature image data associated with the detected target.

[0108] This embodiment improves the performance of target attribute recognition under different degrees of occlusion without reducing the effect of target image attribute recognition when the image is not occluded.

[0109] To facilitate understanding of the embodiments of this application, the following description uses face-based age group and gender attribute detection as an example. It should be understood that this application is not limited to face-based age group and gender attribute detection, and can also be applied to attribute detection of other targets.

[0110] In face detection technology:

[0111] Method 1: Obtain the region of interest of the face based on face detection and key point detection, then use feature extraction and gray-level histogram statistics to characterize the gender features of the face, and finally obtain the gender recognition result through a trained gender classifier;

[0112] Method 2 involves collecting multiple images of a person to form training samples, obtaining the corresponding gender expectation value through a recognition network, and training until convergence by ensuring that the expectation value of the same person is the same. This improves the robustness of gender recognition for the same person in different environments.

[0113] Method 3 is a pedestrian gender recognition method. It extracts and analyzes pedestrian features from multiple angles, builds a convolutional neural network to train and obtain weights corresponding to different human body features, and finally performs gender recognition through a fully connected layer.

[0114] Method 4: Input the face image into a trained age judgment model to obtain probability values ​​for different ages, and then sum them by weight to obtain the final estimated age;

[0115] Method 5: Randomly crop the collected resume portrait images to increase the model's generalization ability and enhance its adaptability to scenes with varying lighting, shadows, and occlusions, in order to solve the problem of portrait age prediction.

[0116] Method 6: Based on a trained recognition model targeting multiple age groups, determine the consumer's gender and age range.

[0117] The aforementioned face detection methods suffer from a common problem: they all directly obtain face regions through face detection and keypoint detection without assessing the degree of occlusion in the input image. Therefore, if a face is occluded, the gender recognition model cannot correctly determine the gender of the input face image. Even when incorporating features such as clothing and body shape to identify gender, it still lacks specificity. While randomly cropping the face image can enhance the model's generalization ability, random cropping can only simulate a very limited number of real-world occlusion scenarios, limiting the model's practical scene coverage. Therefore, there is an urgent need to address the need for targeted gender and age determination for occluded faces in different scenarios.

[0118] See Figure 3 As shown, Figure 3 This is a schematic flowchart illustrating a face-based attribute detection method according to an embodiment of this application. The method includes:

[0119] Step 301: Acquire the first image data captured by the camera or other device;

[0120] Step 302: Based on the acquired first image data, perform face detection to obtain second image data including faces, and preprocess the acquired second image data, such as image enhancement or image alignment.

[0121] Step 303: Based on the preprocessed second image, obtain the visibility of a specified region in the second image.

[0122] As an example, a semantic segmentation model is used to perform semantic segmentation on a face image to obtain segmented regions for each part. For each segmented region, the ratio of the area of ​​the unoccluded area in the segmented region to the area of ​​the segmented region is calculated to obtain the visibility of the segmented region. The visibility of the segmented region is used to characterize the degree to which the segmented region is occluded.

[0123] For example, according to semantic segmentation, a face can be segmented into: the eye region, the nose region, the mouth region, the eyebrow region, and the skin region excluding the T-shaped region formed by the eyes, nose, mouth, and eyebrows.

[0124] Visibility includes: eye visibility, nose visibility, mouth visibility, eyebrow visibility, and skin visibility.

[0125] in,

[0126] Eye visibility is defined as the ratio of the area of ​​the unobstructed region of the eye to the total area of ​​the eye region.

[0127] Nose visibility is defined as the ratio of the area of ​​the unobstructed region of the nose to the total area of ​​the nose region.

[0128] Mouth visibility is defined as the ratio of the area of ​​the unobstructed portion of the mouth region to the total area of ​​the mouth region.

[0129] Eyebrow visibility is defined as the ratio of the area of ​​the unobstructed portion of the eyebrow region to the total area of ​​the eyebrow region.

[0130] Skin visibility is the ratio of the area of ​​the unobstructed region of the skin to the total area of ​​the skin region.

[0131] As another example, the second image data is a T-shaped region image data formed by the eyes, nose, eyebrows, and mouth. Using a semantic segmentation model, the T-shaped region image data is segmented into T-shaped regions composed of the eyes, nose, eyebrows, and mouth. The visibility of the T-shaped region is obtained by determining the ratio of the area of ​​the unoccluded area within the T-shaped region to the total area of ​​the T-shaped region. This method helps reduce the complexity of region segmentation and improves the efficiency of attribute recognition.

[0132] The aforementioned obstructions can come from objects such as glasses, masks, and hair.

[0133] It should be understood that this embodiment is not limited to the above-mentioned part segmentation and the above-mentioned part visibility. In specific applications, it can be set according to the application scenario and needs.

[0134] Step 304: Determine the face visibility assessment result based on the visibility of each part in the face image. This assessment result is used to characterize the degree to which the face is occluded as a whole.

[0135] Given the diverse types and varying areas of facial occlusion in real-world scenarios, and the different impacts of occluded regions on attribute recognition, it is necessary to measure facial visibility under arbitrary occlusion conditions. As an example, the visibility of each part is input into a graph convolutional neural network (GCN) model to obtain a facial visibility score representing the evaluation results.

[0136] See Figure 4 As shown, Figure 4This diagram illustrates how a semantic segmentation network (SSN) and a graph network network (GCN) model can be used to obtain a face visibility score. The face image is input into the trained SSN to obtain the visibility of each part. This visibility is then input into the GCN model, which autonomously learns the weights of each part's occlusion severity through a graph structure network. Finally, the GCN model outputs a face visibility score to characterize the severity of face occlusion. The SSN can be trained using real-world occluded training data. The face visibility score is a continuous value between 0 and 1, where 0 indicates the most severe occlusion and 1 indicates no occlusion. The relationship between the visibility of each part and the overall visibility score satisfies the following formula:

[0137] visible=w1*eye_visible+w2*nose_visible+w3*mouth_visible+w4*eyebrow_visible+w5*skin_visible

[0138] Among them, w1, w2, w3, w4, and w5 are the respective weights of eye_visible, nose_visible, mouth_visible, eyebrow_visible, and skin_visible, respectively.

[0139] Step 305: Based on the face visibility assessment results, match the corresponding attribute recognition model;

[0140] In this step, attribute recognition models are automatically matched based on face visibility scores.

[0141] As an example, the attribute recognition model includes: a first attribute recognition model for recognizing attributes of unobstructed normal faces, a second attribute recognition model for recognizing attributes of partially occluded faces, a third attribute recognition model for recognizing attributes of fully occluded faces, and a fourth attribute recognition model for recognizing attributes of faces with the head completely occluded.

[0142] in,

[0143] When the first and second attribute recognition models perform attribute recognition, they are mainly based on facial features, which are mostly related to contours, shapes, textures, and angles. Each attribute is located in a specific area of ​​the face. The attributes of the face image are predicted by extracting the facial features.

[0144] When performing attribute recognition, the third and fourth attribute recognition models mainly rely on additional information other than the face, including feature image data related to age group and gender such as body shape, clothing, jewelry, and dressing style, and combine this information to predict attributes.

[0145] Matching strategies can be:

[0146] If the face visibility assessment result is greater than the set first threshold, it means that there is no occluded face, and the attribute recognition is directly performed through the first attribute recognition model; wherein, the first attribute recognition model is pre-trained with unoccluded face data.

[0147] If the face visibility assessment result is between the first threshold and the second threshold, it indicates that there is a partially occluded face, and the second attribute recognition model is used for attribute recognition. Wherein, if the first threshold is greater than the second threshold, the second attribute recognition model is trained to make attribute judgments on the facial features separately.

[0148] If the face visibility assessment result is less than the second threshold, it is considered an invalid recognition area. A binary classification model is used to determine whether the face or the head is completely obscured. If the face is completely obscured, a third attribute recognition model is used for attribute recognition. If the head is completely obscured, a fourth attribute recognition model is used for attribute recognition.

[0149] As an example, a matching model can be constructed for matching attribute recognition models. The face visibility assessment results are classified into four categories: no occlusion, partial occlusion, complete occlusion of the face, and complete occlusion of the head. The matching attribute recognition model is then output. In this way, during model training, the matching model is trained together with the matching attribute recognition model, which helps to improve the performance of attribute recognition.

[0150] Step 306: Input the face image into the matched attribute recognition model to obtain the final attribute recognition result.

[0151] Since the third attribute recognition model or the fourth attribute recognition model also needs to integrate additional information beyond the face image when performing attribute recognition, the additional images, including those other than the face image, are also input into the third attribute recognition model or the fourth attribute recognition model.

[0152] As an example, attribute recognition results can include gender, age group, and facial expression. The gender prediction results include female and male; the age group prediction results include child, teenager, youth, middle-aged, and elderly. Specifically, children are 0-7 years old, teenagers are 7-17 years old, youth are 17-40 years old, middle-aged are 40-65 years old, and elderly are >65 years old; facial expressions include laughing, smiling, angry, crying, etc.

[0153] like Figure 4The test image, after semantic segmentation, yielded visibility scores of 0.99 for the eyes, 0.99 for the nose, 0.98 for the mouth, 0.99 for the eyebrows, and 0.93 for the skin. The overall facial visibility score obtained using the GCN model was 0.94, indicating partial occlusion. The matched second attribute recognition model performed well in predicting attributes based on facial information. The final prediction result of the model trained on the occluded face was female and a teenager.

[0154] like Figure 5 The test image, calculated using a segmentation network, showed the following facial feature visibility: eyes: 0, nose: 0.87, mouth: 0.99, eyebrows: 0, and skin: 0.78. The overall facial visibility obtained through GCN was 0.80. This image also shows that the eyes have a significant impact on the overall facial feature visibility. Using the aforementioned matching strategy, a partially occluded facial attribute prediction model was selected, and the model's final prediction result was male and middle-aged.

[0155] This embodiment of the method for detecting attributes of occluded faces takes a practical application approach, considering the richness of occluded face data scenarios. It outputs the occlusion ratio of a set part through a semantic segmentation model, then calculates a comprehensive score for face visibility through machine learning. A matching model automatically matches the corresponding scene attribute recognition model, and finally outputs the attribute recognition result using the matched model. This embodiment comprehensively utilizes face-related information, effectively breaking down the complex attribute recognition task across all scenarios based on information importance and matching strategies. It automatically selects an attribute recognition model for occluded face attribute recognition, improving the accuracy of attribute recognition in occluded scenarios while maintaining the attribute recognition effect for unoccluded faces. Furthermore, it demonstrates strong practicality and generalization.

[0156] See Figure 6 As shown, Figure 6 This is a schematic diagram of an image-based target attribute detection device. The device includes:

[0157] The image acquisition module is used to acquire the first image data.

[0158] The first detection module is used to perform target detection based on the first image data to obtain the detected target.

[0159] The second detection module is used to obtain the target visibility assessment result of the detected target based on the second image data of the detected target. The target visibility assessment result is used to characterize the degree to which the detected target is occluded.

[0160] Based on the target visibility assessment results, a corresponding attribute recognition model is matched. The attribute recognition model varies depending on the degree of occlusion of the overall detected target.

[0161] According to the matched attribute recognition model, attribute recognition is performed on the second image data to obtain the attribute recognition result.

[0162] The second detection module includes:

[0163] The segmentation region visibility determination submodule is used to perform semantic segmentation on the second image data using a semantic segmentation model to obtain at least one segmented region, and for each segmented region, determine the visibility of each segmented region.

[0164] The target visibility determination submodule uses a graph convolutional neural network model to obtain the weight of the visibility of each segmented region. The weight is used to characterize the degree of occlusion of the segmented region on the detected target. For the visibility of each segmented region, the weight of the visibility of the segmented region is used for weighting. All weighted segmented region visibilitys are accumulated to obtain the target visibility evaluation result.

[0165] The matching submodule is used to match the corresponding attribute recognition model based on the target visibility assessment result.

[0166] The attribute recognition submodule performs attribute recognition on the second image data according to the matched attribute recognition model to obtain the attribute recognition result.

[0167] See Figure 7 As shown, Figure 7 This is another schematic diagram of an image-based target attribute detection device. The device includes an image acquisition unit, a memory, and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the image-based target attribute detection method described in the embodiments of this application.

[0168] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0169] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0170] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image-based target attribute detection method described in this application.

[0171] This invention also provides a facial recognition system, comprising an image acquisition device including a memory, a processor, and an image module for acquiring image data. The memory stores a computer program, which, when executed by the processor, implements the steps of any of the image-based target attribute detection methods. The system further includes an access control device for performing access control based on attribute recognition results. For the device / network-side device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple; relevant details can be found in the descriptions of the method embodiments.

[0172] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target attribute detection method based on images, characterized in that, The method includes: Acquire the first image data. Based on the first image data, target detection is performed to obtain the second image data of the detected target. Based on the second image data of the detected target, the target visibility assessment result is obtained, whereby the target visibility assessment result is used to characterize the degree to which the detected target is occluded. Based on the target visibility assessment results, a corresponding attribute recognition model is matched. The attribute recognition model varies depending on the degree of occlusion of the overall detected target. According to the selected attribute recognition model, attribute recognition is performed on the second image data to obtain the attribute recognition result; The target visibility assessment results are obtained as follows: The second image data is segmented to obtain at least one segmented region. Based on the segmented region, the visibility of the segmented region in the detected target is obtained, wherein the visibility of the segmented region is the ratio of the area of ​​the unobstructed area in the segmented region to the area of ​​the segmented region. The visibility of each segmented region is input into a graph convolutional neural network model. The graph structure network autonomously learns the weights of each segmented region's occlusion level, which characterize the degree of occlusion of the segmented region on the detected target. For the visibility of each segmented region, a weighted average is calculated using the visibility weights of that segmented region. The target visibility assessment result is obtained by summing up the visibility of all weighted segmented regions.

2. The method as described in claim 1, characterized in that, The step of segmenting the second image data to obtain at least one segmented region includes: Using a semantic segmentation model, the second image data is semantically segmented to obtain at least one segmented region. The step of obtaining the visibility of the segmented region in the detected target based on the segmented region includes: For each segmented region, determine the visibility of each segmented region to obtain the segmented region visibility of each segmented region.

3. The method as described in claim 2, characterized in that, The second image data is image data of a T-shaped region formed by the eyes, nose, eyebrows, and mouth; The step of using a semantic segmentation model to perform semantic segmentation on the second image data to obtain at least one segmented region includes: Using a semantic segmentation model, the T-shaped region image data is segmented into a T-shaped region consisting of the eyes, nose, eyebrows, and mouth. For each segmented region, determining the visibility of each segmented region, and obtaining the segmented region visibility of each segmented region includes: The visibility of the T-shaped region is obtained by determining the ratio of the area of ​​the unobstructed area to the total area of ​​the T-shaped region.

4. The method as described in claim 1, characterized in that, The step of matching the corresponding attribute recognition model based on the target visibility assessment result includes: If the target visibility assessment result is greater than the set first threshold, then the first attribute recognition model is matched. The first attribute recognition model is used for attribute recognition of unobstructed targets. If the target visibility assessment result is between the first threshold and the second threshold, then a second attribute recognition model is matched. This second attribute recognition model is used for attribute recognition of partially occluded targets; wherein, the second threshold is less than the first threshold. If the target visibility assessment result is less than the second threshold, then it is determined whether the detected target is completely occluded, or whether the target itself and the set area outside the target are both occluded. If the target itself is completely occluded, then the third attribute recognition model is matched. The third attribute recognition model is used for attribute recognition when the target itself is completely occluded. If the target itself and the set range outside the target are both occluded, then the fourth attribute recognition model is matched. The third attribute recognition model is used for attribute recognition when the target itself and the set range outside the target are both occluded.

5. The method as described in claim 1, characterized in that, The step of matching the corresponding attribute recognition model based on the target visibility assessment result includes: The target visibility assessment result is classified into three categories: no occlusion, partially occluded, completely occluded, and occluded both the target itself and a set range outside the target itself, using a matching model for matching attribute recognition model. The matched attribute recognition model is then output. The matching model is trained together with the matched attribute recognition model, and the matching parameters of the matching model and the model parameters of the matched attribute recognition model are learned through the loss function.

6. The method as described in claim 4 or 5, characterized in that, The attribute recognition model is a neural network model. The step of performing attribute recognition on the second image data according to the matched attribute recognition model to obtain the attribute recognition result includes: In the unobstructed scenario, the second image data is used to perform attribute recognition on the first attribute recognition model for attribute recognition of unobstructed targets, and the attribute recognition result is obtained. In cases of partial occlusion, a second attribute recognition model, designed for attribute identification of partially occluded targets, is used to perform attribute identification on the second image data, yielding the attribute identification results. When the target itself is completely occluded, a third attribute recognition model, designed for attribute recognition when the target is completely occluded, is used to perform attribute recognition on the second image data and the supplementary image data, yielding the attribute recognition results. When the target itself and the set range outside the target are both occluded, additional image data associated with the detected target are acquired. The second image data and the additional image data are then subjected to attribute recognition by a fourth attribute recognition model used for attribute recognition when the target itself and the set range outside the target are both occluded, and the attribute recognition result is obtained. The additional image data consists of feature image data associated with the detected target.

7. The method as described in claim 6, characterized in that, The target attribute detection is facial attribute detection. The segmented region includes at least one of the following: an eye region, a nose region, an eyebrow region, a mouth region, and a skin region, wherein the skin region is the area of ​​the face excluding the T-shaped area formed by the eyes, nose, eyebrows, and mouth. The step of matching the corresponding attribute recognition model based on the target visibility assessment result includes: The target visibility assessment result is classified into four categories: unobstructed face, partially obstructed face, completely obstructed face, and completely obstructed head, and the matched attribute recognition model is output after classifying the matching model for the matching attribute recognition model. The step of performing attribute recognition on the second image data according to the matched attribute recognition model to obtain the attribute recognition result includes: In cases where the face is unobstructed, the second image data is used to perform attribute recognition on the second image data using a first attribute recognition model for unobstructed facial attribute recognition, thus obtaining the attribute recognition result. When a part of the face is obscured, a second attribute recognition model for partially obscured facial attributes is used to perform attribute recognition on the second image data, yielding the attribute recognition result. When the face is completely occluded, a third attribute recognition model, used for attribute recognition in cases of complete facial occlusion, is used to perform attribute recognition on the second image data and the attribute feature image data, yielding the attribute recognition result. In the case where the head is completely occluded, the fourth attribute recognition model used for attribute recognition when the head is completely occluded is used to perform attribute recognition on the second image data and attribute feature image data to obtain the attribute recognition result. The attribute recognition results include age group and gender, and the attribute image data is feature image data associated with age group and gender.

8. A target attribute detection device based on an image, characterized in that, The device includes: The image acquisition module is used to acquire the first image data. The first detection module is used to perform target detection based on the first image data to obtain second image data of the detected target. The second detection module is used to obtain the target visibility assessment result of the detected target based on the second image data of the detected target. The target visibility assessment result is used to characterize the degree to which the detected target is occluded. Based on the target visibility assessment results, a corresponding attribute recognition model is matched. The attribute recognition model varies depending on the degree of occlusion of the overall detected target. According to the matched attribute recognition model, attribute recognition is performed on the second image data to obtain the attribute recognition result; The target visibility assessment results are obtained as follows: The second image data is segmented to obtain at least one segmented region. Based on the segmented region, the visibility of the segmented region in the detected target is obtained, wherein the visibility of the segmented region is the ratio of the area of ​​the unobstructed area in the segmented region to the area of ​​the segmented region. The visibility of each segmented region is input into a graph convolutional neural network model. The graph structure network autonomously learns the weights of each segmented region's occlusion level, which characterize the degree of occlusion of the segmented region on the detected target. For the visibility of each segmented region, a weighted average is calculated using the visibility weights of that segmented region. The target visibility assessment result is obtained by summing up the visibility of all weighted segmented regions.

9. A facial recognition system, characterized in that, The system includes an image acquisition device, which includes a memory, a processor, and an image module for acquiring image data. The memory stores a computer program, which, when executed by the processor, implements the steps of the image-based target attribute detection method as described in any one of claims 1 to 7.

10. The facial recognition system as described in claim 9, characterized in that, The system also includes: access control equipment for access control based on attribute recognition results.

Images