Face image classification method and system
Through adversarial enhancement and feature extraction technology, combined with lighting change data, the problem of insufficient accuracy of long-distance face image classification under background interference is solved, and high-precision classification under light change and background interference is achieved.
Patent Information
- Application Number
- CN202510299257.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-13
- Publication Date
- 2025-07-04
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to realize the effective classification of long-distance face images under background interference, especially in the case of lighting changes and background interference, the classification accuracy is insufficient.
Adversarial enhancement is performed through the face acquisition device, global features and local key features of the face are extracted, and the classification boundaries and granularity are determined based on the lighting change data, and classification labels are output.
Under background interference, the accuracy and stability of long-distance face image classification is improved, and the light interference is effectively filtered, which improves the accuracy and robustness of the classification.
Smart Images
Figure CN120260096A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of image classification. More specifically, this application relates to a face image classification method and system. Background Art
[0002] Image classification is a core task in the field of computer vision, aiming to assign images to predefined categories. Its process usually includes image preprocessing, feature extraction, and classifier training. Traditional methods rely on manually designed features such as edges and textures. However, in recent years, the emergence of deep learning, especially convolutional neural networks, has greatly improved the classification accuracy. Convolutional neural networks can automatically learn multi-level features from images and effectively handle various complex image tasks. Image classification is widely used in fields such as face recognition, autonomous driving, and security monitoring, helping intelligent systems understand and recognize visual information.
[0003] Existing face image classification technologies mainly rely on deep learning, especially convolutional neural networks, which classify by automatically learning facial features and can effectively handle problems such as illumination, expression changes, and angle deviations. This has enabled face recognition to be widely used in fields such as security and identity verification. However, the challenges faced in long-distance face classification are more complex. Since the resolution of images taken at a long distance is relatively low, facial features are often blurred, and in this case, interference in the background (such as illumination conditions) will further affect the classification accuracy. Therefore, how to achieve long-distance face image classification under background interference has become a difficult problem faced by the industry. Summary of the Invention
[0004] This application provides a face image classification method and system, which can achieve long-distance face image classification under background interference.
[0005] In a first aspect, this application provides a face image classification method, including the following steps: Collect a face image to be classified through a face acquisition device, and then perform adversarial enhancement on the face image to obtain an adversarially enhanced face sample set; Extract the global face features from the face sample set, extract local features from the key point regions of the face in the face image to obtain the local key features of the face image, and determine the classification feature value of the face image through the global face features and the local key features; Determine the classification association relationship between the global face features and the local key features according to the position distribution between each face key point in the face image, and determine the classification boundary of the face image in the classification process through the classification association relationship and the classification feature value of the face image; Obtain the illumination change data during the acquisition of the face image, and determine the illumination interference degree of the face image during the acquisition process through the illumination gradients at different times in the illumination change data; Determine the classification granularity of the face attributes in the face image according to the illumination interference degree of the face image during the acquisition process and the classification boundary of the face image during the classification process, and then output the classification label of the face image according to the classification granularity.
[0006] In some embodiments, extracting the global face features from the face sample set specifically includes: Perform key point detection on each face sample image in the face sample set to obtain multiple face key points in each face sample image; Determine the geometric structure relationship between the face key points in each face sample image according to the position information of the multiple face key points in each face sample image; Determine the global face features through the geometric structure relationship between the face key points in all face sample images.
[0007] In some embodiments, extracting local features from the face key point region in the face image to obtain the local key features of the face image specifically includes: Extract the face key region from the face image; Perform multi-scale convolution operations on the face key region to obtain feature maps of different scales of the face image; Generate multiple local attention maps of the face image based on the local self-attention mechanism and the feature maps of different scales of the face image, and then obtain the local key features of the face image.
[0008] In some embodiments, determining the classification feature value of the face image through the global face features and the local key features specifically includes: Perform an initial classification on the face image based on a preset face classification model to obtain the category feature value of the face image; Perform feature compensation on the category feature value of the face image according to the local key features and the global face features to obtain the classification feature value of the face image.
[0009] In some embodiments, determining the classification association relationship between the global face features and the local key features according to the position distribution between the face key points in the face image specifically includes: Perform association analysis on the position distribution between the face key points in the face image through the global face features to obtain the association feature value between the position distribution and the global face features; Determine the dependent feature value between the angular relationship and the local key feature according to the local key feature and the angular relationship between each facial key point in the facial image.
[0010] Determine the classification association relationship between the global facial feature and the local key feature through the correlation feature value and the dependent feature value.
[0011] In some embodiments, determining the classification boundary of the facial image in the classification process through the classification association relationship and the classification feature value of the facial image specifically includes: Determine the classification restriction condition of the facial image in the classification process through the classification association relationship and the positional relationship between each facial key point in the facial image; Determine the classification boundary of the facial image in the classification process according to the classification feature value of the facial image and the classification restriction condition of the facial image in the classification process.
[0012] In some embodiments, determining the classification granularity of the facial attribute in the facial image according to the light interference degree of the facial image in the acquisition process and the classification boundary of the facial image in the classification process specifically includes: Perform confidence evaluation on the category feature value of the facial image according to the classification boundary of the facial image in the classification process to obtain the classification confidence of the facial image; Determine the classification granularity of the facial attribute in the facial image through the light interference degree of the facial image in the acquisition process and the classification confidence of the facial image.
[0013] In a second aspect, the present application provides a facial image classification system, including: An enhancement module, configured to collect a facial image to be classified through a facial acquisition device, and then perform adversarial enhancement on the facial image to obtain an adversarial enhanced facial sample set; A processing module, configured to extract the global facial feature from the facial sample set, perform local feature extraction on the facial key point region in the facial image to obtain the local key feature of the facial image, and determine the classification feature value of the facial image through the global facial feature and the local key feature; The processing module is further configured to determine the classification association relationship between the global facial feature and the local key feature according to the positional distribution between each facial key point in the facial image, and determine the classification boundary of the facial image in the classification process through the classification association relationship and the classification feature value of the facial image; The processing module is further configured to obtain the illumination change data during the acquisition of the face image, and determine the illumination interference degree during the acquisition of the face image according to the illumination gradients at different times in the illumination change data; An execution module, configured to determine the classification granularity of the face attributes in the face image according to the illumination interference degree during the acquisition of the face image and the classification boundary during the classification of the face image, and further output the classification label of the face image according to the classification granularity.
[0014] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned face image classification method.
[0015] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned face image classification method is implemented.
[0016] The technical solutions provided by the disclosed embodiments of the present application have the following beneficial effects: In the face image classification method and system provided by the present application, a face image to be classified is collected by a face acquisition device, and then the face image is adversarially enhanced to obtain an adversarially enhanced face sample set; global face features are extracted from the face sample set, local feature extraction is performed on the face key point regions in the face image to obtain the local key features of the face image, and the classification feature value of the face image is determined through the global face features and the local key features; the classification association relationship between the global face features and the local key features is determined according to the position distribution between the face key points in the face image, and the classification boundary during the classification of the face image is determined through the classification association relationship and the classification feature value of the face image; the illumination change data during the acquisition of the face image is obtained, and the illumination interference degree during the acquisition of the face image is determined according to the illumination gradients at different times in the illumination change data; the classification granularity of the face attributes in the face image is determined according to the illumination interference degree during the acquisition of the face image and the classification boundary during the classification of the face image, and further the classification label of the face image is output according to the classification granularity.
[0017] It can be seen that in this application, the classification granularity of the facial attributes in the face image can be determined according to the illumination interference degree during the acquisition process of the face image and the classification boundary during the classification process of the face image. First, by performing adversarial enhancement on the face image to be classified, an adversarial enhanced face sample set is obtained. This operation not only improves the diversity and robustness of the face image data but also enhances the ability to cope with background interference. Second, the classification feature value of the face image is obtained through the global face feature and the local key feature. The classification feature value represents the decision parameter value for refining the classification of the facial attributes in the face image. The classification feature value helps to combine the overall contour information of the face with the detailed features of the key parts, and can still effectively extract high-discrimination feature information in the case of long-distance background interference, thereby improving the accuracy of face classification at a long distance. Then, the classification boundary during the classification process of the face image is determined through the classification association relationship and the classification feature value of the face image. The classification boundary represents the hyperplane for refining the classification of the facial attributes in the feature space, and the classification boundary realizes the effective discrimination of long-distance facial attributes under background interference. Further, the illumination interference degree during the acquisition process of the face image is evaluated through the illumination gradient in the illumination change data, which helps to effectively filter out the illumination interference in a long-distance scene with significant light changes and improve the stability of face image classification. Furthermore, the classification granularity of the facial attributes in the face image is obtained by combining the illumination interference degree during the acquisition process of the face image and the classification boundary during the classification process of the face image. The classification granularity represents the fine degree of the classification of the facial attributes when classifying the face image. By using the classification granularity, the error of face image classification at a long distance under background interference is solved. Finally, the classification label of the face image is output according to the classification granularity. In summary, the solution of this application can achieve long-distance face image classification under background interference. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 is an exemplary flowchart of a face image classification method according to some embodiments of the present application; Figure 2 is a schematic flowchart of determining the global face feature according to some embodiments of the present application; Figure 3 is a schematic flowchart of determining the classification association relationship according to some embodiments of the present application; Figure 4 is a schematic structural diagram of a face image classification system according to some embodiments of the present application; Figure 5 is a schematic structural diagram of a computer device for implementing the face image classification method according to some embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0019] To better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.
[0020] Referring to Figure 1 , which is an exemplary flowchart of a face image classification method shown according to some embodiments of the present application. The face image classification method 100 mainly includes the following steps: In step 101, a face image to be classified is collected by a face acquisition device, and then the face image is adversarially enhanced to obtain an adversarially enhanced face sample set.
[0021] It should be noted that the face acquisition device in the present application is a high-definition camera.
[0022] In some embodiments, the adversarially enhancing the face image to obtain an adversarially enhanced face sample set can be implemented by the following steps: Performing image enhancement on the face image to obtain multiple enhanced face images; Based on a preset generative adversarial network model, adversarial generation is performed on each enhanced face image, and then an adversarially enhanced face sample set is obtained.
[0023] Specifically, when implementing, performing image enhancement on the face image to obtain multiple enhanced face images can be implemented by the following method, that is: using the image enhancement algorithm of OpenCV to perform operations such as rotation, scaling, cropping, brightness adjustment, color transformation, noise addition, etc. on the face image, so as to obtain multiple face images after image enhancement, and taking the face images after image enhancement as enhanced face images. In other embodiments, other methods can also be used to implement, which will not be elaborated here.
[0024] It should be noted that the generative adversarial network model in the present application can be obtained by training on a large-scale face dataset (such as CelebA, FFHQ), or directly using existing open-source models, such as StyleGAN, StarGAN, AttGAN, etc. During training, real face images are used as inputs, the generator continuously optimizes to generate high-quality forged faces, the discriminator distinguishes between true and false and provides feedback, and the two are adversarially trained until the generator can deceive the discriminator. After training is completed, the weights of the generator can be saved and directly loaded for use in subsequent tasks.
[0025] In specific implementation, adversarial generation is performed on each enhanced face image based on a preset generative adversarial network model, and then the face sample set after adversarial enhancement can be obtained in the following manner, that is: input each enhanced face image into the generative adversarial network model to obtain multiple adversarially generated face images, and use the set composed of all the adversarially generated face images as the face sample set after adversarial enhancement. Among them, each adversarially generated face image in the face sample set is used as a face sample image. In other embodiments, other methods can also be used for implementation, which is not limited here.
[0026] In step 102, extract the global face features from the face sample set, perform local feature extraction on the key point area of the face in the face image to obtain the local key features of the face image, and determine the classification feature value of the face image through the global face features and the local key features.
[0027] In some embodiments, refer to Figure 2 As shown, this figure is a schematic flowchart for determining the global face features in some embodiments of the present application. In this embodiment, the extraction of the global face features from the face sample set can be implemented through the following steps: First, in step 1021, perform key point detection on each face sample image in the face sample set to obtain multiple face key points in each face sample image; Secondly, in step 1022, determine the geometric structure relationship between the face key points in each face sample image according to the position information of the multiple face key points in each face sample image; Then, in step 1023, determine the global face features through the geometric structure relationship between the face key points in all face sample images.
[0028] In specific implementation, the key point detection on each face sample image in the face sample set to obtain multiple face key points in each face sample image can be implemented in the following manner, that is: use the Dlib library in OpenCV to extract multiple face key points from each face sample image in the face sample set, so as to obtain multiple face key points in each face sample image, where the face key points include parts such as the corners of the eyes, the tip of the nose, the corners of the mouth, and the eyebrows.
[0029] It should be noted that the position information of the face key points in the present application can be represented by the position coordinates of the face key points.
[0030] In specific implementation, the geometric structure relationship between each facial key point in each facial sample image can be determined according to the position information of multiple facial key points in each facial sample image, which can be implemented in the following manner, that is: for each facial sample image, calculate the Euclidean distance between every two facial key points in the facial sample image, select the maximum Euclidean distance and the minimum Euclidean distance from all the Euclidean distances, and use the difference between the maximum Euclidean distance and the minimum Euclidean distance as the geometric structure relationship between each facial key point in the facial sample image, so as to obtain the geometric structure relationship between each facial key point in each facial sample image. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.
[0031] It should be noted that the geometric structure relationship described in this application represents the scale parameter of the geometric structure formed by each facial key point in the facial sample image.
[0032] In specific implementation, the global facial feature can be determined according to the geometric structure relationship between each facial key point in all facial sample images, which can be implemented in the following manner, that is: sum up the geometric structure relationships between each facial key point in all facial sample images, and use the obtained sum value as the global facial feature. It should be noted that the global facial feature described in this application represents the eigenvalue of the overall difference in the geometric structure between each facial key point on the face in the facial image to be classified. In other embodiments, other methods can also be used for implementation, which are not limited here.
[0033] In some embodiments, local feature extraction of the facial key point region in the facial image to obtain the local key feature of the facial image can be implemented by the following steps: Extract the facial key region from the facial image; Perform multi-scale convolution operations on the facial key region to obtain feature maps of different scales of the facial image; Generate multiple local attention maps of the facial image based on the local self-attention mechanism and the feature maps of different scales of the facial image, so as to obtain the local key feature of the facial image.
[0034] In specific implementation, the facial key region can be extracted from the facial image in the following manner, that is: use the Dlib library in OpenCV to detect the region formed by eyes, nose and mouth in the facial image, and use the obtained region as the facial key region. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.
[0035] It should be noted that the facial key region described in this application represents the facial region formed by important parts of the face in the facial image.
[0036] In specific implementation, multi-scale convolution operations are performed on the key face region to obtain feature maps of different scales of the face image, which can be implemented in the following manner, that is: convolution kernels of different sizes (such as 3x3, 5x5, 7x7, 9x9, etc.) are used to perform convolution operations on the key face region respectively, so as to obtain feature maps of the face image at different scales. In other embodiments, other methods can also be used for implementation, which is not limited here.
[0037] In specific implementation, based on the local self-attention mechanism and the feature maps of different scales of the face image, multiple local attention maps of the face image are generated, and then the local key features of the face image can be obtained in the following manner, that is: First, select a feature map of a certain scale from the face image as the selected feature map, calculate the attention weights of the selected feature map using the local self-attention mechanism, and then use the attention weights to perform feature weighting on the selected feature map (for example, local attention map = attention weight × selected feature map), and use the feature map obtained after feature weighting as a local attention map of the face image, and continue to perform feature weighting on the remaining feature maps, so as to obtain multiple local attention maps of the face image; Then, select a local attention map as the selected local attention map, perform max pooling operation on the selected local attention map, then calculate the mean value of all the values obtained after the max pooling operation, and use the obtained mean value as the max pooling eigenvalue of the selected local attention map, and continue to determine the max pooling eigenvalues of the remaining local attention maps, and then sum the max pooling eigenvalues of all local attention maps, and use the obtained sum value as the local key features of the face image. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.
[0038] It should be noted that the local key features described in this application represent parameters that distinguish the significant features of the key face region in the face image to be classified.
[0039] In some embodiments, the classification eigenvalue of the face image can be determined by the global face feature and the local key feature through the following steps: Based on a preset face classification model, the face image is initially classified to obtain the class eigenvalue of the face image; According to the local key feature and the global face feature, the class eigenvalue of the face image is compensated for features to obtain the classification eigenvalue of the face image.
[0040] It should be noted that the preset face classification model in this application can use a large amount of face image data to perform classification training and model tuning on the VGGFace2 model, and use the model obtained after classification training and model tuning as the face classification model. Among them, the face classification model can classify and judge face attributes (such as age).
[0041] In specific implementation, the initial classification of the face image is performed based on a preset face classification model, and the category feature value of the face image can be obtained in the following manner: input the face image into the preset face classification model, and use the output result of the face classification model as the category feature value of the face image. Here, the category feature value represents the identification value of the specific classification of the face attributes in the face image. In other embodiments, other methods can also be used to implement this, which will not be elaborated here.
[0042] In specific implementation, the feature compensation of the category feature value of the face image is performed according to the local key features and the face global features, and the classification feature value of the face image can be obtained in the following manner: First, divide the face global feature by the local key feature, then multiply the obtained value by the category feature value of the face image, and use the obtained multiplied value as the classification feature value of the face image. In other embodiments, other methods can also be used to implement this, which is not limited here.
[0043] It should be noted that in this application, the classification feature value represents the decision parameter value for the refined classification of the face attributes in the face image.
[0044] In step 103, the classification association relationship between the face global feature and the local key feature is determined according to the position distribution between each face key point in the face image, and the classification boundary in the classification process of the face image is determined through the classification association relationship and the classification feature value of the face image.
[0045] In some embodiments, as shown in Figure 3 This figure is a schematic flowchart of determining the classification association relationship in some embodiments of this application. In this embodiment, the classification association relationship between the face global feature and the local key feature can be determined according to the position distribution between each face key point in the face image, and the following steps can be used to implement this: Perform association analysis on the position distribution between each face key point in the face image through the face global feature to obtain the association feature value between the position distribution and the face global feature; Determine the dependence feature value between the angle relationship and the local key feature according to the local key feature and the angle relationship between each face key point in the face image.
[0046] Determine the classification association relationship between the face global feature and the local key feature through the association feature value and the dependence feature value.
[0047] It should be noted that the position distribution described in this application represents a set of relative distances of each facial key point in the facial image, where the relative distance can be represented by the Euclidean distance between every two facial key points.
[0048] In specific implementation, the correlation analysis of the position distribution between each facial key point in the facial image is performed through the facial global feature. The correlation feature value between the position distribution and the facial global feature can be implemented in the following manner, that is: calculate the Pearson correlation coefficient between the facial global feature and the position distribution of each facial key point in the facial image, and use the obtained Pearson correlation coefficient as the correlation feature value between the position distribution and the facial global feature. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.
[0049] It should be noted that the correlation feature value described in this application represents a feature parameter of the correlation between the facial global feature and the position distribution of each facial key point in the facial image.
[0050] In specific implementation, the dependence feature value between the angular relationship and the local key feature can be determined according to the angular relationship between the local key feature and each facial key point in the facial image in the following manner, that is: First, calculate the angles between each facial key point in the facial image, and use the obtained angles as the angular relationship between each facial key point. Then, divide the difference between the local key feature and the angular relationship by the local key feature, and use the obtained value as the dependence feature value between the angular relationship and the local key feature. In other embodiments, other methods can also be used for implementation, which will not be limited here.
[0051] It should be noted that the dependence feature value described in this application represents a parameter describing the dependence strength between the angular relationship between each facial key point and the local key feature.
[0052] In specific implementation, the classification association relationship between the facial global feature and the local key feature can be determined through the correlation feature value and the dependence feature value in the following manner, that is: multiply the correlation feature value by the dependence feature value, and use the obtained value as the classification association relationship between the facial global feature and the local key feature. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.
[0053] It should be noted that the classification association relationship described in this application represents a feature parameter of the mutual dependence between the facial global feature and the local key feature in the process of facial image classification.
[0054] In some embodiments, determining the classification boundary of the face image during the classification process based on the classification association relationship and the classification feature values of the face image can be achieved through the following steps: Determine the classification restriction conditions of the face image during the classification process based on the classification association relationship and the positional relationship between each face key point in the face image; Determine the classification boundary of the face image during the classification process based on the classification feature values of the face image and the classification restriction conditions of the face image during the classification process.
[0055] It should be noted that the positional relationship between each face key point in this application represents the parameters of the overall layout of each face key point in the face image.
[0056] Specifically, determining the classification restriction conditions of the face image during the classification process based on the association relationship and the positional relationship between each face key point in the face image can be achieved in the following manner, that is: First, obtain the positional distribution between each face key point in the face image, sum all the relative distances in the positional distribution, and use the obtained sum value as the positional relationship between each face key point in the face image. Then, divide the positional relationship by the association relationship, and use the obtained division value as the classification restriction conditions of the face image during the classification process. In other embodiments, other methods can also be used to achieve this, which will not be elaborated here.
[0057] It should be noted that the classification restriction conditions in this application represent the conditional parameters that restrict the refinement categories of face attributes in the face image during the classification process.
[0058] In specific implementation, the classification boundary of the face image in the classification process can be determined according to the classification feature value of the face image and the classification constraint condition in the classification process of the face image by the following method, that is: First, collect multiple face labeled images, where the face labeled image is a face image labeled with face attributes. Determine the classification feature value and the classification constraint condition of each face labeled image respectively according to the above method. Then, subtract the classification feature value of the face image from the classification feature values of all face labeled images and take the absolute value, and use the values obtained after taking the absolute value as the classification feature deviation values of the corresponding face labeled images. Further, subtract the classification constraint condition of the face image from the classification constraint conditions of all face labeled images and take the absolute value, and use the values obtained after taking the absolute value as the classification constraint deviation values of the corresponding face labeled images. Finally, form the feature label pairs of the corresponding face labeled images with the classification feature deviation values of each face labeled image, the classification constraint deviation values of each face labeled image, and the face attributes of each face labeled image (for example, the feature label pair of the face labeled image = (classification feature deviation value, classification constraint deviation value, face attribute)), so as to obtain multiple feature label pairs, and use all the feature label pairs as the training data of the support vector machine. Further optimize and train the support vector machine, and use the hyperplane of the support vector machine obtained after the end of the optimization training as the classification boundary of the face image in the classification process. In other embodiments, other methods can also be used to implement this, which is not limited here.
[0059] It should be noted that the classification boundary described in this application represents a hyperplane for refining the classification of face attributes in the feature space.
[0060] In step 104, obtain the light change data of the face image during the acquisition process, and determine the light interference degree of the face image during the acquisition process through the light gradients at different times in the light change data.
[0061] In specific implementation, obtain the light change data of the face image during the acquisition process through the ambient light sensor on the face acquisition device.
[0062] It should be noted that the light change data described in this application represents an ordered data composed of the ambient light intensity of the face image in chronological order during the acquisition process, where the time interval for collecting the light intensity is 1 / 30 second, and the acquisition process of the face image represents the time period from the start of face image acquisition to the end of face image acquisition.
[0063] In some embodiments, determining the light interference degree of the face image during the acquisition process through the light gradients at different times in the light change data can be implemented by the following steps: Cluster all pixel values in the face image to obtain multiple data clusters of pixel values; Determine the brightness change uniformity of the face image during the acquisition process according to the light intensity gradients at different times in the light change data; Determine the light interference degree of the face image during the acquisition process according to each data cluster and the brightness change uniformity.
[0064] In specific implementation, clustering all pixel values in the face image to obtain multiple data clusters of pixel values can be implemented in the following way, that is: use an existing clustering algorithm (such as hierarchical clustering algorithm) to cluster all pixel values in the face image, so as to obtain multiple data clusters of pixel values. In other embodiments, other methods can also be used to implement, which is not limited here.
[0065] It should be noted that in this application, the light intensity gradient refers to the rate of change of the ambient light intensity over time, and the light intensity gradients at different times can be obtained by performing differential calculation on the light change data.
[0066] In specific implementation, determining the brightness change uniformity of the face image during the acquisition process according to the light intensity gradients at different times in the light change data can be implemented in the following way, that is: calculate the standard deviation of the light intensity gradients at different times in the light change data, and use the obtained standard deviation as the brightness change uniformity of the face image during the acquisition process. In other embodiments, other methods can also be used to implement, which will not be elaborated here.
[0067] It should be noted that in this application, the brightness change uniformity represents a measure of the uniformity of the change in the ambient light intensity of the face image during the acquisition process. The larger the brightness change uniformity, the lower the uniformity of the change in the ambient light intensity of the face image during the acquisition process, and vice versa.
[0068] In specific implementation, determining the light interference degree of the face image during the acquisition process according to each data cluster and the brightness change uniformity can be implemented in the following way, that is: calculate the distance between the centers of every two data clusters, then sum up all the obtained distances, further divide the sum value by the brightness change uniformity, and use the obtained quotient value as the light interference degree of the face image during the acquisition process. In other embodiments, other methods can also be used to implement, which is not limited here.
[0069] It should be noted that in this application, the light interference degree represents the degree of influence of the light change during the face image acquisition process on the face image classification. The larger the light interference degree, the higher the degree of influence of the light change during the face image acquisition process on the face image classification, and vice versa.
[0070] In step 105, the classification granularity of the facial attributes in the face image is determined according to the illumination interference degree during the acquisition process of the face image and the classification boundary during the classification process of the face image, and then the classification label of the face image is output according to the classification granularity.
[0071] In some embodiments, the determination of the classification granularity of the facial attributes in the face image according to the illumination interference degree during the acquisition process of the face image and the classification boundary during the classification process of the face image can be implemented by the following steps: Perform a confidence evaluation on the class feature values of the face image according to the classification boundary during the classification process of the face image to obtain the classification confidence of the face image; Determine the classification granularity of the facial attributes in the face image through the illumination interference degree during the acquisition process of the face image and the classification confidence of the face image.
[0072] Specifically, when implemented, the confidence evaluation of the class feature values of the face image according to the classification boundary during the classification process of the face image to obtain the classification confidence of the face image can be implemented in the following manner, that is: First, perform an initial classification on all face labeled images based on a preset face classification model to obtain the class feature values of each face labeled image. Second, perform a weighted sum of the weight vector of the classification boundary and the class feature values of each face labeled image, then add up all the values obtained from the weighted sums, and use the added value as the global confidence feature value. Then, perform a weighted sum of the weight vector of the classification boundary and the class feature value of the face image, and use the value obtained from the weighted sum as the local confidence feature value. Finally, divide the local confidence feature value by the global confidence feature value, and use the obtained value as the classification confidence of the face image. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.
[0073] It should be noted that the classification confidence in this application represents the credibility of the face image in subdividing facial attributes. The greater the classification confidence, the higher the credibility of the face image in subdividing facial attributes, and vice versa.
[0074] Specifically, when implemented, the determination of the classification granularity of the facial attributes in the face image through the illumination interference degree during the acquisition process of the face image and the classification confidence of the face image can be implemented in the following manner, that is: Divide the classification confidence of the face image by the illumination interference degree during the acquisition process of the face image, and use the obtained value as the classification granularity of the facial attributes in the face image. In other embodiments, other methods can also be used for implementation, which is not limited here.
[0075] It should be noted that the classification granularity in this application represents the fineness of the facial attribute classification when classifying the face image.
[0076] In specific implementation, outputting the classification label of the face image according to the classification granularity can be implemented in the following manner, that is: obtaining the category feature value of the face image, multiplying the category feature value by the classification granularity, and using the obtained multiplied value as the calibration value of the face attribute in the face image. Then, comparing the calibration value with the face attribute category label, so as to obtain the age interval label where the calibration value of the face attribute in the face image is located in the face attribute category label, and using the obtained age interval label as the classification label of the face image, where the face attribute label divides different age interval labels (for example: different stages such as 5 - 10 years old, 10 - 15 years old, 15 - 20 years old, 20 - 25 years old, 25 - 30 years old, 30 - 35 years old, 35 - 40 years old, 40 - 45 years old, 45 - 50 years old, 50 - 55 years old, 55 - 60 years old, 60 - 65 years old, 65 - 70 years old, 70 - 75 years old, 75 - 80 years old, 80 - 85 years old, over 90 years old, etc.).
[0077] It should be noted that the classification label in this application represents the final label of the refined classification interval of the face attribute in the face image.
[0078] In addition, on the other hand of this application, in some embodiments, this application provides a face image classification system. Refer to Figure 4 , this figure is a schematic structural diagram of the face image classification system shown in some embodiments of this application. The face image classification system 400 includes: an enhancement module 401, a processing module 402, and an execution module 403, which are described as follows respectively: Enhancement module 401. In this application, the acquisition module 401 is mainly used to collect the face image to be classified through a face acquisition device, and then perform adversarial enhancement on the face image to obtain an adversarial enhanced face sample set; Processing module 402. In this application, the processing module 402 is used to extract the global face feature from the face sample set, perform local feature extraction on the face key point area in the face image to obtain the local key feature of the face image, and determine the classification feature value of the face image through the global face feature and the local key feature; It should be noted that the processing module 402 in this application is further used to determine the classification association relationship between the global face feature and the local key feature according to the position distribution between each face key point in the face image, and determine the classification boundary of the face image in the classification process through the classification association relationship and the classification feature value of the face image; In addition, the processing module 402 in this application is further used to obtain the illumination change data during the acquisition process of the face image, and determine the illumination interference degree of the face image during the acquisition process through the illumination gradient at different moments in the illumination change data; Execution module 403. In this application, the execution module 403 is mainly used to determine the classification granularity of the facial attributes in the face image according to the illumination interference degree during the acquisition process of the face image and the classification boundary during the classification process of the face image, and then output the classification label of the face image according to the classification granularity.
[0079] In addition, this application also provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned face image classification method.
[0080] In some embodiments, refer to Figure 5 , this figure is a schematic structural diagram of a computer device for implementing the face image classification method according to some embodiments of this application. The face image classification method in the above embodiments can be implemented by Figure 5 the computer device shown. The computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.
[0081] The processor 501 can be a general-purpose central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more for controlling the execution of the face image classification method in this application.
[0082] The communication bus 502 can be used to transmit information between the above components.
[0083] The memory 503 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), or other types of dynamic storage devices that can store information and instructions. It can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disks, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 503 can exist independently and be connected to the processor 501 through the communication bus 502. The memory 503 can also be integrated with the processor 501.
[0084] Among them, the memory 503 is used to store the program code for executing the solution of this application and is controlled by the processor 501 to execute. The processor 501 is used to execute the program code stored in the memory 503. The program code can include one or more software modules. The methods described in the above method embodiments can be implemented by one or more software modules in the program code in the processor 501 and the memory 503.
[0085] The communication interface 504 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.
[0086] In a specific implementation, as an embodiment, the computer device can include multiple processors, and each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).
[0087] The computer device described above may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.
[0088] In addition, the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned face image classification method is implemented.
[0089] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0090] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.
Claims
1. A face image classification method, characterized in that, Including the following steps: Collect a face image to be classified through a face acquisition device, and then perform adversarial enhancement on the face image to obtain an adversarially enhanced face sample set; Extract the global face features from the face sample set, extract local features from the face key point regions in the face image to obtain the local key features of the face image, and determine the classification feature value of the face image through the global face features and the local key features; Determine the classification association relationship between the global face features and the local key features according to the position distribution between each face key point in the face image, and determine the classification boundary of the face image during the classification process through the classification association relationship and the classification feature value of the face image; Obtain the illumination change data during the acquisition of the face image, and determine the illumination interference degree of the face image during the acquisition process through the illumination gradient at different times in the illumination change data; Determine the classification granularity of the face attributes in the face image according to the illumination interference degree of the face image during the acquisition process and the classification boundary of the face image during the classification process, and then output the classification label of the face image according to the classification granularity.
2. The method according to claim 1, wherein Specifically, extracting the global face features from the face sample set includes: Perform key point detection on each face sample image in the face sample set to obtain multiple face key points in each face sample image; Determine the geometric structure relationship between each face key point in each face sample image according to the position information of the multiple face key points in each face sample image; Determine the global face features through the geometric structure relationship between each face key point in all face sample images.
3. The method according to claim 1, wherein Specifically, extracting local features from the face key point regions in the face image to obtain the local key features of the face image includes: Extract the face key region from the face image; Perform multi-scale convolution operations on the face key region to obtain feature maps of different scales of the face image; Generate multiple local attention maps of the face image based on the local self-attention mechanism and the feature maps of different scales of the face image, and then obtain the local key features of the face image.
4. The method according to claim 1, wherein Specifically, determining the classification feature value of the face image through the global face features and the local key features includes: Perform an initial classification on the face image based on a preset face classification model to obtain the category feature value of the face image; Perform feature compensation on the category feature value of the face image according to the local key features and the global face features to obtain the classification feature value of the face image.
5. The method according to claim 1, characterized in that, Specifically, determining the classification association relationship between the global face features and the local key features according to the position distribution between each face key point in the face image includes: Perform association analysis on the position distribution between each face key point in the face image through the global face features to obtain the association feature value between the position distribution and the global face features; Determine the dependent feature value between the angular relationship and the local key features according to the local key features and the angular relationship between each facial key point in the facial image; Determine the classification association relationship between the global facial feature and the local key features through the correlation feature value and the dependent feature value.
6. The method according to claim 1, characterized in that, Determine the classification boundary of the facial image during the classification process through the classification association relationship and the classification feature value of the facial image, specifically including: Determine the classification restriction conditions of the facial image during the classification process through the classification association relationship and the positional relationship between each facial key point in the facial image; Determine the classification boundary of the facial image during the classification process according to the classification feature value of the facial image and the classification restriction conditions of the facial image during the classification process.
7. The method according to claim 1, wherein Determine the classification granularity of the facial attributes in the facial image according to the illumination interference degree during the acquisition process of the facial image and the classification boundary of the facial image during the classification process, specifically including: Perform a confidence evaluation on the category feature value of the facial image according to the classification boundary of the facial image during the classification process to obtain the classification confidence of the facial image; Determine the classification granularity of the facial attributes in the facial image through the illumination interference degree during the acquisition process of the facial image and the classification confidence of the facial image.
8. A face image classification system, characterized in that, Include: An enhancement module, configured to collect a facial image to be classified through a facial acquisition device, and then perform adversarial enhancement on the facial image to obtain an adversarially enhanced facial sample set; A processing module, configured to extract the global facial feature from the facial sample set, perform local feature extraction on the facial key point region in the facial image to obtain the local key features of the facial image, and determine the classification feature value of the facial image through the global facial feature and the local key features; The processing module is further configured to determine the classification association relationship between the global facial feature and the local key features according to the positional distribution between each facial key point in the facial image, and determine the classification boundary of the facial image during the classification process through the classification association relationship and the classification feature value of the facial image; The processing module is further configured to obtain the illumination change data during the acquisition process of the facial image, and determine the illumination interference degree during the acquisition process of the facial image through the illumination gradient at different times in the illumination change data; An execution module, configured to determine the classification granularity of the facial attributes in the facial image according to the illumination interference degree during the acquisition process of the facial image and the classification boundary of the facial image during the classification process, and then output the classification label of the facial image according to the classification granularity.
9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the facial image classification method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the facial image classification method according to any one of claims 1 to 7.