Bottom-up hierarchical face attribute recognition method and system

Abstract

The present invention provides a bottom-up hierarchical facial attribute recognition method, comprising: performing image analysis on a facial image to obtain facial information; obtaining recognition results for underlying facial attributes from the facial information; and, based on the semantic relationships between the underlying facial attributes and the upper-level facial attributes, obtaining recognition results for the upper-level facial attributes from the recognition results for the underlying facial attributes. The present invention can efficiently and accurately obtain facial attribute recognition results and is highly scalable.

Description

Technical Field

[0001] The present invention belongs to the field of facial attribute recognition technology, involving technologies such as artificial intelligence, computer vision and statistics. More specifically, the present invention relates to a bottom-up hierarchical facial attribute recognition method and system. Background Art

[0002] Facial attributes, such as wide palpebral fissures, round eyes, thick lips, and slender brows, can be used to describe a face in detail. Facial attribute recognition involves analyzing facial images to obtain relevant information about these attributes. With the emergence of massive amounts of data and the rapid development of hardware, a variety of facial attribute recognition methods have been proposed.

[0003] Most existing facial attribute recognition methods rely on supervised machine learning methods (hereinafter referred to as machine learning methods). This type of method usually uses facial images annotated with facial attributes (labels) and designs different machine learning algorithms to obtain the relationship between facial attributes or the mapping relationship between facial images and facial attributes. In existing facial attribute recognition methods, labeling is usually done manually by labelers. Different labelers may have different understandings of facial attributes, which leads to the introduction of errors in the facial attribute labeling process. In addition, the automatic learning process in the machine learning method is difficult to fully obtain the relationship between facial attributes and the mapping relationship between facial attributes and facial images. Therefore, existing facial attribute recognition methods have problems such as low recognition accuracy.

[0004] Machine learning methods typically require constructing a corresponding dataset for each facial attribute, containing both positive and negative examples. As the number of facial attributes increases, the amount of data or labels required also increases. This necessitates adding a correspondingly large number of positive and negative examples for machine training. Consequently, existing facial attribute recognition methods suffer from low scalability. Summary of the Invention

[0005] In order to overcome the problems existing in the above-mentioned prior art, the first aspect of the present invention provides a bottom-up hierarchical facial attribute recognition method, including: step 1) performing image analysis on a facial image to obtain facial information; step 2) obtaining recognition results of underlying facial attributes from the facial information; and step 3) obtaining recognition results of upper-level facial attributes from the recognition results of the underlying facial attributes based on the semantic relationship between the underlying facial attributes and the upper-level facial attributes.

[0006] In the above method, step 2) includes: step 21) for each underlying facial attribute among all underlying facial attributes, mapping the underlying facial attribute to a corresponding part in the facial information, and obtaining the semantic absolute value of the underlying facial attribute based on the corresponding part in the facial information; and, step 22) for each underlying facial attribute among all underlying facial attributes, obtaining the recognition result of the underlying facial attribute based on the comparison result of the semantic absolute value of the underlying facial attribute and the statistical mean of the underlying facial attribute.

[0007] In the above method, step 22) includes performing the following operations for each underlying facial attribute among all underlying facial attributes: calculating the difference between the semantic absolute value of the underlying facial attribute and the statistical mean of the underlying facial attribute; calculating the ratio of the difference to the statistical mean of the underlying facial attribute; and comparing the ratio with a predetermined threshold. If the ratio is greater than the predetermined threshold, setting the underlying facial attribute as a positive attribute, otherwise setting the underlying facial attribute as a negative attribute.

[0008] The above method may also include setting up an expandable underlying facial attribute information database, which stores the mapping formula and statistical mean of each underlying facial attribute among all underlying facial attributes, wherein the mapping formula of each underlying facial attribute is used to map the underlying facial attribute to the corresponding part in the facial information and obtain the semantic absolute value of the underlying facial attribute based on the corresponding part in the facial information.

[0009] In the above method, step 3) includes performing the following operations for each upper-level facial attribute among all upper-level facial attributes: obtaining the recognition results of one or more underlying facial attributes that have a semantic relationship with the upper-level facial attribute; and, based on the semantic relationship between the one or more underlying facial attributes and the upper-level facial attribute, obtaining the recognition result of the upper-level facial attribute from the recognition results of the one or more underlying facial attributes.

[0010] The above method may also include setting up an expandable bottom-level-upper-level facial attribute semantic relationship library, which stores formulas for expressing the semantic relationship between each upper-level facial attribute in all upper-level facial attributes and one or more bottom-level facial attributes.

[0011] In the above method, step 1) includes: performing at least face detection, feature point positioning, facial component segmentation, iris segmentation, pupil segmentation and three-dimensional face reconstruction on the facial image to obtain the facial information; wherein the facial information at least includes: a face position frame, facial feature point positions, facial component masks, iris masks, pupil masks and a three-dimensional face shape.

[0012] The second aspect of the present invention provides a bottom-up hierarchical facial attribute recognition system, comprising: a facial information processing module for performing image analysis on a facial image to obtain facial information; a bottom-level facial attribute recognition module for obtaining recognition results of bottom-level facial attributes from the facial information; and an upper-level facial attribute recognition module for obtaining recognition results of upper-level facial attributes from the recognition results of the bottom-level facial attributes based on the semantic relationship between the bottom-level facial attributes and the upper-level facial attributes.

[0013] A third aspect of the present invention provides a computer-readable medium having a computer program stored thereon, which implements the above-mentioned bottom-up hierarchical face attribute recognition method when the computer program is executed by a processor.

[0014] The fourth aspect of the present invention provides an electronic device, comprising: a processor; and a memory, on which a computer program is stored and coupled to the processor, so that when the computer program is executed by the processor, the electronic device implements the above-mentioned bottom-up hierarchical facial attribute recognition method.

[0015] The embodiments of the present invention can achieve the following beneficial effects:

[0016] Based on the intuitive interpretability of facial attributes, the present invention divides facial attributes into low-level facial attributes and high-level facial attributes. The low-level facial attributes are identified based on the facial information analyzed from the facial image, and the high-level facial attributes are identified based on the recognition results of the low-level facial attributes based on the semantic relationship between the low-level facial attributes and the high-level facial attributes. In the low-level facial attribute recognition, detailed and comprehensive facial information is obtained by analyzing the facial image, and the semantic absolute value of each low-level facial attribute is obtained from the facial information, and the low-level facial attributes are combined with the basic facial information. The low-level facial attributes are identified by comparing the semantic absolute value of the low-level facial attributes with the statistical mean, avoiding the errors introduced by labeling in the machine learning method, so that the low-level facial attribute recognition results can be accurately obtained. In the high-level facial attribute recognition, the semantic relationship between the low-level facial attributes and the high-level facial attributes is utilized to avoid the errors that may occur in the automatic learning process of the machine learning method, thereby achieving efficient recognition of the high-level facial attributes.

[0017] The present invention can independently identify each underlying facial attribute and add new underlying facial attribute recognition tasks without affecting the recognition of other underlying facial attributes. This can be achieved by efficiently expanding the underlying facial attribute information library. Furthermore, the present invention can expand upper-level facial attribute recognition tasks simply by expanding the semantic relationships between underlying facial attributes and upper-level facial attributes, for example, by expanding the underlying-upper-level facial attribute semantic relationship library. Compared to machine learning methods, the present invention eliminates the need to add large amounts of data or labels, allowing for the expansion of facial attribute recognition tasks at a relatively low cost, resulting in high scalability.

[0018] It should be understood that the foregoing general description and the following detailed description are illustrative and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present invention and, together with the description, serve to explain the principles of the invention, in which:

[0020] Figure 1 A flowchart of a bottom-up hierarchical face attribute recognition method according to an embodiment of the present invention is shown;

[0021] Figure 2 A schematic diagram showing a facial information analysis and underlying facial attribute recognition process according to an embodiment of the present invention is shown;

[0022] Figure 3 A schematic diagram showing the semantic relationship between some underlying facial attributes and upper-level facial attributes according to one embodiment of the present invention;

[0023] Figure 4 A flowchart of an upper-layer face attribute recognition process according to an embodiment of the present invention is shown;

[0024] Figure 5 A block diagram of a bottom-up hierarchical face attribute recognition system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0026] Through research, the inventors discovered that facial attributes are intuitively interpretable. For example, the facial attribute "palpebral fissure" can be intuitively interpreted as the distance between the inner and outer corners of the eye, and this facial attribute can be obtained by measuring the number of pixels between the two corners of the eye; the "inner and outer canthal angles" can be intuitively interpreted as the angle between the upper and lower eyelid edges, and this facial attribute can be obtained by calculating the angle between the upper and lower eyelid edges. The recognition results of these facial attributes can be obtained by analyzing facial images to obtain facial information and applying simple geometric and statistical methods. They are referred to as underlying facial attributes in the present invention. In addition, there are some other attributes in facial images, such as "round eyes". This facial attribute can be intuitively interpreted as the shape of the eyes being round, and can be determined based on the palpebral fissure, the ratio of the palpebral fissure, and the inner and outer canthal angles. Such facial attributes depend on other facial attributes, have semantic relationships with other facial attributes, and can be obtained by simply combining other facial attributes. They are referred to as "upper-layer facial attributes" in the present invention.

[0027] In view of the intuitive interpretability of facial attributes, the underlying facial attributes can be obtained through facial image analysis and the upper-level facial attributes can be identified based on the underlying facial attributes. The present invention divides facial attributes into underlying facial attributes and upper-level facial attributes, identifies the underlying facial attributes based on the facial information in the facial image, and identifies the upper-level facial attributes based on the recognition results of the underlying facial attributes based on the semantic relationship between the underlying facial attributes and the upper-level facial attributes, thereby efficiently and accurately obtaining facial attribute recognition results and improving scalability.

[0028] One aspect of the present invention provides a bottom-up hierarchical facial attribute recognition method, which can be executed at a computing device with computing capabilities, and the computing device can be located at a client or a server.

[0029] Figure 1 The flowchart of the bottom-up hierarchical face attribute recognition method according to one embodiment of the present invention is schematically shown. The method comprises: face information analysis in a face image, bottom-level face attribute recognition based on the face information, and top-level face attribute recognition based on the bottom-level face attribute recognition result. Figure 1 The various steps of the bottom-up hierarchical face attribute recognition method are described in detail.

[0030] Step S11: Analyze facial information in the facial image.

[0031] Performing image analysis on the input face image can obtain the face information in the face image. Figure 2As shown, various types of image analysis can be performed on the input face image, including but not limited to: face detection, feature point positioning, face part segmentation, iris segmentation, pupil segmentation and three-dimensional face reconstruction. Based on these image analyses, multiple face information can be obtained from the face image, namely, face location box, facial feature point location, face part mask, iris mask, pupil mask and three-dimensional face shape. In these image analyses, face detection takes a face image as input and a face position frame as output, which can be a 4-dimensional vector representing the position of the face in the face image; feature point positioning takes a face image and a face position frame (i.e., the output of face detection) as input, and multiple (e.g., 81) facial feature point positions as output, each facial feature point position can be a 2-dimensional vector representing the position of the facial feature point in the face image; face part segmentation takes a face image as input and a face part mask as output, which can be a vector of the same size as the face image, used to identify multiple (e.g., 19) facial parts, including background, face, left eye, right eye, left eyebrow, Right eyebrow, left ear, right ear, necklace, earrings, glasses, hair, hat, upper lip, inside of mouth, lower lip, nose, neck and clothing; iris segmentation takes the eye area in the face image obtained based on the position of facial feature points as input and outputs an iris mask, which can be a vector of the same size as the eye area; pupil segmentation takes the eye area in the face image obtained based on the position of facial feature points as input and outputs a pupil mask, which can be a vector of the same size as the eye area; 3D face reconstruction takes the face image and the position of facial feature points as input and outputs a 3D face shape, which can be represented as a spatial coordinate point, i.e., a vector of coordinate points * 3. The multiple facial information obtained from the above image analysis can be spliced ​​together to form a multidimensional vector to represent the facial information in the face image.

[0032] Step S12: Identify underlying facial attributes based on facial information.

[0033] The underlying facial attribute recognition involves obtaining the recognition results of the underlying facial attributes from the facial information, and includes the following sub-steps:

[0034] S121. For each underlying facial attribute among all underlying facial attributes, map the underlying facial attribute to a corresponding portion in the facial information, and obtain a semantic absolute value of the underlying facial attribute based on the corresponding portion in the facial information.

[0035] As mentioned above, since facial attributes are intuitively interpretable, each underlying facial attribute (e.g., Figure 2The underlying facial attributes 1-N shown in are mapped to the corresponding parts of the facial information, so that the semantic absolute value corresponding to the underlying facial attribute is obtained according to the corresponding parts of the facial information represented in the form of a vector. In one example, for the underlying facial attribute "large palpebral fissure", it can be mapped to the position of the facial feature points in the facial information based on the semantic information, and the distance between the upper and lower eyelid margins can be obtained according to the position of the facial feature points, and the distance is used as the semantic absolute value corresponding to the underlying facial attribute "large palpebral fissure". In another example, for the underlying facial attribute "large eyes", it can be mapped to the position of the facial feature points or the facial component mask in the facial information based on the semantic information, and the area of ​​the eyes can be obtained according to the facial feature points or the facial component mask, and the area is used as the semantic absolute value corresponding to the underlying facial attribute "large eyes".

[0036] In this embodiment, a mapping formula can be pre-set to map each underlying facial attribute to a corresponding portion of the facial information, and to obtain the semantic absolute value corresponding to the underlying facial attribute based on the corresponding portion of the facial information. For example, for the underlying facial attribute "large palpebral fissure," the mapping formula is used to calculate the distance between the upper and lower eyelid edges based on the locations of the facial feature points; this distance is the semantic absolute value corresponding to "large palpebral fissure." The mapping formula can be stored in a dictionary-based underlying facial attribute information library. This dictionary-based underlying facial attribute information library can improve the efficiency of retrieval of underlying facial attributes.

[0037] S122. For each underlying facial attribute among all underlying facial attributes, obtain a recognition result of the underlying facial attribute based on a comparison result of the semantic absolute value of the underlying facial attribute and the statistical mean of the underlying facial attribute.

[0038] First, for each underlying facial attribute, calculate the difference between the corresponding semantic absolute value obtained in S121 and the statistical mean of the underlying facial attribute in the crowd, and calculate the ratio of the difference to the statistical mean, as shown in the following formula:

[0039]

[0040] Here, s represents the semantic absolute value corresponding to the underlying facial attribute, and m represents the statistical mean of the underlying facial attribute. Similar to the mapping formula, the statistical mean of the underlying facial attribute can also be stored in the underlying facial attribute information library, where the key is the name of the underlying facial attribute and the value is the mapping formula and the statistical mean of the underlying facial attribute.

[0041] Secondly, compare the calculated ratio ρ with a predetermined threshold. If ρ is greater than the predetermined threshold (for example, 20%), it is determined that the underlying facial attribute in the facial image is a positive attribute, represented by 1; if ρ is less than or equal to the predetermined threshold, it is determined that the underlying facial attribute in the facial image is a negative attribute, represented by 0.

[0042] The recognition results of all underlying facial attributes in a facial image can be stored in a underlying facial attribute set. This underlying facial attribute set can be in dictionary format, where the key is the name of the underlying facial attribute and the value is the recognition result of the underlying facial attribute. As mentioned above, the recognition result can be 1 or 0, with 1 indicating a positive attribute and 0 indicating a negative attribute. This improves the efficiency of retrieval of underlying facial attributes when subsequently identifying higher-level facial attributes.

[0043] Step S13: upper-layer facial attribute recognition based on the underlying facial attribute recognition results.

[0044] The upper-level facial attribute recognition is to obtain the recognition result of the upper-level facial attribute from the recognition result of the lower-level facial attribute based on the semantic relationship between the lower-level facial attributes and the upper-level facial attributes. Figure 3 The semantic relationship between some underlying face attributes and upper-level face attributes is schematically shown, such as Figure 3 As shown in the figure, the upper-level facial attributes include "willow-shaped eyebrows" and "straight eyebrows". The upper-level facial attribute "willow-shaped eyebrows" has a semantic relationship with the underlying facial attributes "small eyebrow angle", "large eyebrow curvature" and "small eyebrow tip angle". This means that the upper-level facial attribute "willow-shaped eyebrows" depends on the three underlying facial attributes "small eyebrow angle", "large eyebrow curvature" and "small eyebrow tip angle" and none of them can be missing. Figure 3 The upper-layer facial attribute "straight eyebrows" has a semantic relationship with the underlying facial attributes "large eyebrow angle", "small eyebrow curvature", and "large eyebrow tip angle", which means that the upper-layer facial attribute "straight eyebrows" depends on the three underlying facial attributes "large eyebrow angle", "small eyebrow curvature", and "large eyebrow tip angle", and none of them can be missing. When, in step S12, it is identified that the underlying facial attributes "small eyebrow angle", "large eyebrow curvature", and "small eyebrow tip angle" in the facial image are all positive attributes (expressed as 1), and the underlying facial attributes "large eyebrow angle", "small eyebrow curvature", and "large eyebrow tip angle" have negative attributes (expressed as 0), it can be obtained from the semantic relationship between the underlying facial attributes and the upper-layer facial attributes that the upper-layer facial attribute "willow-leaf eyebrows" in the facial image is a positive attribute, expressed as 1, and the upper-layer facial attribute "straight eyebrows" is a negative attribute, expressed as 0.

[0045] In this embodiment, the correspondence between the underlying facial attributes and the upper-level facial attributes can be stored in a dictionary-style underlying-upper-level facial attribute semantic relationship library, where the key is the name of the upper-level facial attribute and the value is a formula representing the semantic relationship between the upper-level facial attribute and the underlying facial attribute. For example, the semantic relationship between the upper-level facial attribute "willow-shaped eyebrows" and the underlying facial attributes "small eyebrow angle," "high eyebrow curvature," and "small eyebrow tip angle" can be expressed as follows:

[0046] P{willow-shaped eyebrows}=P{small eyebrow angle}&P{large eyebrow curvature}&P{small eyebrow tip angle}(2)

[0047] Here, P{small eyebrow angle} represents the recognition result for the underlying facial attribute "small eyebrow angle," P{large eyebrow curvature} represents the recognition result for the underlying facial attribute "large eyebrow curvature," and P{small eyebrow tip angle} represents the recognition result for the underlying facial attribute "small eyebrow tip angle." When all of these underlying facial attributes are positive (i.e., all represent 1), the recognition result for the upper-layer facial attribute "willow-shaped eyebrows" is also positive (i.e., 1). When at least one of these underlying facial attributes is negative, the recognition result for the upper-layer facial attribute "willow-shaped eyebrows" is negative (i.e., 0).

[0048] Figure 4 The flowchart of the upper-layer face attribute recognition process according to one embodiment of the present invention is schematically shown, wherein the recognition results of all the underlying face attributes in the face image are stored in the underlying face attribute set, and the correspondence between the underlying face attributes and the upper-layer face attributes is stored in the underlying-upper-layer face attribute semantic relationship library. Figure 4 As shown, the recognition process includes performing the following sub-steps for each upper-level face attribute in all upper-level face attributes:

[0049] S131. For the upper-layer facial attribute, search the lower-layer-upper-layer facial attribute semantic relationship library for a formula representing the semantic relationship between the upper-layer facial attribute and the lower-layer facial attribute, and obtain one or more lower-layer facial attributes having a semantic relationship with the upper-layer facial attribute.

[0050] S132. For the one or more underlying facial attributes found, search for recognition results of the one or more underlying facial attributes in the underlying facial attribute set.

[0051] S133. Substitute the recognition results of one or more underlying facial attributes into the formula representing the semantic relationship between the upper-layer facial attribute and the underlying facial attribute in the underlying-upper-layer facial attribute semantic relationship library, thereby obtaining the recognition result of the upper-layer facial attribute.

[0052] The above embodiment provides a bottom-up hierarchical facial attribute recognition method. Based on the intuitive interpretability of facial attributes, facial attributes are divided into bottom-level facial attributes and top-level facial attributes. The bottom-level facial attributes are identified based on facial information in a facial image. Based on the semantic relationship between the bottom-level facial attributes and the top-level facial attributes, the top-level facial attributes are identified based on the recognition results of the bottom-level facial attributes. In bottom-level facial attribute recognition, detailed and comprehensive facial information is obtained by analyzing the facial image, and the semantic absolute value of each bottom-level facial attribute is obtained from the facial information. The bottom-level facial attributes are then combined with the basic facial information. The bottom-level facial attributes are identified by comparing the semantic absolute value of the bottom-level facial attributes with the statistical mean, avoiding the errors introduced by labeling in machine learning methods, thereby accurately obtaining the bottom-level facial attribute recognition results. In top-level facial attribute recognition, the semantic relationship between the bottom-level facial attributes and the top-level facial attributes is utilized to avoid errors that may occur during the automatic learning process of the machine learning method, thereby achieving efficient recognition of the top-level facial attributes.

[0053] The above embodiment can independently identify each underlying facial attribute, and can add new underlying facial attribute recognition tasks without affecting the recognition of other underlying facial attributes, which can be achieved by efficiently expanding the underlying facial attribute information library; in addition, the upper-level facial attribute recognition task can be expanded only by expanding the semantic relationship between the underlying facial attributes and the upper-level facial attributes, that is, expanding the underlying-upper-level facial attribute semantic relationship library. Compared with the machine learning method, there is no need to add a large amount of data or labels, and the expansion of the facial attribute recognition task can be completed at a relatively low cost, and the scalability is high.

[0054] It should be noted that some exemplary methods are depicted as flow charts. Although the flow charts depict operations as being performed sequentially, it is understood that many of the operations can be performed in parallel, simultaneously, or synchronously, and the order of the operations can be rearranged. Processing can terminate when an operation is completed and can include additional steps not included in the figures or embodiments.

[0055] Another aspect of the present invention provides a bottom-up hierarchical face attribute recognition system.

[0056] Figure 5The following schematically illustrates a block diagram of a hierarchical facial attribute recognition system according to one embodiment of the present invention. The system includes a facial information processing module, a low-level facial attribute recognition module, and a high-level facial attribute recognition module. The facial information processing module is used to perform image analysis on an input facial image to obtain facial information; the low-level facial attribute recognition module is used to obtain recognition results for low-level facial attributes from the facial information; and the high-level facial attribute recognition module is used to obtain recognition results for high-level facial attributes from the recognition results for low-level facial attributes based on the semantic relationship between the low-level and high-level facial attributes.

[0057] Specifically, the facial information processing module can perform multiple image analyses on the input facial image, including but not limited to: face detection, feature point location, facial component segmentation, iris segmentation, pupil segmentation, and three-dimensional face reconstruction, thereby obtaining multiple facial information based on these analyses, namely, a face location frame, facial feature point locations, facial component masks, iris masks, pupil masks, and three-dimensional face shape. The underlying facial attribute recognition module maps each underlying facial attribute among all underlying facial attributes to a corresponding portion of the facial information, and obtains the semantic absolute value of the underlying facial attribute based on the corresponding portion of the facial information; and the underlying facial attribute recognition module obtains the recognition result of each underlying facial attribute among all underlying facial attributes based on the comparison result of the semantic absolute value of the underlying facial attribute and the statistical mean of the underlying facial attribute. The upper-layer facial attribute recognition module is used to obtain the recognition results of one or more underlying facial attributes that have a semantic relationship with the upper-layer facial attribute; based on the semantic relationship between one or more underlying facial attributes and the upper-layer facial attribute, the recognition result of the upper-layer facial attribute is obtained from the recognition results of one or more underlying facial attributes.

[0058] It should be understood that although several modules or units of the system are described above, the division of modules and units is not limited to this. In fact, the features and functions of two or more modules or units described above can also be implemented in one module or unit. Conversely, the features and functions of one module or unit described above can also be further divided into multiple modules or units to implement.

[0059] Another aspect of the present invention provides a computer system for an electronic device for implementing an embodiment of the present invention. The computer system may include a bus, and a processor, memory, input devices (such as a keyboard, mouse, sensor, etc.), output devices (such as a display, printer, speaker, etc.), communication interfaces (such as a parallel port, serial port, modem, network card, etc.) and other devices (such as detachable devices, drive devices, etc.) coupled to the bus.

[0060] The memory (such as ROM, PROM, EEPROM, RAM, SRAM, etc.) is used to store data and computer instructions or programs, including computer instructions or programs for implementing the bottom-up hierarchical face attribute recognition method described above. The processor is used to execute a series of actions specified by the computer instructions or programs, such as executing the computer instructions or programs stored in the memory. When the processor executes the computer instructions or programs stored in the memory, the computer system can implement the embodiment of the bottom-up hierarchical face attribute recognition method described above, including Figure 1 、 4 Follow the steps shown in .

[0061] Another aspect of the present invention provides a computer-readable medium, which includes but is not limited to: a floppy disk, a hard disk, a magnetic tape, other magnetic media, a CDROM, a CDRW, a DVD, other optical media, a punched card, other physical media, a ROM, a PROM, an EEPROM, a RAM, a SRAM, or other computer-readable media, and a transmission medium (such as a coaxial cable, an optical fiber cable, a carrier wave, etc.). The computer-readable medium can be included in the above-mentioned computer system, or it can be an uninstalled separate medium. The computer-readable medium is used to carry computer instructions or programs, including computer instructions or programs for implementing the bottom-up hierarchical face attribute recognition method described above. When the computer instructions or programs in the computer-readable medium are read and executed by a processor (for example, a processor in the above-mentioned computer system), the method embodiments described above can be implemented (for example, causing the above-mentioned computer system to implement), including Figure 1 、 4 Follow the steps shown in .

[0062] Although the present invention has been described through preferred embodiments, it should be understood that the present invention is not limited to the embodiments described above and shown in the drawings, and that various changes and modifications may be made by those skilled in the art without departing from the scope of the invention.

Claims

1. A bottom-up hierarchical face attribute recognition method, characterized in that: The method comprises: Step 1) Perform image analysis on the face image to obtain face information; Step 2) obtaining recognition results of underlying facial attributes from the facial information, wherein step 2) includes: Step 21) for each of all underlying facial attributes, mapping the underlying facial attribute to a corresponding portion of the facial information, and obtaining a semantic absolute value of the underlying facial attribute based on the corresponding portion of the facial information; and Step 22) for each of all underlying facial attributes, obtaining a recognition result of the underlying facial attribute based on a comparison result of the semantic absolute value of the underlying facial attribute and the statistical mean of the underlying facial attribute, where the recognition result of the underlying facial attribute is a positive attribute or a negative attribute; and Step 3) obtaining recognition results of upper-layer facial attributes from the recognition results of the lower-layer facial attributes based on the semantic relationship between the lower-layer facial attributes and the upper-layer facial attributes, wherein step 3) includes performing the following operations for each of all upper-layer facial attributes: Obtaining recognition results of one or more underlying facial attributes having a semantic relationship with the upper-layer facial attributes; Obtaining a recognition result of the upper-layer facial attribute based on recognition results of one or more lower-layer facial attributes having a semantic relationship with the upper-layer facial attribute; The method further comprises: An extensible underlying facial attribute information database is provided, wherein the underlying facial attribute information database stores a mapping formula and a statistical mean of each underlying facial attribute among all underlying facial attributes, wherein the mapping formula of each underlying facial attribute is used to map the underlying facial attribute to a corresponding portion of the facial information and obtain a semantic absolute value of the underlying facial attribute based on the corresponding portion of the facial information; An extensible bottom-upper-layer face attribute semantic relationship library is provided, wherein the bottom-upper-layer face attribute semantic relationship library stores a formula for representing a semantic relationship between each upper-layer face attribute in all upper-layer face attributes and one or more bottom-layer face attributes.

2. The method according to claim 1, characterized in that Step 22) includes performing the following operations for each of all underlying facial attributes: Calculate the difference between the absolute semantic value of the underlying facial attribute and the statistical mean of the underlying facial attribute; Calculating the ratio of the difference to the statistical mean of the underlying facial attributes; as well as The ratio is compared with a predetermined threshold, and if the ratio is greater than the predetermined threshold, the underlying facial attribute is set as a positive attribute; otherwise, the underlying facial attribute is set as a negative attribute.

3. The method according to claim 1, characterized in that Step 1) includes: Perform at least face detection, feature point location, face part segmentation, iris segmentation, pupil segmentation, and three-dimensional face reconstruction on the face image to obtain the face information; wherein the face information at least includes: a face location frame, facial feature point locations, face part masks, iris masks, pupil masks, and a three-dimensional face shape.

4. A bottom-up hierarchical face attribute recognition system for implementing the method according to any one of claims 1 to 3, characterized in that: The system comprises: A face information processing module is used to perform image analysis on the face image to obtain face information; An underlying facial attribute recognition module, configured to obtain recognition results of underlying facial attributes from the facial information; and The upper-layer facial attribute recognition module is used to obtain the recognition results of the upper-layer facial attributes from the recognition results of the lower-layer facial attributes according to the semantic relationship between the lower-layer facial attributes and the upper-layer facial attributes.

5. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

6. An electronic device, characterized in that: include: processor; as well as A memory having a computer program stored thereon and coupled to the processor, which enables the electronic device to implement the method according to any one of claims 1 to 3 when the computer program is executed by the processor.

Images