A character recognition method, device, equipment and storage medium
By using multi-level convolutional processing units to extract and fuse convolutional features in painted portrait images, the problem of low accuracy in portrait recognition is solved, and higher recognition accuracy is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2023-01-16
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies have low accuracy in recognizing people in painted portraits, mainly due to inconsistent painting styles and exaggerated facial features.
By extracting features from the facial region of a person using at least two levels of convolutional processing units in a feature extraction network, and fusing convolutional features from different levels, combined with deep semantic information and shallow texture location information, person recognition is performed.
It improves the accuracy of figure recognition in painted portrait images by enhancing the accuracy of recognition through rich facial feature extraction.
Smart Images
Figure CN116071802B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of image processing technology, specifically to a method, apparatus, device, and storage medium for human recognition. Background Technology
[0002] Portrait painting can depict real people by exaggerating or minimizing salient facial features. With the rapid development of deep learning technology, facial recognition of real people can be achieved through face detection and feature matching, and real-world face recognition models have reached a high accuracy rate. However, portrait recognition in drawing is still under development. The main reasons are: First, different artists have different styles in drawn portraits (such as comics), so even for the same character, different artists will depict different forms; second, paintings may exaggerate facial features, such as creating an unusually long nose, while in a normal photograph, facial features are more regular. These two reasons result in portrait recognition being less effective than real-world face recognition. Summary of the Invention
[0003] This disclosure provides a method, apparatus, device, and storage medium for human identification.
[0004] According to one aspect of this disclosure, a person recognition method is provided, the method comprising:
[0005] The facial region of a person is extracted based on at least two layers of convolutional processing units in the human feature extraction network, and the convolutional features output by at least two layers of convolutional processing units are obtained; the facial region of the person belongs to the target painted human image to be identified.
[0006] The convolutional features output from at least two levels of convolutional processing units are fused to obtain the target facial features of the person's facial region;
[0007] Based on the target facial features and the facial features of the painted figures in the painted figure image database, the person in the target painted figure image is identified.
[0008] According to one aspect of this disclosure, a person recognition device is provided, the device comprising:
[0009] The convolutional feature determination module is used to extract features from the facial region of a person based on at least two layers of convolutional processing units in the person feature extraction network, and to obtain the convolutional features output by at least two layers of convolutional processing units; the facial region of the person belongs to the target painted person image to be identified;
[0010] The target facial feature determination module is used to fuse the convolutional features output by at least two layers of convolutional processing units to obtain the target facial features of the person's facial region.
[0011] The person recognition module is used to identify the person in the target painted figure image based on the target's facial features and the facial features of painted figure images in the painted figure image library.
[0012] According to another aspect of this disclosure, an electronic device is provided, the electronic device comprising:
[0013] At least one processor; and
[0014] A memory communicatively connected to the at least one processor; wherein,
[0015] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the person recognition method according to any embodiment of this disclosure.
[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform the person recognition method described in any embodiment of this disclosure.
[0017] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the person recognition method described in any embodiment of this disclosure.
[0018] The technology disclosed herein can improve the accuracy of person recognition.
[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0021] Figure 1 This is a flowchart of a person recognition method provided according to an embodiment of the present disclosure;
[0022] Figure 2A This is a flowchart of another person recognition method provided according to an embodiment of the present disclosure;
[0023] Figure 3 This is a flowchart of yet another person recognition method provided according to an embodiment of the present disclosure;
[0024] Figure 4 This is a flowchart of yet another person recognition method provided according to an embodiment of the present disclosure;
[0025] Figure 5 This is a schematic diagram of the structure of a person recognition device according to an embodiment of the present disclosure;
[0026] Figure 6 This is a block diagram of an electronic device used to implement the person recognition method of the embodiments of this disclosure. Detailed Implementation
[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0028] It should be noted that the terms "first," "second," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0029] Furthermore, it should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of target painted figures images and painted figures images in the painted figures image library involved in the technical solution of the present invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0030] Figure 1 This is a flowchart illustrating a person recognition method according to an embodiment of this disclosure. This embodiment applies to the identification of people in drawn person images (such as cartoon character images). The method can be executed by a person recognition device, which can be implemented in software and / or hardware and integrated into an electronic device that carries person recognition functionality, such as a server. Figure 1 As shown, the person recognition method in this embodiment may include:
[0031] S101, based on at least two layers of convolutional processing units in the human feature extraction network, feature extraction is performed on the human face region to obtain convolutional features output by at least two layers of convolutional processing units.
[0032] In this embodiment, the facial region of the person belongs to the target painted figure image to be identified, that is, the facial region of the person in the target painted figure image. The target painted figure image refers to the painted figure image for which person recognition is required; optionally, the target painted figure image can be a figure image drawn using exaggerated techniques; it should be noted that the form of the target painted figure image is not specifically limited, and can be a cartoon character image, an oil painting character image, a traditional Chinese painting character image, etc.
[0033] A character feature extraction network is a network used to extract features from the facial region of a painted figure. It can be a convolutional neural network. Optionally, the character feature extraction network can include at least two layers of convolutional processing units, with the at least two layers of convolutional processing units cascaded in sequence. Each convolutional processing unit can contain at least one convolutional layer.
[0034] Specifically, the facial region of a person can be input into a human feature extraction network, and processed by at least two levels of convolutional processing units in the human feature extraction network to obtain the convolutional features output by each level of convolutional processing units.
[0035] It should be noted that different levels of convolutional processing units extract different convolutional features. The higher the level of the convolutional processing unit, the more semantic information is extracted into the convolutional features; the lower the level of the convolutional processing unit, the more information such as texture location is extracted into the convolutional features.
[0036] S102, fuse the convolutional features output by at least two layers of convolutional processing units to obtain the target facial features of the person's face region.
[0037] In this embodiment, the target facial features are used to characterize the features of the facial region of a person, and can be represented in matrix or vector form.
[0038] Specifically, based on a preset fusion method, the convolutional features output by at least two layers of convolutional processing units can be fused to obtain the target facial features of the person's face region. For example, the convolutional features output by at least two layers of convolutional processing units can be summed, and the summed features can be used as the target facial features of the person's face region. Alternatively, the convolutional features output by at least two convolutional processing units can be concatenated, and the concatenated features can be normalized to obtain the target facial features of the person's face region.
[0039] S103, Identify the person in the target painting image based on the facial features of the target and the facial features of the painting images in the painting image database.
[0040] Specifically, for each facial feature of each painted figure image in the painting figure image library, the target facial feature can be compared with the facial feature. Based on the comparison result, the person in the target painted figure image is identified. That is, if the target facial feature is similar to the facial feature, that is, the similarity between the target facial feature and the facial feature is greater than the similarity threshold, then the person corresponding to the facial feature is determined to be the person in the target painted figure image. The similarity threshold can be set by those skilled in the art according to the actual situation.
[0041] The technical solution provided in this disclosure extracts features from the facial region of a person using at least two levels of convolutional processing units in a human feature extraction network. This yields convolutional features output by at least two levels of convolutional processing units. These convolutional features are then fused to obtain target facial features of the facial region. Finally, based on these target facial features and facial features of painted figures in a database of painted figures, the person in the target painted figure image is identified. This technical solution, by fusing convolutional features from different levels, ensures that the obtained facial features contain both deep semantic features and shallow information such as texture location, enriching the facial features and thus improving the accuracy of person recognition.
[0042] Based on the above embodiments, as an optional method of this disclosure, the facial region of the target painted figure image can also be segmented to obtain the figure's facial region; wherein, the facial region includes facial features, hairstyle region and head shape region.
[0043] Specifically, the facial region of a target painted figure image can be segmented using an object detection network to obtain the face region. The object detection network, which extracts the facial region from the painted figure image, can be a pre-trained deep neural network.
[0044] It is understandable that the segmented facial region includes the facial features, hairstyle, and head shape. Compared to ordinary face recognition which only extracts the facial features, in this disclosure, due to the exaggerated nature of the person, the hairstyle and head shape can also fully reflect the person's characteristics. Therefore, when segmenting the facial region of the target drawn person image, retaining the person's hairstyle and head shape can ensure that the facial features extracted later are richer and more diverse.
[0045] In addition, as an alternative method of this disclosure, the painted figure image to be identified can also be detected to determine whether there is a painted face in the painted figure image. If there is, the painted figure image is determined to be the target painted figure image.
[0046] Figure 2A This is a flowchart of another person recognition method provided according to an embodiment of this disclosure; based on the above embodiments, this embodiment further optimizes the step of "fusing the convolutional features output by at least two levels of convolutional processing units to obtain the target facial features of the person's facial region," providing an optional implementation scheme. For example... Figure 2A As shown, the person recognition method in this embodiment may include:
[0047] S201, based on at least two layers of convolutional processing units in the human feature extraction network, feature extraction is performed on the human face region to obtain convolutional features output by at least two layers of convolutional processing units.
[0048] Among them, the facial area of the person is the target drawn figure image to be identified.
[0049] S202, obtain deep convolution features and shallow convolution features from the convolution features output by at least two levels of convolution processing units.
[0050] In this embodiment, deep convolutional features refer to the convolutional features output by higher-level convolutional processing units, which contain more semantic information. Shallow convolutional features refer to the convolutional features output by lower-level convolutional processing units, which contain information such as the texture position of the lower layers.
[0051] An alternative approach is to use the convolutional features output by the highest-level convolutional unit in at least two convolutional processing units as deep convolutional features, and the convolutional features output by the lowest-level convolutional unit in at least two convolutional processing units as shallow convolutional features. For example, a human feature extraction network contains five convolutional processing units: a first-level convolutional unit, a second-level convolutional unit, a third-level convolutional unit, a fourth-level convolutional unit, and a fifth-level convolutional unit, with these units cascaded. The convolutional features output by the first-level convolutional unit are used as shallow convolutional features; the convolutional features output by the fifth-level convolutional unit are used as deep convolutional features.
[0052] Another alternative approach is to use the convolutional features output by the higher-level convolutional processing units in at least two convolutional processing units as deep convolutional features, and the convolutional features output by the lower-level convolutional processing units in at least two convolutional processing units as shallow convolutional features. For example, a human feature extraction network contains five levels of convolutional processing units, namely, the first level, the second level, the third level, the fourth level, and the fifth level, with the convolutional processing units of each level cascaded. The first three levels (the first, second, and third levels) of convolutional processing units are considered lower-level convolutional processing units, and the last two levels (the fourth and fifth levels) are considered higher-level convolutional processing units. Choose any convolutional processing unit from the lower-level convolutional processing units and use its output convolutional features as shallow convolutional features. That is, choose any convolutional processing unit from the first, second, and third levels and use its output convolutional features as shallow convolutional features. Choose any convolutional processing unit from the higher-level convolutional processing units and use its output convolutional features as deep convolutional features. That is, choose any convolutional processing unit from the fourth and fifth levels and use its output convolutional features as deep convolutional features.
[0053] S203 performs a convolution operation on the deep convolution features to obtain the first deformation bias.
[0054] In this embodiment, the first deformation bias is used to shift grid points during deformable convolution of shallow convolution features. It can be represented by a sampling score map, which can be represented in matrix form.
[0055] Specifically, a regular convolution operation, denoted as CONV, is performed on the deep convolution features to obtain the offset field offsetfield. The offset field is then processed to obtain the first deformation offset offset.
[0056] S204. Based on the first deformation bias and shallow convolution features, the first shallow deformation features are obtained.
[0057] In this embodiment, the first shallow deformation feature refers to the feature obtained after performing a deformable convolution operation on the shallow convolution feature, which can be represented in matrix or vector form.
[0058] Specifically, using the first deformation bias, deformable convolution operation, denoted as DCN, is performed on the shallow convolution features to obtain the first shallow deformation features.
[0059] S205 fuses the deep convolution features and the first shallow deformation features to obtain the target facial features of the person's face region.
[0060] Specifically, a preset fusion method can be used to fuse deep convolutional features and first shallow deformation features to obtain the target facial features of the person's face region. For example, the deep convolutional features can be upsampled so that the feature dimensions of the deep convolutional features and the first shallow deformation features are the same. Then, the upsampled deep convolutional features and the first shallow deformation features are summed, and the sum is used as the target facial features of the person's face region. Alternatively, the upsampled deep convolutional features and the first shallow deformation features can be concatenated, and the concatenated features are used as the target facial features of the person's face region.
[0061] S206, Identify the person in the target painted figure image based on the facial features of the target and the facial features of the painted figure images in the painted figure image database.
[0062] The technical solution provided in this disclosure extracts features from the facial region of a person using at least two layers of convolutional processing units in a human feature extraction network, obtaining convolutional features output by at least two layers of convolutional processing units. Then, deep convolutional features and shallow convolutional features are obtained from the convolutional features output by at least two layers of convolutional processing units. A convolution operation is performed on the deep convolutional features to obtain a first deformation bias. Based on the first deformation bias and the shallow convolutional features, a first shallow deformation feature is obtained. The deep convolutional features and the first shallow deformation feature are fused to obtain the target facial features of the person's facial region. Then, based on the target facial features and the facial features of painted figures in the painted figure image library, the person in the target painted figure image is identified. Compared to the traditional deformable convolution method that uses the current layer (shallow convolution feature) to calculate the deformation bias, the present disclosure uses deep convolution feature to obtain the first deformation bias, making the determination of the first deformation feature more supervised, thereby obtaining a better first shallow deformation feature, which can effectively extract the target facial features and improve the accuracy of person recognition.
[0063] Figure 3 This is a flowchart of another person recognition method provided according to an embodiment of this disclosure. Based on the above embodiments, this embodiment further optimizes the step of "fusing the convolutional features output by at least two levels of convolutional processing units to obtain the target facial features of the person's facial region," providing an optional implementation scheme. For example... Figure 3 As shown, the person recognition method in this embodiment may include:
[0064] S301, based on at least two layers of convolutional processing units in the human feature extraction network, performs feature extraction on the human face region to obtain convolutional features output by at least two layers of convolutional processing units.
[0065] Among them, the facial area of the person is the target drawn figure image to be identified.
[0066] S302, for each adjacent convolutional processing unit, the convolutional features output by the higher-level convolutional processing unit are taken as deep features, and the convolutional features output by the lower-level convolutional processing unit are taken as shallow features. The deep features and shallow features are fused to obtain the fused features corresponding to the adjacent convolutional processing unit.
[0067] In this embodiment, deep features refer to the convolutional features output by the higher-level convolutional processing unit in adjacent layers. Shallow features refer to the convolutional features output by the lower-level convolutional processing unit in adjacent layers.
[0068] Specifically, for each adjacent convolutional processing unit, the convolutional features output by the higher-level convolutional processing unit are taken as deep features, and the convolutional features output by the lower-level convolutional processing unit are taken as shallow features. Then, a preset fusion method is used to fuse the deep and shallow features to obtain the fused features corresponding to the adjacent convolutional processing unit. For example, the deep and shallow features can be summed or concatenated to obtain the fused features corresponding to the adjacent convolutional processing unit. For example, a feature extraction network for people contains five layers of convolutional processing units: the first layer, the second layer, the third layer, the fourth layer, and the fifth layer, cascaded sequentially. For adjacent first-layer and second-layer convolutional processing units, the convolutional features output by the first-layer unit are shallow features, and the convolutional features output by the second-layer unit are deep features. These deep and shallow features are then combined... By performing line fusion, the fusion features corresponding to the first and second level convolutional processing units are obtained; and so on, the fusion features corresponding to the second and third level convolutional processing units, the third and fourth level convolutional processing units, and the fourth and fifth level convolutional processing units are obtained, that is, the fusion features of four levels can be obtained.
[0069] S303 fuses the fusion features corresponding to each adjacent convolutional processing unit to obtain the target facial features.
[0070] One alternative approach is to fuse the fusion features corresponding to adjacent convolutional processing units based on a preset fusion method to obtain the target facial features. For example, the fusion features corresponding to adjacent convolutional processing units can be summed sequentially, and the summed result can be used as the target facial features. Alternatively, the fusion features corresponding to adjacent convolutional processing units can be concatenated sequentially, and the concatenated result can be used as the target facial features.
[0071] Another alternative approach is to select deep and shallow fusion features from the fusion features corresponding to adjacent convolutional processing units. Specifically, the fusion features corresponding to adjacent convolutional processing units at higher levels can be used as deep fusion features, while the fusion features corresponding to adjacent convolutional processing units at lower levels can be used as shallow fusion features. For example, the fusion features corresponding to the fourth and fifth level convolutional processing units can be used as deep fusion features, while the fusion features corresponding to the first and second level convolutional processing units can be used as shallow fusion features. Then, the deep and shallow fusion features are fused to obtain the facial region of the person.
[0072] S304, Identify the person in the target painted figure image based on the target's facial features and the facial features of the painted figure images in the painted figure image database.
[0073] The technical solution provided in this disclosure extracts features from the facial region of a person using at least two convolutional processing units in a human feature extraction network. This yields convolutional features output by at least two convolutional processing units. Then, for each adjacent convolutional processing unit, the convolutional features output by the higher-level unit are used as deep features, and the convolutional features output by the lower-level unit are used as shallow features. These deep and shallow features are fused to obtain the fused features corresponding to that adjacent convolutional processing unit. The fused features corresponding to each adjacent convolutional processing unit are then fused together to obtain the target facial features. Finally, based on the target facial features and the facial features of painted figures in a database of painted figures, the person in the target painted figure image is identified. This technical solution, by fusing deep and shallow features, enriches the fused features. Furthermore, a secondary fusion of the fused features corresponding to each adjacent level results in the final target facial features possessing rich semantic and positional texture information, thereby improving the accuracy of person recognition.
[0074] Based on the above embodiments, as an optional approach of this disclosure, deep features and shallow features are fused to obtain fused features corresponding to the adjacent convolutional processing units, including:
[0075] A convolution operation is performed on the deep features to obtain the second deformation bias; based on the second deformation bias and the shallow features, the second shallow deformation features are obtained; the deep features and the second shallow deformation features are fused to obtain the fused features corresponding to the convolution processing unit of the adjacent layer.
[0076] The second shallow deformation feature refers to the feature obtained after performing deformable convolution operation on the shallow features, which can be represented in matrix or vector form.
[0077] Specifically, a second deformation bias is used to perform deformable convolution operation on the shallow convolution features to obtain the second shallow deformation features.
[0078] Specifically, a preset fusion method can be used to fuse deep features and second shallow deformation features to obtain the target facial features of the person's face region. For example, the deep features can be upsampled so that the feature dimensions of the deep features and the second shallow deformation features are the same, and then the upsampled deep features and the second shallow deformation features are summed, with the summed result used as the target facial features of the person's face region; or, the upsampled deep features and the second shallow deformation features can be concatenated, with the concatenated feature used as the target facial features of the person's face region.
[0079] Understandably, compared to the traditional deformable convolution method that uses the current layer (shallow convolution feature) to calculate the deformation bias, this disclosure uses deep features to obtain the second deformation bias, making the determination of the second deformation feature more supervised, thereby obtaining a better second shallow deformation feature, which can effectively extract the target facial features and lay the foundation for improving the accuracy of person recognition.
[0080] Figure 4 This is a flowchart of another person recognition method provided according to an embodiment of this disclosure. Based on the above embodiments, this embodiment further optimizes the process of "identifying the person in the target painted figure image based on the target's facial features and the facial features of painted figure images in a painted figure image database," providing an optional implementation scheme. For example... Figure 4 As shown, the person recognition method in this embodiment may include:
[0081] S401, based on at least two layers of convolutional processing units in the human feature extraction network, performs feature extraction on the human face region to obtain convolutional features output by at least two layers of convolutional processing units.
[0082] The facial area of the figure belongs to the target drawn figure image to be identified.
[0083] S402, fuses the convolutional features output by at least two layers of convolutional processing units to obtain the target facial features of the person's face region.
[0084] S403, cluster the facial features of the painted figures in the painting figure image database to obtain at least one feature cluster center corresponding to a figure.
[0085] In this embodiment, the feature cluster center refers to the cluster center after clustering the facial features of each person.
[0086] Specifically, a preset clustering method can be used to cluster the facial features of the painted figures in the image library to obtain the feature cluster center for each figure. The preset clustering method can be k-means clustering, etc., and this disclosure does not impose any specific limitations.
[0087] S404, Match the target facial features with the feature cluster centers corresponding to at least one person.
[0088] Specifically, the target facial features can be matched with the feature cluster centers corresponding to each person. For example, the similarity between the target walking facial features and the feature cluster centers corresponding to each person can be determined.
[0089] S405, Based on the matching results, identify the figures in the target painting figure image.
[0090] Specifically, based on the similarity in the matching results, the person in the target painting image is identified. For each person in the painting image database, if the similarity between the target facial features and the corresponding feature cluster center of that person is greater than a set threshold, then that person is identified as a person in the target painting image.
[0091] The technical solution provided in this disclosure extracts features from the facial region of a person using at least two levels of convolutional processing units in a human feature extraction network. This yields convolutional features output by these two levels of units. These convolutional features are then fused to obtain the target facial features of the person's face. Furthermore, the facial features of painted figures in a database of painted figures are clustered to obtain feature cluster centers corresponding to at least one person. The target facial features are then matched with these feature cluster centers, and based on the matching results, the person in the target painted figure image is identified. This clustering-based approach to person matching reduces false positives and false negatives, thereby improving the accuracy of person matching and recognition.
[0092] Figure 5This is a schematic diagram of a person recognition device according to an embodiment of this disclosure. This embodiment is applicable to situations where people in drawn figures (such as cartoon characters) are identified. The device can be implemented using software and / or hardware and can be integrated into an electronic device that carries person recognition functionality, such as a server. Figure 5 As shown, the person recognition device 500 in this embodiment may include:
[0093] The convolutional feature determination module 501 is used to extract features from the facial region of a person based on at least two layers of convolutional processing units in the person feature extraction network, and obtain the convolutional features output by at least two layers of convolutional processing units; the facial region of the person belongs to the target painted person image to be identified;
[0094] The target facial feature determination module 502 is used to fuse the convolutional features output by at least two layers of convolutional processing units to obtain the target facial features of the person's facial region.
[0095] The person recognition module 503 is used to identify the person in the target painting image based on the target's facial features and the facial features of the painting images in the painting image library.
[0096] The technical solution provided in this disclosure extracts features from the facial region of a person using at least two levels of convolutional processing units in a human feature extraction network. This yields convolutional features output by at least two levels of convolutional processing units. These convolutional features are then fused to obtain target facial features of the facial region. Finally, based on these target facial features and facial features of painted figures in a database of painted figures, the person in the target painted figure image is identified. This technical solution, by fusing convolutional features from different levels, ensures that the obtained facial features contain both deep semantic features and shallow information such as texture location, enriching the facial features and thus improving the accuracy of person recognition.
[0097] Furthermore, the target facial feature determination module 502 is specifically used for:
[0098] Extract deep convolutional features and shallow convolutional features from the convolutional features output by at least two levels of convolutional processing units;
[0099] Perform a convolution operation on the deep convolutional features to obtain the first deformation bias;
[0100] Based on the first deformation bias and shallow convolution features, the first shallow deformation features are obtained;
[0101] The deep convolution features and the first shallow deformation features are fused to obtain the target facial features of the person's face region.
[0102] Furthermore, the target facial feature determination module 502 includes:
[0103] The fusion feature determination unit is used to, for each adjacent convolutional processing unit, take the convolutional features output by the higher-level convolutional processing unit as deep features and the convolutional features output by the lower-level convolutional processing unit as shallow features, and fuse the deep features and shallow features to obtain the fusion feature corresponding to the convolutional processing unit of that adjacent level.
[0104] The target facial feature determination unit is used to fuse the fusion features corresponding to each adjacent convolutional processing unit to obtain the target facial features.
[0105] Furthermore, the feature fusion determination unit is specifically used for:
[0106] Convolution operations are performed on deep features to obtain the second deformation bias;
[0107] The second shallow deformation features are obtained based on the second deformation bias and shallow features;
[0108] The deep features and the second shallow deformation features are fused to obtain the fused features corresponding to the convolutional processing units of the adjacent layers.
[0109] Furthermore, the person recognition module 503 is specifically used for:
[0110] Cluster the facial features of the painted figures in the painting figure image database to obtain the feature cluster center corresponding to at least one figure;
[0111] Match the target facial features with the feature cluster centers corresponding to at least one person;
[0112] Based on the matching results, the figures in the target painting image are identified.
[0113] Furthermore, the device also includes:
[0114] The facial region determination module is used to cut out the facial region of the target drawn figure image to obtain the facial region; the facial region includes the facial features, hairstyle region and head shape region.
[0115] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0116] Figure 6 This is a block diagram of an electronic device used to implement the person recognition method of the embodiments of this disclosure. Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0117] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0118] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0119] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the person recognition method. For example, in some embodiments, the person recognition method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the person recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the person recognition method by any other suitable means (e.g., by means of firmware).
[0120] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0121] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0122] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0124] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0125] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0126] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.
[0127] Cloud computing refers to a technology system that enables access to a shared pool of physical or virtual resources via a network. These resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for applications such as artificial intelligence and blockchain, as well as for model training.
[0128] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0129] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A person recognition method, comprising: The facial region of a person is extracted based on at least two layers of convolutional processing units in the human feature extraction network, and the convolutional features output by at least two layers of convolutional processing units are obtained; the facial region of the person belongs to the target painted human image to be identified. The convolutional features output from at least two levels of convolutional processing units are fused to obtain the target facial features of the person's facial region, including: Extract deep convolutional features and shallow convolutional features from the convolutional features output by at least two levels of convolutional processing units; A convolution operation is performed on the deep convolutional features to obtain a first deformation bias; wherein, the first deformation bias is used to offset and move grid points during deformable convolution of the shallow convolutional features; The first shallow deformation feature is obtained based on the first deformation bias and the shallow convolution feature; The deep convolutional features and the first shallow deformation features are fused to obtain the target facial features of the person's facial region; Based on the target facial features and the facial features of the painted figures in the painted figure image database, the person in the target painted figure image is identified.
2. The method according to claim 1, wherein, The process of fusing the convolutional features output by at least two levels of convolutional processing units to obtain the target facial features of the person's facial region includes: For each adjacent convolutional processing unit, the convolutional features output by the higher-level convolutional processing unit are taken as deep features, and the convolutional features output by the lower-level convolutional processing unit are taken as shallow features. The deep features and the shallow features are fused to obtain the fused features corresponding to the adjacent convolutional processing unit. The fusion features corresponding to each adjacent convolutional processing unit are fused to obtain the target facial features.
3. The method according to claim 2, wherein, The deep features and the shallow features are fused to obtain the fused features corresponding to the adjacent convolutional processing units, including: The deep features are convolved to obtain the second deformation bias; The second shallow deformation feature is obtained based on the second deformation bias and the shallow feature; The deep features and the second shallow deformation features are fused to obtain the fused features corresponding to the convolutional processing units of the adjacent layers.
4. The method according to claim 1, wherein, The step of identifying the person in the target painted figure image based on the target facial features and the facial features of painted figure images in the painted figure image database includes: Cluster the facial features of the painted figures in the painting figure image database to obtain the feature cluster center corresponding to at least one figure; The target facial features are matched with the feature cluster centers corresponding to at least one person; Based on the matching results, the figures in the target painting image are identified.
5. The method according to claim 1, further comprising: The facial region of the target drawn figure image is segmented to obtain the figure's facial region; wherein, the facial region includes the facial features, hairstyle region and head shape region.
6. A person recognition device, comprising: The convolutional feature determination module is used to extract features from the facial region of a person based on at least two layers of convolutional processing units in the person feature extraction network, and to obtain the convolutional features output by at least two layers of convolutional processing units; the facial region of the person belongs to the target painted person image to be identified; The target facial feature determination module is used to fuse the convolutional features output by at least two layers of convolutional processing units to obtain the target facial features of the person's facial region, including: Extract deep convolutional features and shallow convolutional features from the convolutional features output by at least two levels of convolutional processing units; A convolution operation is performed on the deep convolutional features to obtain a first deformation bias; wherein, the first deformation bias is used to offset and move grid points during deformable convolution of the shallow convolutional features; The first shallow deformation feature is obtained based on the first deformation bias and the shallow convolution feature; The deep convolutional features and the first shallow deformation features are fused to obtain the target facial features of the person's facial region; The person recognition module is used to identify the person in the target painted figure image based on the target's facial features and the facial features of painted figure images in the painted figure image library.
7. The apparatus according to claim 6, wherein, The target facial feature determination module includes: The fusion feature determination unit is used to, for each adjacent convolutional processing unit, take the convolutional features output by the higher-level convolutional processing unit as deep features and the convolutional features output by the lower-level convolutional processing unit as shallow features, and fuse the deep features and the shallow features to obtain the fusion feature corresponding to the adjacent convolutional processing unit. The target facial feature determination unit is used to fuse the fusion features corresponding to each adjacent convolutional processing unit to obtain the target facial features.
8. The apparatus according to claim 7, wherein, The fusion feature determination unit is specifically used for: The deep features are convolved to obtain the second deformation bias; The second shallow deformation feature is obtained based on the second deformation bias and the shallow feature; The deep features and the second shallow deformation features are fused to obtain the fused features corresponding to the convolutional processing units of the adjacent layers.
9. The apparatus according to claim 6, wherein, The person recognition module is specifically used for: Cluster the facial features of the painted figures in the painting figure image database to obtain the feature cluster center corresponding to at least one figure; The target facial features are matched with the feature cluster centers corresponding to at least one person; Based on the matching results, the figures in the target painting image are identified.
10. The apparatus according to claim 6, further comprising: The facial region determination module is used to cut out the facial region of the target drawn figure image to obtain the facial region; wherein, the facial region includes the facial features, hairstyle region and head shape region.
11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the person recognition method according to any one of claims 1-5.
12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the person recognition method according to any one of claims 1-5.
13. A computer program product comprising a computer program that, when executed by a processor, implements the person recognition method according to any one of claims 1-5.