Method, device, electronic device and storage medium for constructing facial feature library

Abstract

The present disclosure relates to the field of Internet technology, and more particularly to a method, apparatus, electronic device, and storage medium for constructing a facial feature library. For each first entity tag in a first entity tag set, multiple first person images corresponding to the first entity tag are obtained, and the multiple first person images are preprocessed to obtain multiple facial features. For each first entity tag, the multiple facial features corresponding to the first entity tag are subjected to a preset screening process to obtain a first facial feature set. A similarity comparison is performed on each two first facial feature sets in the multiple first facial feature sets. If the similarity comparison results of the two first facial feature sets meet a preset condition, any two first facial feature sets are merged. A facial feature library is constructed based on the newly obtained multiple first facial feature sets and the first entity tag set. The present disclosure can efficiently construct a more complete facial feature library and improve the recognition accuracy of person entity tags.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, device, electronic device and storage medium for constructing a facial feature library. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present disclosure as recited in the claims. No portion of the description herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] With the development of the mobile video industry, massive amounts of videos are posted online every day. These videos contain a large number of film and television mashups, character collections, and other content. However, many of these videos do not have clear entity labels (such as names) for the characters. By performing face recognition on the characters in the video images, specific entity labels can be obtained. These entity labels are beneficial for video platforms to retrieve and distribute videos and recommend videos that users may like. On the other hand, they can also eliminate videos of people with negative influences and prevent these videos from entering the video platform.

[0004] Face recognition for people usually faces problems such as changing makeup, changing environments, and changing angles. Therefore, in order to better identify the entity labels of people in video images, it is necessary to build a more complete face feature library and provide more person images for the entity labels of each person to be identified.

[0005] However, the construction of facial feature libraries is time-consuming and labor-intensive. How to improve the construction efficiency and richness of facial feature libraries is a problem that needs to be solved. Summary of the Invention

[0006] The embodiments of the present disclosure provide a method, device, electronic device, and storage medium for constructing a facial feature library, which are used to efficiently construct a more complete facial feature library. When subsequently identifying the entity labels of people in a video, more person images can be provided for the entity labels of each person, thereby improving the recognition accuracy.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for constructing a facial feature library, comprising:

[0008] For each first entity tag in the first entity tag set, obtain multiple first person images corresponding to the first entity tag, and preprocess the multiple first person images to obtain multiple facial features; wherein the preprocessing includes at least face detection and facial feature extraction;

[0009] For each first entity tag, performing a preset screening process on multiple facial features corresponding to the first entity tag to obtain a first facial feature set;

[0010] Performing a similarity comparison on each two first facial feature sets in the plurality of first facial feature sets, and merging the arbitrary two first facial feature sets if the similarity comparison results of the two first facial feature sets meet a preset condition in each similarity comparison;

[0011] A facial feature library is constructed based on the newly obtained multiple first facial feature sets and the first entity label set.

[0012] Optionally, performing a preset screening process on the multiple facial features corresponding to the first entity tag to obtain a first facial feature set includes:

[0013] performing clustering processing on the plurality of facial features corresponding to the first entity label to obtain a plurality of first facial feature clusters, each first facial feature cluster including a cluster center;

[0014] For each cluster center, if the similarity between the cluster center and each other cluster center does not meet the first similarity condition, deleting the first facial feature cluster where the cluster center is located;

[0015] Based on the remaining multiple first facial feature clusters, the first facial feature set is obtained.

[0016] Optionally, obtaining the first facial feature set based on the remaining multiple first facial feature clusters includes:

[0017] For each remaining first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and the corresponding cluster center, and removing facial features whose similarity does not meet a second similarity condition from the first facial feature cluster;

[0018] The first facial feature set is obtained based on the newly obtained multiple first facial feature clusters.

[0019] Optionally, obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters includes:

[0020] For each newly obtained first facial feature cluster, if the number of facial features in the first facial feature cluster does not meet the quantity condition, the first facial feature cluster is deleted.

[0021] Optionally, obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters includes:

[0022] For each newly obtained first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and a plurality of other facial features, and removing the other facial features whose similarities meet a third similarity condition from the first facial feature cluster;

[0023] Based on the multiple newly obtained first facial feature clusters, the first facial feature set is obtained.

[0024] Optionally, performing a similarity comparison on each two first facial feature sets in the plurality of first facial feature sets, and merging the two first facial feature sets if the similarity comparison result of the two first facial feature sets meets a preset condition in each similarity comparison, includes:

[0025] For each of the two first face feature sets, perform the following operations:

[0026] Obtaining multiple cluster centers of a first facial feature set, and obtaining multiple cluster centers of another first facial feature set;

[0027] Determine the similarity between each cluster center of the first facial feature set and multiple cluster centers of the other first facial feature set to obtain a similarity vector;

[0028] constructing an inter-class similarity matrix based on the obtained multiple similarity vectors, determining target similarities in the inter-class similarity matrix that reach a first similarity threshold, and using the number of the target similarities as the similarity comparison result;

[0029] If the similarity comparison result meets a preset condition, the one first facial feature set and the other first facial feature set are merged.

[0030] Optionally, before merging the first facial feature set with the other first facial feature set, the method further includes:

[0031] Constructing a first intra-class similarity matrix based on the similarity between each cluster center of the first facial feature set and each other cluster center of the first facial feature set;

[0032] Constructing a second intra-class similarity matrix based on the similarity between each cluster center of the another first facial feature set and each of the other cluster centers of the another first facial feature set;

[0033] If it is determined based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix that a target cluster center in the one first facial feature set and each cluster center in the other first facial feature set satisfy a first preset relationship, then the first facial feature cluster in which the target cluster center is located is removed;

[0034] If, based on the first intra-class similarity matrix, the second intra-class similarity matrix and the inter-class similarity matrix, it is determined that the target cluster center in the other first facial feature set and the cluster centers of the one first facial feature set satisfy a second preset relationship, then the first facial feature cluster where the target cluster center is located is eliminated.

[0035] Optionally, the method further includes:

[0036] Acquire a second entity tag set and a second person image set, wherein each second entity tag in the second entity tag set corresponds to at least one second person image in the second person image set;

[0037] For each second person image, if a target facial feature that matches the facial feature in the second person image is retrieved from the facial feature library, the facial feature in the second person image is saved under the target entity label corresponding to the target facial feature in the facial feature library.

[0038] Optionally, if a target facial feature matching the facial feature in the second person image is retrieved from the facial feature library, saving the facial feature in the second person image to a target entity tag corresponding to the target facial feature in the facial feature library includes:

[0039] If a target facial feature matching the facial feature in the second person image is retrieved from the facial feature library, determining whether the target facial feature satisfies at least one of the following conditions: a target entity label corresponding to the target facial feature is consistent with a second entity label corresponding to the second person image, and a similarity between the target facial feature and the facial feature in the second person image is less than a second similarity threshold;

[0040] If so, the facial features in the second person image are saved under the target entity label in the facial feature library.

[0041] Optionally, the method further includes:

[0042] For each second person image, if no target facial features matching the facial features in the second person image are retrieved from the facial feature library, the second person image is used as a candidate person image to be stored in the library;

[0043] Obtaining a new second entity label set based on the second labels corresponding to the plurality of candidate person images, and preprocessing the plurality of candidate person images to obtain a second facial feature set; wherein the preprocessing includes at least face detection and facial feature extraction;

[0044] The second facial feature set and the new second entity tag set are saved to a facial feature temporary library, where each second entity tag in the new second entity tag set corresponds to at least one facial feature in the second facial feature set.

[0045] Optionally, the step of saving the second facial feature set and the new second entity label set to a temporary facial feature library includes:

[0046] performing clustering processing on the plurality of facial features in the second facial feature set to obtain a plurality of second facial feature clusters, each second facial feature cluster including a cluster center;

[0047] For each second facial feature cluster, if the similarity between the candidate facial feature in the second facial feature cluster and the corresponding cluster center does not meet the fourth similarity condition, then the candidate facial feature is removed from the second facial feature cluster;

[0048] Based on the newly obtained multiple second facial feature clusters, a new second facial feature set is obtained, and the new second facial feature set and the new second entity label set are correspondingly saved in the facial feature temporary library.

[0049] Optionally, the method further includes:

[0050] For each second entity tag in the new second entity tag set, performing the preset screening process on multiple facial features corresponding to the second entity tag to obtain a third facial feature set;

[0051] A plurality of third facial feature sets and second entity labels corresponding to each of the plurality of third facial feature sets are obtained and saved in the facial feature library.

[0052] Optionally, obtaining a plurality of first person images corresponding to the first entity tag includes:

[0053] For any first web page, identify page content in the first web page, and if a person image and person entity information matching any first entity tag are identified, use the identified person image as the first person image corresponding to any first entity tag;

[0054] A plurality of first person images corresponding to each of the first entity tags are identified from the plurality of first web pages.

[0055] Optionally, obtaining the second entity tag set and the second person image set includes:

[0056] For any second web page, identify page content in the second web page, and if a person image and person entity information are identified, obtain a second entity tag based on the person entity information, and use the identified person image as a second person image corresponding to the second entity tag;

[0057] The second entity tag set is obtained based on the second entity tags obtained from the plurality of second network pages, and the second person image set is obtained based on at least one second person image corresponding to each second entity tag.

[0058] In a second aspect, the present disclosure also provides a device for constructing a facial feature library, including:

[0059] A first acquisition module is configured to acquire, for each first entity tag in the first entity tag set, a plurality of first person images corresponding to the first entity tag, and preprocess the plurality of first person images to obtain a plurality of facial features; wherein the preprocessing includes at least face detection and facial feature extraction;

[0060] a screening module configured to perform a preset screening process on a plurality of facial features corresponding to each first entity tag to obtain a first facial feature set;

[0061] a comparison module, configured to perform a similarity comparison on each two first facial feature sets in the plurality of first facial feature sets, and in each similarity comparison, if the similarity comparison result of the two first facial feature sets meets a preset condition, then merge the two first facial feature sets;

[0062] A construction module is used to construct a facial feature library based on the newly obtained multiple first facial feature sets and the first entity label set.

[0063] Optionally, the screening module is further configured to:

[0064] performing clustering processing on the plurality of facial features corresponding to the first entity label to obtain a plurality of first facial feature clusters, each first facial feature cluster including a cluster center;

[0065] For each cluster center, if the similarity between the cluster center and each other cluster center does not meet the first similarity condition, deleting the first facial feature cluster where the cluster center is located;

[0066] Based on the remaining multiple first facial feature clusters, the first facial feature set is obtained.

[0067] Optionally, when obtaining the first facial feature set based on the remaining multiple first facial feature clusters, the screening module is further configured to:

[0068] For each remaining first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and the corresponding cluster center, and removing facial features whose similarity does not meet a second similarity condition from the first facial feature cluster;

[0069] The first facial feature set is obtained based on the newly obtained multiple first facial feature clusters.

[0070] Optionally, when obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters, the screening module is further configured to:

[0071] For each newly obtained first facial feature cluster, if the number of facial features in the first facial feature cluster does not meet the quantity condition, the first facial feature cluster is deleted.

[0072] Optionally, when obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters, the screening module is further configured to:

[0073] For each newly obtained first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and a plurality of other facial features, and removing the other facial features whose similarities meet a third similarity condition from the first facial feature cluster;

[0074] Based on the multiple newly obtained first facial feature clusters, the first facial feature set is obtained.

[0075] Optionally, the comparison module is further configured to:

[0076] For each of the two first face feature sets, perform the following operations:

[0077] Obtaining multiple cluster centers of a first facial feature set, and obtaining multiple cluster centers of another first facial feature set;

[0078] Determine the similarity between each cluster center of the first facial feature set and multiple cluster centers of the other first facial feature set to obtain a similarity vector;

[0079] constructing an inter-class similarity matrix based on the obtained multiple similarity vectors, determining target similarities in the inter-class similarity matrix that reach a first similarity threshold, and using the number of the target similarities as the similarity comparison result;

[0080] If the similarity comparison result meets a preset condition, the one first facial feature set and the other first facial feature set are merged.

[0081] Optionally, a culling module is also included to:

[0082] Constructing a first intra-class similarity matrix based on the similarity between each cluster center of the first facial feature set and each other cluster center of the first facial feature set;

[0083] Constructing a second intra-class similarity matrix based on the similarity between each cluster center of the another first facial feature set and each of the other cluster centers of the another first facial feature set;

[0084] If it is determined based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix that a target cluster center in the one first facial feature set and each cluster center in the other first facial feature set satisfy a first preset relationship, then the first facial feature cluster in which the target cluster center is located is removed;

[0085] If, based on the first intra-class similarity matrix, the second intra-class similarity matrix and the inter-class similarity matrix, it is determined that the target cluster center in the other first facial feature set and the cluster centers of the one first facial feature set satisfy a second preset relationship, then the first facial feature cluster where the target cluster center is located is eliminated.

[0086] Optionally, the device further includes:

[0087] A second acquisition module is configured to acquire a second entity tag set and a second character image set, wherein each second entity tag in the second entity tag set corresponds to at least one second character image in the second character image set;

[0088] The first updating module is used to save the facial features in the second person image to the target entity label corresponding to the target facial features in the facial feature library for each second person image if a target facial feature matching the facial features in the second person image is retrieved from the facial feature library.

[0089] Optionally, the first update module is further configured to:

[0090] If a target facial feature matching the facial feature in the second person image is retrieved from the facial feature library, determining whether the target facial feature satisfies at least one of the following conditions: a target entity label corresponding to the target facial feature is consistent with a second entity label corresponding to the second person image, and a similarity between the target facial feature and the facial feature in the second person image is less than a second similarity threshold;

[0091] If so, the facial features in the second person image are saved under the target entity label in the facial feature library.

[0092] Optionally, the device further includes a storage module, configured to:

[0093] For each second person image, if no target facial features matching the facial features in the second person image are retrieved from the facial feature library, the second person image is used as a candidate person image to be stored in the library;

[0094] Obtaining a new second entity label set based on the second labels corresponding to the plurality of candidate person images, and preprocessing the plurality of candidate person images to obtain a second facial feature set; wherein the preprocessing includes at least face detection and facial feature extraction;

[0095] The second facial feature set and the new second entity tag set are saved to a facial feature temporary library, where each second entity tag in the new second entity tag set corresponds to at least one facial feature in the second facial feature set.

[0096] Optionally, when saving the second facial feature set and the new second entity label set to the facial feature temporary library, the saving module is further configured to:

[0097] performing clustering processing on the plurality of facial features in the second facial feature set to obtain a plurality of second facial feature clusters, each second facial feature cluster including a cluster center;

[0098] For each second facial feature cluster, if the similarity between the candidate facial feature in the second facial feature cluster and the corresponding cluster center does not meet the fourth similarity condition, then the candidate facial feature is removed from the second facial feature cluster;

[0099] Based on the newly obtained multiple second facial feature clusters, a new second facial feature set is obtained, and the new second facial feature set and the new second entity label set are correspondingly saved in the facial feature temporary library.

[0100] Optionally, the device further includes a second updating module, configured to:

[0101] For each second entity tag in the new second entity tag set, performing the preset screening process on multiple facial features corresponding to the second entity tag to obtain a third facial feature set;

[0102] A plurality of third facial feature sets and second entity labels corresponding to each of the plurality of third facial feature sets are obtained and saved in the facial feature library.

[0103] Optionally, the first acquisition module is further configured to:

[0104] For any first web page, identify page content in the first web page, and if a person image and person entity information matching any first entity tag are identified, use the identified person image as the first person image corresponding to any first entity tag;

[0105] A plurality of first person images corresponding to each of the first entity tags are identified from the plurality of first web pages.

[0106] Optionally, the second acquisition module is further configured to:

[0107] For any second web page, identify page content in the second web page, and if a person image and person entity information are identified, obtain a second entity tag based on the person entity information, and use the identified person image as a second person image corresponding to the second entity tag;

[0108] The second entity tag set is obtained based on the second entity tags obtained from the plurality of second network pages, and the second person image set is obtained based on at least one second person image corresponding to each second entity tag.

[0109] In a third aspect, an embodiment of the present disclosure further provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor implements the steps of any one of the facial feature library construction methods of the first aspect.

[0110] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program. When the computer program runs on an electronic device, the electronic device executes the steps of any one of the facial feature library construction methods of the first aspect.

[0111] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, comprising a computer program, which is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any one of the above-mentioned facial feature library construction methods.

[0112] The method for constructing a facial feature library provided by the embodiments of the present disclosure has at least the following beneficial effects:

[0113] According to the solution provided by the embodiments of the present disclosure, when constructing a facial feature library, a first entity tag set including first entity tags of multiple persons is first obtained. For each first entity tag, multiple first person images corresponding to the first entity tag are obtained, and face detection and facial feature extraction are performed on each of the first person images to obtain multiple facial features. Then, a preset screening process is performed on the multiple facial features corresponding to each first entity tag to obtain a first facial feature set corresponding to each first entity tag, so as to complete the facial feature screening within each first entity tag. In addition, considering that a person may have multiple first entity tags, it is necessary to determine whether to merge the first facial feature sets of the multiple first entity tags. Specifically, a similarity comparison is performed on each two first facial feature sets in the multiple first facial feature sets. In each similarity comparison, if the similarity comparison result of the two first facial feature sets meets a preset condition, any two first facial feature sets are merged. Finally, a facial feature library is constructed based on the newly obtained multiple first facial feature sets and the first entity tag set.

[0114] Based on the above-mentioned scheme of the embodiment of the present disclosure, a more complete facial feature library can be efficiently constructed. When subsequently identifying the entity labels of characters in the video, more character images can be provided for the entity labels of each character, thereby improving the recognition accuracy.

[0115] Other features and advantages of the present disclosure will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present disclosure. The purposes and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0117] Figure 1 A flowchart of a method for constructing a facial feature library provided in an embodiment of the present disclosure;

[0118] Figure 2 A flowchart of another method for constructing a facial feature library provided in an embodiment of the present disclosure;

[0119] Figure 3 A flowchart of another method for constructing a facial feature library provided in an embodiment of the present disclosure;

[0120] Figure 4 A flowchart of another method for constructing a facial feature library provided in an embodiment of the present disclosure;

[0121] Figure 5 A flowchart of another method for constructing a facial feature library provided in an embodiment of the present disclosure;

[0122] Figure 6 A logical schematic diagram of constructing a facial feature library provided in an embodiment of the present disclosure;

[0123] Figure 7 A logical schematic diagram of constructing a facial feature library provided in an embodiment of the present disclosure;

[0124] Figure 8 A schematic diagram of a process for constructing a facial feature library provided in an embodiment of the present disclosure;

[0125] Figure 9 A schematic diagram of a facial feature set screening process provided by an embodiment of the present disclosure;

[0126] Figure 10 A schematic diagram of another apparatus for constructing a facial feature library provided by an embodiment of the present disclosure;

[0127] Figure 11 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0128] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the present disclosure will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present disclosure, rather than all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present disclosure without creative effort are intended to fall within the scope of protection of the present disclosure.

[0129] In order to help those skilled in the art better understand the technical solution of this application, some terms involved in this application are introduced below.

[0130] Face detection: The face area in the video image is extracted through an algorithm to obtain an image containing only the face, usually a square rectangle.

[0131] Face recognition: For a face image, its identity label is determined by comparing it with the portraits in the library.

[0132] Facial feature extraction: Analyze facial images and extract their features. The feature is usually a vector of fixed dimension.

[0133] Face feature database: A database obtained by extracting features from a batch of face images with identity labels and storing them.

[0134] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0135] In addition, the terms "comprises" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0136] The following introduces the design concept of the embodiment of the present disclosure.

[0137] As mentioned earlier, entity labels for people in videos play a crucial role. For videos without clear entity labels for people, facial recognition can be performed on the people in the video images to obtain specific entity labels. Face recognition often presents challenges such as varying makeup, environments, and angles. To better identify entity labels for people in video images, a more comprehensive facial feature library is necessary. However, building a facial feature library is time-consuming and labor-intensive. Improving the efficiency and richness of this library remains a challenge.

[0138] In view of this, the embodiments of the present disclosure provide a method for constructing a facial feature library, which utilizes the current mature facial recognition technology to automatically complete the construction of a large-scale facial feature library, thereby improving the construction efficiency and the richness of the facial feature library, thereby improving the recognition accuracy of the entity labels of the characters in the video.

[0139] In the technical solution disclosed herein, the collection, dissemination, and use of data comply with relevant national laws and regulations. For example, the data includes images of people.

[0140] The following describes the method for constructing a facial feature library in accordance with an embodiment of the present application in conjunction with the accompanying drawings and specific implementation methods.

[0141] The facial feature library construction method of the disclosed embodiments can be executed independently by a terminal device or a server, or by a server and a terminal device in combination. Terminal devices include, but are not limited to, electronic devices such as desktop computers, mobile phones, computers, smart home appliances, intelligent voice interaction devices, and in-vehicle terminals. The server can be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0142] The following is an introduction to the method for constructing a facial feature library in an embodiment of the present disclosure.

[0143] See Figure 1 As shown, the embodiment of the present disclosure provides a method for constructing a facial feature library, comprising the following steps S101-S104:

[0144] Step S101: for each first entity tag in the first entity tag set, obtain multiple first person images corresponding to the first entity tag, and preprocess the multiple first person images respectively to obtain multiple facial features; wherein the preprocessing includes at least face detection and facial feature extraction.

[0145] The first entity tag set may include a plurality of pre-defined first entity tags for a plurality of persons. The first entity tags may be names or titles of the persons, and each person may correspond to one or more first entity tags. The first person images corresponding to each first entity tag may be obtained in a predetermined manner, such as from the Internet or through other legal means.

[0146] Optionally, when obtaining multiple first person images corresponding to each first entity tag from the network, the following steps A1-A2 may be performed:

[0147] A1. For any first web page, identify the page content of the first web page. If a person image and person entity information matching any first entity tag are identified, use the identified person image as the first person image corresponding to the any first entity tag.

[0148] In an embodiment of the present disclosure, the first web page can be a web page in any web platform with access rights. If a person image is recognized in a first web page, and at the same time, person entity information matching any first entity tag is extracted based on natural language processing analysis, the recognized person image can be used as the first person image corresponding to the first entity tag.

[0149] It should be noted that when a person image is recognized on a first web page and two or more first entity tags are recognized at the same time, the recognized person image can be used as the first person image corresponding to these first entity tags at the same time, because the first person images corresponding to each first entity tag will be screened subsequently.

[0150] A2. Identify multiple first person images corresponding to each first entity tag from multiple first web pages.

[0151] Based on the above step A1, the page contents in a large number of first web pages can be identified respectively, and corresponding first entity tags and corresponding first character images can be obtained from multiple first web pages respectively. Finally, multiple first character images corresponding to each first entity tag can be obtained.

[0152] After obtaining multiple first person images corresponding to each first entity label, face detection and facial feature extraction are performed on each of the multiple first person images for each first entity label to obtain multiple facial features. By performing face detection on each of the multiple first person images, images without faces are eliminated, and facial images and facial key points of each first person image containing a face are obtained. Each facial image is corrected based on its facial key points; for example, based on a facial key point template, the facial key points in the facial image are mapped to the template. Furthermore, a facial feature extraction algorithm is used to extract features from each corrected facial image to obtain facial features.

[0153] Step S102: For each first entity tag, a preset screening process is performed on multiple facial features corresponding to the first entity tag to obtain a first facial feature set.

[0154] Considering the facial features corresponding to each first entity tag obtained in step S102, there may be facial features that do not match the first entity tag or repeated facial features, and these facial features need to be deleted.

[0155] In the disclosed embodiments, for each first entity tag, multiple facial features corresponding to the first entity tag can be screened based on a preset screening method, with facial features irrelevant to the first entity tag removed to ensure consistency and diversity of the facial features. The screened facial features are then combined into a first facial feature set. The screening process will be further described in the following embodiments.

[0156] Step S103 , performing a similarity comparison on every two first facial feature sets in the plurality of first facial feature sets. In each similarity comparison, if the similarity comparison result of the two first facial feature sets meets a preset condition, any two first facial feature sets are merged.

[0157] In this step, considering that there may be more than one first entity label corresponding to a person, if the above first entity label set contains multiple first entity labels of the same person, the first face feature sets of these first entity labels need to be merged.

[0158] Specifically, based on a preset method, the similarity between every two first face feature sets is compared. If the similarity comparison result of a certain two first face feature sets meets the preset condition, these two first face feature sets are merged to ensure that each person finally obtained corresponds to one first face feature set. Among them, the similarity comparison method of every two first face feature sets and the above preset conditions can be set as needed, and will be introduced exemplarily in the following embodiments of the present disclosure.

[0159] Step S104, construct a face feature library based on the newly obtained multiple first face feature sets and the first entity label set.

[0160] Through the above solution of the embodiments of the present disclosure, a more complete face feature library can be efficiently constructed. When identifying the entity labels of characters in a subsequent video, more character images can be provided for the entity labels of each person, improving the recognition accuracy.

[0161] The process of performing preset screening on multiple face features corresponding to the first entity label in step S102 above will be introduced below.

[0162] In some embodiments, as Figure 2 shown, the above step S102 performs preset screening on multiple face features corresponding to the first entity label to obtain the first face feature set, which may include the following steps S1021 - S1023:

[0163] Step S1021, perform clustering on multiple face features corresponding to the first entity label to obtain multiple first face feature clusters, and each first face feature cluster includes a clustering center.

[0164] Among them, a clustering algorithm can be used to perform clustering on multiple face features. The clustering algorithm includes but is not limited to meanshift, K-Means, etc. Specifically, taking the meanshift algorithm as an example, a smaller radius r, such as 0.8, is used to cluster N face features {F1, F2, F3, F4... F N}, to obtain K clustering centers, where K < N, and the value of K is calculated by the meanshift algorithm. The face features that are not clustering centers are classified into a clustering center F k to form a set of face feature clusters C k(i.e. the first facial feature cluster), and delete the facial features that do not belong to any facial feature cluster.

[0165] Meanshift is an iterative clustering algorithm that calculates the mean shift vector of a sample point (i.e., a facial feature) within a radius r, continuously moves the cluster center to the place with the highest density, and finally finds the local maximum point of probability density. These local maximum points are the cluster centers F k , and at each cluster center F k During the movement, the facial features passed by constitute the cluster center F k The facial feature cluster C k .

[0166] Step S1022: For each cluster center, if the similarity between the cluster center and each other cluster center does not meet the first similarity condition, the first facial feature cluster where the cluster center is located is deleted.

[0167] For example, the first similarity condition may be reaching a first threshold, such as 0.5, which is not limited thereto.

[0168] In this step, the similarity between each cluster center and each of the other cluster centers is calculated. If the similarity between a cluster center and any of the other cluster centers does not exceed (is less than or equal to) a first threshold, it can be considered that the first facial feature cluster of this cluster center is too different from the first facial feature clusters of the other cluster centers and needs to be deleted. The similarity between each of the above two cluster centers can be calculated using a similarity algorithm. The following example uses the similarity algorithm to calculate cosine similarity.

[0169] Assuming there are K cluster centers, calculate the cosine similarity of these cluster centers pairwise to form an intra-class similarity matrix M with dimension K*K intra , where the eigenvector F of cluster center a is a and the eigenvector F of cluster center b b The cosine similarity can be calculated by the following formula (1):

[0170]

[0171] Traverse the intra-class similarity matrix M row by row intra If the cosine similarity between a cluster center and other cluster centers does not exceed the first threshold T1, the first facial feature cluster where the cluster center is located is deleted.

[0172] Through the above implementation, the multiple facial features of each first entity tag can be cleaned internally, and first facial feature clusters that do not meet the conditions can be deleted, so that the screened first facial feature clusters match the corresponding first entity tag.

[0173] Step S1023: Obtain a first facial feature set based on the remaining multiple first facial feature clusters.

[0174] Through the above step S1022, after deleting the first facial feature clusters that are significantly different from other first facial feature clusters, the remaining multiple first facial feature clusters can be combined into a first facial feature set; and further screening can be performed on the remaining multiple first facial feature clusters to further ensure that the screened first facial feature clusters match the corresponding first entity labels.

[0175] Optionally, when the remaining plurality of first facial feature clusters are further screened, the step S1023 of obtaining the first facial feature set based on the remaining plurality of first facial feature clusters may include the following steps A1-A2:

[0176] A1. For each remaining first facial feature cluster, determine the similarity between each facial feature in the first facial feature cluster and the corresponding cluster center, and remove facial features whose similarity does not meet the second similarity condition from the first facial feature cluster.

[0177] For example, the second similarity condition may be reaching a second threshold, such as 0.6, which is not limited thereto.

[0178] A2. Obtain a first facial feature set based on the newly obtained multiple first facial feature clusters.

[0179] In the above step 1022, after deleting the first facial feature clusters containing cluster centers that do not meet the conditions among the multiple cluster centers, for each remaining first facial feature cluster, the facial features in the first facial feature cluster whose similarity with the cluster center does not reach the second threshold can also be deleted to eliminate the facial features in each first facial feature cluster that have a large difference with its cluster center.

[0180] For example: for a first face feature cluster C k , calculate each facial feature and cluster center F k similarity, and delete the facial features whose similarity does not reach the second threshold T2 to eliminate noise data.

[0181] In order to make each first face feature cluster C k The number of facial features in the cluster reaches a certain value to ensure that the first facial feature cluster C kThe distances between the facial features in the cluster are within a reasonable range. The minimum number of facial features in the first facial feature cluster can be set, for example, to 2. If the first facial feature cluster C k The number of facial features in is less than the minimum number of facial features, then the first facial feature cluster C can be deleted. k .

[0182] Optionally, when obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters in the above step A2, for each newly obtained first facial feature cluster, if the number of facial features in the first facial feature cluster does not meet the quantity condition, the first facial feature cluster is deleted.

[0183] In some embodiments, the above step A2 obtains the first facial feature set based on the newly obtained multiple first facial feature clusters, and may further include the following steps:

[0184] A21. For each newly obtained first facial feature cluster, determine the similarity between each facial feature in the first facial feature cluster and multiple other facial features, and remove other facial features whose similarities meet the third similarity condition from the first facial feature cluster.

[0185] For example, the third similarity condition may be reaching a third threshold, such as 0.95, which is not limited thereto.

[0186] A22. Obtain a first facial feature set based on the multiple first facial feature clusters newly obtained again.

[0187] After deleting the facial features with a large gap between each first facial feature cluster and the cluster center in the above step A1, considering that there may be very similar facial features among multiple facial features in each first facial feature cluster, in order to avoid duplication, only one of the multiple very similar facial features can be retained.

[0188] For example: for a first face feature cluster C k The similarity of multiple facial features in the cluster is calculated pairwise to obtain the intra-cluster similarity matrix Mcluster. Mcluster is traversed row by row. If there is a facial feature whose similarity with a certain facial feature reaches the third threshold T3, the facial feature is deleted to remove very similar facial features and avoid duplication.

[0189] Based on the screening process of the above-mentioned embodiment of the present application, the final first facial feature set corresponding to each first entity tag can be obtained. In order to ensure that each first facial feature set has a certain amount of data, a minimum number of facial features in the first facial feature set can be set, for example, to 5. If the number of facial features in a first facial feature set is less than the minimum number of facial features, it is considered to be unreliable and the first facial feature set can be deleted, thereby ensuring that the facial features in each first facial feature set are both consistent and diverse.

[0190] The following embodiment introduces the specific process of merging the two first facial feature sets in the above step S103.

[0191] After performing internal data cleaning on the multiple facial features corresponding to each first entity tag through the above embodiment, data cleaning can also be performed between the first entity tags. Specifically, the first facial feature set of one first entity tag is analyzed and compared with the first facial feature sets of other first entity tags, and the first facial feature sets belonging to the same person are merged to further improve the accuracy of the data in the constructed facial feature library.

[0192] In some embodiments, as Figure 3 As shown, in the above step S103, each two first facial feature sets in the plurality of first facial feature sets are compared for similarity. In each similarity comparison, if the similarity comparison result of the two first facial feature sets meets a preset condition, the two first facial feature sets are merged, which may include the following steps S1031-S1035:

[0193] Step S1031: For every two first facial feature sets, execute the following steps S1032-S1035.

[0194] Wherein, multiple first facial feature sets obtained after screening through the above steps S1021-S1023 are obtained, and the following steps are performed for every two first facial feature sets.

[0195] Step S1032: Acquire multiple cluster centers of a first facial feature set, and acquire multiple cluster centers of another first facial feature set.

[0196] In this step, the multiple cluster centers of each first facial feature set are obtained after screening the multiple cluster centers in the first facial feature set in the above step S1022.

[0197] Step S1033 : Determine the similarity between each cluster center of a first facial feature set and multiple cluster centers of another first facial feature set to obtain a similarity vector.

[0198] Step S1034 : constructing an inter-class similarity matrix based on the obtained multiple similarity vectors, determining target similarities in the inter-class similarity matrix that reach a first similarity threshold, and taking the number of target similarities as a similarity comparison result.

[0199] For example, assuming that the first face feature set A of a first entity tag contains K1 cluster centers, and the first face feature set B of another first entity tag contains K2 cluster centers, for each cluster center in the first face feature set A, the similarity with the K2 cluster centers in the first face feature set B is calculated one by one to obtain a similarity vector with a dimension of K2. The obtained K1 similarity vectors with a dimension of K2 are combined into an inter-class similarity matrix but The dimension is K1*K2; traverse statistics The number of similarities that reach the similarity threshold is used as the inter-class similarity matrix The number of target similarities that reaches the similarity threshold, for example, the similarity threshold is 0.6, which is not limited.

[0200] Step S1035: If the similarity comparison result meets the preset condition, one first facial feature set is merged with another first facial feature set.

[0201] In this step, the preset condition may be: the proportion of target similarities that reach a similarity threshold in the inter-class similarity matrix reaches a preset ratio. For example, if the target similarity is greater than 80%, it is considered that the two first entity labels correspond to the same person, and the corresponding two first facial feature sets are merged.

[0202] If the target similarity does not meet the preset conditions, it can be considered that there is no overlap in the facial features between the two first entity tags, and the first facial feature set of one of the first entity tags is compared with the first facial feature set of the next first entity tag.

[0203] In some embodiments, considering that a first person image may contain multiple persons, resulting in different first entity labels corresponding to the same facial features, the same facial features need to be deleted. For the two first facial feature sets determined to be merged through steps S1031-S1035 above, internal data cleaning can be continued to ensure that the facial features in the merged first facial feature set are not repeated.

[0204] Optionally, before merging one first facial feature set with another first facial feature set in step S1035, the following steps B1-B4 may be performed:

[0205] B1. constructing a first intra-class similarity matrix based on the similarity between each cluster center of a first facial feature set and each other cluster center of the first facial feature set;

[0206] The first intra-class similarity matrix is ​​similar to the intra-class similarity matrix M in the above embodiment. intra The construction method is similar. Assuming that the above-mentioned first face feature set includes K cluster centers, the cosine similarity of these cluster centers is calculated pairwise to form a first-class intra-class similarity matrix with a dimension of K*K.

[0207] B2. Construct a second intra-class similarity matrix based on the similarity between each cluster center of another first facial feature set and each other cluster center of another first facial feature set.

[0208] The calculation method of the second intra-category similarity matrix is ​​similar to that of the first intra-category similarity matrix, and will not be repeated here.

[0209] B3. If, based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix, it is determined that the target cluster center in one first facial feature set and the cluster centers of another first facial feature set satisfy a first preset relationship, then the first facial feature cluster in which the target cluster center is located is removed;

[0210] The inter-class similarity matrix is ​​obtained in step S1034 above, that is, for each cluster center in the first face feature set A, the similarity with the K2 cluster centers in the first face feature set B is calculated one by one to obtain a similarity vector with a dimension of K2. The obtained K1 similarity vectors with a dimension of K2 are combined to form an inter-class similarity matrix.

[0211] For example: The first intra-class similarity matrix of the first face feature set A is The second-class similarity matrix of the first face feature set B is The inter-class similarity matrix is Calculate separately The mean of each row and the mean of each column in , we get and Recalculate The mean of each row in (excluding the elements of the column with the same sequence number as the current row) is obtained calculate The mean of each row in (excluding the elements of the column with the same sequence number as the current row) is obtained

[0212] In turn The i-th element in If it is greater than , it means that the i-th cluster center of the first face feature set A is more similar to the first face feature set B, and the first face feature cluster where the i-th cluster center in the first face feature set A is located is deleted.

[0213] B4. If, based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix, it is determined that the target cluster center in another first facial feature set and the cluster centers of a first facial feature set satisfy the second preset relationship, then the first facial feature cluster where the target cluster center is located is eliminated.

[0214] Similar to the above step B3, The jth element in If it is greater than , it means that the j-th cluster center in the first face feature set B is more similar to the first face feature set A, and the first face feature cluster where the j-th cluster center in the first face feature set B is located is deleted.

[0215] Based on the above embodiments of the present disclosure, a diverse and reliable facial feature library is constructed. In the following embodiments, new character images and corresponding new entity tags can be acquired in real time, and the constructed facial feature library can be updated regularly to enrich the facial feature library.

[0216] In some embodiments, as Figure 4 As shown, based on steps S101-S104 in the above embodiment, the following steps S105-S106 may also be performed:

[0217] Step S105: acquiring a second entity tag set and a second person image set, wherein each second entity tag in the second entity tag set corresponds to at least one second person image in the second person image set;

[0218] Optionally, obtaining the second entity tag set and the second person image set in step S105 may include the following steps C1-C2:

[0219] C1. For any second web page, identify the page content of the second web page. If a person image and person entity information are identified, obtain a second entity tag based on the person entity information, and use the identified person image as the second person image corresponding to the second entity tag.

[0220] In the disclosed embodiment, the second web page can be a web page on any web platform with access rights. In the above embodiment, the first web page can be a historical web page, and the second web page here can be a new web page. If a person image is recognized in a second web page and person entity information is extracted based on natural language processing analysis, a second entity label and its corresponding second person image are obtained based on the person entity information and the person image.

[0221] C2. Obtain a second entity tag set based on each second entity tag obtained from a plurality of second web pages, and obtain a second character image set based on at least one second character image corresponding to each second entity tag.

[0222] Step S106: For each second person image, if a target facial feature that matches the facial feature in the second person image is retrieved from the facial feature library, the facial feature in the second person image is saved under the target entity label corresponding to the target facial feature in the facial feature library.

[0223] Face detection is performed on each second person image. If a face is detected, facial feature extraction is performed on the facial image to obtain facial features. A facial feature library is then searched to determine whether a target facial feature that matches the facial features is present. For example, if the similarity between two facial features reaches a third similarity threshold, the two facial features are considered to match. The second similarity threshold can be set as desired, for example, to 0.6, and is not limited thereto.

[0224] Optionally, to further ensure that the facial features in the second person image match the target entity label, when a target facial feature matching the facial features in the second person image is retrieved from the facial feature library, it may be further determined whether the target facial feature satisfies at least one of the following two conditions:

[0225] In the first condition, the target entity label corresponding to the target facial feature is consistent with the second entity label corresponding to the second person image;

[0226] The second condition is that the similarity between the target facial features and the facial features in the second person image is less than a second similarity threshold; wherein, the second similarity threshold can be set higher, for example, 0.9 or 0.95, etc., and there is no limitation on this. By limiting this condition, it is possible to avoid repeatedly storing very similar facial features.

[0227] If the target facial feature meets at least one of the above two conditions, the facial feature in the second person image is saved under the target entity label in the facial feature library.

[0228] Step S107 : For each second person image, if no target facial features matching the facial features in the second person image are retrieved from the facial feature library, the second person image is used as a candidate person image to be stored in the library.

[0229] Step S108: obtaining a new second entity label set based on the second labels corresponding to the plurality of candidate person images, and preprocessing the plurality of candidate person images to obtain a second facial feature set; wherein the preprocessing includes at least face detection and facial feature extraction.

[0230] In this step, the process of pre-processing the multiple candidate character images is similar to the implementation process of step S101 in the above embodiment of the present disclosure, and will not be repeated here.

[0231] Step S109: Save the second facial feature set and the new second entity tag set to a temporary facial feature library, where each second entity tag in the new second entity tag set corresponds to at least one facial feature in the second facial feature set.

[0232] Optional, such as Figure 5 As shown, in step S109, saving the second facial feature set and the new second entity label set to the facial feature temporary library may include the following steps S1091-S1093:

[0233] Step S1091 , clustering the multiple facial features in the second facial feature set to obtain multiple second facial feature clusters, each of which includes a cluster center.

[0234] Specifically, multiple facial features may be clustered based on a clustering algorithm. For example, the clustering algorithm is kmeans, and the number of cluster centers is set to be the same as the number of second entity labels, that is, each second entity label corresponds to a second facial feature cluster.

[0235] Step S1092: For each second facial feature cluster, if the similarity between the candidate facial feature in the second facial feature cluster and the corresponding cluster center does not meet the fourth similarity condition, the candidate facial feature is removed from the second facial feature cluster.

[0236] The fourth similarity condition may be reaching a fourth threshold, such as 0.6, which is not limited thereto.

[0237] Step S1093: Obtain a new second facial feature set based on the newly obtained multiple second facial feature clusters, and save the new second facial feature set and the new second entity label set to the facial feature temporary library accordingly.

[0238] In some embodiments, the new second facial feature set and the new second entity label set in the temporary facial feature library can be periodically saved to the constructed facial feature library. Specifically, the following steps D1-D2 can be performed:

[0239] D1. For each second entity tag in the new second entity tag set, perform a preset screening process on multiple facial features corresponding to the second entity tag to obtain a third facial feature set.

[0240] This step is similar to the implementation process of step S102 in the above embodiment of the present disclosure, that is, clustering multiple facial features to obtain multiple third facial feature clusters, and then continuing to perform screening processing, which will not be repeated here.

[0241] D2. Obtain multiple third facial feature sets and second entity labels corresponding to each of the multiple third facial feature sets and save them to a facial feature library.

[0242] In the disclosed embodiment, steps D1 and D2 may be performed on the new second facial feature set and the new second entity tag set in the temporary facial feature library at regular intervals, such as a few days or a week. To ensure accuracy, when performing step D1, a minimum number of facial features in the third facial feature cluster may be set, for example, to 5, with no particular limit. When performing step D2, a minimum number of facial features in the third facial feature set corresponding to each second entity tag may be set, for example, to 10, with no particular limit.

[0243] The following is an illustrative introduction to the method for constructing and updating the facial feature library according to the embodiment of the present disclosure with reference to specific examples.

[0244] like Figure 6 As shown, in the construction phase, a large amount of Internet historical data (page content in historical network pages) is used to build a system based on the face feature library to build an initial face feature library. Subsequently, at fixed intervals, for new Internet data (page content in new network pages), the system is updated based on the face feature library. If it is determined that the facial features in the newly added character image belong to the characters already in the face feature library (with the above-mentioned first entity label), the facial features in the character image are added to the face feature library, or, when the facial features in the character image are repeated with the facial features already in the face feature library, the facial features in the character image are discarded; if it is determined that the facial features in the newly added character image do not belong to the characters already in the face feature library, the character image is first stored in the face feature temporary library, and the character images in the face feature temporary library are regularly summarized. When the number of character images reaches the set number, the face feature temporary library can be merged into the face feature library to complete the update of the face feature library.

[0245] The following is an exemplary introduction to the construction process of the facial feature library construction system.

[0246] The facial feature library construction system generates a person entity tag (corresponding to the first entity tag in the above embodiment) based on a pre-acquired list of people. Specifically, a large number of person images are obtained from historical Internet data to form a person image set corresponding to each person entity tag. The large number of obtained person images often have the following problems:

[0247] 1. There is no face in the character image and it needs to be deleted;

[0248] 2. Multiple identical character images need to be deduplicated;

[0249] 3. The person in the person image does not match the person entity label and needs to be deleted;

[0250] 4. A person may have more than one common name, which results in multiple character image sets corresponding to one person, which need to be merged;

[0251] 5. There are multiple characters in a character image at the same time, resulting in mixing between the character image sets corresponding to different characters, which need to be deleted and merged.

[0252] To solve the above problems, the embodiment of the present disclosure cleans and filters the character image set corresponding to multiple character entity labels through the following process to complete the construction of a reliable face feature library, such as Figure 7 As shown, the specific process includes the following:

[0253] 1) Face detection: Perform face detection on the person image set corresponding to each person entity label, remove images without faces, obtain a face image set, and correct the face image based on the facial key points in each face image;

[0254] 2) Facial feature extraction: For each person entity label, a facial feature extraction algorithm is used to extract features from each corrected facial image to obtain facial features.

[0255] 3) Internal cleaning of person entity labels: Cluster the multiple facial features of each person entity label to obtain multiple cluster centers, and delete facial features that are irrelevant to the person entity label to ensure the consistency and diversity of the facial features of each person entity label;

[0256] 4) Cleaning between person entity labels: Calculate the similarity between the facial feature set of each person entity label and the facial feature sets of other person entity labels, merge different facial feature sets that should be merged, and remove mixed facial features with a certain degree of similarity to obtain the final facial feature library.

[0257] The above-mentioned face detection and face feature extraction processes will be introduced exemplarily below.

[0258] For each set of person images corresponding to a person entity label, use a face detector and a feature extractor on the set of person images to extract a set of N face features, denoted as {F1, F2, F3, F4... F N}. Each F n is a one-dimensional feature vector (f1, f2, f3,... f M ), and the vector length M is determined by the feature extractor, for example, it is 256 or 512.

[0259] The internal data cleaning process of the person entity label will be introduced exemplarily below.

[0260] Based on the above process, for each person entity label, N face features are initially obtained. Most of these face features belong to the same person, but due to the clutter of network data, there may be some noisy data. First, analyze and screen these N face features to eliminate the noisy data. As Figure 8 shown, the main screening process is as follows:

[0261] a1. Adopt a clustering algorithm, such as the meanshift algorithm, and use a smaller radius r, such as 0.8, to cluster the N face features {F1, F2, F3, F4... F N} of a person entity label to obtain K clustering centers, where K < N and is calculated by the meanshift algorithm. Classify the other face features into a clustering center F k , form a set of face feature clusters C k , and delete the face features that do not belong to any face feature cluster.

[0262] a2. Calculate the similarity between each pair of these K clustering centers, such as cosine similarity, to form an intra-class similarity matrix M intra with a dimension of K * K, where the similarity calculation formula for the feature vector F a and the feature vector F b is as shown in Equation (1) in the above embodiment.

[0263] Traverse the matrix M intra row by row. If the similarity between a certain clustering center and other clustering centers is less than the first threshold T1 = 0.5, it is considered that the face features in the face feature cluster where this clustering center is located have too large a gap from the other face features under this person entity label, and delete the face feature cluster where this clustering center is located;

[0264] a3. For a face feature cluster C k, calculate each facial feature to the cluster center F k similarity, deleting facial features with a similarity less than a second threshold value T2=0.6 to eliminate noise data. If the number of facial features in the facial feature cluster is less than the minimum number of facial features in the cluster, for example, 2, then deleting the facial feature cluster to ensure that the distance between facial features in the facial feature cluster is within a reasonable range;

[0265] a4. Face feature cluster C k The remaining facial features in the cluster are similarity calculated pairwise to obtain the intra-cluster similarity matrix M cluster , traverse M row by row cluster If there is a facial feature whose similarity to the current facial feature is greater than the third threshold T3=0.95, the facial feature is deleted to remove very similar facial features and avoid duplicate storage;

[0266] Through the above screening, the number of facial features corresponding to each person entity label finally obtained is determined. If it is less than the minimum number of facial features in the class, for example 5, it is considered unreliable and the facial feature set corresponding to the person entity label is deleted.

[0267] Through the above steps, the facial feature set corresponding to each retained person entity label has a certain amount of data and has a certain consistency and diversity.

[0268] The following is an example introduction to the data cleaning process between person entity labels.

[0269] Data cleaning between person entity labels refers to analyzing and comparing the facial feature set of a person entity label with the facial feature sets of other person entity labels, merging the facial feature sets belonging to the same person, and eliminating overlap between facial features of different people, further improving the accuracy of the data in the facial feature library. The specific process includes:

[0270] b1. Get multiple cluster centers of the facial feature set A of a person entity label, set the number of cluster centers to K1, and calculate the cosine similarity of each cluster center of the facial feature set A with multiple cluster centers of the facial feature set B of other person entity labels one by one to obtain the inter-class similarity matrix Assume that the number of cluster centers of face feature set B is K2, then The dimension is K1*K2;

[0271] b2. Traverse the inter-class similarity matrix statistics The number of elements whose values ​​are greater than the second threshold value T2=0.6, if the values ​​of all elements are less than 0.6, it is considered that the facial features of the two facial feature sets have no overlap, and the facial feature set corresponding to the next person entity label is compared. If If the number of elements with a value greater than 0.6 is greater than or equal to 80%, the two facial feature sets are considered to belong to the same person, and the two facial feature sets are merged. The merged facial feature set is then cleaned again for the person entity label internal data.

[0272] b3. Inter-class similarity matrix Find the mean of each row and each column respectively, and we get and Then the intra-class similarity matrix of face feature set A is and the intra-class similarity matrix of face feature set B Find the mean of each row separately (excluding the elements of the column with the same sequence number as the current row), and get and In turn The i-th element in If the former is greater than the latter, it means that the i-th cluster center of character A is more similar to that of character B, and the i-th cluster center of character A and its corresponding feature cluster are directly deleted. The jth element in If the former is greater than the latter, it means that the j-th cluster center in the face feature set B is more similar to the face feature set A. Then directly delete the j-th cluster center in the face feature set B and its corresponding face feature cluster.

[0273] Based on the above steps, a diverse and reliable facial feature library is constructed.

[0274] The following is an exemplary introduction to the updating process of the facial feature library updating system.

[0275] The online platform generates a massive amount of new content every day, generating new character images. Updating the facial feature library based on these new character images can further improve the timeliness and accuracy of face recognition. The facial feature library update system obtains new character images and corresponding new character entity labels from the online platform and regularly updates the facial feature library. The specific process includes:

[0276] c1. Regularly obtain web pages from the network platform with access rights, and extract person entity label information from the page content of a web page based on natural language processing analysis. If the person entity label information is extracted and the person image is recognized at the same time, a new person entity label set labelA = {label_1, label_2...label_n} and a new person image set imgA = {img_1, img_2...img_m} are obtained.

[0277] c2. Send the person images in imgA into the face feature library for retrieval in sequence, set the recognition threshold to 0.6, if the label lablei is retrieved and lablei belongs to labelA, delete labeli in labelA, if the recognition confidence is less than the third threshold T3 = 0.95, save the facial features in the current person image to the face feature library; if lablei does not belong to labelA, delete the current person image directly.

[0278] c3. After step c2 above, the remaining person entity label set (img_1, img_2...img_m) is recorded as labelB, and the remaining person image set is recorded as imgB. After extracting facial features from the person images in imgB, clustering is performed using a clustering algorithm, such as kmeans. The number of cluster centers is consistent with the number of person entity labels in labelB. The distance from each facial feature to its nearest cluster center is calculated. If it is greater than the second threshold T2=0.6, the current facial feature is deleted. The remaining facial features are saved in a temporary facial feature library. At the same time, each facial feature corresponds to a person entity label in labelB.

[0279] c4. Every fixed time (such as a few days or a week), the facial feature set corresponding to each person entity label in the temporary facial feature library is cleaned using the internal data cleaning process of the person entity label. To ensure accuracy, the minimum number of facial features in a facial feature cluster is set, for example, to 5, and the minimum number of facial features in the facial feature set of each person entity label is set, for example, to 10. After the facial feature set corresponding to each person entity label is cleaned by the internal data of the person entity label, if the number of remaining facial features reaches 10, the person entity label and its corresponding facial feature set are saved to the facial feature library. At the same time, the person entity label and its corresponding facial feature set in the temporary facial feature library are deleted.

[0280] Through the above-mentioned updating process, the internal diversity of character entities is updated and the timeliness of new characters entering the database is achieved, which further improves the accuracy of the character facial feature database.

[0281] The disclosed embodiments propose a method for rapidly and automatically constructing and updating a large-scale facial feature library, enabling the construction and updating of this library with minimal or no human intervention. This method utilizes a dual-dimensional filtering scheme for both intra- and inter-entity label cleaning, along with a clustering algorithm and multi-dimensional similarity matrix analysis, to effectively filter data and construct a highly accurate and diverse facial feature library.

[0282] Based on the same inventive concept, the embodiment of the present disclosure also provides a facial feature library construction device. The principle of solving the problem by this device is similar to the method of the above embodiment. Therefore, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0283] like Figure 9 As shown, the embodiment of the present disclosure further provides a device for constructing a facial feature library, comprising:

[0284] A first acquisition module 91 is configured to, for each first entity tag in the first entity tag set, acquire multiple first person images corresponding to the first entity tag, and preprocess the multiple first person images to obtain multiple facial features; wherein the preprocessing includes at least face detection and facial feature extraction;

[0285] A screening module 92 is configured to perform a preset screening process on the multiple facial features corresponding to each first entity tag to obtain a first facial feature set;

[0286] a comparison module 93 for performing a similarity comparison between each two first facial feature sets in the plurality of first facial feature sets, and merging any two first facial feature sets if the similarity comparison result of the two first facial feature sets meets a preset condition in each similarity comparison;

[0287] The construction module 94 is used to construct a facial feature library based on the newly obtained multiple first facial feature sets and the first entity label set.

[0288] Optionally, the screening module 92 is further configured to:

[0289] performing clustering processing on the multiple facial features corresponding to the first entity label to obtain multiple first facial feature clusters, each first facial feature cluster including a cluster center;

[0290] For each cluster center, if the similarity between the cluster center and each other cluster center does not meet the first similarity condition, then the first facial feature cluster where the cluster center is located is deleted;

[0291] A first facial feature set is obtained based on the remaining multiple first facial feature clusters.

[0292] Optionally, when obtaining the first facial feature set based on the remaining multiple first facial feature clusters, the screening module 92 is further configured to:

[0293] For each remaining first facial feature cluster, determine the similarity between each facial feature in the first facial feature cluster and the corresponding cluster center, and remove facial features whose similarity does not meet the second similarity condition from the first facial feature cluster;

[0294] A first facial feature set is obtained based on the newly obtained multiple first facial feature clusters.

[0295] Optionally, when obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters, the screening module 92 is further configured to:

[0296] For each newly obtained first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and a plurality of other facial features, and removing the other facial features whose similarities meet a third similarity condition from the first facial feature cluster;

[0297] Based on the multiple newly obtained first facial feature clusters, a first facial feature set is obtained.

[0298] Optionally, when obtaining the first facial feature set based on the multiple newly obtained first facial feature clusters, the screening module 92 is further configured to:

[0299] For each newly obtained first facial feature cluster, if the number of facial features in the first facial feature cluster does not meet the quantity condition, the first facial feature cluster is deleted.

[0300] Optionally, the comparison module 93 is further configured to:

[0301] For each of the two first face feature sets, perform the following operations:

[0302] Obtaining multiple cluster centers of a first facial feature set, and obtaining multiple cluster centers of another first facial feature set;

[0303] Determine the similarity between each cluster center of a first facial feature set and multiple cluster centers of another first facial feature set to obtain a similarity vector;

[0304] constructing an inter-class similarity matrix based on the obtained multiple similarity vectors, determining target similarities in the inter-class similarity matrix that reach a first similarity threshold, and using the number of target similarities as a similarity comparison result;

[0305] If the similarity comparison result meets the preset conditions, one first facial feature set is merged with another first facial feature set.

[0306] Optional, such as Figure 10 As shown, it also includes a rejection module 95 for:

[0307] Constructing a first intra-class similarity matrix based on the similarity between each cluster center of a first facial feature set and each other cluster center of the first facial feature set;

[0308] Constructing a second intra-class similarity matrix based on the similarity between each cluster center of another first facial feature set and each other cluster center of another first facial feature set;

[0309] If it is determined based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix that a target cluster center in one first facial feature set satisfies a first preset relationship with each cluster center in another first facial feature set, then the first facial feature cluster where the target cluster center is located is removed;

[0310] If, based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix, it is determined that the target cluster center in another first facial feature set and the cluster centers of a first facial feature set satisfy the second preset relationship, then the first facial feature cluster where the target cluster center is located is eliminated.

[0311] Optionally, the device further comprises:

[0312] A second acquisition module 96 is configured to acquire a second entity tag set and a second character image set, wherein each second entity tag in the second entity tag set corresponds to at least one second character image in the second character image set;

[0313] The first update module 97 is used to save the facial features in the second person image to the target entity label corresponding to the target facial features in the facial feature library if a target facial feature matching the facial features in the second person image is retrieved from the facial feature library for each second person image.

[0314] Optionally, the first updating module 97 is further configured to:

[0315] If a target facial feature that matches the facial feature in the second person image is retrieved from the facial feature library, determining whether the target facial feature satisfies at least one of the following conditions: a target entity label corresponding to the target facial feature is consistent with a second entity label corresponding to the second person image, and a similarity between the target facial feature and the facial feature in the second person image is less than a second similarity threshold;

[0316] If satisfied, the facial features in the second person image are saved under the target entity label in the facial feature library.

[0317] Optionally, the device further includes a storage module 98, configured to:

[0318] For each second person image, if no target facial features matching the facial features in the second person image are retrieved from the facial feature library, the second person image is used as a candidate person image to be stored in the library;

[0319] Obtaining a new second entity label set based on the second labels corresponding to the plurality of candidate person images, and preprocessing the plurality of candidate person images to obtain a second facial feature set; wherein the preprocessing includes at least face detection and facial feature extraction;

[0320] The second facial feature set and the new second entity tag set are saved to a temporary facial feature library, where each second entity tag in the new second entity tag set corresponds to at least one facial feature in the second facial feature set.

[0321] Optionally, when saving the second facial feature set and the new second entity label set to the facial feature temporary library, the saving module 98 is further configured to:

[0322] performing clustering processing on the plurality of facial features in the second facial feature set to obtain a plurality of second facial feature clusters, each of the second facial feature clusters including a cluster center;

[0323] For each second facial feature cluster, if the similarity between the candidate facial feature in the second facial feature cluster and the corresponding cluster center does not meet the fourth similarity condition, then the candidate facial feature is removed from the second facial feature cluster;

[0324] Based on the newly obtained multiple second facial feature clusters, a new second facial feature set is obtained, and the new second facial feature set and the new second entity label set are correspondingly saved in a facial feature temporary library.

[0325] Optionally, the device further includes a second updating module 99, configured to:

[0326] For each second entity tag in the new second entity tag set, perform a preset screening process on multiple facial features corresponding to the second entity tag to obtain a third facial feature set;

[0327] The obtained multiple third facial feature sets and the second entity labels corresponding to the multiple third facial feature sets are saved in the facial feature library.

[0328] Optionally, the first obtaining module 91 is further configured to:

[0329] For any first web page, identify page content in the first web page, and if a person image and person entity information matching any first entity tag are identified, use the identified person image as the first person image corresponding to any first entity tag;

[0330] A plurality of first person images corresponding to each first entity tag are identified from the plurality of first web pages.

[0331] Optionally, the second obtaining module 96 is further configured to:

[0332] For any second web page, identify page content in the second web page, and if a person image and person entity information are identified, obtain a second entity tag based on the person entity information, and use the identified person image as a second person image corresponding to the second entity tag;

[0333] A second entity tag set is obtained based on each second entity tag obtained from a plurality of second network pages, and a second character image set is obtained based on at least one second character image corresponding to each second entity tag.

[0334] For the convenience of description, the above parts are divided into modules according to their functions and described separately. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software or hardware.

[0335] Regarding the device in the above embodiment, the specific execution method of each module has been described in detail in the embodiment of the method, and will not be elaborated here.

[0336] Based on the same inventive concept, the embodiment of the present disclosure also provides an electronic device, the principle of solving the problem of the electronic device is similar to the method of the above embodiment, so the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0337] See Figure 11 As shown, the electronic device may include a processor 1102 and a memory 1101. The memory 1101 provides the processor 1102 with program instructions and data stored in the memory 1101. In the embodiment of the present disclosure, the memory 1101 may be used to store the program for constructing the facial feature library in the embodiment of the present disclosure.

[0338] The processor 1102 calls the program instructions stored in the memory 1101, and the processor 1102 is used to execute the method in any of the above method embodiments, for example Figure 2 The illustrated embodiment provides a method for constructing a facial feature library.

[0339] The specific connection medium between the memory 1101 and the processor 1102 is not limited in the embodiment of the present disclosure. Figure 11 In the embodiment, the memory 1101 and the processor 1102 are connected via a bus 1103. The bus 1103 is connected to the processor 1102 via a bus 1103. Figure 11The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus 1103 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0340] The memory may include read-only memory (ROM) and random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0341] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processing processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0342] An embodiment of the present disclosure further provides a computer storage medium, in which a computer program is stored. A processor of an electronic device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the electronic device executes the facial feature library construction method in any of the above method embodiments.

[0343] In a specific implementation process, computer storage media may include: Universal Serial Bus Flash Drive (USB), mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0344] Based on the same inventive concept as the above-described method embodiments, this embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above-described methods for constructing a facial feature library.

[0345] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0346] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0347] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0348] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0349] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0350] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A method for constructing a facial feature library, characterized in that: include: For each first entity tag in the first entity tag set, obtain multiple first person images corresponding to the first entity tag, and preprocess the multiple first person images to obtain multiple facial features; wherein the preprocessing includes at least face detection and facial feature extraction; For each first entity tag, clustering the multiple facial features corresponding to the first entity tag to obtain multiple first facial feature clusters, each first facial feature cluster including a cluster center. For each cluster center, if the similarity between the cluster center and each other cluster center does not reach a first threshold, deleting the first facial feature cluster in which the cluster center is located; for each remaining first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and the corresponding cluster center, and removing facial features whose similarity does not reach a second threshold from the first facial feature cluster; and obtaining a first facial feature set based on the multiple newly obtained first facial feature clusters; Based on the multiple cluster centers of the multiple first facial feature sets, each two first facial feature sets in the multiple first facial feature sets are compared for similarity; in each similarity comparison, if the similarity comparison results of any two first facial feature sets meet a preset condition, the any two first facial feature sets are merged; A facial feature library is constructed based on the newly obtained multiple first facial feature sets and the first entity label set.

2. The method according to claim 1, characterized in that The step of obtaining the first facial feature set based on the newly obtained plurality of first facial feature clusters includes: For each newly obtained first facial feature cluster, if the number of facial features in the first facial feature cluster does not meet the quantity condition, the first facial feature cluster is deleted.

3. The method according to claim 1, characterized in that The step of obtaining the first facial feature set based on the newly obtained plurality of first facial feature clusters includes: For each newly obtained first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and a plurality of other facial features, and removing the other facial features whose similarities meet a third similarity condition from the first facial feature cluster; Based on the multiple newly obtained first facial feature clusters, the first facial feature set is obtained.

4. The method according to claim 1, wherein The similarity comparison of each two first facial feature sets in the plurality of first facial feature sets is performed, and in each similarity comparison, if the similarity comparison result of the two first facial feature sets meets a preset condition, the two first facial feature sets are merged, including: For each of the two first face feature sets, perform the following operations: Obtaining multiple cluster centers of a first facial feature set, and obtaining multiple cluster centers of another first facial feature set; Determine the similarity between each cluster center of the first facial feature set and multiple cluster centers of the other first facial feature set to obtain a similarity vector; constructing an inter-class similarity matrix based on the obtained multiple similarity vectors, determining target similarities in the inter-class similarity matrix that reach a first similarity threshold, and using the number of the target similarities as the similarity comparison result; If the similarity comparison result meets a preset condition, the one first facial feature set and the other first facial feature set are merged.

5. The method according to claim 4, characterized in that Before merging the first facial feature set with the other first facial feature set, the method further includes: Constructing a first intra-class similarity matrix based on the similarity between each cluster center of the first facial feature set and each other cluster center of the first facial feature set; Constructing a second intra-class similarity matrix based on the similarity between each cluster center of the another first facial feature set and each of the other cluster centers of the another first facial feature set; If it is determined based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix that a target cluster center in the one first facial feature set and each cluster center in the other first facial feature set satisfy a first preset relationship, then the first facial feature cluster in which the target cluster center is located is removed; If, based on the first intra-class similarity matrix, the second intra-class similarity matrix and the inter-class similarity matrix, it is determined that the target cluster center in the other first facial feature set and the cluster centers of the one first facial feature set satisfy a second preset relationship, then the first facial feature cluster where the target cluster center is located is eliminated.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Acquire a second entity tag set and a second person image set, wherein each second entity tag in the second entity tag set corresponds to at least one second person image in the second person image set; For each second person image, if a target facial feature that matches the facial feature in the second person image is retrieved from the facial feature library, the facial feature in the second person image is saved under the target entity label corresponding to the target facial feature in the facial feature library.

7. The method according to claim 6, characterized in that If a target facial feature matching the facial feature in the second person image is retrieved from the facial feature library, the facial feature in the second person image is saved under a target entity tag corresponding to the target facial feature in the facial feature library, including: If a target facial feature matching the facial feature in the second person image is retrieved from the facial feature library, determining whether the target facial feature satisfies at least one of the following conditions: a target entity label corresponding to the target facial feature is consistent with a second entity label corresponding to the second person image, and a similarity between the target facial feature and the facial feature in the second person image is less than a second similarity threshold; If so, the facial features in the second person image are saved under the target entity label in the facial feature library.

8. The method according to claim 6, characterized in that The method further comprises: For each second person image, if no target facial features matching the facial features in the second person image are retrieved from the facial feature library, the second person image is used as a candidate person image to be stored in the library; Obtaining a new second entity label set based on the second labels corresponding to the plurality of candidate person images, and preprocessing the plurality of candidate person images to obtain a second facial feature set; wherein the preprocessing includes at least face detection and facial feature extraction; The second facial feature set and the new second entity tag set are saved to a facial feature temporary library, where each second entity tag in the new second entity tag set corresponds to at least one facial feature in the second facial feature set.

9. The method according to claim 8, characterized in that Saving the second facial feature set and the new second entity label set to a temporary facial feature library includes: performing clustering processing on the plurality of facial features in the second facial feature set to obtain a plurality of second facial feature clusters, each second facial feature cluster including a cluster center; For each second facial feature cluster, if the similarity between the candidate facial feature in the second facial feature cluster and the corresponding cluster center does not meet the fourth similarity condition, then the candidate facial feature is removed from the second facial feature cluster; Based on the newly obtained multiple second facial feature clusters, a new second facial feature set is obtained, and the new second facial feature set and the new second entity label set are correspondingly saved in the facial feature temporary library.

10. The method according to claim 8, characterized in that The method further comprises: For each second entity tag in the new second entity tag set, performing the preset screening process on multiple facial features corresponding to the second entity tag to obtain a third facial feature set; A plurality of third facial feature sets and second entity labels corresponding to each of the plurality of third facial feature sets are obtained and saved in the facial feature library.

11. The method according to any one of claims 1 to 5, characterized in that The obtaining of a plurality of first person images corresponding to the first entity tag includes: For any first web page, identify page content in the first web page, and if a person image and person entity information matching any first entity tag are identified, use the identified person image as the first person image corresponding to any first entity tag; A plurality of first person images corresponding to each of the first entity tags are identified from the plurality of first web pages.

12. The method according to claim 6, characterized in that The obtaining of the second entity tag set and the second person image set includes: For any second web page, identify page content in the second web page, and if a person image and person entity information are identified, obtain a second entity tag based on the person entity information, and use the identified person image as a second person image corresponding to the second entity tag; The second entity tag set is obtained based on the second entity tags obtained from the plurality of second network pages, and the second person image set is obtained based on at least one second person image corresponding to each second entity tag.

13. A facial feature library construction device, characterized in that: include: A first acquisition module is configured to acquire, for each first entity tag in the first entity tag set, a plurality of first person images corresponding to the first entity tag, and preprocess the plurality of first person images to obtain a plurality of facial features; wherein the preprocessing includes at least face detection and facial feature extraction; a screening module configured to cluster, for each first entity tag, multiple facial features corresponding to the first entity tag to obtain multiple first facial feature clusters, each first facial feature cluster including a cluster center; and for each cluster center, if the similarity between the cluster center and each other cluster center does not reach a first threshold, deleting the first facial feature cluster in which the cluster center is located; for each remaining first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and the corresponding cluster center, and removing from the first facial feature cluster facial features whose similarity does not reach a second threshold; and obtaining a first facial feature set based on the multiple newly obtained first facial feature clusters; a comparison module, configured to perform a similarity comparison between each two first facial feature sets in the plurality of first facial feature sets based on the respective plurality of cluster centers of the plurality of first facial feature sets, and merge the two first facial feature sets if the similarity comparison results of any two first facial feature sets meet a preset condition in each similarity comparison; A construction module is used to construct a facial feature library based on the newly obtained multiple first facial feature sets and the first entity label set.

14. The device according to claim 13, characterized in that When obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters, the screening module is further configured to: For each newly obtained first facial feature cluster, if the number of facial features in the first facial feature cluster does not meet the quantity condition, the first facial feature cluster is deleted.

15. The device according to claim 13, characterized in that When obtaining the first facial feature set based on the newly obtained multiple first facial feature clusters, the screening module is further configured to: For each newly obtained first facial feature cluster, determining the similarity between each facial feature in the first facial feature cluster and a plurality of other facial features, and removing the other facial features whose similarities meet a third similarity condition from the first facial feature cluster; Based on the multiple newly obtained first facial feature clusters, the first facial feature set is obtained.

16. The device according to claim 13, characterized in that The comparison module is also used for: For each of the two first face feature sets, perform the following operations: Obtaining multiple cluster centers of a first facial feature set, and obtaining multiple cluster centers of another first facial feature set; Determine the similarity between each cluster center of the first facial feature set and multiple cluster centers of the other first facial feature set to obtain a similarity vector; constructing an inter-class similarity matrix based on the obtained multiple similarity vectors, determining target similarities in the inter-class similarity matrix that reach a first similarity threshold, and using the number of the target similarities as the similarity comparison result; If the similarity comparison result meets a preset condition, the one first facial feature set and the other first facial feature set are merged.

17. The device according to claim 16, characterized in that Also includes culling modules for: Constructing a first intra-class similarity matrix based on the similarity between each cluster center of the first facial feature set and each other cluster center of the first facial feature set; Constructing a second intra-class similarity matrix based on the similarity between each cluster center of the another first facial feature set and each of the other cluster centers of the another first facial feature set; If it is determined based on the first intra-class similarity matrix, the second intra-class similarity matrix, and the inter-class similarity matrix that a target cluster center in the one first facial feature set and each cluster center in the other first facial feature set satisfy a first preset relationship, then the first facial feature cluster in which the target cluster center is located is removed; If, based on the first intra-class similarity matrix, the second intra-class similarity matrix and the inter-class similarity matrix, it is determined that the target cluster center in the other first facial feature set and the cluster centers of the one first facial feature set satisfy a second preset relationship, then the first facial feature cluster where the target cluster center is located is eliminated.

18. The device according to any one of claims 13 to 17, characterized in that The device further comprises: A second acquisition module is configured to acquire a second entity tag set and a second character image set, wherein each second entity tag in the second entity tag set corresponds to at least one second character image in the second character image set; The first updating module is used to save the facial features in the second person image to the target entity label corresponding to the target facial features in the facial feature library for each second person image if a target facial feature matching the facial features in the second person image is retrieved from the facial feature library.

19. The device according to claim 18, characterized in that The first update module is further configured to: If a target facial feature matching the facial feature in the second person image is retrieved from the facial feature library, determining whether the target facial feature satisfies at least one of the following conditions: a target entity label corresponding to the target facial feature is consistent with a second entity label corresponding to the second person image, and a similarity between the target facial feature and the facial feature in the second person image is less than a second similarity threshold; If so, the facial features in the second person image are saved under the target entity label in the facial feature library.

20. The device according to claim 18, characterized in that The device further includes a storage module, configured to: For each second person image, if no target facial features matching the facial features in the second person image are retrieved from the facial feature library, the second person image is used as a candidate person image to be stored in the library; Obtaining a new second entity label set based on the second labels corresponding to the plurality of candidate person images, and preprocessing the plurality of candidate person images to obtain a second facial feature set; wherein the preprocessing includes at least face detection and facial feature extraction; The second facial feature set and the new second entity tag set are saved to a facial feature temporary library, where each second entity tag in the new second entity tag set corresponds to at least one facial feature in the second facial feature set.

21. The device according to claim 20, characterized in that When the second facial feature set and the new second entity label set are saved in the facial feature temporary library, the saving module is further configured to: performing clustering processing on the plurality of facial features in the second facial feature set to obtain a plurality of second facial feature clusters, each second facial feature cluster including a cluster center; For each second facial feature cluster, if the similarity between the candidate facial feature in the second facial feature cluster and the corresponding cluster center does not meet the fourth similarity condition, then the candidate facial feature is removed from the second facial feature cluster; Based on the newly obtained multiple second facial feature clusters, a new second facial feature set is obtained, and the new second facial feature set and the new second entity label set are correspondingly saved in the facial feature temporary library.

22. The device according to claim 20, characterized in that The device further includes a second updating module, configured to: For each second entity tag in the new second entity tag set, performing the preset screening process on multiple facial features corresponding to the second entity tag to obtain a third facial feature set; A plurality of third facial feature sets and second entity labels corresponding to each of the plurality of third facial feature sets are obtained and saved in the facial feature library.

23. The device according to any one of claims 13 to 17, characterized in that The obtaining of a first person image set corresponding to the first entity label includes: For any web page among the plurality of web pages, identifying page content in the web page, and if first image information and person entity information matching any of the first entity tags are identified, obtaining a first person image corresponding to any of the first entity tags based on the first image information; Based on a plurality of first person images corresponding to each first entity tag obtained from a plurality of the web pages, a first person image set corresponding to the first entity tag is obtained.

24. The device according to claim 18, wherein The obtaining of the second entity tag set and the second person image set includes: For any web page among the plurality of web pages, identifying page content in the web page, and if the page content includes person entity information and second image information, obtaining a second entity tag based on the person entity information, and obtaining a second person image corresponding to the second entity tag based on the second image information; The second entity tag set is obtained based on the second entity tags obtained from the plurality of web pages, and the second person image set is obtained based on at least one second person image corresponding to each second entity tag.

25. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor executes the steps of any one of the methods of claims 1 to 12.

26. A computer-readable storage medium, characterized in that The device stores a computer program, which, when executed on an electronic device, enables the electronic device to execute the steps of any one of the methods of claims 1 to 12.

Images