A training method, device and equipment for facial recognition model

Abstract

The embodiments of this specification disclose a method, device and equipment for training a facial recognition model. By obtaining a training sample including facial key points; dividing the training sample into multiple sub-images; determining the category to which any sub-image belongs according to the key points included in the sub-image, generating multiple image subsets corresponding to multiple different categories; using an initial model to determine the local feature vectors corresponding to the multiple image subsets, fusing the local feature vectors to generate a global feature vector corresponding to the training sample; performing classification training on the initial model according to the global feature vector to generate a usable target model. Thus, for the trained target model, when in use, the image subset contained in each image to be identified can be used to quantify its similarity with the corresponding area of ​​the image to be confirmed.

Description

Technical Field

[0001] This specification relates to the field of Internet technology, and in particular to a training method, device and equipment for a facial recognition model. Background Art

[0002] With the great progress of deep learning, facial recognition technology has made significant progress and is widely used in practical applications of identity authentication, such as mobile phone unlocking, gate authentication, payment, etc. However, since the facial recognition model trained on a large scale has always been a black box system, it has poor interpretability, especially for close relatives or people with similar faces, the differences are quite subtle and lack quantitative interpretable basis.

[0003] Based on this, a training scheme for a facial recognition model with high interpretability of recognition results is needed. Summary of the invention

[0004] One or more embodiments of the present specification provide a facial recognition model training method, apparatus, device, and storage medium to solve the following technical problem: a facial recognition model training scheme with high interpretability of recognition results is needed.

[0005] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0006] In a first aspect, an embodiment of the present specification provides a method for training a facial recognition model, comprising: obtaining a training sample including facial key points; dividing the training sample into a plurality of sub-images; determining the category to which any sub-image belongs based on the key points included in the sub-image, and generating a plurality of image subsets corresponding to a plurality of different categories; using an initial model to determine local feature vectors corresponding to the plurality of image subsets, and fusing the local feature vectors to generate a global feature vector corresponding to the training sample; performing classification training on the initial model based on the global feature vector to generate a usable target model.

[0007] In a second aspect, an embodiment of the present specification provides a training device for a facial recognition model, comprising: an acquisition module, which acquires training samples including facial key points; a division module, which divides the training samples into multiple sub-images; a classification module, which determines the category to which any sub-image belongs based on the key points included in the sub-image, and generates multiple image subsets corresponding to multiple different categories; a generation module, which uses an initial model to determine local feature vectors corresponding to the multiple image subsets, and fuses the local feature vectors to generate a global feature vector corresponding to the training sample; and a training module, which performs classification training on the initial model based on the global feature vector to generate a usable target model.

[0008] In a third aspect, an electronic device provided by an embodiment of this specification includes:

[0009] at least one processor; and,

[0010] a memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0012] In a fourth aspect, an embodiment of the present specification provides a non-volatile computer storage medium storing computer executable instructions. When a computer reads the computer executable instructions in the storage medium, the instructions enable one or more processors to execute the method described in the first aspect.

[0013] At least one of the above technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects: by obtaining a training sample including facial key points; dividing the training sample into multiple sub-images; determining the category to which any sub-image belongs according to the key points included in the sub-image, and generating multiple image subsets corresponding to multiple different categories; using an initial model to determine the local feature vectors corresponding to the multiple image subsets, fusing the local feature vectors to generate the global feature vector corresponding to the training sample; and classifying and training the initial model according to the global feature vector to generate a usable target model. Thus, for the trained target model, when used, it is possible to quantify the similarity between the image subset contained in each image to be identified and the corresponding area of ​​the image to be confirmed, give a clear similarity calculation and explainable reasons, and improve the interpretability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0015] Figure 1 A flowchart of a method for training a facial recognition model provided in an embodiment of this specification;

[0016] Figure 2a A schematic diagram of dividing a sub-image used in the embodiments of this specification;

[0017] Figure 2bA schematic diagram of another method of dividing sub-images used in the embodiments of this specification;

[0018] Figure 3 A schematic diagram of a target model for user identification provided in an embodiment of this specification;

[0019] Figure 4 A schematic diagram of the structure of a facial recognition model training device provided in an embodiment of this specification;

[0020] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0021] The embodiments of this specification provide a method, apparatus, device and storage medium for training a facial recognition model.

[0022] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0023] In the first aspect, Figure 1 As shown, Figure 1 A flow chart of a method for training a facial recognition model provided in an embodiment of this specification specifically includes the following steps:

[0024] S101: Obtain training samples including facial key points.

[0025] A training sample is a facial image. In practical applications, the set of training samples can be a large-scale set of facial images, and the number of training samples contained in the set can reach billions. For example, tens of millions of users with different identities can be obtained, and about 100 facial images are given to each user as training samples.

[0026] That is, the training samples contain user IDs. The training results require that the model can accurately classify any given training samples into the category corresponding to the user ID.

[0027] For any given training sample, a mature key point detection model can be used to calibrate the key points of the head, shoulders and face contained therein. For example, a key point detection model based on the Dlib library or a deep alignment network (DAN) can be used to calibrate the key points. While calibrating the key points, the key point detection model can also give the semantics of any key point (i.e., the sub-region to which the key point belongs).

[0028] For example, for key point 1, its corresponding semantics may be "around the corner of the eye", for key point 2, its corresponding semantics may be "nose", and so on.

[0029] Furthermore, before performing key point recognition on any training sample, it can also be preprocessed, for example, the background part of the training sample can be removed (the area outside the human body contour can be blackened) and only the head and shoulder area of ​​the person is retained, thereby obtaining a preprocessed training sample so that the key point detection model can better perform key point recognition.

[0030] S103: Divide the training sample into a plurality of sub-images.

[0031] The specific division method can be determined according to actual needs, and the sub-images obtained by division may have overlapping parts or may not have overlapping parts.

[0032] For example, in one division method, the key points of the face and head and shoulders can be ignored, and only a specified window size with a fixed length and width can be used to divide the training sample into multiple sub-images of the specified window size starting from the coordinate origin of the training sample and using a specified step size. Figure 2a As shown, Figure 2a A schematic diagram of dividing a sub-image adopted in the embodiments of this specification.

[0033] In this division method, a sub-image may include key points belonging to different regions at the same time. For example, sub-image 2 may include key points corresponding to the periphery of the eye corners and the nose at the same time. Also, the number of sub-images obtained by division may be pre-specified.

[0034] In another division method, the training sample can be first divided into multiple sub-regions with certain semantics. For example, the training sample is segmented based on pixel coordinates to generate corresponding sub-regions such as "nose", "eyes", "mouth", etc., and then the image is segmented in each sub-region to generate multiple sub-images.

[0035] In this division method, the key points in the sub-images obtained by division are generally of the same category, that is, the key points in the sub-images obtained by division in the sub-region "nose" all correspond to "nose", and the key points in the sub-images obtained by division in the sub-region "eyes" all correspond to "eyes". Figure 2b As shown, Figure 2b This is a schematic diagram of another method of dividing sub-images used in the embodiments of this specification.

[0036] In this division method, since each sub-image basically contains key points of the same category, it is easier to determine the category corresponding to the sub-image.

[0037] S105 , determining the category to which any sub-image belongs according to the key points included in the sub-image, and generating a plurality of image subsets corresponding to a plurality of different categories.

[0038] As mentioned above, under different division methods, the categories of key points that may be contained in the sub-image are diverse. Figure 2a Under the division method shown in the figure, some sub-images may include key points of three to four categories at the same time. At this time, it is necessary to clearly classify the sub-images to generate corresponding multiple image subsets, or they can also be called image ptachs.

[0039] It should be noted that each category of image subsets also has clear semantics. For example, the semantics of image subset 1 may be "nose", while the semantics of image subset 2 may be "around the corner of the eye", and so on.

[0040] Corresponding to the aforementioned division method, different methods can also be used to determine the category of the image subset. Figure 2a In the illustrated division method, the facial sub-region to which the key points included in the sub-image belong can be determined; and the facial sub-region to which the key points with the largest number belong determines the category to which the sub-image belongs.

[0041] For example, assuming that the facial sub-regions to which the key points included in a sub-image obtained belong are sub-region 1, sub-region 2 and sub-region 3 respectively, and the sub-image contains a total of 10 key points, among which the number of key points belonging to sub-regions 1, 2 and 3 are 2, 5 and 3 respectively, then it can be known that in the sub-image, sub-region 2 contains the largest number of key points, and the category to which the sub-image belongs is determined as "the category corresponding to sub-region 2".

[0042] Furthermore, if the Figure 2bIt is easy to understand that in this division method, since the sub-regions have been pre-divided, the key points with the largest number contained in any sub-image are actually of the same category.

[0043] For example, for any sub-image obtained by dividing in sub-region 1, the facial sub-region containing the largest number of key points is obviously "sub-region 1". At this time, the category to which the sub-image belongs can be directly determined as "the category corresponding to sub-region 1".

[0044] S107, using an initial model to determine local feature vectors corresponding to the multiple image subsets, and fusing the local feature vectors to generate a global feature vector corresponding to the training sample.

[0045] After obtaining a plurality of image subsets, the initial model may be used to extract features from the sub-images contained in each image subset, thereby obtaining a local feature vector corresponding to each image subset.

[0046] Since each image subset corresponds to a sub-region with clear semantics, each local feature vector also has clear semantics. For example, the local feature vector extracted from the image subset corresponding to the "eye periphery" can be clearly regarded as having the semantics of "eye periphery" and can be compared with the "eye periphery" feature vector extracted from other images for similarity. The specific similarity calculation can be performed using calculation methods such as Euclidean distance or cosine distance.

[0047] The specific form of the initial model used can be built based on actual needs. For example, an initial model based on automatic encoding and decoding can be used to encode the sub-images contained in the image subset to obtain input features, and then decode the input features to restore the corresponding predicted sub-images.

[0048] Since an image subset usually contains a large number of sub-images, for each sub-image, after the aforementioned encoding and decoding, the restored image is still a partial image in a sub-region, that is, for a sub-image, the sub-feature vector corresponding to the sub-image can be obtained during the encoding and decoding process. Then, the N sub-images corresponding to an image subset can be repeatedly encoded and decoded to generate the corresponding N sub-feature vectors, so that the multiple sub-feature vectors corresponding to the N sub-images can be fused (for example, the mean can be superimposed) to generate the local feature vector corresponding to the image subset, and the encoding and decoding model can be used, so that the face recognition training is no longer strongly dependent on the final softmax classification calculation.

[0049] S109, performing classification training on the initial model according to the global feature vector to generate a usable target model.

[0050] The specific training method is that after obtaining the global feature vector, another vector with the greatest similarity to the global feature vector is found and classified into the user ID to which the other vector belongs. Since the training sample itself carries the user ID, the accuracy of the classification result can be finally determined.

[0051] When the accuracy of the classification result does not meet the preset requirements, the target vector of the user to whom the training sample belongs can be obtained first, and the encoder and decoder can be trained according to the difference between the global feature vector and the target vector until the accuracy of the classification result meets the preset requirements. The model at this time is regarded as a usable target model.

[0052] By obtaining a training sample including facial key points; dividing the training sample into multiple sub-images; determining the category to which any sub-image belongs according to the key points included in the sub-image, generating multiple image subsets corresponding to multiple different categories; using an initial model to determine the local feature vectors corresponding to the multiple image subsets, fusing the local feature vectors to generate the global feature vector corresponding to the training sample; and performing classification training on the initial model according to the global feature vector to generate a usable target model. Thus, for the trained target model, when used, it is possible to quantify the similarity between the image subset contained in each image to be identified and the corresponding area of ​​the image to be confirmed, give a clear similarity calculation and explainable reasons, and improve the interpretability of the model.

[0053] In one implementation, before the training sample is divided into a plurality of sub-images, an alignment operation may be performed on the training sample to generate aligned training samples. Furthermore, during the division, the aligned training sample may be divided into a plurality of sub-images.

[0054] The specific alignment operation may include rotating and correcting the tilted image so that the head is aligned in the center (i.e., the facial symmetry axis is the center vertical line of the training sample). In addition, the image may be scaled so that a specific area in the face area is in a specific position. For example, the training sample may be scaled or translated so that the nose vertex is in the center of the image, etc. Through the alignment operation, the semantics of each area of ​​any training sample can be made the same, which is conducive to the subsequent classification of the sub-images obtained by division.

[0055] In one embodiment, before the initial model is used to encode and decode the sub-image, the sub-image in the image subset may be randomly partially occluded. The size of the occluded area may be set according to actual needs. For example, experience shows that randomly occluding 70% to 80% of the area in the sub-image has a better effect on the convergence of the model training.

[0056] The occlusion method can be to first determine the number of pixels contained in the sub-image, and then randomly select 70% to 80% of adjacent pixels, and cover the selected pixels with pixels of the same color (for example, white or blue, etc.) to complete the occlusion of the sub-image.

[0057] In the actual face scanning process, users often use masks or veils to cover part of their faces. Some malicious users may also use partial facial features to attack. Therefore, encoding and decoding through partial occlusion to achieve feature extraction can effectively deal with the problems of masked faces and occluded faces in reality.

[0058] In one embodiment, when the local feature vectors in the initial model are fused to generate the global feature vectors corresponding to the training samples, the weight distribution parameters of the local feature vectors corresponding to each image subset can also be trained. That is, for m image subsets, there can be a trainable weight distribution parameter W=(W1, W2, ... Wm) in the model, and any W1 to Wm is the weight of the local feature vector extracted from the corresponding image subset.

[0059] Therefore, during the training process, the feature expression of each image subset can be fine-tuned by adjusting the encoder and decoder, and the global feature vector obtained by the final fusion can be fine-tuned by training the weight distribution parameters. In practical applications, some areas of certain faces are usually more important for facial recognition. Therefore, the local feature vectors extracted from the corresponding image subsets usually have higher weight values. For example, the area around the eyes and the nose usually have higher weight values.

[0060] After the target model is trained, the target model can be used to identify the image to be identified. Specifically, the process includes: obtaining the image to be identified; obtaining the image to be confirmed corresponding to the image to be identified; using the target model to generate multiple first local feature vectors corresponding to the image to be identified, and using the target model to generate multiple second local feature vectors corresponding to the image to be confirmed; respectively determining multiple local similarities between the multiple first local feature vectors and the multiple second local feature vectors; fusing the multiple local similarities to generate a recognition result.

[0061] In the aforementioned process, the image to be identified is the facial image obtained by the device scanning the face, and the image to be confirmed is the facial image with the user identification pre-stored in the server or device.

[0062] In actual applications, when the user usually performs facial recognition through the client of his own device, the image to be confirmed can be obtained from the server based on the user identifier corresponding to the client; in scenarios such as clocking in or facial authentication, several images with a high similarity to the image to be recognized can be first screened from the pre-stored images as the images to be confirmed.

[0063] Furthermore, the target model can adopt similar steps as those in the aforementioned training to perform similar processing on the image to be recognized and the image to be confirmed: first, the key points in the image are identified, and the image to be recognized and the image to be confirmed are divided to obtain multiple sub-images containing multiple key points, and then the sub-images are classified based on the key points to obtain corresponding multiple image subsets with semantics, and then the sub-images in the image subsets are encoded and decoded to extract the local feature vectors corresponding to each sub-image.

[0064] Through the above-mentioned method, multiple first local feature vectors and multiple second local feature vectors of the same number and corresponding to the sub-regions can be obtained respectively, and the first local feature vectors and the second local feature vectors corresponding to the sub-regions have the same semantics (for example, they can be around the corners of the eyes).

[0065] Thus, the similarity of the local feature vectors of each corresponding area can be integrated and compared to comprehensively judge the similarity of the image to be identified and the image to be confirmed. For example, the similarity of each local feature vector can be directly added to obtain the similarity of the image to be identified and the image to be confirmed. When the similarity of the image to be identified and the image to be confirmed is greater than a preset threshold, it is considered that the two are the same user, otherwise, they are not the same user.

[0066] like Figure 3 shown. Figure 3 A schematic diagram of a target model for user identification provided in an embodiment of this specification. In this schematic diagram, the image to be identified and the image to be confirmed are divided into five sub-regions, namely eyes, eye corners, nose, mouth, and ears, and then the similarities of the five sub-regions are added to determine whether the image to be identified and the image to be confirmed belong to the same user.

[0067] In one embodiment, when fusing the multiple local similarities to generate the recognition result, the aforementioned weight distribution parameters can also be used for fusion. It should be noted that in this process, the weight distribution parameters are fixed parameters that have been trained and will not change based on the different images to be recognized. For example, assuming that the weight distribution parameter is W = (1, 1, 2, 3, 1), and the similarities of the five sub-areas calculated at the same time are S = (0.7, 0.8, 0.9, 0.9, 0.6), then the final similarity is W dot product S = 0.7 + 1 * 0.8 + 2 * 0.9 + 3 * 0.9 + 0.6 = 6.8. In this way, the role of local feature vectors that have significant reference significance in facial recognition can be highlighted, making the recognition results more accurate.

[0068] In one embodiment, a specific part of the local feature vector can also be selected for comprehensive determination. For example, the user may partially block the face, and at this time, the features of the mouth area and the ear area in the image to be identified cannot be extracted. In this case, only the local feature vectors corresponding to other sub-areas can be considered for similarity comparison for comprehensive determination.

[0069] Continuing with the previous example, at this time, if it is assumed that the weight distribution parameters of the five sub-regions of eyes, eye corners, nose, mouth, and ears remain unchanged, and are still W = (1, 1, 2, 3, 1), then the weight parameters of the mouth region and the ear region can be omitted. At this time, the dot product of W = (1, 1, 2) and the similarity S = (0.7, 0.8, 0.9) of the other three sub-regions is used for comprehensive judgment. If the dot product result and the mean of the number of sub-regions exceed the preset value, it can be determined that the recognition result belongs to the same user.

[0070] In one embodiment, the similarity of the designated sub-region may also be used for determination. If the local feature vector in the designated sub-region cannot be extracted in the image to be identified (i.e., the designated sub-region may be obscured), it is deemed that no determination can be made, and a prompt is issued, requiring the user to display the obscured sub-region.

[0071] For example, you can pre-specify that the four sub-regions of the nose, eyes, eye corners, and mouth are used for similarity recognition. If the user wears a mask, the local feature vector corresponding to the mouth may not be obtained. At this time, the user can be prompted to take off the mask, show the mouth, and rescan. For another example, you can pre-specify that any three of the four sub-regions of the nose, eyes, eye corners, and mouth are used for similarity recognition. Thus, a prompt is issued when only local feature vectors of two sub-regions can be obtained from the image to be recognized. In this way, the area required for facial recognition can be adjusted based on the needs of the actual scene to improve the user experience.

[0072] In addition, based on the aforementioned scheme, if misrecognition occurs during the user identification process, for example, some similar faces of different people are judged to be the same person, at this time, in the embodiment of the present specification, it is possible to extract background data to clarify the target model based on which areas of similarity caused the misrecognition, thereby being able to quantify the explainable basis, improve the user's sense of security in facial recognition, and facilitate the promotion and iterative optimization of the target model.

[0073] In the second aspect, based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 4 , Figure 5 shown.

[0074] Figure 4 A schematic diagram of a facial recognition model training device provided in an embodiment of this specification, the device comprising:

[0075] An acquisition module 401 acquires a training sample including facial key points;

[0076] A division module 403 divides the training sample into a plurality of sub-images;

[0077] A classification module 405, which determines the category to which any sub-image belongs according to the key points included in the sub-image, and generates a plurality of image subsets corresponding to a plurality of different categories;

[0078] A generating module 407, which uses an initial model to determine local feature vectors corresponding to the plurality of image subsets, and fuses the local feature vectors to generate a global feature vector corresponding to the training sample;

[0079] The training module 409 performs classification training on the initial model according to the global feature vector to generate a usable target model.

[0080] Optionally, the division module 403 divides the training sample into multiple sub-images of a specified window size using a specified step size; or divides the training sample into multiple sub-regions with certain semantics, and for any sub-region, divides the sub-region into multiple sub-images of a specified window size.

[0081] Optionally, the device further comprises an alignment module 411, which performs an alignment operation on the training samples to generate aligned training samples; accordingly, dividing the training samples into a plurality of sub-images comprises: dividing the aligned training samples into a plurality of sub-images.

[0082] Optionally, the classification module 405 determines the facial sub-region to which the key points included in the sub-image belong; and determines the facial sub-region to which the key points with the largest number belong as the category to which the sub-image belongs.

[0083] Optionally, the generation module 407, for any image subset, uses the initial model to encode and decode the sub-images contained in the image subset to generate a sub-feature vector corresponding to the sub-image; and fuses multiple sub-feature vectors corresponding to multiple sub-images to generate a local feature vector corresponding to the image subset.

[0084] Optionally, the generating module 407 performs occlusion of random areas of the sub-image to generate an occluded sub-image; and uses an initial model to encode and decode the occluded sub-image.

[0085] Optionally, the generating module 407 determines the weight distribution parameters corresponding to the local feature vectors, performs weighted merging of the local feature vectors according to the weight distribution parameters, and generates the global feature vector corresponding to the training sample; correspondingly, the training module 409 trains the weight distribution parameters according to the global feature vector.

[0086] Optionally, the device also includes a recognition module 413, which obtains an image to be recognized; obtains an image to be confirmed corresponding to the image to be recognized; uses the target model to generate multiple first local feature vectors corresponding to the image to be recognized, and uses the target model to generate multiple second local feature vectors corresponding to the image to be confirmed; respectively determines multiple local similarities between the multiple first local feature vectors and the multiple second local feature vectors; and fuses the multiple local similarities to generate a recognition result.

[0087] Optionally, the generating module 407 adopts a weight distribution parameter corresponding to the local feature vector determined by the target model; weights the multiple local similarities according to the weight distribution parameter to generate a recognition result.

[0088] In the third aspect, Figure 5 As shown, Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification, the device includes:

[0089] at least one processor; and,

[0090] a memory communicatively connected to the at least one processor; wherein,

[0091] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0092] In the fourth aspect, based on the same idea, the embodiment of this specification also provides a non-volatile computer storage medium corresponding to the above method, storing computer executable instructions. When the computer reads the computer executable instructions in the storage medium, the instructions enable one or more processors to execute the method described in the first aspect.

[0093] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0094] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0095] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0096] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0097] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0098] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0101] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0102] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0103] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0104] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0105] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0106] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0107] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0108] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A method for training a facial recognition model, comprising: Obtain training samples including facial key points; Dividing the training sample into a plurality of sub-images; Determining the category to which any sub-image belongs according to the key points included in the sub-image, and generating a plurality of image subsets corresponding to a plurality of different categories; Using an initial model to determine local feature vectors corresponding to the plurality of image subsets, and fusing the local feature vectors to generate a global feature vector corresponding to the training sample; Performing classification training on the initial model according to the global feature vector to generate a usable target model for face recognition; The step of fusing the local feature vectors to generate a global feature vector corresponding to the training sample includes: Determine the weight distribution parameters corresponding to the local feature vectors, perform weighted merging on the local feature vectors according to the weight distribution parameters, and generate a global feature vector corresponding to the training sample; Correspondingly, classification training is performed on the initial model according to the global feature vector, including: training the weight distribution parameters according to the global feature vector.

2. The method of claim 1, wherein: Dividing the training sample into a plurality of sub-images, including: Using a specified step size, the training sample is divided into a plurality of sub-images of a specified window size; Alternatively, the training sample is divided into a plurality of sub-regions with certain semantics, and for any sub-region, the sub-region is divided into a plurality of sub-images with a specified window size.

3. The method of claim 1, wherein: Before dividing the training sample into a plurality of sub-images, the method further comprises: Performing an alignment operation on the training samples to generate aligned training samples; Correspondingly, dividing the training sample into a plurality of sub-images includes: dividing the aligned training sample into a plurality of sub-images.

4. The method of claim 1, wherein: Determining the category to which any sub-image belongs according to the key points included in the sub-image includes: Determine the facial sub-region to which the key points included in the sub-image belong; The facial sub-region to which the most key points belong determines the category to which the sub-image belongs.

5. The method according to claim 1, wherein the initial model is used to determine the local feature vectors corresponding to the plurality of image subsets, comprising: For any image subset, the initial model is used to encode and decode the sub-images contained in the image subset to generate the sub-feature vectors corresponding to the sub-images; Multiple sub-feature vectors corresponding to multiple sub-images are fused to generate a local feature vector corresponding to the image subset.

6. The method of claim 5, wherein: The initial model is used to encode and decode the sub-images contained in the image subset, including: Performing masking on the sub-image in random areas to generate a masked sub-image; The initial model is used to encode and decode the blocked sub-image.

7. The method of claim 1, further comprising: Obtain an image to be recognized; Acquire the image to be confirmed corresponding to the image to be identified; Using the target model to generate a plurality of first local feature vectors corresponding to the image to be identified, and using the target model to generate a plurality of second local feature vectors corresponding to the image to be confirmed; respectively determining a plurality of local similarities between the plurality of first local feature vectors and the plurality of second local feature vectors; The multiple local similarities are combined to generate a recognition result.

8. The method of claim 7, wherein: Fusion of the multiple local similarities to generate a recognition result includes: Using the weight distribution parameters corresponding to the local feature vector determined by the target model; The multiple local similarities are weighted according to the weight distribution parameter to generate a recognition result.

9. A training device for a facial recognition model, comprising: An acquisition module is used to acquire training samples including facial key points; A division module, dividing the training sample into a plurality of sub-images; A classification module, which determines the category to which any sub-image belongs according to the key points included in the sub-image, and generates a plurality of image subsets corresponding to a plurality of different categories; A generation module, which uses an initial model to determine local feature vectors corresponding to the plurality of image subsets, and fuses the local feature vectors to generate a global feature vector corresponding to the training sample; A training module, performing classification training on the initial model according to the global feature vector to generate a usable target model for face recognition; The generation module determines the weight distribution parameters corresponding to the local feature vectors, performs weighted combination of the local feature vectors according to the weight distribution parameters, and generates the global feature vector corresponding to the training sample; correspondingly, the training module trains the weight distribution parameters according to the global feature vector.

10. In the device as described in claim 9, the division module adopts a specified step size to divide the training sample into multiple sub-images of a specified window size; or divides the training sample into multiple sub-regions with certain semantics, and for any sub-region, divides the sub-region into multiple sub-images of a specified window size.

11. The apparatus according to claim 9, further comprising an alignment module, performing an alignment operation on the training samples to generate aligned training samples; accordingly, dividing the training samples into a plurality of sub-images, comprising: The aligned training samples are divided into a plurality of sub-images.

12. The device as claimed in claim 9, wherein the classification module determines the facial sub-region to which the key points included in the sub-image belong; and determines the facial sub-region to which the key points with the largest number belong as the category to which the sub-image belongs.

13. The device as described in claim 9, wherein the generation module, for any image subset, uses the initial model to encode and decode the sub-images contained in the image subset to generate a sub-feature vector corresponding to the sub-image; and fuses multiple sub-feature vectors corresponding to multiple sub-images to generate a local feature vector corresponding to the image subset.

14. The device as claimed in claim 13, wherein the generating module performs random area occlusion on the sub-image to generate the occluded sub-image; and uses the initial model to encode and decode the occluded sub-image.

15. The device as described in claim 9 further includes a recognition module, which obtains an image to be recognized; obtains an image to be confirmed corresponding to the image to be recognized; uses the target model to generate multiple first local feature vectors corresponding to the image to be recognized, and uses the target model to generate multiple second local feature vectors corresponding to the image to be confirmed; respectively determines multiple local similarities between the multiple first local feature vectors and the multiple second local feature vectors; and fuses the multiple local similarities to generate a recognition result.

16. The device as claimed in claim 15, wherein the generation module adopts the weight distribution parameters corresponding to the local feature vector determined by the target model; weights the multiple local similarities according to the weight distribution parameters to generate the recognition result.

17. An electronic device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images