Model training method, kinship identification method, electronic device and storage medium

Abstract

The embodiments of the present application relate to the field of face recognition and disclose a model training method, a kinship identification method, an electronic device, and a storage medium. The model training method includes: constructing a family face feature extraction model for extracting family face features from a family image sample; wherein the family image sample includes multiple face images belonging to the same family; the family face feature extraction model is provided with a genetic type parameter for fusing the individual face features of the multiple face images to form the family face feature; the genetic type parameter represents the degree of dominance of a preset genetic type in the family to which the family image sample belongs; constructing a loss function based on the distance between the family face feature of each family and the corresponding family label and the distance between the family face feature and the individual face feature of each face image in the corresponding family image sample, and training the family face feature extraction model.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of face recognition, and in particular to a model training method, a kinship recognition method, an electronic device, and a storage medium. Background Art

[0002] Kinship recognition based on facial images is currently widely used in fields such as paternity testing and missing child tracing. Currently, a commonly used technical solution for kinship recognition is kinship recognition based on a convolutional neural network model. This method inputs multiple training image sets into a convolutional neural network and trains it using a binary classification loss function to produce a kinship recognition model.

[0003] However, this method only relies on the training of the binary classification loss function, and the supervision signal of the training process is weak, resulting in low recognition accuracy. Summary of the Invention

[0004] The purpose of the implementation method of this application is to provide a model training method, a kinship identification method, an electronic device and a storage medium, which obtains the family facial features of a family by deeply mining the genetic characteristics of each organ in the family, and effectively improves the accuracy of kinship identification based on the family facial features.

[0005] To solve the above technical problems, an embodiment of the present application provides a model training method, including: constructing a family facial feature extraction model for extracting family facial features from family image samples; wherein the family image samples include multiple facial images belonging to the same family, and the family facial feature extraction model is provided with a genetic type parameter for fusing the individual facial features of the multiple facial images to form the family facial features, and the genetic type parameter represents the degree of dominance of a preset genetic type in the family to which the family image sample belongs; constructing a loss function based on the distance between the family facial features of each family and the corresponding family label and the distance between the family facial features and the facial features of each facial image in the corresponding family image sample, and training the family facial feature extraction model; wherein the facial features of each facial image in the family image sample are obtained through the trained individual facial feature extraction model.

[0006] An embodiment of the present application also provides a method for identifying kinship relationships, including: obtaining facial features of a facial image to be tested; obtaining family image data of a family to be matched, and inputting the family image data of the family to be matched into a trained family facial feature extraction model to obtain family facial features of the family to be matched, wherein the family facial feature extraction model is obtained by the model training method as described above; comparing the family facial features of the family to be matched with the facial features of the facial image to be tested to determine the family to which the facial image to be tested belongs.

[0007] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method mentioned in the above embodiment, or can execute the kinship identification method mentioned in the above embodiment.

[0008] The embodiments of the present application also provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the model training method mentioned in the above embodiments, or can execute the kinship identification method mentioned in the above embodiments.

[0009] The model training method provided in the embodiment of the present application inputs multiple facial images of the same family into a family facial feature extraction model, extracts individual facial features of the facial images, and fuses multiple individual facial features to form family facial features through the genetic type parameters in the family facial feature extraction model. That is to say, the present application uses the genetic type parameters that represent the degree of dominance of the preset genetic type in the family to focus on the individual facial features involved in the genetic type with a stronger degree of dominance in the family facial features, while weakening the individual facial features involved in the genetic type with a weaker degree of dominance. In this way, the genetic relationship between multiple facial images of each family is fully explored, so that the obtained family facial features not only cover all individual facial features in the family image sample, but also give some emphasis to different individual facial features according to the degree of genetic dominance. In addition, the present application constructs a loss function based on the distance between the family facial features of each family and the corresponding family label and the distance between the family facial features and the individual facial features of each facial image in the corresponding family image sample, trains the family facial feature extraction model, and continuously corrects the family facial features through multiple trainings, so that the family facial features are more in line with the actual situation of the family, thereby improving the accuracy of the family facial features.

[0010] The kinship identification method provided in the embodiment of the present application obtains family image data of the family to be matched, obtains the family facial features of the family to be matched through a trained family facial feature extraction model, and compares the facial features of the facial image to be tested with the family facial features of the family to be matched to determine the family to which the facial image to be tested belongs, thereby avoiding the problem of long time and low recognition efficiency caused by the need to compare the facial image to be tested with each facial image in the family to be matched. At the same time, since the family facial features cover the facial features of all facial images of the family, they can be determined in one comparison, and the recognition accuracy is high.

[0011] In addition, the model training method provided in the embodiment of the present application, the preset genetic type parameters are obtained by the following steps: facial images belonging to the same family are respectively combined according to the preset genetic types to obtain a genetic image group of images involved in each of the genetic types; facial image pairs that meet the corresponding genetic type are selected from the genetic image group, and the similarity between the facial images in each of the facial image pairs is calculated to obtain multiple similarity values; the genetic type parameters of the genetic type to which the genetic image group belongs are determined based on the similarity values. The present application groups the images involved into image pairs according to the genetic characteristics of different genetic types, explores the number of image pairs that meet the genetic characteristics in the family by calculating the similarity between the image pairs, and then calculates and quantifies the more prominent genetic types of each family, thereby providing a theoretical basis for the subsequent fusion of individual facial features, so that the obtained family facial features are more consistent with the family's genetic characteristics.

[0012] In addition, the model training method provided by the embodiment of the present application, the feature fusion network includes: a first fusion layer and a second fusion layer; the individual facial features of each facial image in the family image sample are fused through the feature fusion network to form the family facial features, including: using the first fusion layer to screen and combine the individual facial features of each facial image in the family image sample to form multiple feature groups corresponding to preset genetic types, and fusing the maximum number of non-repeating individual facial features contained in each feature group to form a fused facial feature corresponding to each feature group; the features contained in each feature group are at least one group of individual facial feature pairs that meet the corresponding genetic type; using the second fusion layer to fuse each of the fused facial features to form a family facial feature; wherein, during the fusion process, the weight of each fused facial feature in the family facial feature is adjusted by the genetic type parameter corresponding to each fused facial feature; the loss function used for training the second fusion layer is constructed based on the distance between the family facial feature and the corresponding family label and the distance between the family facial feature and the facial features of each facial image in the corresponding family image sample. This application obtains family facial features by fusing individual facial features twice. In the first fusion, multiple individual facial features in the feature group are fused to obtain fused facial features. In the second fusion, the fused facial features of feature groups corresponding to different genetic types are fused according to genetic type parameters. In this way, individual facial features belonging to the same genetic type are fused, avoiding the influence of feature fusion without genetic relationship on the accuracy of family facial features. The fusion of fused facial features of different genetic types fully considers the genetic complexity and diversity between family facial images, which is more in line with natural genetic laws.

[0013] In addition, the model training method provided by the embodiment of the present application, wherein the maximum number of non-repeating individual facial features contained in each of the feature groups are fused to form fused facial features corresponding to each feature group, includes: weighted fusion of the maximum number of non-repeating individual facial features contained in each of the feature groups according to the preset weight of each individual facial feature to form fused facial features corresponding to each feature group; wherein the preset weight is determined based on the number of repetitions of the individual facial feature in the feature group, and the higher the number of repetitions, the greater the weight. The feature group of the present application is composed of multiple feature pairs, and the weight of an individual facial feature during fusion is determined according to the number of feature pairs in which the feature is located (i.e., the number of repetitions). That is to say, the higher the number of repetitions of an individual facial feature, the more individual facial features that have a genetic relationship with it, the more important the individual facial feature is in the sample, and the more genetic information it covers, and thus the greater the weight during fusion, and the more stable and reliable the fused facial feature obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0015] Figure 1 is a flowchart of the model training method provided by the embodiments of this application;

[0016] Figure 2 is a flowchart of a kinship identification method provided by an embodiment of the present application;

[0017] Figure 3 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] To make the purpose, technical solutions, and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that many technical details are provided in each embodiment of the present application to help readers better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0019] The following examples illustrate the implementation details of the model training of this embodiment. The following content is only provided for the convenience of understanding and is not required for the implementation of this solution.

[0020] The embodiment of the present application relates to a model training method, such as Figure 1 Shown, including:

[0021] Step 101, constructing a family facial feature extraction model for extracting family facial features from family image samples; wherein the family image samples include multiple facial images belonging to the same family, and the family facial feature extraction model is provided with a genetic type parameter for fusing individual facial features of the multiple facial images to form family facial features, and the genetic type parameter represents the degree of dominance of a preset genetic type in the family to which the family image sample belongs.

[0022] In this embodiment, family image samples can be constructed by family face image sets. Each family corresponds to a family face image set, and each family face image set contains multiple face images corresponding to different members of the family. For example, the face image set of family A is represented by A=(a1, a2, ..., a n ), the B family face image set is expressed as B = (b1, b2, ..., b n ), multiple face images are selected from the family face image set to form a family image sample. For example: Family A image sample 1, Family A image sample 2, Family A image sample 3, Family B image sample 1, Family B image sample 2, Family B image sample 3.

[0023] In one embodiment, the preset genetic type parameters are obtained by the following steps: facial images belonging to the same family are combined according to preset genetic types to obtain genetic image groups of images involved in each genetic type; facial image pairs that meet the corresponding genetic type are selected from the genetic image groups, and the similarity between the facial images in each facial image pair is calculated to obtain multiple similarity values; and the genetic type parameters of the genetic type to which the genetic image group belongs are determined based on the similarity values.

[0024] The images involved in a genetic type can be all possible images that meet the corresponding genetic type, that is, each of these images can correspond to at least one other image to form the kinship relationship required for the genetic type; the preset genetic types include at least: male inheritance, female inheritance, cross inheritance, and skipping inheritance. Male inheritance refers to the inheritance of relevant genes or relevant characteristics from male to male, that is, from father to son, and from son to grandson. Female inheritance refers to the inheritance of relevant genes or relevant characteristics from female to female, that is, from mother to daughter, and then from daughter to the next generation of females. Skipping inheritance means that relevant genes or relevant characteristics are not inherited in every generation, but only after two or three generations, for example: from father to granddaughter, from mother to grandson, from father to grandson, and so on. Cross inheritance refers to the transmission of relevant genes or relevant characteristics from one generation to the next generation of different genders, such as from a male to his daughter, or from a female to her son.

[0025] When the genetic type corresponding to the genetic image group is male inheritance, the facial image pairs that meet the corresponding genetic type are selected from the genetic image group as image pairs consisting of a middle-aged male facial image and an elderly male facial image, and / or image pairs consisting of a middle-aged male facial image and a male child facial image; when the genetic type corresponding to the genetic image group is female inheritance, the facial image pairs that meet the corresponding genetic type are selected from the genetic image group as image pairs consisting of a middle-aged female facial image and an elderly female facial image, and / or image pairs consisting of a middle-aged female facial image and a female child facial image; when the genetic type corresponding to the genetic image group is female inheritance, the facial image pairs that meet the corresponding genetic type are selected from the genetic image group as image pairs consisting of a middle-aged female facial image and an elderly female facial image, and / or image pairs consisting of a middle-aged female facial image and a female child facial image; When the inheritance type is cross-inheritance, the facial image pairs that meet the corresponding genetic type are selected from the genetic image group as image pairs consisting of male child facial images and female middle-aged facial images, and / or image pairs consisting of female child facial images and male middle-aged facial images, and / or image pairs consisting of male middle-aged facial images and female elderly facial images, and / or image pairs consisting of female middle-aged facial images and male elderly facial images; when the inheritance type of the genetic image group is skipped-generation inheritance, the facial image pairs that meet the corresponding genetic type are selected from the genetic image group as image pairs consisting of child facial images and elderly facial images.

[0026] Specifically, the genetic image group is obtained by screening and combining facial images of the same family according to the genetic characteristics of each genetic type, and obtaining multiple image groups. Each image group includes multiple image pairs that meet the genetic characteristics. The similarity between the two facial images in each image pair is calculated to obtain multiple similarity values, and the genetic type parameters of the genetic image group are calculated based on the multiple similarity values.

[0027] It should be noted that a family has a set of the aforementioned genetic type parameters (including the first genetic type parameter, the second genetic type parameter, etc.), and the facial images involved in calculating the genetic type parameters are all facial images in the family facial image set of the family. This application calculates the genetic type parameters to explore which genetic characteristics are most prominent and which are more common in the family, thereby determining the weights of different individual facial features when forming the family facial characteristics.

[0028] For example, the genetic type corresponding to the first genetic image group is male inheritance, the genetic type corresponding to the second genetic image group is female inheritance, the genetic type corresponding to the third genetic image group is cross-inheritance, and the genetic type corresponding to the fourth genetic image group is skipped-generation inheritance; all facial images in the first genetic image group are male, all facial images in the second genetic image group are female, the facial images in the third genetic image group include all males and all females, and the facial images in the fourth genetic image group include all children's facial images and all elderly people's facial images. For the first image group, based on the genetic characteristics of father-to-son and son-to-grandson transmission, middle-aged male facial images and elderly male facial images are combined into image pairs, and middle-aged male facial images and male children's facial images are combined into image pairs. The similarity between the two facial images in each image pair is calculated, and then the average of the multiple similarity values ​​is used as the genetic type parameter of the genetic type (male inheritance) of the first genetic image group. The genetic type parameter calculation method for other genetic image groups is similar.

[0029] In addition, multiple facial images in a family image sample may belong to the same person, or may belong to at least two different people in the same family. That is, multiple facial images in a family image sample may belong to the same person, or some of the facial images may belong to different people, or all of the facial images may belong to different people. For example, family image sample A1 includes four facial images, represented by A1 = (a1, a2, a3, a4), and these four facial images are facial images of different people; family image sample A2 includes four facial images, represented by A2 = (a1, a1, a3, a4), and of these four facial images, two belong to the same person, and two belong to different people. It should be noted that multiple facial images belonging to the same person can be facial images taken from different angles, at different ages, with different resolutions, and with different noise levels. This embodiment constructs multiple samples of different composition types, including samples composed of facial images of different people and samples composed of multiple images of the same face. This not only expands the amount of training data, but also improves the stability of the model, so that the model can extract accurate family facial features when facing various types of family image samples.

[0030] Step 102 , constructing a loss function based on the distance between the family facial features of each family and the corresponding family label and the distance between the family facial features and the individual facial features of each facial image in the corresponding family image sample, and training the family facial feature extraction model.

[0031] In this embodiment, the present application inputs multiple family image samples of a family into the family facial feature extraction model to obtain family facial features. Therefore, the family facial features of the family also include multiple family facial features, and the number of family facial features is consistent with the number of family image samples of the family. The purpose of training the family facial feature extraction model is to make the obtained family facial features as close as possible to the individual facial features of each facial image in the corresponding family image samples, and the family facial features are as close as possible to the family label of the corresponding family. For example: Family A includes 4 family image samples A1 = (a1, a2, a3, a4), A2 = (a1, a3, a7, a9), A3 = (a1, a1, a8, a9), A4 = (a2, a2, a2, a2), and the family label of Family A is F A 0, each family image sample gets a family face feature F A 1. F A 2. F A 3. F A 4. The individual facial features of each face image in the family image sample A1 are f1, f2, f3, and f4. The facial feature representation methods of the face images in the other image samples are similar. Then the loss function is the family facial feature F A 1 is as close as possible to each of the individual face features f1, f2, f3, f4, and the family face feature F A 2 is as close as possible to each of the individual face features f1, f3, f7, f9, and the family face feature F A 3 is as close as possible to each of the individual face features f1, f8, f9, and the family face feature F A 4 is as close as possible to the individual face feature f2 and the family face feature F A 1. F A 2. F A 3. F A Each of the 4 is associated with the family label F A 0 as close as possible.

[0032] Specifically, the loss function used in the training of the family face feature extraction model in this embodiment is:

[0033]

[0034]

[0035] in, i F is the family face feature of the i-th family, i f j is the individual facial feature of the jth image of the ith family, and n in formula (1) is the number of facial images in the family image sample corresponding to the family facial feature.i F * is the family label of the i-th family, i F s is the family face feature of the sth family image sample of the ith family, and n in formula (2) is the number of family image samples of the ith family.

[0036] It should be noted that during the training process of the family facial feature extraction model, the family facial feature extraction model is connected to a classifier to classify the extracted features and implement the loss constraint of formula (2). When the family facial feature extraction model is applied, the classifier can be connected or not, depending on the actual application scenario. Generally speaking, if a classifier is connected during application, it will be different from the classifier used during training, and the family facial feature data registered in the classifier during application will be more and more extensive.

[0037] The specific classification algorithm used by the classifier can be any one of a supervised classification algorithm, an unsupervised classification algorithm, a semi-supervised classification algorithm, and the like. Supervised classification means that all samples used to train the classifier have been labeled manually or by other means, that is, the family label of each family in this application has been labeled in advance by manual or other means. Unsupervised classification means that all samples are not labeled, and the classification algorithm needs to use the sample's own information to complete the classification learning task. This method is usually called clustering, that is, the family label of each family in this application needs to be learned and labeled by the classifier itself. Semi-supervised classification means that only a part of the training samples have class labels, and the classification algorithm needs to use labeled samples and unlabeled samples to learn classification at the same time. The result of training with two types of samples is better than training with only labeled samples.

[0038] In one embodiment, a family facial feature extraction model includes: a feature extraction network and a feature fusion network; constructing a family facial feature extraction model for extracting family facial features from family image samples, including: using the feature extraction network to extract individual facial features of each facial image in the family image sample; fusing the individual facial features of each facial image in the family image sample through the feature fusion network to form a family facial feature; wherein, during the fusion process, the weight of each individual facial feature in the family facial feature is adjusted by the pre-set genetic type parameters of the family to which the family image sample belongs.

[0039] The feature extraction network extracts individual facial features from each face image in the family image sample, and the feature fusion network fuses multiple individual facial features to obtain family facial features. Specifically, the feature extraction network can adopt any one of the following structures, including convolutional neural network, residual network, transformer network, or a combination thereof. The feature fusion network can adopt a feature pyramid network, ParseNet network, VGG19 network, or the like.

[0040] Furthermore, the feature fusion network includes: a first fusion layer and a second fusion layer; the individual facial features of each facial image in the family image sample are fused through the feature fusion network to form a family facial feature, including: using the first fusion layer to screen and combine the individual facial features of each facial image in the family image sample to form multiple feature groups corresponding to the preset genetic type, and fusing the maximum number of non-repeating individual facial features contained in each feature group to form a fused facial feature corresponding to each feature group; the features contained in each feature group are at least one group of individual facial feature pairs that meet the corresponding genetic type; using the second fusion layer to fuse each fused facial feature to form a family facial feature; wherein, during the fusion process, the weight of each fused facial feature in the family facial feature is adjusted by the genetic type parameter corresponding to each fused facial feature; the loss function used for training the second fusion layer is constructed based on the distance between the family facial feature and the corresponding family label and the distance between the family facial feature and the facial feature of each facial image in the corresponding family image sample.

[0041] In this embodiment, similar to the genetic image group, multiple individual facial features are screened and combined to form multiple feature groups corresponding to the genetic types. For example, the family image sample of family A includes 10 facial images A1, A2, A3, ..., A10. The corresponding individual facial features are f1, f2, f3, ..., f10. Assuming that the preset genetic types include male inheritance, female inheritance and cross inheritance, three feature groups are formed accordingly, namely the first feature group (corresponding to male inheritance), the second feature group (corresponding to female inheritance) and the third feature group (corresponding to cross inheritance). For the first feature group, the individual facial features (f1, f3, f8) corresponding to the male middle-aged face images (assuming A1, A3, A8) and the individual facial features (f2, f6) corresponding to the male elderly face images (assuming A2, A6) are combined into multiple feature pairs, and the individual facial features (f1, f3, f8) corresponding to the male middle-aged face images (A1, A3, A8) and the individual facial features (f10) corresponding to the male child face images (assuming A10) are combined into multiple feature pairs. It can be understood that there may be overlapping individual facial features between different feature pairs. The combination method of multiple feature pairs contained in other feature groups is similar.

[0042] The maximum number of non-repeating individual facial features within a feature group is fused to form a fused facial feature corresponding to each feature group. Specifically, individual facial features f1, f3, f8, f2, f6, and f10 within the first feature group are fused to form a first fused facial feature. Similarly, the maximum number of non-repeating individual facial features within the second feature group is fused to form a second fused facial feature, and the maximum number of non-repeating individual facial features within the third feature group is fused to form a third fused facial feature. A second fusion layer is then used to fuse the first, second, and third fused facial features based on the genetic type parameters corresponding to each fused facial feature to form a family facial feature. Specifically, during fusion, the first fused facial feature is weighted by the male genetic type parameter, the second fused facial feature is weighted by the female genetic type parameter, and the third fused facial feature is weighted by the cross-genetic type parameter.

[0043] It is worth mentioning that, when forming a feature group, the present application can select some genetic types from the preset genetic types, and form a feature group according to the genetic characteristics of the selected genetic types. The specific selection method can include two kinds. The first kind is to arrange the family genetic type parameters from large to small, and select the genetic types corresponding to the preset number from the first order to form a feature group. For example, if the preset number is 3, the genetic types corresponding to the first (maximum value), the second (second largest value), and the third (third largest value) are selected to form three feature groups. The second kind is to select the genetic types whose genetic type parameters are greater than the preset threshold to form a feature group.

[0044] In addition, the maximum number of non-repeating individual facial features contained in each feature group are fused to form a fused facial feature corresponding to each feature group, including: weighted fusion of the maximum number of non-repeating individual facial features contained in each feature group according to a preset weight of each individual facial feature to form a fused facial feature corresponding to each feature group; wherein the preset weight is determined based on the number of repetitions of the individual facial features in the feature group, and the higher the number of repetitions, the greater the weight. For example, if a feature group contains four feature pairs, including: f1-f2, f1-f5, f1-f7, and f2-f3, when fusing the individual facial features f1, f2, f3, f5, and f7 in the feature group, assuming that the weight of the individual facial features increases by 0.15 each time the individual facial features are repeated, the weight of f1 is 0.45 (f1 is repeated three times), the weight of f2 is 0.3 (f2 is repeated twice), and the weights of f3, f5, and f7 are all 0.15 (f3, f5, and f7 are all repeated once).

[0045] It should be noted that the more times an individual facial feature is repeated within a feature group, the more individual facial features that may have a genetic relationship with it, and the more important the individual facial feature is in the family image sample. Therefore, the greater the weight during fusion, the more stable and reliable the obtained fused facial feature.

[0046] The model training method provided in the embodiment of the present application inputs multiple facial images of the same family into a family facial feature extraction model, extracts individual facial features of the facial images, and fuses multiple individual facial features to form family facial features through the genetic type parameters in the family facial feature extraction model. That is to say, the present application uses the genetic type parameters that represent the degree of dominance of the preset genetic type in the family to focus on the individual facial features involved in the genetic type with a stronger degree of dominance in the family facial features, while weakening the individual facial features involved in the genetic type with a weaker degree of dominance. In this way, the genetic relationship between multiple facial images of each family is fully explored, so that the obtained family facial features not only cover all individual facial features in the family image sample, but also give some emphasis to different individual facial features according to the degree of genetic dominance. In addition, the present application constructs a loss function based on the distance between the family facial features of each family and the corresponding family label and the distance between the family facial features and the individual facial features of each facial image in the corresponding family image sample, trains the family facial feature extraction model, and continuously corrects the family facial features through multiple trainings, so that the family facial features are more in line with the actual situation of the family, thereby improving the accuracy of the family facial features.

[0047] The embodiment of the present application relates to a method for identifying kinship, such as Figure 2 Shown, including:

[0048] Step 201: Acquire facial features of a face image to be tested.

[0049] In this embodiment, the facial features of the facial image to be tested can be obtained according to other models, or can be obtained according to the feature extraction network in the family facial feature extraction model of this application. Specifically, the facial feature extraction model or network can adopt any of the existing feature extraction models, such as: convolutional neural network model, residual network model, feature extraction model based on attention mechanism, etc.

[0050] It should be noted that the feature extraction network used to obtain the facial features of the face image to be tested should be the same feature extraction network used to obtain the individual facial features of each face image in the corresponding family image sample used in the loss function when training the family face feature extraction model.

[0051] Step 203: Obtain family image data of the family to be matched, and input the family image data of the family to be matched into a trained family facial feature extraction model to obtain family facial features of the family to be matched, wherein the family facial feature extraction model is obtained by the model training method described in the above embodiment.

[0052] Step 203 : Compare the facial features of the family to be matched with the facial features of the facial image to be tested to determine the family to which the facial image to be tested belongs.

[0053] In this embodiment, when comparing the family facial features of the family to be matched with the facial features of the face image to be tested, it can be determined whether the two features belong to the same family using methods such as Mahalanobis distance, Euclidean distance, and cosine distance.

[0054] The kinship identification method provided in the embodiment of the present application obtains family image data of the family to be matched, obtains the family facial features of the family to be matched through a trained family facial feature extraction model, and compares the facial features of the facial image to be tested with the family facial features of the family to be matched to determine the family to which the facial image to be tested belongs, thereby avoiding the problem of long time and low recognition efficiency caused by the need to compare the facial image to be tested with each facial image in the family to be matched. At the same time, since the family facial features cover the facial features of all facial images of the family, they can be determined in one comparison, and the recognition accuracy is high.

[0055] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0056] The embodiments of the present application relate to an electronic device, such as Figure 3 Shown, including:

[0057] At least one processor 301; and a memory 302 communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 to enable the at least one processor 301 to perform the model training mentioned in the above embodiment, or to perform the kinship identification method mentioned in the above embodiment.

[0058] The electronic device includes: one or more processors 301 and a memory 302, Figure 3A processor 301 is used as an example. The processor 301 and the memory 302 may be connected via a bus or other means. Figure 3 The example of a bus connection is taken as an example. Memory 302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. For example, the algorithms corresponding to the various processing strategies in the strategy space in the embodiment of the present application are stored in memory 302. Processor 301 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 302, that is, implements the above-mentioned model training method or kinship recognition method.

[0059] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 302 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 302 may optionally include a memory remotely located relative to the processor 301, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0060] One or more modules are stored in the memory 302 and, when executed by one or more processors 301 , execute the model training method in any of the above embodiments, or can execute the kinship identification method mentioned in the above embodiments.

[0061] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0062] The embodiments of the present application relate to a computer-readable storage medium storing a computer program, which implements the above method embodiments when executed by a processor.

[0063] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0064] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A model training method, characterized in that: include: Constructing a family facial feature extraction model for extracting family facial features from family image samples; wherein the family image samples include multiple facial images belonging to the same family, and the family facial feature extraction model is provided with a genetic type parameter for fusing individual facial features of the multiple facial images to form the family facial features, wherein the genetic type parameter represents the degree of dominance of a preset genetic type in the family to which the family image samples belong; A loss function is constructed based on the distance between the family facial features of each family and the corresponding family label and the distance between the family facial features and the individual facial features of each facial image in the corresponding family image sample to train the family facial feature extraction model.

2. The model training method according to claim 1, characterized in that The genetic type parameters are obtained by the following steps: Combining facial images belonging to the same family according to preset genetic types to obtain genetic image groups of images related to each genetic type; Selecting a pair of facial images that conform to a corresponding genetic type from the genetic image group, and calculating the similarity between the facial images in each of the facial image pairs to obtain a plurality of similarity values; A genetic type parameter of the genetic type to which the genetic image group belongs is determined according to the similarity value.

3. The model training method according to claim 2, characterized in that The inheritance types include: male inheritance, female inheritance, cross inheritance and skipping inheritance; When the genetic type corresponding to the genetic image group is male inheritance, the facial image pair selected from the genetic image group that meets the corresponding genetic type is an image pair consisting of a middle-aged male facial image and an elderly male facial image, and / or an image pair consisting of a middle-aged male facial image and a male child facial image; When the genetic type corresponding to the genetic image group is female, the facial image pair selected from the genetic image group that meets the corresponding genetic type is an image pair consisting of a middle-aged female facial image and an elderly female facial image, and / or an image pair consisting of a middle-aged female facial image and a female child facial image; When the genetic type corresponding to the genetic image group is cross-inheritance, the facial image pairs that meet the corresponding genetic type selected from the genetic image group are image pairs consisting of a male child facial image and a female middle-aged facial image, and / or image pairs consisting of a female child facial image and a male middle-aged facial image, and / or image pairs consisting of a male middle-aged facial image and a female elderly facial image, and / or image pairs consisting of a female middle-aged facial image and a male elderly facial image; When the inheritance type of the genetic image group is skipped-generation inheritance, the face image pair selected from the genetic image group that meets the corresponding inheritance type is an image pair consisting of a child face image and an elderly face image.

4. The model training method according to claim 1, characterized in that The family face feature extraction model includes: a feature extraction network and a feature fusion network; the family face feature extraction model constructed for extracting family face features from family image samples includes: extracting individual facial features of each facial image in the family image sample using the feature extraction network; The individual facial features of each facial image in the family image sample are fused through the feature fusion network to form the family facial features; wherein, during the fusion process, the weight of each individual facial feature in the family facial features is adjusted by the pre-set genetic type parameters of the family to which the family image sample belongs.

5. The model training method according to claim 4, characterized in that The feature fusion network includes: a first fusion layer and a second fusion layer; The step of fusing the individual facial features of each facial image in the family image sample through the feature fusion network to form the family facial features includes: The first fusion layer is used to screen and combine the individual facial features of each facial image in the family image sample to form a plurality of feature groups corresponding one to one with the preset genetic types, and a maximum number of non-repeated individual facial features contained in each feature group are fused to form fused facial features corresponding to each feature group; the features contained in each feature group are at least one set of individual facial feature pairs that satisfy the corresponding genetic type; The second fusion layer is used to fuse the fused facial features to form a family facial feature; wherein, during the fusion process, the weight of each fused facial feature in the family facial feature is adjusted by the genetic type parameter corresponding to each fused facial feature; the loss function used for training the second fusion layer is constructed based on the distance between the family facial feature and the corresponding family label and the distance between the family facial feature and the facial features of each facial image in the corresponding family image sample.

6. The model training method according to claim 5, characterized in that The fusing of the maximum number of non-repeated individual facial features contained in each of the feature groups to form fused facial features corresponding to each feature group includes: The maximum number of non-repeating individual facial features contained in each of the feature groups are weightedly fused according to the preset weight of each individual facial feature to form a fused facial feature corresponding to each feature group; wherein the preset weight is determined based on the number of repetitions of the individual facial feature in the feature group, and the higher the number of repetitions, the greater the weight.

7. The model training method according to claim 1, characterized in that The multiple facial images in the family image sample are facial images belonging to the same person, or are facial images belonging to at least two different persons in the same family.

8. A method for identifying kinship, characterized in that: include: Obtaining facial features of the face image to be tested; Acquiring family image data of the family to be matched, and inputting the family image data of the family to be matched into a trained family facial feature extraction model to obtain family facial features of the family to be matched, wherein the family facial feature extraction model is obtained by the model training method according to any one of claims 1 to 7; The family facial features of the family to be matched are compared with the facial features of the facial image to be tested to determine the family to which the facial image to be tested belongs.

9. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the model training method as described in any one of claims 1 to 7, or to execute the kinship identification method as described in claim 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the model training method according to any one of claims 1 to 7, or implements the kinship identification method according to claim 8.

Images