Relative relationship verification method and device, computing equipment and storage medium

Abstract

The invention provides a relativity verification method and device, computing equipment and a storage medium, and the method comprises the steps: obtaining at least two images, inputting each image into a feature extraction model, obtaining the output features of a plurality of specified residual blocks of each image and the global character features of each image, and carrying out the verification of the relativity for any image, combining output features of a plurality of specified residual blocks of the image to obtain a first combined feature, inputting the first combined feature into a local attention model to obtain a character local feature, and combining the character local feature of each image and the character global feature of each image to obtain a second combined feature; and inputting the second combined features into a relativity verification model to obtain a relativity verification result among the figures. And extracting the character local features with significant differences of the images, combining the character local features with the character global features, and inputting the combined character local features and character global features into a relative relationship verification model to obtain an accurate relative relationship verification result.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a kinship verification method, a kinship verification device, a computing device, and a computer-readable storage medium. Background Art

[0002] Kinship verification is an important image recognition technology. Kinship verification refers to the use of image recognition technology to verify whether the people in the image to be tested are related. Kinship verification can help people find missing children, deal with case disputes, etc. Kinship verification is currently a very challenging research field and has attracted widespread attention from researchers. Its research results can be widely applied to various fields such as national economy and people's livelihood. How to efficiently and accurately obtain the kinship between different people has become the research focus of the above fields. Summary of the invention

[0003] In view of this, the embodiment of the present application provides a kinship verification method to solve the technical defects existing in the prior art. The embodiment of the present application also provides a kinship verification device, a computing device, and a computer-readable storage medium.

[0004] According to a first aspect of an embodiment of the present application, a kinship verification method is provided, comprising: Acquire at least two images, wherein each image includes a different person; Input each image into the feature extraction model to obtain output features of multiple specified residual blocks of each image and global features of the person in each image; For any image, the output features of multiple specified residual blocks of the image are combined to obtain a first combined feature, and the first combined feature is input into a local attention model to obtain a local feature of the person, wherein the local attention model is a multi-head attention model; Combining the local features of the characters in each image and the global features of the characters in each image to obtain a second combined feature; The second combined feature is input into the kinship verification model to obtain the kinship verification results between the characters.

[0005] Optionally, the step of acquiring at least two images comprises: Acquire at least two initial images, wherein each initial image includes a different person; Based on the characters, each initial image is preprocessed to obtain at least two images.

[0006] Optionally, there are multiple feature extraction models, and each feature extraction model has the same model structure and the same model parameters; The steps of inputting each image into a feature extraction model to obtain output features of a plurality of specified residual blocks of each image and global features of a person in each image include: Each image is input into each feature extraction model in parallel to obtain output features of multiple specified residual blocks of each image and global features of the person in each image.

[0007] Optionally, for any image, the step of combining output features of multiple specified residual blocks of the image to obtain a first combined feature includes: For any image, output features of multiple specified residual blocks of the image are input into the channel layer, and the output features of the multiple specified residual blocks of the image are cross-combined through the channel layer to obtain a first combined feature.

[0008] Optionally, the local attention model includes a spatial attention sub-model and an attention learning sub-model; The step of inputting the first combined feature into the local attention model to obtain the local feature of the person includes: Perform channel shuffling on the first combination feature; Use 1*1 convolution kernel to fuse the first combination features after channel shuffling to obtain fused features; Split the fused features to obtain a preset number of split features; Input each split feature into the spatial attention sub-model respectively to obtain the spatial attention matrix corresponding to each split feature; Each spatial attention matrix is ​​input into the attention learning sub-model respectively to obtain multiple local features of the character.

[0009] Optionally, the step of inputting each spatial attention matrix into the attention learning sub-model to obtain multiple character local features includes: Performing a column-wise sum operation on the first spatial attention matrix, and constructing a second spatial attention matrix based on the summation result, wherein the first spatial attention matrix is ​​any one of the spatial attention matrices; Transpose the second spatial attention matrix to obtain a third spatial attention matrix; Determine the maximum value in the third spatial attention matrix as the matrix center of the first spatial attention matrix; According to the matrix center of each spatial attention matrix, under the constraint of a preset loss function, the local features of the character corresponding to each spatial attention matrix are determined respectively, wherein the constraints of the preset loss function include the aggregation of each weight in the same spatial attention matrix to the matrix center of the spatial attention matrix, and the separation of matrix centers of different spatial attention matrices from each other.

[0010] Optionally, the step of combining the local features of the person in each image and the global features of the person in each image to obtain a second combined feature includes: Combine k local features of characters in each image to obtain k local combined features of characters; Combine the global features of the characters in each image to obtain a global combined feature of the character; The k local combined features of the characters and the global combined features of the characters are combined to obtain a second combined feature.

[0011] Optionally, the training method of the kinship verification model includes: Obtaining a training sample set, wherein the training sample set includes a plurality of training images, and the training images include a sample person and a sample family label and a sample person label of the sample person; Dividing the training sample set to obtain a plurality of training subsets, wherein each training subset includes a plurality of training images; Input each training image in the first training subset into the initial verification model, obtain the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image, wherein the class center is updated based on a preset class center update function during the calculation process, and the first training subset is any one of the multiple training subsets; According to the distance information between each training image and the class center in the first training subset, a family-level multi-center loss value is calculated using a preset family-level multi-center loss function; Calculate the household classification loss value based on the predicted household category and sample household label of each training image; Calculate the person classification loss value based on the predicted person category and sample person label of each training image; The total loss value is calculated based on the household-level multi-center loss value, household classification loss value, and person classification loss value; Based on the total loss value, the model parameters of the initial verification model are adjusted, and the step of inputting each training image in the first training subset into the initial verification model is returned to execute, and the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image are obtained, until the preset training stop condition is reached, and the trained kinship verification model is obtained.

[0012] According to a second aspect of an embodiment of the present application, a kinship verification device is provided, including: An acquisition module is configured to acquire at least two images, wherein each image includes a different person; A feature extraction module is configured to input each image into a feature extraction model to obtain output features of a plurality of specified residual blocks of each image and global features of a person in each image; An input module is configured to combine output features of multiple specified residual blocks of any image to obtain a first combined feature, and input the first combined feature into a local attention model to obtain a local feature of a person, wherein the local attention model is a multi-head attention model; A feature combination module is configured to combine the local features of the characters in each image and the global features of the characters in each image to obtain a second combined feature; The verification module is configured to input the second combined feature into a kinship verification model to obtain kinship verification results between the characters.

[0013] Optionally, the acquisition module is further configured to: Acquire at least two initial images, wherein each initial image includes a different person; Based on the characters, each initial image is preprocessed to obtain at least two images.

[0014] Optionally, there are multiple feature extraction models, and each feature extraction model has the same model structure and the same model parameters; The feature extraction module is further configured as follows: Each image is input into each feature extraction model in parallel to obtain output features of multiple specified residual blocks of each image and global features of the person in each image.

[0015] Optionally, the input module is further configured to: For any image, output features of multiple specified residual blocks of the image are input into the channel layer, and the output features of the multiple specified residual blocks of the image are cross-combined through the channel layer to obtain a first combined feature.

[0016] Optionally, the local attention model includes a spatial attention sub-model and an attention learning sub-model; The input module is further configured as: Perform channel shuffling on the first combination feature; Use 1*1 convolution kernel to fuse the first combination features after channel shuffling to obtain fused features; Split the fused features to obtain a preset number of split features; Input each split feature into the spatial attention sub-model respectively to obtain the spatial attention matrix corresponding to each split feature; Each spatial attention matrix is ​​input into the attention learning sub-model respectively to obtain multiple local features of the character.

[0017] Optionally, the input module is further configured to: Performing a column-wise sum operation on the first spatial attention matrix, and constructing a second spatial attention matrix based on the summation result, wherein the first spatial attention matrix is ​​any one of the spatial attention matrices; Transpose the second spatial attention matrix to obtain a third spatial attention matrix; Determine the maximum value in the third spatial attention matrix as the matrix center of the first spatial attention matrix; According to the matrix center of each spatial attention matrix, under the constraint of a preset loss function, the local features of the character corresponding to each spatial attention matrix are determined respectively, wherein the constraints of the preset loss function include the aggregation of each weight in the same spatial attention matrix to the matrix center of the spatial attention matrix, and the separation of matrix centers of different spatial attention matrices from each other.

[0018] Optionally, the feature combination module is further configured to: Combine k local features of characters in each image to obtain k local combined features of characters; Combine the global features of the characters in each image to obtain a global combined feature of the character; The k local combined features of the characters and the global combined features of the characters are combined to obtain a second combined feature.

[0019] Optionally, the device further comprises: a training module; The training module is configured as follows: Obtaining a training sample set, wherein the training sample set includes a plurality of training images, and the training images include a sample person and a sample family label and a sample person label of the sample person; Dividing the training sample set to obtain a plurality of training subsets, wherein each training subset includes a plurality of training images; Input each training image in the first training subset into the initial verification model, obtain the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image, wherein the class center is updated based on a preset class center update function during the calculation process, and the first training subset is any one of the multiple training subsets; According to the distance information between each training image and the class center in the first training subset, a family-level multi-center loss value is calculated using a preset family-level multi-center loss function; Calculate the household classification loss value based on the predicted household category and sample household label of each training image; Calculate the person classification loss value based on the predicted person category and sample person label of each training image; The total loss value is calculated based on the household-level multi-center loss value, household classification loss value, and person classification loss value; Based on the total loss value, the model parameters of the initial verification model are adjusted, and the step of inputting each training image in the first training subset into the initial verification model is returned to execute, and the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image are obtained, until the preset training stop condition is reached, and the trained kinship verification model is obtained.

[0020] According to a third aspect of an embodiment of the present application, there is provided a computing device, including: a memory and a processor; Among them, the memory is used to store computer executable instructions, and the above-mentioned kinship verification method is implemented when the processor executes the computer executable instructions.

[0021] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions implement the above-mentioned kinship verification method when executed by a processor.

[0022] According to a fifth aspect of an embodiment of the present application, a chip is provided, which stores computer instructions, and the computer instructions implement the above-mentioned kinship verification method when executed by the chip.

[0023] The kinship verification method provided by the present application obtains at least two images, inputs each image into a feature extraction model, obtains output features of multiple specified residual blocks of each image and global features of the person in each image, then for any image, combines the output features of multiple specified residual blocks of the image to obtain a first combined feature, inputs the first combined feature into a local attention model, obtains local features of the person, then combines the local features of the person in each image and the global features of the person in each image to obtain a second combined feature, inputs the second combined feature into a kinship verification model, and obtains kinship verification results between the persons. Through the above-mentioned kinship verification method based on deep learning, efficient verification of kinship between persons is achieved. By using the local attention model, local features with significant differences in each image can be automatically extracted without the need for image key point detection. The local features of the person with significant differences in each image and the global features of the person are respectively combined and input into the kinship verification model, which can make the output results of the kinship verification model more accurate and ensure the accuracy of the kinship verification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a kinship verification method provided according to an embodiment of the present application is shown; Figure 2 A schematic diagram of the structure of a local attention model provided according to an embodiment of the present application is shown; Figure 3A schematic diagram of the effect of a loss function of an attention learning sub-model provided according to an embodiment of the present application is shown; Figure 4 A distribution diagram of a single-center feature space and a multi-center feature space provided according to an embodiment of the present application is shown; Figure 5 A structural diagram of a kinship verification model provided according to an embodiment of the present application is shown; Figure 6 A schematic diagram of the structure of a kinship verification device provided according to an embodiment of the present application is shown; Figure 7 A structural block diagram of a computing device provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0026] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0027] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0028] First, the terms involved in one or more embodiments of the present invention are explained.

[0029] Residual Network (ResNet): In mathematical statistics, residual refers to the difference between the actual observed value and the estimated value. The residual network is composed of a series of residual blocks. During the training process of the neural network model, under the premise that the neural network can converge, as the depth of the neural network increases, the neural network model will degenerate. Adding a residual network to the neural network model effectively solves the problem of neural network degradation and has been widely used in image recognition, image segmentation and other fields.

[0030] Metric learning: It is a spatial mapping method that can learn a feature space in which all data is converted into a feature vector. The distance between feature vectors of similar samples is small, while the distance between feature vectors of dissimilar samples is large, thereby distinguishing the data.

[0031] Multi-head attention model: The multi-head attention model consists of multiple attention heads, each of which focuses on different features of the input information.

[0032] Face recognition technology is an important biometric technology, which is widely used in various fields of people's daily life. Kinship verification based on face images refers to the use of image processing technology to complete the verification of whether the face images to be tested have a kinship relationship. Usually, 1:1 face kinship verification is to input two face images, and use the kinship verification algorithm to obtain whether the two input images have a kinship relationship. Face kinship verification includes three types of kinship pairs: the kinship verification between grandparents and grandchildren is called the next-generation kinship verification; the kinship verification between parents and children is called the next-generation kinship verification; the kinship verification between brothers and sisters is called the same-generation kinship verification. Face kinship verification is a specific application of face recognition technology. Compared with face recognition, kinship verification is more difficult. Face image data with complete kinship is more difficult to collect, which makes the face kinship verification problem a huge challenge. Face kinship verification has important application value. The use of this technology can help find missing children, analyze family images to extract family relationships, and aggregate family images in albums.

[0033] However, existing face kinship verification methods are all based on metric learning methods. However, using a metric learning feature center for different members of a family will result in all people in the same family being verified as having a kinship relationship. However, there is no kinship relationship between husband and wife, so it is unreasonable to identify them as having a kinship relationship. In the face kinship training, the present application adds a family-level multi-center loss function to each family to separate the family members who do not have kinship characteristics in each family.

[0034] Moreover, due to differences in gender and age, the facial features between relatives extracted using conventional face recognition models have large differences. The conventional method for verifying kinship relationships is to use a metric function to compare pairs of facial features and calculate the differences between the final features. This type of method cannot model the facial features of relatives and has a low accuracy rate.

[0035] Based on the above technical problems, in this application, a kinship verification method is provided. This application also relates to a kinship verification device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0036] Figure 1 A flowchart of a kinship verification method provided according to an embodiment of the present application is shown, which specifically includes the following steps: S102: Acquire at least two images.

[0037] Different persons are included in at least two images, wherein the person refers to an object to be verified for kinship. When verifying the kinship of at least two different persons, it is first necessary to obtain images containing different persons, and then verify the kinship based on at least two images containing different persons. The format of the image can be ".jpg", ".png", ".raw" and other formats, and this specification does not specifically limit the format of the image.

[0038] There are many ways to obtain at least two images. The user can directly input the at least two images into the execution entity for kinship verification, or the user can store the at least two images in a database of a server or terminal in advance. When the user sends a kinship verification instruction to the execution entity, the execution entity obtains the images from the server or the interrupted database for kinship verification. This specification does not impose any limitation on the method of obtaining at least two images.

[0039] In one implementation of the embodiment of this specification, step S102 can be specifically implemented in the following manner: Acquire at least two initial images, wherein each initial image includes a different person; Based on the characters included in each initial image, each initial image is preprocessed to obtain at least two images.

[0040] When verifying the kinship of at least two different persons, it is first necessary to obtain images including different persons as initial images for kinship verification, and then pre-process the at least two initial images obtained based on the persons included in each initial image, so as to obtain at least two images including different persons.

[0041] The reason why at least two initial images need to be preprocessed separately is that each initial image includes image areas other than the characters, and the positions of the characters in the initial images are different, which is not conducive to extracting the features of the characters during the kinship verification process, and thus leads to inaccurate kinship verification results of the images.

[0042] Based on the characters included in each initial image, preprocessing the at least two acquired initial images respectively refers to cropping and correcting the at least two initial images respectively.

[0043] Specifically, after cropping at least two initial images, image areas other than the person in the at least two initial images can be eliminated, and only the image area including the person in each initial image is retained. Specifically, the at least two initial images can be cropped using a face detection algorithm, for example, the at least two initial images can be cropped using a multi-task cascaded convolutional neural network (MTCNN). After cropping at least two initial images, the person in each initial image needs to be corrected, and specifically, the cropped initial images can be straightened using a face key point detection algorithm. For example, for the at least two initial images after cropping, when the face key point detection algorithm is used to detect that the key points in the cropped initial images are not on the same horizontal line, the key points of each initial image are adjusted to the same horizontal line through image processing such as scaling, rotating, and stretching. Through the above method, when the kinship verification is performed on each person, the positions of the person input into the kinship verification model are consistent, which is conducive to the extraction of person features, thereby improving the accuracy of the kinship verification result.

[0044] In the above embodiment, at least two initial images are obtained, and then the initial images are preprocessed based on the characters included in the initial images, thereby obtaining at least two images. Through the above method, each initial image is cropped and corrected respectively, which is conducive to more accurately extracting the feature information of the image when verifying the kinship relationship of at least two different characters, thereby improving the accuracy of the kinship verification result.

[0045] S104, inputting each image into a feature extraction model respectively, obtaining output features of a plurality of designated residual blocks of each image and global features of the person in each image.

[0046] The feature extraction model is a network model used to extract feature information of people in an image. Any network model that can extract feature information of people in an image can be used as a feature extraction model. This application does not limit the type of feature extraction model.

[0047] In this embodiment, the residual network (ResNet) model can be used to extract features of people in the image. Specifically, the residual network structure of ResNet50, which is a combination of a convolutional neural network model (CNN) and ResNet, is used to extract features of people in the image.

[0048] The residual network structure of ResNet50 includes multiple residual blocks, which are used to extract high-level and low-level features of the image, so that the high-level and low-level features of the image can be subsequently fused using the convolution kernel in CNN to obtain the local features of the characters in each image.

[0049] The feature extraction model also includes a global average pooling layer (GAP), which is used to extract the global features of the characters in the image. Using the global average pooling layer to extract the global features of the characters in the image can reduce the parameter information in the feature extraction model and avoid the overfitting of the feature extraction model.

[0050] In an implementation of the embodiment of this specification, there are multiple feature extraction models, and the model structures and model parameters of the feature extraction models are the same. Accordingly, step S104 can be specifically implemented in the following manner: Each image is input into each feature extraction model in parallel to obtain output features of multiple specified residual blocks of each image and global features of the person in each image.

[0051] After preprocessing at least two initial images to obtain at least two images, the execution subject inputs each image into a feature extraction model to obtain output features of multiple specified residual blocks of each image and global features of the person in each image.

[0052] Among them, the execution subject inputs each image into the feature extraction model respectively, and can use a serial method to input each image into the feature extraction model in turn, and based on the feature extraction model, obtain the output features of multiple specified residual blocks of each image and the global features of the characters in each image in turn; or it can simultaneously establish multiple feature extraction models with the same structure and parameters, and then input each image into each feature extraction model in parallel, and then simultaneously obtain the output features of multiple specified residual blocks of each image and the global features of the characters in each image.

[0053] In this embodiment, multiple feature extraction models with the same structure and parameters are established at the same time, wherein the number of feature extraction models depends on the number of images. Then, at least two preprocessed images are input into each feature extraction model in parallel, and high-level features and low-level features of each image are extracted using multiple specified residual blocks in the feature extraction model. The global average pooling layer in the feature extraction model is used to extract the global features of the characters in each image.

[0054] In the above implementation, each image is input into each feature extraction model in parallel to obtain the output features of multiple specified residual blocks of each image and the global features of the characters of each image. By the above method, each image is input into multiple feature extraction models with the same structure and parameters in parallel, so that the output features of multiple specified residual blocks of each image and the global features of the characters of each image can be obtained at the same time, thereby improving the efficiency of character feature extraction; at the same time, each image is input into the global average pooling layer of the feature extraction model to extract the global features of the characters, which can reduce the parameter information in the feature extraction model and avoid the phenomenon of overfitting of the feature extraction model.

[0055] S106. For any image, combine the output features of multiple specified residual blocks of the image to obtain a first combined feature, input the first combined feature into a local attention model, and obtain a local feature of the character.

[0056] After using the feature extraction model to obtain the output features of multiple specified residual blocks of each image, for any image, it is necessary to combine the output features of multiple specified residual blocks of the image, and then input the output features of the combined multiple specified residual blocks into the local attention model, so as to obtain the local features of the characters in each image.

[0057] In this embodiment, the local attention model is a multi-head attention model, which includes multiple attention heads, and each attention head focuses on different output features of multiple specified residual blocks after combination. The multi-head attention model can automatically identify the significant difference areas of the characters in the image without the need to detect key points of the image, and then extract local features for the significant difference areas of different characters in each image, thereby improving the accuracy of kinship verification.

[0058] In one implementation of the embodiment of the present specification, in step S106, for any image, the step of combining output features of multiple specified residual blocks of the image to obtain a first combined feature can be specifically implemented in the following manner: For any image, output features of multiple specified residual blocks of the image are input into the channel layer, and the output features of the multiple specified residual blocks of the image are cross-combined through the channel layer to obtain a first combined feature.

[0059] In this embodiment, after high-level features and low-level features of any image are extracted using multiple specified residual blocks in the feature extraction model, the high-level features and low-level features of any image are input into the channel layer of the feature extraction model for cross-combination, and the cross-combined high-level features and low-level features are obtained as the first combined features.

[0060] In the above-mentioned embodiment, for any image, the high-level features and low-level features of any image extracted by using multiple specified residual blocks in the feature extraction model will be very small and scattered, which is not conducive to further extracting the local features of the image. Therefore, it is necessary to cross-combine the high-level features and low-level features of any image extracted by multiple specified residual blocks at the channel layer in the feature extraction model, so as to obtain the first combined features after combination, which is more conducive to the subsequent extraction of local features of the image.

[0061] In one implementation of the embodiment of the present specification, the local attention model includes a spatial attention sub-model and an attention learning sub-model. Accordingly, in step S106, the step of inputting the first combined feature into the local attention model to obtain the local feature of the person can be specifically implemented in the following manner: Perform channel shuffling on the first combination feature; Use 1*1 convolution kernel to fuse the first combination features after channel shuffling to obtain fused features; Split the fused features to obtain a preset number of split features; Input each split feature into the spatial attention sub-model respectively to obtain the spatial attention matrix corresponding to each split feature; Each spatial attention matrix is ​​input into the attention learning sub-model respectively to obtain multiple local features of the character.

[0062] Figure 2 A structural schematic diagram of a local attention model provided according to an embodiment of the present application is shown, which includes multi-scale combined features, channel layers, 1*1 convolution kernels, 3*3 convolution kernels, spatial attention sub-models (SAM, Spatial Attention Mechanisms), spatial attention matrices and attention learning sub-models (ALM, Attention Learning Module).

[0063] In this embodiment, after obtaining the first combined feature, channel shuffling is performed on the first combined feature, and the 1*1 convolution kernel in the feature extraction model is used to further fuse the first combined feature after channel shuffling to obtain fused features of at least two images, and then the fused features of at least two images are evenly split into a preset number of split features in the channel layer of the feature extraction model, wherein the first combined feature can be a multi-scale combined feature.

[0064] Then, each split feature is input into the spatial attention sub-model respectively, and the multi-head attention mechanism in the spatial attention sub-model is used to obtain the spatial attention matrix corresponding to each split feature. The spatial attention matrix corresponding to each split feature is input into the attention learning sub-model to obtain multiple local features of the characters in at least two images.

[0065] In the above implementation, a 1*1 convolution kernel is used to further fuse the first combined features to obtain fused features, and then the fused features are input into the channel layer and split into a preset number of split features, and each split feature is input into the spatial attention sub-model respectively. The multi-head attention mechanism is used to facilitate the identification of image areas with large differences in at least two images, thereby being more conducive to kinship verification of the images.

[0066] In one implementation of the embodiment of this specification, the step of inputting each spatial attention matrix into the attention learning sub-model to obtain multiple local features of a person can be specifically implemented in the following manner: Performing a column-wise sum operation on the first spatial attention matrix, and constructing a second spatial attention matrix based on the summation result, wherein the first spatial attention matrix is ​​any one of the spatial attention matrices; Transpose the second spatial attention matrix to obtain a third spatial attention matrix; Determine the maximum value in the third spatial attention matrix as the matrix center of the first spatial attention matrix; According to the matrix center of each spatial attention matrix, under the constraint of a preset loss function, the local features of the character corresponding to each spatial attention matrix are determined respectively, wherein the constraints of the preset loss function include the aggregation of each weight in the same spatial attention matrix to the matrix center of the spatial attention matrix, and the separation of matrix centers of different spatial attention matrices from each other.

[0067] Figure 3 A schematic diagram of the effect of a loss function of an attention learning sub-model provided according to an embodiment of the present application is shown.

[0068] In this embodiment, after a preset number of split features are input into the spatial attention module (SAM), the spatial attention matrix corresponding to each split feature is obtained, and then the k spatial attention matrices are respectively input into the attention learning module (ALM), where the spatial attention matrix is ​​the result of processing by the row softmax function. Therefore, each spatial attention matrix A is first l i The column-wise sum operation S is then used to construct the spatial attention matrix based on the summation results. , specifically, construct the spatial attention matrix As shown in formula (1).

[0069] (1) After getting the spatial attention matrix After that, the spatial attention matrix is ​​transposed to obtain the spatial attention matrix : (2) After getting the spatial attention matrix After that, determine the spatial attention matrix The maximum value in is taken as the matrix center of the spatial attention matrix A: (3) Among them, S represents the summation of the spatial attention matrix; R represents the Reshape operation.

[0070] After determining the matrix center of the spatial attention matrix, the local features of the person corresponding to each spatial attention matrix are determined according to the matrix center under the constraint of a preset loss function, wherein the constraints of the preset loss function include that the weights in the same spatial attention matrix are aggregated to the matrix center of the spatial attention matrix, and the matrix centers of different spatial attention matrices are separated from each other. Specifically, the preset loss function Including the attention weight center aggregation loss function And the attention weight center separation loss function Specifically, the preset loss function can be calculated by formulas (4)-(6): (4) and

[0071] (5) (6) in, Represents the preset loss function, which is given by and The sum of the two loss functions; represents the attention weight center aggregation loss function; represents the attention weight center separation loss function; H, W represent the length and width of the spatial attention matrix; i, j represent the i-th and j-th spatial attention matrices in each spatial attention matrix; k represents the total number of spatial attention matrices; x, y represent the coordinates of each element in each spatial attention matrix; , Represents the center coordinates of each spatial attention matrix.

[0072] In the above implementation, by introducing the attention weight center aggregation loss function, the attention weights in each spatial attention matrix can be aggregated toward the center, and the point-to-point attention matrix difference is weighted using the inverse of the distance from each element point to the center coordinate in the spatial attention matrix, so that the pixel points far from the center coordinate of the spatial attention matrix have smaller weights, and the pixel points close to the center coordinate of the spatial attention matrix have larger weights. At the same time, the attention weights of the pixels close to the center coordinate will also be close to the highest center point. By introducing the attention weight center separation loss function, the centers of attention can be separated from each other, which helps to extract the attention feature maps of different positions of the characters in the image.

[0073] S108: Combine the local features of the person in each image and the global features of the person in each image to obtain a second combined feature.

[0074] After the first combined feature is input into the local attention model to obtain multiple local features of the characters, the local features of each character and the global features of the characters of each image obtained by using the global average pooling layer are combined to obtain the second combined feature, so that the second combined feature can be input into the kinship verification model for kinship verification.

[0075] In one implementation of the embodiment of this specification, step S108 can be specifically implemented in the following manner: Combine k local features of characters in each image to obtain k local combined features of characters; Combine the global features of the characters in each image to obtain a global combined feature of the character; The k local combined features of the characters and the global combined features of the characters are combined to obtain a second combined feature.

[0076] In this embodiment, when performing kinship verification on at least two images, k local features of characters and one global feature of characters in the first image and k local features of characters and one global feature of characters in the second image are firstly extracted.

[0077] Then, a total of 2k local features of the characters in the first image and the second image are combined to obtain k local combined features of the characters; a total of 2 global features of the characters in the first image and the second image are combined to obtain 1 global combined feature of the characters; and the k local combined features of the characters and the 1 global combined feature of the characters are combined to obtain the second combined feature.

[0078] Through the above method, the local features of each character and the global features of the character are combined to obtain a second combined feature, and input into the kinship verification model, which can make the output result of the kinship verification model more accurate and improve the accuracy of kinship verification between images.

[0079] S110, inputting the second combined feature into a kinship verification model to obtain kinship verification results between the characters.

[0080] After combining the local features of each character with the global features of the character to obtain the second combined features, the second combined features are input into the kinship verification model to perform kinship verification to obtain the relationship score values ​​between the characters, and then the score values ​​are normalized to obtain the kinship verification results between the characters.

[0081] In a possible implementation of the embodiments of this specification, the kinship verification model can be used to measure the kinship between faces, learn the family relationship of the fused features, and obtain the kinship / non-kinship score of the character pair through the sigmoid activation function. In addition, the training here uses the BCE loss. This module is similar to a learnable metric function. Through training, the feature relationship of faces in different family relationships can be learned, so it can overcome the limitations of manually designed metric functions and better learn the potential relationship between features.

[0082] Specifically, the relationship score between the characters can be calculated by formula (7): (7) in, Indicates the relationship score between characters; Represents input image 1; Represents input image 2; Representing the local features of the characters in each image; Represents the mapping of the backbone network, which is used to extract the global features of the characters in each image; express operate.

[0083] In a possible implementation of the embodiments of this specification, a training method for a kinship verification model is provided, which specifically includes the following steps: Obtaining a training sample set, wherein the training sample set includes a plurality of training images, and the training images include a sample person and a sample family label and a sample person label of the sample person; Dividing the training sample set to obtain a plurality of training subsets, wherein each training subset includes a plurality of training images; Input each training image in the first training subset into the initial verification model, obtain the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image, wherein the class center is updated based on a preset class center update function during the calculation process, and the first training subset is any one of the multiple training subsets; According to the distance information between each training image and the class center in the first training subset, a family-level multi-center loss value is calculated using a preset family-level multi-center loss function; Calculate the household classification loss value based on the predicted household category and sample household label of each training image; Calculate the person classification loss value based on the predicted person category and sample person label of each training image; The total loss value is calculated based on the household-level multi-center loss value, household classification loss value, and person classification loss value; Based on the total loss value, the model parameters of the initial verification model are adjusted, and the step of inputting each training image in the first training subset into the initial verification model is returned to execute, and the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image are obtained, until the preset training stop condition is reached, and the trained kinship verification model is obtained.

[0084] In a possible implementation of the embodiments of the present specification, taking the training of a kinship verification model as an example, firstly, a training sample set is obtained. The training sample set may be a CASIA dataset. The CASIA dataset is a large-scale face dataset, that is, the CASIA dataset includes multiple training images for identity authentication and face recognition.

[0085] Then, the CASIA dataset is used to train the Base network by combining softmax and center loss. It should be noted that in the initial stage of training the kinship verification model, the center loss cannot be given a large weight, which will lead to a large center loss and the failure of softmax to converge. Therefore, in the process of training the kinship verification model, a warm-up strategy is adopted to dynamically adjust the weight of the center loss.

[0086] For example, in the process of training the Base network, the first 200,000 iterations of training use preset weights (for example, the preset weights are set to 0.5) to train the Base network. After 200,000 iterations, the preset weights are increased to 1.5 times the original weights every 50,000 iterations, and then the Base network is trained with the increased preset weights until the iterations are completed, that is, the training of the Base network is completed.

[0087] After the training of the Base network is completed, only the last fully connected layer of the Base network is left, and the parameters of the network layers other than the fully connected layer are frozen to generate the Base network with frozen parameters. Then, the CASIA dataset is used to iteratively train the frozen Base network, and the updated Base network is obtained by iteratively training the frozen Base network 200,000 times. The network layer parameters in the updated Base network are unfrozen, and the CASIA dataset is used to iteratively train the unfrozen Base network to obtain the final trained kinship verification model. Finally, two Base networks are used in the inference stage to perform the final kinship verification in a shared parameter manner.

[0088] By training the kinship verification model in the above training method, the phenomenon that softmax cannot converge due to large center loss can be avoided, thereby making the output result of the kinship verification model more accurate and improving the accuracy of kinship verification between images.

[0089] In addition, the present application uses a combination of logistic regression and center loss to train the basic network for the kinship verification model, which can generate a feature space with multiple center points. Usually, the metric loss function used in the kinship verification model is a single feature center, that is, all the facial features in the family are aggregated to one center in the feature space, which will cause the distance of the same facial features in the feature space to be greater than the distance of different faces, and the facial features of couples who are not related are close in distance, resulting in misjudgment.

[0090] Figure 4 A schematic diagram of the distribution of a single-center feature space and a multi-center feature space provided according to an embodiment of the present application is shown. Figure 4In the figure, (a) represents a single-center feature space and (b) represents a multi-center feature space. In the single-center feature space and the multi-center feature space, each shape represents a face sample of a different family member, and the dots with asterisks represent the feature centers in the feature space. In the kinship verification model, a single-center feature space is usually used, that is, by optimizing the metric loss, all facial features of the same family can be aggregated, which will also cause the intra-class distance of a person to be greater than the inter-class distance between this person and his relatives. For example, we cannot map the features of the father, mother, and child to the same point. This is because the child has potential similarities with his parents, but the father and mother do not have such similarities.

[0091] Therefore, it is necessary to use the combination of logistic regression and center loss to generate a feature space with multiple center points. Through the above method, the feature boundary between different training sample sets can be increased. The family-level multi-center loss function is specifically calculated by formula (8): (8) in, represents the household-level multi-center loss function; Indicates the number of training samples in each minibatch; Indicates the number of accumulated training samples; Represents the category of feature center in multi-center feature space; Represents the number of feature center categories accumulated in the multi-center feature space.

[0092] In the embodiment of the present specification, the class center needs to be updated based on a preset class center update function during the calculation process, wherein the class center update function is shown in formulas (9)-(11):

[0093] (10) (11) in, Represents the number of training samples in each minibatch, Represents the category of feature center in multi-center feature space; , which is used to prevent the denominator from being zero when updating the categories of multiple feature center points.

[0094] In order to ensure that the backbone network can extract effective facial features, a classification loss function is added after the backbone network. This loss function consists of two parts: character label classification loss and the family label classification loss Specifically, the classification loss function is calculated by formulas (12)-(14): (12) (13) (14) in, is the weight of loss, The classification loss is composed of the cross entropy loss of fid and the mutil-center loss (multi-center loss function); CE is the cross entropy loss function, which is used for face recognition feature classification and is used in series with the softmax function.

[0095] The kinship verification method provided by the present application obtains at least two images, inputs each image into a feature extraction model, obtains output features of multiple specified residual blocks of each image and global features of the person in each image, then for any image, combines the output features of multiple specified residual blocks of the image to obtain a first combined feature, inputs the first combined feature into a local attention model, obtains local features of the person, then combines the local features of the person in each image and the global features of the person in each image to obtain a second combined feature, inputs the second combined feature into a kinship verification model, and obtains kinship verification results between the persons. Through the above-mentioned kinship verification method based on deep learning, efficient verification of kinship between persons is achieved. By using the local attention model, local features with significant differences in each image can be automatically extracted without the need for image key point detection. The local features of the person with significant differences in each image and the global features of the person are respectively combined and input into the kinship verification model, which can make the output results of the kinship verification model more accurate and ensure the accuracy of the kinship verification results.

[0096] Figure 5 A structural diagram of a kinship verification model provided according to an embodiment of the present application is shown.

[0097] The first face image and the second face image are input into the kinship verification model to determine whether the first face image and the second face image have a kinship relationship. First, the first face image and the second face image whose kinship relationship needs to be verified are cropped and corrected, and input into the backbone network composed of the improved network structure of resnet50. After passing through the GAP (global average pooling layer), the final global face features are obtained. The first backbone network and the second backbone network have the same structure and share parameters, so they are also called twin networks. At the same time, the output features of C3, C4 and C5 of the residual block are combined at the channel layer to obtain multi-scale combined features of the face for extracting local features of the face. The above combined features are respectively input into PAM (local attention model) to obtain k local features of the face, and then the total 2*k local features of the first face image and the second face image are combined, and the global features output by the backbone network are also combined to obtain k+1 combined features. The above k+1 combined features are input into the kinship verification model to finally obtain the kinship score between the faces.

[0098] Corresponding to the above method embodiment, the present application also provides a kinship verification device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of a kinship verification device according to an embodiment of the present application. Figure 6 As shown, the device comprises: The acquisition module 602 is configured to acquire at least two images, wherein each image includes a different person; The feature extraction module 604 is configured to input each image into a feature extraction model to obtain output features of a plurality of specified residual blocks of each image and global features of the person in each image; The input module 606 is configured to combine the output features of multiple specified residual blocks of any image to obtain a first combined feature, and input the first combined feature into a local attention model to obtain a local feature of the person, wherein the local attention model is a multi-head attention model; The feature combination module 608 is configured to combine the local features of the characters in each image and the global features of the characters in each image to obtain a second combined feature; The verification module 610 is configured to input the second combined feature into a kinship verification model to obtain kinship verification results between the characters.

[0099] Optionally, the acquisition module 602 is further configured to: Acquire at least two initial images, wherein each initial image includes a different person; Based on the characters, each initial image is preprocessed to obtain at least two images.

[0100] Optionally, there are multiple feature extraction models, and each feature extraction model has the same model structure and the same model parameters; The feature extraction module 604 is further configured to: Each image is input into each feature extraction model in parallel to obtain output features of multiple specified residual blocks of each image and global features of the person in each image.

[0101] Optionally, the input module 606 is further configured to: For any image, output features of multiple specified residual blocks of the image are input into the channel layer, and the output features of the multiple specified residual blocks of the image are cross-combined through the channel layer to obtain a first combined feature.

[0102] Optionally, the local attention model includes a spatial attention sub-model and an attention learning sub-model; The input module 606 is further configured to: Perform channel shuffling on the first combination feature; Use 1*1 convolution kernel to fuse the first combination features after channel shuffling to obtain fused features; Split the fused features to obtain a preset number of split features; Input each split feature into the spatial attention sub-model respectively to obtain the spatial attention matrix corresponding to each split feature; Each spatial attention matrix is ​​input into the attention learning sub-model respectively to obtain multiple local features of the character.

[0103] Optionally, the input module 606 is further configured to: Performing a column-wise sum operation on the first spatial attention matrix, and constructing a second spatial attention matrix based on the summation result, wherein the first spatial attention matrix is ​​any one of the spatial attention matrices; Transpose the second spatial attention matrix to obtain a third spatial attention matrix; Determine the maximum value in the third spatial attention matrix as the matrix center of the first spatial attention matrix; According to the matrix center of each spatial attention matrix, under the constraint of a preset loss function, the local features of the character corresponding to each spatial attention matrix are determined respectively, wherein the constraints of the preset loss function include the aggregation of each weight in the same spatial attention matrix to the matrix center of the spatial attention matrix, and the separation of matrix centers of different spatial attention matrices from each other.

[0104] Optionally, the feature combination module 608 is further configured to: Combine k local features of characters in each image to obtain k local combined features of characters; Combine the global features of the characters in each image to obtain a global combined feature of the character; The k local combined features of the characters and the global combined features of the characters are combined to obtain a second combined feature.

[0105] Optionally, the device further comprises: a training module; The training module is configured as follows: Obtaining a training sample set, wherein the training sample set includes a plurality of training images, and the training images include a sample person and a sample family label and a sample person label of the sample person; Dividing the training sample set to obtain a plurality of training subsets, wherein each training subset includes a plurality of training images; Input each training image in the first training subset into the initial verification model, obtain the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image, wherein the class center is updated based on a preset class center update function during the calculation process, and the first training subset is any one of the multiple training subsets; According to the distance information between each training image and the class center in the first training subset, a family-level multi-center loss value is calculated using a preset family-level multi-center loss function; Calculate the household classification loss value based on the predicted household category and sample household label of each training image; Calculate the person classification loss value based on the predicted person category and sample person label of each training image; The total loss value is calculated based on the household-level multi-center loss value, household classification loss value, and person classification loss value; Based on the total loss value, the model parameters of the initial verification model are adjusted, and the step of inputting each training image in the first training subset into the initial verification model is returned to execute, and the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image are obtained, until the preset training stop condition is reached, and the trained kinship verification model is obtained.

[0106] The kinship verification device provided by the present application obtains at least two images, inputs each image into a feature extraction model, obtains output features of multiple specified residual blocks of each image and global features of the person in each image, then for any image, combines the output features of multiple specified residual blocks of the image to obtain a first combined feature, inputs the first combined feature into a local attention model, obtains local features of the person, then combines the local features of the person in each image and the global features of the person in each image to obtain a second combined feature, inputs the second combined feature into a kinship verification model, and obtains kinship verification results between the persons. Through the above-mentioned kinship verification method based on deep learning, efficient verification of kinship between persons is achieved. By using the local attention model, local features with significant differences in each image can be automatically extracted without the need for image key point detection. The local features of the person with significant differences in each image and the global features of the person are respectively combined and input into the kinship verification model, which can make the output result of the kinship verification model more accurate and ensure the accuracy of the kinship verification result.

[0107] The above is a schematic scheme of a kinship verification device of the present embodiment. It should be noted that the technical scheme of the kinship verification device and the technical scheme of the kinship verification method mentioned above belong to the same concept. For details not described in detail in the technical scheme of the kinship verification device, please refer to the description of the technical scheme of the kinship verification method mentioned above. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0108] Figure 7 The structure block diagram of a computing device 700 provided according to an embodiment of the present application is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0109] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world microwave interconnection access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0110] In one embodiment of the present application, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0111] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 700 may also be a mobile or stationary server.

[0112] The processor 720 is used to execute computer executable instructions of the above-mentioned kinship verification method.

[0113] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned kinship verification method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned kinship verification method.

[0114] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which are used in a kinship verification method when executed by a processor.

[0115] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned kinship verification method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned kinship verification method.

[0116] An embodiment of the present application discloses a chip storing computer instructions, which implement the steps of the aforementioned kinship verification method when executed by a processor.

[0117] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0118] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0119] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A method for verifying kinship, characterized in that: include: Acquire at least two images, wherein each of the images includes a different person; Inputting each of the images into a feature extraction model respectively to obtain output features of a plurality of specified residual blocks of each of the images and global features of the person in each of the images; For any image, combining output features of multiple specified residual blocks of the image to obtain a first combined feature, and inputting the first combined feature into a local attention model to obtain a local feature of a person, wherein the local attention model is a multi-head attention model; Combining the local features of the characters in each of the images and the global features of the characters in each of the images to obtain a second combined feature; The second combined feature is input into a kinship verification model to obtain kinship verification results between the characters.

2. The method according to claim 1, characterized in that The step of acquiring at least two images comprises: Acquire at least two initial images, wherein each of the initial images includes a different person; Based on the character, each of the initial images is preprocessed to obtain at least two images.

3. The method according to claim 1, characterized in that There are multiple feature extraction models, and each of the feature extraction models has the same model structure and the same model parameters; The step of inputting each of the images into a feature extraction model to obtain output features of a plurality of specified residual blocks of each of the images and global features of the person in each of the images comprises: The images are input into the feature extraction models in parallel to obtain output features of a plurality of designated residual blocks of the images and global features of the characters of the images.

4. The method according to any one of claims 1 to 3, characterized in that The step of combining output features of multiple specified residual blocks of any image to obtain a first combined feature includes: For any image, output features of multiple specified residual blocks of the image are input into a channel layer, and the output features of the multiple specified residual blocks of the image are cross-combined through the channel layer to obtain a first combined feature.

5. The method according to any one of claims 1 to 3, characterized in that: The local attention model includes a spatial attention sub-model and an attention learning sub-model; The step of inputting the first combined feature into a local attention model to obtain a local feature of a person includes: performing channel shuffling on the first combined features; Using a 1*1 convolution kernel to perform feature fusion on the first combined features after channel shuffling, to obtain a fused feature; Splitting the fused features to obtain a preset number of split features; Input each of the split features into the spatial attention sub-model respectively to obtain a spatial attention matrix corresponding to each of the split features; Each of the spatial attention matrices is input into the attention learning sub-model to obtain multiple local features of the character.

6. The method according to claim 5, characterized in that The step of inputting each of the spatial attention matrices into the attention learning sub-model to obtain a plurality of local features of a person includes: Performing a column-wise summation operation on the first spatial attention matrix, and constructing a second spatial attention matrix based on the summation result, wherein the first spatial attention matrix is ​​any one of the spatial attention matrices; Performing a transposition operation on the second spatial attention matrix to obtain a third spatial attention matrix; Determine a maximum value in the third spatial attention matrix as the matrix center of the first spatial attention matrix; According to the matrix center of each of the spatial attention matrices, under the constraints of a preset loss function, the local features of the character corresponding to each of the spatial attention matrices are determined respectively, wherein the constraints of the preset loss function include the aggregation of each weight in the same spatial attention matrix to the matrix center of the spatial attention matrix, and the separation of matrix centers of different spatial attention matrices from each other.

7. The method according to any one of claims 1 to 3, characterized in that The step of combining the local features of the characters in each of the images and the global features of the characters in each of the images to obtain a second combined feature comprises: Combining k local features of the characters in each of the images to obtain k local combined features of the characters; Combining the global features of the characters in each of the images to obtain a global combined feature of the character; The k local combined features of the character and the global combined feature of the character are combined to obtain the second combined feature.

8. The method according to claim 1, characterized in that The training method of the kinship verification model includes: Acquire a training sample set, wherein the training sample set includes a plurality of training images, and the training images include sample persons and sample family labels and sample person labels of the sample persons; Dividing the training sample set to obtain a plurality of training subsets, wherein each of the training subsets includes a plurality of training images; Input each training image in the first training subset into the initial verification model, obtain the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image, wherein the class center is updated based on a preset class center update function during the calculation process, and the first training subset is any one of the multiple training subsets; Calculate the family-level multi-center loss value using a preset family-level multi-center loss function according to the distance information between each training image and the class center in the first training subset; Calculating a household classification loss value according to the predicted household category and the sample household label of each training image; Calculating a person classification loss value according to the predicted person category and the sample person label of each training image; Calculating a total loss value according to the family-level multi-center loss value, the family classification loss value, and the person classification loss value; Based on the total loss value, the model parameters of the initial verification model are adjusted, and the step of inputting each training image in the first training subset into the initial verification model to obtain the distance information between each training image in the first training subset and the class center, the predicted family category and the predicted person category of each training image is returned to be executed until the preset training stop condition is reached to obtain the trained kinship verification model.

9. A kinship verification device, characterized in that: include: An acquisition module is configured to acquire at least two images, wherein each of the images includes a different person; A feature extraction module is configured to input each of the images into a feature extraction model to obtain output features of a plurality of specified residual blocks of each of the images and global features of the person in each of the images; An input module is configured to combine output features of multiple specified residual blocks of any image to obtain a first combined feature, and input the first combined feature into a local attention model to obtain a local feature of a person, wherein the local attention model is a multi-head attention model; A feature combination module is configured to combine the local features of the characters in each of the images and the global features of the characters in each of the images to obtain a second combined feature; The verification module is configured to input the second combined feature into a kinship verification model to obtain kinship verification results between the characters.

10. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: Computer instructions are stored, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images