Feature extraction network, training method of liveness detection model and liveness detection method

Through the training method of feature extraction network, a pre-domain classifier and actual probability training extracted subnet is used to form domain invariant features, which solves the problem of poor live detection effect of deep learning technology on different data domains and improves the generalization of the model Capability and detection accuracy.

CN113591736BInactive Publication Date: 2024-12-03BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202110888934.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-03
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Deep learning technology has poor live detection effect on different data domains, mainly due to the low generalization ability of the model and the failure to effectively consider the intrinsic distribution relationships of different data domains, resulting in feature learning bias.

Method used

Using the training method of feature extraction network, through the combination of the first extracted subnet and the second extracted subnet, training is performed using a predetermined domain classifier and actual probability to ensure that the extracted features have generalization capabilities for different data domains and form domains. variable features so that the live detection model can be effectively generalized in the new data domain.

Benefits of technology

The model's generalization ability of different data domains is improved, the detection accuracy and reliability of the live detection model in the new data domain is ensured, and labor costs and the complexity of data acquisition are reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113591736B_ABST
    Figure CN113591736B_ABST
Patent Text Reader

Abstract

The present disclosure provides a feature extraction network, a training method for a liveness detection model, and a liveness detection method, which relate to the field of artificial intelligence, specifically to the field of computer vision and deep learning, and can be applied to scenarios such as face recognition and liveness detection. The training method of the feature extraction network includes: inputting a sample into a first extraction subnetwork in a feature extraction network to obtain a feature map; the sample has a first label indicating the actual probability that the sample comes from multiple data domains; inputting the feature map into a second extraction subnetwork in the feature extraction network to obtain domain-invariant features; based on the feature map and the domain-invariant features, using a domain classifier to obtain a first probability and a second probability that the sample comes from multiple data domains; based on the first probability, the actual probability, and a first weight for the actual probability, training the first extraction subnetwork; and based on the second probability, the actual probability, and a second weight for the actual probability that is greater than the first weight, training the second extraction subnetwork.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of computer vision and deep learning technology, and can be applied to scenarios such as face recognition and liveness detection. Background Art

[0002] With the development of computer technology and network technology, deep learning technology has been widely used in many fields. At present, deep learning technology can bring good test results for a certain data domain. However, it cannot guarantee good test results on different data domains. In other words, the generalization ability of deep learning technology models is usually low. Summary of the invention

[0003] Based on this, the present disclosure provides a method for training a feature extraction network that improves generalization capability, a method for training a liveness detection model, and a liveness detection method, device, equipment, and storage medium.

[0004] According to one aspect of the present disclosure, a training method for a feature extraction network is provided, wherein the feature extraction network includes a first extraction subnetwork and a second extraction subnetwork, and the training method includes: inputting a first sample image into the first extraction subnetwork to obtain a feature map of the first sample image; wherein the first sample image has a first label indicating an actual probability that the first sample image comes from multiple data domains; inputting the feature map into the second extraction subnetwork to obtain a first domain-invariant feature of the first sample image; based on the feature map and the first domain-invariant feature, using a predetermined domain classifier to obtain a first probability and a second probability that the first sample image comes from multiple data domains, respectively; training the first extraction subnetwork based on the first probability, the actual probability and a first predetermined weight for the actual probability; and training the second extraction subnetwork based on the second probability, the actual probability and a second predetermined weight for the actual probability, wherein the first predetermined weight is greater than the second predetermined weight.

[0005] According to another aspect of the present disclosure, a training method for a liveness detection model is provided, wherein the liveness detection model includes a feature extraction submodule and a classification submodule; the method includes: inputting a second sample image into a feature extraction network to obtain a second domain-invariant feature of the second sample image; wherein the second sample image includes a target object, and the second sample image has a second label indicating an actual category of the target object; inputting the second domain-invariant feature into a classification network to obtain a predicted category of the target object in the second sample image; and training the liveness detection model based on the actual category and the predicted category, wherein the feature extraction network is trained using the aforementioned feature extraction network training method, and the actual category includes a real category or a non-real category.

[0006] According to another aspect of the present disclosure, a liveness detection method is provided, comprising: inputting an image to be detected into a feature extraction network included in a liveness detection model to obtain a third domain invariant feature of the image to be detected; and inputting the third domain invariant feature into a classification network included in the liveness detection model to obtain a category of a target object in the image to be detected, wherein the liveness detection model is trained using the training method of the liveness detection model described above.

[0007] According to another aspect of the present disclosure, a training device for a feature extraction network is provided, wherein the feature extraction network includes a first extraction subnetwork and a second extraction subnetwork, and the training device includes: a first feature extraction module, used to input a first sample image into the first extraction subnetwork to obtain a feature map of the first sample image; wherein the first sample image has a first label indicating an actual probability that the first sample image comes from multiple data domains; a second feature extraction module, used to input the feature map into the second extraction subnetwork to obtain a first domain-invariant feature of the first sample image; a probability acquisition module, used to obtain a first probability and a second probability that the first sample image comes from multiple data domains using a predetermined domain classifier based on the feature map and the first domain-invariant feature, respectively; a first training module, used to train the first extraction subnetwork based on the first probability, the actual probability and a first predetermined weight for the actual probability; and a second training module, used to train the second extraction subnetwork based on the second probability, the actual probability and a second predetermined weight for the actual probability, wherein the first predetermined weight is greater than the second predetermined weight.

[0008] According to another aspect of the present disclosure, a training device for a liveness detection model is provided, wherein the liveness detection model includes a feature extraction submodule and a classification submodule, and the device includes: a third feature extraction module, used to input a second sample image into a feature extraction network to obtain a second domain-invariant feature of the second sample image; wherein the second sample image includes a target object, and the second sample image has a second label indicating an actual category of the target object; a first category acquisition module, used to input the second domain-invariant feature into a classification network to obtain a predicted category of the target object in the second sample image; and a seventh training module, used to train the liveness detection model based on the actual category and the predicted category, wherein the feature extraction submodule is trained using the training device of the aforementioned feature extraction network, and the actual category includes a real category or a non-real category.

[0009] According to another aspect of the present disclosure, a liveness detection device is provided, comprising: a fourth feature extraction module, used to input the image to be detected into a feature extraction network included in a liveness detection model, to obtain a third domain invariant feature of the image to be detected; and a second category acquisition module, used to input the third domain invariant feature into a classification network included in the liveness detection model, to obtain the category of the target object in the image to be detected, wherein the liveness detection model is trained using the training device of the aforementioned liveness detection model.

[0010] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute at least one of the following methods provided by the present disclosure: a training method for a feature extraction network, a training method for a liveness detection model, and a liveness detection method.

[0011] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute at least one of the following methods provided by the present disclosure: a training method for a feature extraction network, a training method for a liveness detection model, and a liveness detection method.

[0012] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements at least one of the following methods provided by the present disclosure: a training method for a feature extraction network, a training method for a liveness detection model, and a liveness detection method.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0015] Figure 1 It is a schematic diagram of application scenarios of a feature extraction network, a training method of a liveness detection model, and a liveness detection method and device according to an embodiment of the present disclosure;

[0016] Figure 2 is a flow chart of a method for training a feature extraction network according to an embodiment of the present disclosure;

[0017] Figure 3 is a structural diagram of a feature extraction network according to an embodiment of the present disclosure;

[0018] Figure 4 is a schematic diagram of the principle of training a feature extraction network according to an embodiment of the present disclosure;

[0019] Figure 5 is a flowchart of a method for training a liveness detection model according to an embodiment of the present disclosure;

[0020] Figure 6 is a schematic diagram of the principle of training a living body detection model according to an embodiment of the present disclosure;

[0021] Figure 7 is a flow chart of a liveness detection method according to an embodiment of the present disclosure;

[0022] Figure 8 is a structural block diagram of a training device for a feature extraction network according to an embodiment of the present disclosure;

[0023] Fig. 9 is a structural block diagram of a training device for a living body detection model according to an embodiment of the present disclosure;

[0024] Fig.10 is a structural block diagram of a living body detection device according to an embodiment of the present disclosure; and

[0025] Fig.11 It is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] Deep learning technology has achieved great success in liveness detection. Although the deep learning technology can achieve good test results in a certain data domain, the test results are often poor when performing liveness detection on images in different data domains. This is due to the poor generalization ability of the model. In the process of implementing the concept of the present disclosure, the inventors found that the reason for the poor generalization ability of the model is that the inherent distribution relationship of data in different data domains is not considered, so the features learned by the model are biased.

[0028] In order to improve the generalization of the model, we can adopt methods such as expanding data collection, domain adaptation technology, and using generative adversarial networks to perform domain augmentation processing on the source domain data.

[0029] Specifically, the method of expanding data collection can use images collected by more data collection scenarios and more diverse data collection devices to train the model to increase the spatial range of the source domain. This method has a long cycle from data collection to data reflow and data processing, and it is difficult to ensure data diversity during the data collection process. The execution process not only consumes manpower and material resources, but also makes it difficult for the collected data to meet expectations.

[0030] The use of domain adaptation technology can reduce the difference between the source domain and the target domain and map the source domain and the target domain to the same feature space. However, if this domain adaptation technology is used, it is necessary to perform an iteration every time a new domain is returned to make the model adapt to the new domain, which has a high labor cost.

[0031] The method of domain augmentation through generative adversarial networks can generate features of different domains in order to enhance the model's generalization ability for unknown domains. This method uses noise perturbations to increase the types of generated data domains. It does not require data reflux and multiple collections, but instead uses data augmentation to increase the probability of hitting unknown data domains. However, using this method, there is still a high probability of encountering unknown domains that are not generalized by the model.

[0032] In order to overcome the problems existing in the above methods, the present disclosure aims to train a feature extraction network from the perspective of domain generalization through existing source data, so that the extracted features can align the data distribution of different domains, eliminate the features related to the data domain in the features extracted by the model, and map the target object (such as a face) of the unknown data domain to the vicinity of the shared feature space, so that the liveness detection model with the feature extraction network can be well generalized to the new data domain. Specifically, the present disclosure provides a training method for a feature extraction network, including a feature extraction stage, a probability determination stage and a model training stage. In the feature extraction stage, a first sample image is input into a first extraction subnetwork in a feature extraction network to obtain a feature map of the first sample image, and the feature map is input into a second extraction subnetwork in the feature extraction network to obtain a first domain invariant feature of the first sample image. Among them, the first sample image has a first label indicating the actual probability that the first sample image comes from multiple data domains. In the probability determination stage, based on the feature map and the first domain invariant feature, a predetermined domain classifier is used to obtain a first probability and a second probability that the first sample image comes from multiple data domains. In the model training phase, the first extraction subnetwork is trained based on the first probability, the actual probability, and a first predetermined weight for the actual probability, and the second extraction subnetwork is trained based on the second probability, the actual probability, and a second predetermined weight for the actual probability, wherein the first predetermined weight is greater than the second predetermined weight.

[0033] The following will be combined Figure 1The application scenarios of the method and device provided by the present disclosure are described.

[0034] Figure 1 It is a schematic diagram of application scenarios of a feature extraction network, a training method of a liveness detection model, and a liveness detection method and device according to an embodiment of the present disclosure.

[0035] like Figure 1 As shown, the application scenario 100 of this embodiment may include a resident 110, a building 120, an image acquisition device 130, and an electronic device 140. The image acquisition device 130 may be connected to the electronic device 140 through a network or a short-range communication protocol.

[0036] The entrance door of the building 120 may be, for example, a smart door. The electronic device 140 may, for example, control the opening and closing of the entrance door. The image acquisition device 130 may, for example, be used to capture a facial image of the resident 110 walking toward the entrance door, and send the facial image to the electronic device 140, which performs liveness detection and facial recognition on the facial image. When it is determined that the face in the facial image is the real face of the predetermined user, an opening instruction may be sent to the entrance door to realize the intelligent opening of the entrance door.

[0037] In one embodiment, the electronic device 140 may be, for example, a terminal device provided in the building 120 and having a display screen, or may be a remote control server, etc.

[0038] In one embodiment, the electronic device 140 may use a liveness detection model to detect whether a face in a face image is a real face. The liveness detection model may be trained by a server that is in communication with the electronic device 140. Alternatively, the liveness detection model may be pre-trained by the electronic device 140.

[0039] It should be noted that the training method of the feature extraction network and the training method of the liveness detection model provided in the present disclosure can generally be executed by the electronic device 140, or by a server connected to the electronic device 140 in communication. Accordingly, the training device of the feature extraction network and the training device of the liveness detection model provided in the present disclosure can be set in the electronic device 140, or in a server connected to the electronic device 140 in communication. The liveness detection method provided in the present disclosure can be executed by the electronic device 140. Accordingly, the liveness detection device provided in the present disclosure can be set in the electronic device 140.

[0040] It should be understood that Figure 1 The number and types of image acquisition devices, buildings, and electronic devices, and the number of residents in the figure are only illustrative. Depending on the implementation requirements, there can be any number and type of image acquisition devices, buildings, and electronic devices, and there can also be any number of residents.

[0041] Understandably, Figure 1 The application scenarios are only examples to facilitate understanding of the present disclosure. The method of the present disclosure can also be applied to various scenarios such as target classification, target detection, target recognition, target segmentation and target prediction.

[0042] The following will be combined Figure 1 , through the following Figure 2 to Figure 4 The training method of the feature extraction network provided in the present disclosure is described in detail.

[0043] Figure 2 It is a flowchart of a method for training a feature extraction network according to an embodiment of the present disclosure.

[0044] like Figure 2 As shown, the feature extraction network training method 200 of this embodiment may include operations S210 to S250.

[0045] In operation S210, a first sample image is input into a first extraction subnetwork in a feature extraction network to obtain a feature map of the first sample image.

[0046] According to an embodiment of the present disclosure, the first extraction subnetwork may adopt, for example, a feature extraction architecture in a U-Net network, or may adopt any encoder structure, which is not limited by the present disclosure. As long as the first extraction subnetwork can be used to extract image features. The operation may input the first sample image into the first extraction subnetwork, and use the features output by the first extraction subnetwork as a feature map.

[0047] In operation S220, the feature map is input into a second extraction subnetwork in the feature extraction network to obtain a first domain invariant feature of the first sample image.

[0048] According to an embodiment of the present disclosure, the second extraction subnetwork may, for example, adopt a decoder structure to filter out domain-invariant features (DIR) from the feature map. The domain-invariant features refer to features that are irrelevant to the domain to which the sample image belongs, that is, the domain-invariant features will not change due to the different domains to which the sample image belongs. Taking the liveness detection scenario as an example, the domain-invariant features may be features of a human face.

[0049] In one embodiment, the second feature extraction network may be provided with a decoupler, for example, to decouple the domain-invariant features from the domain-specific features in the feature map, so as to facilitate the extraction of the domain-invariant features. The domain-specific features (DSR) refer to features that are unique to the data domain to which the sample image belongs, and the domain-specific features may vary due to the different domains to which the sample image belongs. For example, in a liveness detection scenario, the domain-specific features may be features of the environment surrounding the target object, such as weather features.

[0050] In operation S230, a first probability and a second probability that the first sample image comes from a plurality of data domains are obtained using a predetermined domain classifier based on the feature map and the first domain invariant feature, respectively.

[0051] The feature map can be input into a predetermined domain classifier (Domain Classifier), which processes the feature map and outputs the probability that the first sample image belongs to each of the multiple data domains to obtain a first probability vector. The multiple data domains may be, for example, a collection of data domains to which the multiple sample images belong. Similarly, the first domain invariant feature is input into the predetermined domain classifier, which processes the first domain invariant feature and outputs the probability that the first sample image belongs to each of the multiple data domains to obtain a second probability vector. The predetermined domain classifier may be any domain classifier in the related art, and the present disclosure does not limit this.

[0052] In operation S240, a first extraction sub-network is trained based on the first probability, the actual probability, and a first predetermined weight for the actual probability.

[0053] According to an embodiment of the present disclosure, the first sample image may have a first label indicating the actual probability that the first sample image comes from multiple data domains. For example, the label of the first sample image may include information such as the model of the image acquisition device that captured the first sample image. In this way, the actual probability that the first sample image comes from multiple data domains may be: the actual probability that the first sample image comes from the data domain to which it belongs is 1, and the actual probability that it comes from other data domains is 0.

[0054] This embodiment can determine the value of the first loss function according to the first probability, the actual probability and the first predetermined weight. In one embodiment, the first loss function can be a cross entropy loss function, a paradigm loss function or a mean square error loss function, etc. In the first loss function, the term representing the target value (target) can be the product of the first predetermined weight and the actual probability. Then, based on the value of the first loss function, the first extraction subnetwork is trained using a gradient descent algorithm or a back propagation algorithm. Among them, the first predetermined weight can be 1, or any value less than 1 but close to 1, and the present disclosure does not limit this.

[0055] In one embodiment, there may be multiple first sample images, and the multiple first sample images constitute a batch of sample data. This embodiment can statistically calculate the difference between the product of the actual probability obtained based on the batch of sample data and the first predetermined weight and the first probability to obtain the value of the first loss function.

[0056] In one embodiment, when training the first extraction sub-network, it can be trained together with the domain classifier to further improve the accuracy of the domain classifier, which is beneficial to improving the training effect of the first extraction sub-network.

[0057] In operation S250, the second extraction subnetwork is trained based on the second probability, the actual probability and a second predetermined weight for the actual probability. Operation S250 is similar to operation S240, except that the value of the second predetermined weight is less than the value of the first predetermined weight.

[0058] The disclosed embodiment can make the domain-invariant features extracted by the trained second extraction subnetwork indistinguishable from the domain by assigning a smaller weight to the actual probability when training the second extraction subnetwork, thereby improving the accuracy of the domain-invariant features. Therefore, it is helpful to improve the completion accuracy of downstream tasks (such as liveness detection and object classification) performed based on the domain-invariant features.

[0059] In one embodiment, the first sample image can be set according to the downstream task. For example, in the liveness detection task, the first sample image can be an image including a human face. For example, the first sample image can also include both positive samples and negative samples, and this embodiment trains the feature extraction network only based on the domain-invariant features of the positive samples. In this way, it can be avoided that due to the large difference in the distribution of negative samples, it is difficult for the model to optimize the generalizable features, so that the accuracy of downstream tasks performed based on the extracted features cannot be improved. For example, in liveness detection, due to the different attack methods and collection methods of fake faces, it is difficult to aggregate the fake face data of each data domain, which is not conducive to finding a generalizable feature space for fake face data.

[0060] Specifically, the plurality of first sample images include a first sample of a target object of a real category and a second sample of a target object of a non-real category. The target object of the real category may be a real human face, and the target object of the non-real category may be a fake face. The first extraction subnetwork is trained based on the first probability and the actual probability only when the first probability is obtained based on the first sample. In this way, the expression ability of the first extraction subnetwork for the real target object can be enhanced, and the detection accuracy can be improved in liveness detection. Similarly, the second extraction subnetwork can be trained based on the second probability and the actual probability only when the second probability is obtained based on the first sample. In this way, the expression ability of the second extraction subnetwork for domain-invariant features can be enhanced, avoiding the situation where it is difficult to converge due to the large difference in feature distribution of non-real targets in different data domains.

[0061] The multiple first sample images may also be composed of images obtained from multiple data domains, so as to further improve the generalization ability of the model.

[0062] Figure 3 Schematic diagram of the structure of a feature extraction network according to an embodiment of the present disclosure.

[0063] According to the embodiments of the present disclosure, Figure 3 As shown, in addition to the first extraction subnetwork G 311 and the second extraction subnetwork D 312, the feature extraction network in this embodiment 300 may also include a first normalization subnetwork 313 and a second normalization subnetwork 314. The first normalization subnetwork 313 is connected to the first extraction subnetwork G 311 to normalize the feature map extracted by the first extraction subnetwork G 311 to obtain the first standard feature. The second normalization subnetwork 314 is connected to the second extraction subnetwork D 312 to normalize the domain invariant feature extracted by the second extraction subnetwork D 312 to obtain the second standard feature.

[0064] Accordingly, when training the feature extraction network, the first sample image 301 can be input into the first extraction subnetwork G 311, and the feature map obtained by the first extraction subnetwork G 311 is respectively input into the second extraction subnetwork D 312 and the first normalization subnetwork 313, and the domain invariant feature output by the second extraction subnetwork D 312 is input into the second normalization subnetwork 314. After the first standard feature output by the first normalization subnetwork 313 is input into the predetermined domain classifier 320, the predetermined domain classifier 320 outputs the first probability 302. The second standard feature output by the second normalization subnetwork 314 is input into the predetermined domain classifier 305, and the predetermined domain classifier 320 outputs the second probability 303.

[0065] The first normalization subnetwork 313 and the second normalization subnetwork 314 may respectively use a normalization function to normalize the feature map and the domain invariant feature. The normalization function may be a normalize function, etc., which is not limited in the present disclosure.

[0066] The feature extraction network of the embodiment of the present disclosure normalizes the features by setting a normalization network, which can further improve the generalization ability of the feature extraction network for new domain data and further improve the completion accuracy of downstream tasks.

[0067] Figure 4 It is a schematic diagram of the principle of training a feature extraction network according to an embodiment of the present disclosure.

[0068] According to an embodiment of the present disclosure, the overall structure of the feature extraction network can also be trained based on the distance between the domain-invariant features of two samples, that is, the first extraction subnetwork and the second extraction subnetwork are trained based on the distance, so that the feature extraction network selectively learns the commonalities between the features.

[0069] According to an embodiment of the present disclosure, a feature extraction network can be trained based on the distance between domain-invariant features of two second samples from the same data domain, so that the feature extraction network can learn the commonalities between the second samples of the same data domain, make the feature distribution of multiple non-real categories of target objects in the same data domain compact, and make the distribution of data features of multiple data domains in the feature space more uniform.

[0070] Exemplarily, the aforementioned second sample may include multiple samples from at least two data domains among the multiple data domains. Figure 4 As shown, when training the feature extraction model, the embodiment 400 can first determine the first domain invariant features of the two second samples from the same data domain to obtain two first target features. Then, based on the first distance 440 between the two first target features, the feature extraction network is trained using a predetermined loss function 480. The predetermined loss function 480 and the first distance 440 are positively correlated with each other.

[0071] For example, for the second sample 421 and the second sample 422 from the first data domain 411, the first domain invariant feature 431 and the first domain invariant feature 432 can be obtained respectively. The distance between the two first domain invariant features is the first distance 440. Similarly, the first domain invariant feature 433 and the first domain invariant feature 434 can be obtained based on the second sample 423 and the second sample 424 from the second data domain 412, and thus a first distance 440 is obtained. Substituting the first distance 440 into the predetermined loss function 480, the value of the predetermined loss function 480 can be obtained. The predetermined loss function can be a contrast loss function or a triple loss function, etc. The first distance can be represented by Euclidean distance, Manhattan distance, Mahalanobis distance, etc., and the present disclosure does not limit this. It can be understood that if the number of second samples belonging to each data domain is M, and the number of multiple data domains is N, the number of first distances obtained is N*M*(M-1) / 2.

[0072] According to an embodiment of the present disclosure, there may be multiple first samples, and the embodiment may also train the feature extraction network based on the distance between the first domain invariant features of any two first samples, so as to make the feature distribution of the target object of the real category more compact, and improve the model's ability to express the target object of the real category. The reason why the difference in data domains is not considered for the first sample is that the distribution difference of the target object of the real category is usually small, and the feature extraction network can easily learn a compact feature space for the target object of the real category.

[0073] For example, Figure 4 As shown, in this embodiment 400, when training the feature extraction model, the first domain invariant features of any two first samples among the plurality of first samples can be determined first to obtain two second target features. Then, based on the second distance 450 between the two second target features, the feature extraction network is trained using a predetermined loss function. The predetermined loss function 480 is positively correlated with the second distance 450.

[0074] For example, for the first sample 425 to the first sample 426, the first domain invariant feature 435 to the first domain invariant feature 436 can be obtained respectively. The distance between the feature 435 and the feature 436 is the second distance 450. Substituting the second distance 450 into the predetermined loss function 480, the value of the predetermined loss function 480 can be obtained. The predetermined loss function and the second distance are similar to the predetermined loss function and the first distance respectively, and will not be repeated here. It can be understood that, if the number of first samples is set to P, the number of second distances 450 obtained is P(P-1) / 2.

[0075] This embodiment can also train the feature extraction network based on the distance between the first domain invariant feature of the first sample and the first domain invariant feature of the second sample, so that the feature extraction network can learn the distinguishing features between the target objects of the real category and the target objects of the non-real category, so as to improve the feature extraction model to effectively distinguish target objects of different categories.

[0076] For example, Figure 4 As shown, when training the feature extraction model, the embodiment 400 can train the feature extraction network using a predetermined loss function based on the third distance 460 between the first domain invariant feature of the first sample and the first domain invariant feature of the second sample. The predetermined loss function 480 is negatively correlated with the third distance 460. The predetermined loss function and the third distance are similar to the aforementioned predetermined loss function and the first distance, respectively, and are not described in detail here.

[0077] For example, for the first sample 426 and the second sample 421, the first domain invariant feature 436 and the first domain invariant feature 431 can be obtained respectively, and the distance between the feature 436 and the feature 431 is the third distance 460. For the first sample 425 and the second sample 421, the first domain invariant feature 435 and the first domain invariant feature 431 can be obtained respectively, and the distance between the feature 435 and the feature 431 is the third distance 460. By analogy, assuming that the total number of second samples is m*n and the number of first samples is p, the number of third distances 460 obtained is m*n*p.

[0078] According to an embodiment of the present disclosure, a feature extraction network can be trained based on the distance between domain-invariant features of two second samples from different data domains, so that the feature distributions of multiple non-real categories of target objects in different data domains are dispersed, thereby avoiding the problem of difficulty in optimizing and converging the feature extraction network due to the large differences in non-real categories of target objects in different data domains.

[0079] For example, Figure 4 As shown, when training the feature extraction model, the embodiment 400 can first determine the first domain invariant features of the two second samples from different data domains to obtain two third target features. Then, based on the fourth distance 470 between the two third target features, the feature extraction network is trained using a predetermined loss function. The predetermined loss function and the fourth distance 470 are negatively correlated with each other. The predetermined loss function and the fourth distance are similar to the aforementioned predetermined loss function and the first distance, respectively, and will not be repeated here.

[0080] For example, for the second sample 421 from the first data domain 411 and the second sample 423 from the second data domain 412, the first domain invariant feature 431 and the first domain invariant feature 433 can be obtained respectively, and the distance between the feature 431 and the feature 433 is the fourth distance 470. For the second sample 422 from the first data domain 411 and the second sample 423 from the second data domain 412, the first domain invariant feature 432 and the first domain invariant feature 433 can be obtained respectively, and the distance between the feature 432 and the feature 433 is the fourth distance 470. By analogy, assuming that the number of multiple data domains is M, and the number of second samples belonging to each data domain is N, the number of fourth distances 470 obtained is N. 2 *M*(M-1) / 2.

[0081] It can be understood that when a predetermined loss function is used to train a feature extraction model, at least two of the first distance, the second distance, the third distance, and the fourth distance mentioned above can be considered.

[0082] For example, in one embodiment, Figure 4 As shown, the first distance, the second distance, the third distance and the fourth distance can be comprehensively considered. In this way, the feature extraction model can learn a generalizable feature space in which the target object features of the real category are compactly distributed, the target object features of the non-real category of different data domains are dispersed, and the target object features of the non-real category of the same data domain are compactly distributed. In this case, the predetermined loss function can be expressed by the following formula:

[0083]

[0084] in, respectively represent first domain invariant features of two first samples or first domain invariant features of two second samples belonging to the same data domain; Respectively represent the first domain invariant feature of the first sample and the first domain invariant feature of the second sample, or the first domain invariant features of two second samples belonging to different data domains, and m is a predetermined boundary value greater than 0. In the summation formula, the first term is the first distance or the second distance, the second term is the third distance and the fourth distance. i and j are the serial numbers of the first sample images.

[0085] Based on the above-mentioned feature extraction network training method, the present disclosure also provides a method for training a liveness detection model. Figure 5-6 The method is described in detail.

[0086] Figure 5 It is a flowchart of a method for training a living body detection model according to an embodiment of the present disclosure.

[0087] like Figure 5As shown, the training method 500 of the liveness detection model of this embodiment may include operations S510 to S530. The liveness detection model may include a feature extraction network and a classification network. The feature extraction network adopts the feature extraction network trained by the above-mentioned embodiment.

[0088] In operation S510, the second sample image is input into a feature extraction network to obtain a second domain invariant feature of the second sample image.

[0089] According to an embodiment of the present disclosure, the second sample image may include a target object, and the target object may be a real category or a non-real category. For example, the target object may be a real face or a fake face. The method for obtaining the second domain invariant feature of the second sample image is similar to the method for obtaining the first domain invariant feature of the first sample image described above, and will not be repeated here.

[0090] In operation S520, the second domain invariant feature is input into a classification network to obtain a predicted category of the target object in the second sample image.

[0091] According to an embodiment of the present disclosure, the classification network may include, for example, an activation layer and a fully connected layer. The classification network is similar to the classification network of the liveness detection model in the related art, and the present disclosure does not limit this. The data output by the classification network may be a predicted probability value of the target object in the second sample image being a real category and a predicted probability value of being a non-real category. In this embodiment, when the probability value of being a real category is greater than a predetermined threshold, the predicted category may be determined to be a real category, otherwise the predicted category may be determined to be a non-real category. When the target object is a human face, the real category is the real face category, and the non-real category is the fake face category.

[0092] In operation S530 , a living body detection model is trained based on the actual category and the predicted category.

[0093] According to an embodiment of the present disclosure, the second sample image has a second label indicating the actual category of the target object, and the actual category is a real category or a non-real category. For example, if the actual category is a real category, the embodiment can set the actual probability value of the target object being the real category to 1, and set the actual probability value of the target object being the non-real category to 0. Then, according to the actual probability value and the predicted probability value, the value of the second loss function is calculated. The liveness detection model is trained by minimizing the second loss function. Among them, the second loss function is similar to the first loss function described above, and can be a cross entropy loss function, a paradigm loss function, a mean square error loss function, etc.

[0094] The training of the liveness detection model can be to train the classification network and the feature extraction network at the same time.

[0095] Figure 6It is a schematic diagram of the principle of training a living body detection model according to an embodiment of the present disclosure.

[0096] like Figure 6 As shown, the living body detection model in the embodiment 600 includes a feature extraction network 610 and a classification network 620. The feature extraction network 610 includes a first extraction sub-network G 611, a second extraction sub-network D 612, a first normalization sub-network 613 and a second normalization sub-network 614.

[0097] The multiple sample images 601 may include, for example, the aforementioned first sample and the second sample, and each sample image includes the aforementioned first label and the second label. When training a liveness detection model, the multiple sample images 601 may be sequentially input into the feature extraction network 610, and the domain classifier 630 is used to perform domain classification on the feature map and the domain invariant feature of each sample image obtained by the feature extraction network, respectively, to obtain a first probability 602 and a second probability 603 for each sample image. At the same time, the domain invariant feature of each sample image is input into the classification network 620, and the predicted category 604 for each sample image is obtained based on the output of the classification network 620. Based on the first probability 602, the actual probability and the first predetermined weight of the multiple sample images, the value of the aforementioned first loss function for the first extraction subnetwork G 611 may be obtained. Based on the second probability 603, the actual probability and the second predetermined weight, the value of the aforementioned first loss function for the second extraction subnetwork D 612 may be obtained. Based on the predicted category and the actual category of the multiple sample images, the value of the aforementioned second loss function may be obtained.

[0098] At the same time, the second sample includes multiple samples from different data domains. Based on the domain-invariant features of the multiple sample images 601, the value of the predetermined loss function 605 can be calculated.

[0099] After obtaining the values ​​of each loss function, the first extraction subnetwork G 611, the first normalization subnetwork 613 and the domain classifier 630 can be trained based on the value of the first loss function for the first extraction subnetwork G611, and the second extraction subnetwork, the second normalization subnetwork, the classification network and the domain classifier can be trained based on the value of the predetermined loss function 605, the value of the first loss function for the second extraction subnetwork D 612 and the weighted sum of the value of the second loss function.

[0100] In one embodiment, the feature extraction network may be pre-trained based on the value of the first loss function, and then the second extraction sub-network D 612 may be trained based on a predetermined loss function, and finally the feature extraction network and the classification network may be trained based on the second loss function.

[0101] Based on the training method of the liveness detection model provided in the present disclosure, the present disclosure also provides a liveness detection method. Figure 7 The method is described in detail.

[0102] Figure 7 It is a flowchart of a liveness detection method according to an embodiment of the present disclosure.

[0103] like Figure 7 As shown, the living body detection method 700 of this embodiment may include operations S710 to S720.

[0104] In operation S710, the image to be detected is input into a feature extraction network included in a living body detection model to obtain a third domain invariant feature of the image to be detected.

[0105] The liveness detection model is trained using the liveness detection model training method described above. The image to be detected is an image with a target object, for example, an image with a face. The method for obtaining the third domain invariant feature is similar to the method for obtaining the second domain invariant feature described above, and will not be repeated here.

[0106] In operation S720, the third domain invariant feature is input into the classification network included in the living body detection model to obtain the category of the target object in the image to be detected. The method for obtaining the category of the target object is similar to the method for obtaining the predicted category in the previous text, and will not be repeated here.

[0107] Based on the training method of the feature extraction network provided by the present disclosure, the present disclosure also provides a training device for the feature extraction network. Figure 8 Describe in detail.

[0108] Figure 8 It is a structural block diagram of a training device for a feature extraction network according to an embodiment of the present disclosure.

[0109] like Figure 8 As shown, the training device 800 of this embodiment may include a first feature extraction module 810 , a second feature extraction module 820 , a probability acquisition module 830 , a first training module 840 and a second training module 850 .

[0110] The first feature extraction module 810 is used to input the first sample image into the first extraction subnetwork in the feature extraction network to obtain a feature map of the first sample image. The first sample image has a first label indicating the actual probability that the first sample image comes from multiple data domains. In one embodiment, the first feature extraction module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0111] The second feature extraction module 820 is used to input the feature map into the second extraction subnetwork in the feature extraction network to obtain the first domain invariant feature of the first sample image. In one embodiment, the second feature extraction module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0112] The probability acquisition module 830 is used to obtain the first probability and the second probability that the first sample image comes from multiple data domains using a predetermined domain classifier based on the feature map and the first domain invariant feature. In one embodiment, the probability acquisition module 830 can be used to perform the operation S230 described above, which will not be repeated here.

[0113] The first training module 840 is used to train the first extraction subnetwork based on the first probability, the actual probability and the first predetermined weight for the actual probability. In one embodiment, the first training module 840 can be used to perform the operation S240 described above, which will not be repeated here.

[0114] The second training module 850 is used to train the second extraction subnetwork based on the second probability, the actual probability and the second predetermined weight for the actual probability. Wherein, the first predetermined weight is greater than the second predetermined weight. In one embodiment, the second training module 850 can be used to perform the operation S250 described above, which will not be repeated here.

[0115] According to an embodiment of the present disclosure, the feature extraction network also includes a first normalization subnetwork and a second normalization subnetwork. The probability acquisition module 830 may include a first feature acquisition submodule, a second feature acquisition submodule and a probability acquisition submodule. The first feature acquisition submodule is used to input the feature map into the first normalization subnetwork to obtain a first standard feature. The second feature acquisition submodule is used to input the first domain invariant feature into the second normalization subnetwork to obtain a second standard feature. The probability acquisition submodule is used to input the first standard feature and the second standard feature into a predetermined domain classifier, respectively, to obtain a first probability and a second probability, respectively.

[0116] According to an embodiment of the present disclosure, there are multiple first sample images, and the multiple first sample images include: a first sample with a target object of a real category and a second sample with a target object of a non-real category.

[0117] According to an embodiment of the present disclosure, the second sample includes multiple samples from at least two data domains in the multiple data domains. The above-mentioned device 800 may also include a first feature determination module and a first feature determination module. The first feature determination module is used to determine the first domain invariant features of each of the two second samples from the same data domain to obtain two first target features. The first feature determination module is used to train the feature extraction network based on the first distance between the two first target features using a predetermined loss function. Wherein, the predetermined loss function and the first distance are positively correlated with each other.

[0118] According to an embodiment of the present disclosure, there are multiple first samples, and the above-mentioned device 800 may also include a second feature determination module and a fourth training module. The second feature determination module is used to determine the first domain invariant features of any two first samples among the multiple first samples to obtain two second target features. The fourth training module is used to train the feature extraction network based on the second distance between the two second target features using a predetermined loss function. The predetermined loss function is positively correlated with the second distance.

[0119] According to an embodiment of the present disclosure, the apparatus 800 may further include a fifth training module, configured to train the feature extraction network using a predetermined loss function based on a third distance between the first domain invariant feature of the first sample and the first domain invariant feature of the second sample, wherein the predetermined loss function and the third distance are negatively correlated with each other.

[0120] According to an embodiment of the present disclosure, the above-mentioned device 800 may further include a third feature determination module and a sixth training module. The third feature determination module is used to determine the first domain invariant features of the two second samples from different data domains, and obtain two third target features. The sixth training module is used to train the feature extraction network using a predetermined loss function based on a fourth distance between the two third target features. The predetermined loss function is negatively correlated with the fourth distance.

[0121] According to an embodiment of the present disclosure, the above predetermined loss function is expressed by the following formula:

[0122]

[0123] in, respectively represent first domain invariant features of two first samples or first domain invariant features of two second samples belonging to the same data domain; They respectively represent the first domain invariant features of the first sample and the first domain invariant features of the second sample, or the first domain invariant features of two second samples belonging to different data domains, and m is a predetermined boundary value greater than 0.

[0124] According to an embodiment of the present disclosure, the first training module 840 is used to train the first extraction sub-network when the first probability is obtained based on the first sample. Alternatively, the second training module 850 is used to train the second extraction sub-network when the second probability is obtained based on the first sample.

[0125] Based on the training method of the liveness detection model provided by the present disclosure, the present disclosure also provides a training device for the liveness detection model. Fig. 9 Describe in detail.

[0126] Fig. 9 It is a structural block diagram of a training device for a living body detection model according to an embodiment of the present disclosure.

[0127] like Fig. 9 As shown, the training device 900 of the liveness detection model of this embodiment may include a third feature extraction module 910, a first category acquisition module 920 and a seventh training module 930. The liveness detection model includes a feature extraction network and a classification network. The feature extraction network is trained by using the training device of the feature extraction network described above.

[0128] The third feature extraction module 910 is used to input the second sample image into the feature extraction network to obtain a second domain invariant feature of the second sample image. The second sample image includes the target object, and the second sample image has a second label indicating the actual category of the target object, and the actual category includes a real category or a non-real category. In one embodiment, the third feature extraction module 910 can be used to perform the operation S510 described above, which will not be repeated here.

[0129] The first category acquisition module 920 is used to input the second domain invariant feature into the classification network to obtain the predicted category of the target object in the second sample image. In one embodiment, the first category acquisition module 920 can be used to perform the operation S520 described above, which will not be repeated here.

[0130] The seventh training module 930 is used to train the living body detection model based on the actual category and the predicted category. In one embodiment, the seventh training module 930 can be used to perform the operation S530 described above, which will not be repeated here.

[0131] Based on the liveness detection method provided by the present disclosure, the present disclosure also provides a liveness detection device. Fig.10 Describe in detail.

[0132] Fig.10 4 is a structural block diagram of a living body detection device according to an embodiment of the present disclosure.

[0133] like Fig.10As shown, the living body detection device 1000 of this embodiment may include a fourth feature extraction module 1010 and a second category acquisition module 1020 .

[0134] The fourth feature extraction module 1010 is used to input the image to be detected into the feature extraction network included in the liveness detection model to obtain the third domain invariant feature of the image to be detected. The liveness detection model is trained using the training device of the liveness detection model described above. In one embodiment, the fourth feature extraction module 1010 can be used to perform the operation S710 described above, which will not be repeated here.

[0135] The second category acquisition module 1020 is used to input the third domain invariant feature into the classification network included in the living body detection model to obtain the category of the target object in the image to be detected. In one embodiment, the second category acquisition module 1020 can be used to perform the operation S720 described above, which will not be repeated here.

[0136] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0137] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0138] Fig.11 A schematic block diagram of an example electronic device 1100 that can be used to implement the method of an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0139] like Fig.11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0140] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0141] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as at least one of the following methods: a training method for a feature extraction network, a training method for a liveness detection model, and a liveness detection method. For example, in some embodiments, at least one of the training method for a feature extraction network, a training method for a liveness detection model, and a liveness detection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of at least one of the training method of the feature extraction network, the training method of the liveness detection model, and the liveness detection method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform at least one of the following methods: the training method of the feature extraction network, the training method of the liveness detection model, and the liveness detection method by any other appropriate means (e.g., by means of firmware).

[0142] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0144] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0147] A computer system may include a client and a server. The client and the server are generally far away from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. Among them, the server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server can also be a server of a distributed system, or a server combined with a blockchain.

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0149] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for training a feature extraction network, wherein: The feature extraction network includes a first extraction sub-network, a second extraction sub-network, a first normalization sub-network and a second normalization sub-network; the method includes: Inputting a first sample image into the first extraction subnetwork to obtain a feature map of the first sample image; wherein the first sample image has a first label indicating an actual probability that the first sample image comes from multiple data domains; Inputting the feature map into the second extraction subnetwork to obtain a first domain invariant feature of the first sample image; Inputting the feature map into the first normalization subnetwork to obtain a first standard feature; Inputting the first domain invariant feature into the second normalization subnetwork to obtain a second standard feature; Inputting the first standard feature and the second standard feature into a predetermined domain classifier respectively, and obtaining a first probability and a second probability that the first sample image comes from the plurality of data domains respectively; Training the first extraction subnetwork based on the first probability, the actual probability, and a first predetermined weight for the actual probability; and training the second extraction subnetwork based on the second probability, the actual probability and a second predetermined weight for the actual probability, The first predetermined weight is greater than the second predetermined weight.

2. The method according to claim 1, wherein: There are multiple first sample images, and the multiple first sample images include: a first sample with a target object of a real category and a second sample with a target object of a non-real category.

3. The method according to claim 2, wherein: The second sample includes a plurality of samples from at least two data domains of the plurality of data domains; the method further includes: Determine the first domain invariant features of two second samples from the same data domain to obtain two first target features; and Based on the first distance between the two first target features, the feature extraction network is trained using a predetermined loss function, Wherein, the predetermined loss function and the first distance are positively correlated with each other.

4. The method according to claim 2, wherein: There are multiple first samples, and the method further includes: Determine first domain invariant features of any two first samples from the plurality of first samples to obtain two second target features; and Based on the second distance between the two second target features, the feature extraction network is trained using a predetermined loss function, Wherein, the predetermined loss function and the second distance are positively correlated with each other.

5. The method according to any one of claims 2 to 4, further comprising: Based on a third distance between the first domain invariant feature of the first sample and the first domain invariant feature of the second sample, the feature extraction network is trained using a predetermined loss function, Wherein, the predetermined loss function and the third distance are negatively correlated with each other.

6. The method according to claim 3, further comprising: Determine the first domain invariant features of two second samples from different data domains, respectively, to obtain two third target features; as well as Based on the fourth distance between the two third target features, the feature extraction network is trained using the predetermined loss function, Wherein, the predetermined loss function and the fourth distance are negatively correlated with each other.

7. The method according to claim 6, wherein: The predetermined loss function is expressed by the following formula: in, respectively represent first domain invariant features of two first samples or first domain invariant features of two second samples belonging to the same data domain; They respectively represent the first domain invariant features of the first sample and the first domain invariant features of the second sample, or the first domain invariant features of two second samples belonging to different data domains, and m is a predetermined boundary value greater than 0.

8. The method according to claim 2, wherein: Training the first extraction sub-network includes: training the first extraction sub-network when the first probability is obtained based on the first sample; or Training the second extraction sub-network includes: training the second extraction sub-network when the second probability is obtained based on the first sample.

9. A method for training a liveness detection model, wherein: The living body detection model includes a feature extraction network and a classification network; the method includes: Inputting a second sample image into the feature extraction network to obtain a second domain invariant feature of the second sample image; wherein the second sample image includes a target object, and the second sample image has a second label indicating an actual category of the target object; Inputting the second domain invariant feature into the classification network to obtain a predicted category of the target object in the second sample image; and Based on the actual category and the predicted category, the living body detection model is trained, Wherein, the feature extraction network is trained by adopting the method described in any one of claims 1 to 8; and the actual category includes a real category or a non-real category.

10. A method for detecting a living body, comprising: Inputting the image to be detected into a feature extraction network included in the living body detection model to obtain the third domain invariant features of the image to be detected; as well as Inputting the third domain invariant feature into the classification network included in the living body detection model to obtain the category of the target object in the image to be detected, Wherein, the living body detection model is trained by using the method described in claim 9.

11. A training device for a feature extraction network, wherein: The feature extraction network includes a first extraction sub-network, a second extraction sub-network, a first normalization sub-network and a second normalization sub-network; the device includes: A first feature extraction module, configured to input a first sample image into the first extraction subnetwork to obtain a feature map of the first sample image; wherein the first sample image has a first label indicating an actual probability that the first sample image comes from a plurality of data domains; A second feature extraction module, used for inputting the feature map into the second extraction sub-network to obtain a first domain invariant feature of the first sample image; Probability acquisition module, including: A first feature acquisition submodule, used for inputting the feature map into the first normalization subnetwork to obtain a first standard feature; A second feature acquisition submodule, used for inputting the first domain invariant feature into the second normalization subnetwork to obtain a second standard feature; a probability acquisition submodule, used for inputting the first standard feature and the second standard feature into a predetermined domain classifier respectively, and obtaining a first probability and a second probability that the first sample image comes from the plurality of data domains respectively; a first training module, configured to train the first extraction subnetwork based on the first probability, the actual probability, and a first predetermined weight for the actual probability; and a second training module, configured to train the second extraction subnetwork based on the second probability, the actual probability and a second predetermined weight for the actual probability, The first predetermined weight is greater than the second predetermined weight.

12. The device according to claim 11, wherein There are multiple first sample images, and the multiple first sample images include: a first sample with a target object of a real category and a second sample with a target object of a non-real category.

13. The device according to claim 12, wherein: The second sample includes a plurality of samples from at least two data domains of the plurality of data domains; the apparatus further includes: A first feature determination module, used to determine first domain invariant features of two second samples from the same data domain, respectively, to obtain two first target features; and A first feature determination module is used to train the feature extraction network using a predetermined loss function based on a first distance between the two first target features. Wherein, the predetermined loss function and the first distance are positively correlated with each other.

14. The device according to claim 12, wherein: The first samples are multiple, and the device further includes: A second feature determination module is used to determine first domain invariant features of any two first samples among the plurality of first samples to obtain two second target features; and a fourth training module, configured to train the feature extraction network using a predetermined loss function based on the second distance between the two second target features, Wherein, the predetermined loss function and the second distance are positively correlated with each other.

15. The device according to any one of claims 12 to 14, further comprising: a fifth training module, configured to train the feature extraction network using a predetermined loss function based on a third distance between the first domain invariant feature of the first sample and the first domain invariant feature of the second sample, Wherein, the predetermined loss function and the third distance are negatively correlated with each other.

16. The apparatus according to claim 13, further comprising: A third feature determination module is used to determine the first domain invariant features of two second samples from different data domains to obtain two third target features; as well as a sixth training module, configured to train the feature extraction network using the predetermined loss function based on a fourth distance between the two third target features, Wherein, the predetermined loss function and the fourth distance are negatively correlated with each other.

17. The device according to claim 16, wherein: The predetermined loss function is expressed by the following formula: in, respectively represent first domain invariant features of two first samples or first domain invariant features of two second samples belonging to the same data domain; They respectively represent the first domain invariant features of the first sample and the first domain invariant features of the second sample, or the first domain invariant features of two second samples belonging to different data domains, and M is a predetermined boundary value greater than 0.

18. The apparatus of claim 12, wherein: The first training module is used to: train the first extraction sub-network when the first probability is obtained based on the first sample; or The second training module is used to train the second extraction sub-network when the second probability is obtained based on the first sample.

19. A training device for a liveness detection model, wherein: The living body detection model includes a feature extraction network and a classification network; the device includes: a third feature extraction module, configured to input a second sample image into the feature extraction network to obtain a second domain invariant feature of the second sample image; wherein the second sample image includes a target object, and the second sample image has a second label indicating an actual category of the target object; A first category acquisition module, configured to input the second domain invariant feature into the classification network to obtain a predicted category of a target object in the second sample image; and a seventh training module, configured to train the living body detection model based on the actual category and the predicted category, Wherein, the feature extraction network is trained by using the device described in any one of claims 11 to 18; and the actual category includes a real category or a non-real category.

20. A living body detection device, comprising: A fourth feature extraction module, used for inputting the image to be detected into a feature extraction network included in the living body detection model to obtain a third domain invariant feature of the image to be detected; as well as The second category acquisition module is used to input the third domain invariant feature into the classification network included in the living body detection model to obtain the category of the target object in the image to be detected, Wherein, the living body detection model is trained using the device described in claim 19.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

Citation Information

Patent Citations

  • Image classification method based on adversarial fusion multi-source transfer learning

    CN111738315A

  • Few-sample target detection method based on meta-feature and weight adjustment and network model

    CN112364931A

  • Neural network training and target detection method, device and storage medium

    CN112446239A

  • Living body face detection model training method and device, apparatus and storage medium

    CN113052144A

  • Classification model training method and target object classification method

    CN113065614A