Training method of biological detection model, biological detection method and related products

Abstract

Embodiments of the present application provide a biological detection model training method, a biological detection method and related products, wherein the training method comprises: obtaining a plurality of multi-modal sample image inputs corresponding to a plurality of scenes, extracting the features of each multi-modal sample image from a feature extraction network of a training model, and extracting common features and unique features of each multi-modal sample image; inputting the common features into a scene classification network of the training model, and calculating an adversarial loss value; inputting the common features and the unique features into a feature reorganization network of the training model, and obtaining a plurality of reorganized features; inputting a self-organizing feature in the reorganized features into an attack recognition network of the training model, and calculating a classification loss value; inputting the reorganized features into a comparison network of the training model, and calculating a comparison loss value between a mixed feature in the reorganized features and the self-organizing feature; training the training model according to the adversarial loss value, the classification loss value and the comparison loss value to calculate an overall loss value; and obtaining a biological detection model according to the trained training model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a biological detection model training method, a biological detection method, and related products. BACKGROUND

[0002] In recent years, with the continuous development of face recognition systems, biological detection has become an indispensable part of face recognition systems. Biological detection is used to verify whether it is the real user himself. Through biological detection, the face biological characteristics of human beings and other living substances can be distinguished from the face biological characteristics of non-living substances such as photos, silica gel, and plastic, thereby preventing illegal persons from passing face verification by forging face characteristics.

[0003] However, as face recognition systems are increasingly applied to more fields, the scene of biological detection will continue to change, and different types of biological attacks will be encountered. Different types of biological attacks have different representations in different scenes. The existing biological detection model has low detection accuracy for biological attacks after the scene and attack type change, and is difficult to cope with the changing scene and the changing biological attack type. SUMMARY

[0004] The biological detection model training method, the biological detection method, and the related products provided by the embodiments of the present application can provide better detection effects for complex biological attacks in different scenes and better cope with complex biological attacks in new scenes.

[0005] In a first aspect, the embodiments of the present application provide a biological attack detection model training method, which comprises:

[0006] Obtaining a plurality of multi-modal sample images corresponding to a plurality of scenes, wherein the multi-modal sample images contain image information of sample targets to be biologically detected;

[0007] Inputting the plurality of multi-modal sample images into a feature extraction network of a training model to extract common features and unique features of each multi-modal sample image in the plurality of multi-modal sample images;

[0008] Inputting the common features into a scene classification network of the training model to calculate an adversarial loss value of the common features;

[0009] Inputting the common features and the unique features into a feature recombination network of the training model to recombine the common features and the unique features to obtain a plurality of recombined features; wherein each recombined feature is recombined based on one common feature and one unique feature;

[0010] input the self-assembly feature in the plurality of recombined features into an attack recognition network of the training model, and calculate a classification loss value of identifying a biological attack type of the self-assembly feature; wherein the universal feature and the unique feature based on which the self-assembly feature is recombined come from the same multi-modal sample image;

[0011] input the plurality of recombined features into a contrast network of the training model, and calculate a contrast loss value between a mixed feature in the plurality of recombined features and the self-assembly feature, wherein the universal feature and the unique feature based on which the mixed feature is recombined come from different multi-modal sample images;

[0012] calculate a total loss value according to the adversarial loss value, the classification loss value and the contrast loss value, train the training model according to the total loss value, and obtain a trained biological detection model according to a feature extraction network, a feature recombination network and an attack recognition network in the trained training model.

[0013] In a second aspect, an embodiment of the present application provides a biological detection method, and the method comprises:

[0014] obtaining a multi-modal target image of a target scene, wherein the multi-modal target image contains image information of a target to be biologically detected;

[0015] inputting the multi-modal target image into the trained biological detection model, and predicting a biological attack type of the target in the multi-modal target image;

[0016] The biological detection model is trained by the biological detection model training method provided in the foregoing embodiments.

[0017] In a third aspect, an embodiment of the present application provides a biological detection model training device, and the device comprises:

[0018] a sample acquisition module, configured to acquire a plurality of multi-modal sample images corresponding to a plurality of scenes, wherein the multi-modal sample images contain image information of sample targets to be biologically detected;

[0019] a feature extraction module, configured to input the plurality of multi-modal sample images into a feature extraction network of a training model, and extract universal features and unique features of each multi-modal sample image in the plurality of multi-modal sample images;

[0020] an adversarial loss calculation module, configured to input the universal features into a scene classification network of the training model, and calculate an adversarial loss value of the universal features;

[0021] a feature recombination module, configured to input the general features and the specific features into a feature recombination network of the training model, and recombine the general features and the specific features to obtain a plurality of recombined features, wherein each recombined feature is recombined based on one general feature and one specific feature;

[0022] a classification loss module, configured to input a self-recombined feature in the plurality of recombined features into an attack recognition network of the training model, and calculate a classification loss value of identifying a biological attack type of the self-recombined feature, wherein the general feature and the specific feature on which the self-recombined feature is recombined come from the same multi-modal sample image;

[0023] a contrast loss calculation module, configured to input the plurality of recombined features into a contrast network of the training model, and calculate a contrast loss value between a mixed feature in the plurality of recombined features and the self-recombined feature, wherein the general feature and the specific feature on which the mixed feature is recombined come from different multi-modal sample images;

[0024] a training module, configured to calculate a total loss value according to the adversarial loss value, the classification loss value and the contrast loss value, train the training model according to the total loss value, and obtain a trained biological detection model according to a feature extraction network, a feature recombination network and an attack recognition network in the trained training model.

[0025] In a fourth aspect, an embodiment of the present application provides a biological detection device, which comprises:

[0026] an image acquisition module, configured to acquire a multi-modal target image of a target scene, wherein the multi-modal target image contains image information of a target to be biologically detected;

[0027] a prediction module, configured to input the multi-modal target image into the trained biological detection model, and predict a biological attack type of the target in the multi-modal target image;

[0028] wherein the biological detection model is trained by the biological detection model training method provided in the foregoing embodiments.

[0029] In a fifth aspect, an embodiment of the present application provides a computer device, which comprises:

[0030] a processor; and

[0031] a memory arranged to store computer executable instructions configured to be executed by the processor, the executable instructions comprising instructions for performing steps in the biological detection model training method provided in the foregoing embodiments or the biological detection method provided in the foregoing embodiments.

[0032] In a sixth aspect, an embodiment of the present application provides a storage medium, which is used for storing computer executable instructions, and the executable instructions make a computer execute the training method of the biological detection model or the biological detection method provided in the foregoing embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the one or more embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description only illustrate some of the embodiments in the one or more embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0034] Figure 1 A flowchart of the training method of the biological detection model provided in the embodiment of the present application is shown in the figure.

[0035] Figure 2 A structure diagram of the training model used in the training method shown in the figure. Figure 1

[0036] Figure 3 A flowchart of the biological detection method provided in the embodiment of the present application is shown in the figure.

[0037] Figure 4 A module diagram of the training device of the biological detection model provided in the embodiment of the present application is shown in the figure.

[0038] Figure 5 A module diagram of the biological detection device provided in the embodiment of the present application is shown in the figure.

[0039] Figure 6 A structure diagram of the electronic device provided in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0040] In order to make the person skilled in the art better understand the technical solutions in the one or more embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments in the one or more embodiments of the present application, but not all the embodiments. Based on the embodiments in the one or more embodiments of the present application, all the other embodiments obtained by those skilled in the art without any creative effort should belong to the protection scope of the present application.

[0041] It should be noted that, in the case of no conflict, the one or more embodiments in the present application and the features in the embodiments can be combined with each other. The embodiments of the present application will be described in detail below with reference to the drawings and in combination with the embodiments. ​

[0042] As described in the background section, existing biometric detection models suffer from low accuracy in detecting complex biometric attacks after scene changes, making it difficult to cope with complex biometric attacks in new scenarios. To address complex biometric attacks in new scenarios, this application provides a training scheme for a biometric detection model. This scheme acquires multimodal images of the target sample to be detected and uses these multimodal images as sample images to train the training model of the biometric detection model. This allows the trained biometric detection model to fully learn the characteristics of various types of biometric attacks, achieving good detection results for all types of attacks and better handling various complex biometric attacks, thus improving the detection accuracy. Although the scenes and types of biometric attacks in different multimodal sample images may differ, the shape and size of the target region to be detected, such as the face region, are often similar. Multimodal sample images from different scenes and with different attack types share a common semantic space and possess common features. Therefore, this application, during training, uses multimodal sample images... The features are separated into general features and specific features. General features are used to represent common elements of multimodal sample images, such as semantic features and target attribute features. Specific features are used to represent specific elements of multimodal sample images, such as scene-related features and features related to biological attack types. Then, the trained model is jointly optimized through adversarial learning and contrastive learning. Adversarial learning is used to strengthen general features, making them non-specific to specific scenes. Contrastive learning is used to strengthen features related to biological attack types among specific features and suppress features related to scenes among specific features. As a result, the trained biological detection model has better scene generalization and can better adapt to new scenes. When a new scene appears, the biological detection model provided in this application embodiment can effectively detect and identify various complex biological attack types in the new scene.

[0043] Figure 1 This is a flowchart illustrating a training method for a biological detection model provided in an embodiment of this application. Figure 2 for Figure 1 The diagram shows the structure of a training model used in the training method illustrated. (See attached diagram for reference.) Figure 1 , Figure 2 The training process of the biological detection model provided in the embodiments of this application will be described in detail.

[0044] The biological detection model provided in this application embodiment is used to detect the type of biological attack, and it can employ... Figure 1 The method shown is used to train the equipment. Figure 1 The training method shown employs adversarial learning and contrastive learning to jointly optimize the training of the biological detection model. The training model that can be used during training includes, for example... Figure 2The structure shown. As Figure 2 As shown, the training model used by the biological detection model of the embodiment of the application can include: a feature extraction network, a scene classification network, a feature reorganization network, an attack recognition network, and a comparison network; wherein the feature extraction network is used to extract general features and specific features from the multi-modal sample images; the scene classification network is used to predict the scene category of the general features; the feature reorganization network is used to reorganize the general features and specific features of multiple multi-modal sample images to obtain multiple reorganized features; the attack recognition network is used to predict the biological attack category of the self-organizing features in the multiple reorganized features; and the comparison network is used to compare the difference between the self-organizing features and the mixed features corresponding to the self-organizing features in the multiple reorganized features.

[0045] During training, the scene classification network can be used as a scene discriminator to form an adversarial network with the general feature extraction network (equivalent to a general feature generator) in the feature extraction network, to perform adversarial learning on the general features, so that the generated general features are not specific to a specific scene; the comparison network can compare the difference between the mixed features and the corresponding self-organizing features, to enhance the features related to biological attacks in the specific features and suppress the features related to the scene in the specific features. In this way, the joint optimization of adversarial learning and comparison learning can be performed, to improve the generalization of the biological detection model and enhance the detection capability of the biological detection model for complex biological attacks in new scenes.

[0046] The biological detection model trained by the embodiment of the application is trained according to the feature extraction network, the feature reorganization network, and the attack recognition network in the training model, and the training method thereof is as shown in Figure 1 As shown, the biological detection model provided by the embodiment can specifically include the following steps: Figure 1 As shown, the biological detection model provided by the embodiment can specifically include the following steps:

[0047] S102, obtaining multiple multi-modal sample images corresponding to multiple scenes.

[0048] The image information of the sample target in each multi-modal sample image can be image information of a sample target of different types. For example, the image information in some multi-modal sample images can be first type image information, such as image information of a real biological entity face; the image information in some multi-modal sample images can be second type image information, such as image information of a photo-printed face; the image information in some multi-modal sample images can be third type image information, such as image information of an electronic screen displayed face; and the image information in some multi-modal sample images can be fourth type image information, such as image information of a face disguised with a mask.

[0049] The plurality of scenes can be a plurality of existing application scenes of biological detection that are pre-set. The scene conditions, such as the scene background, of each scene can be different. The plurality of scenes correspond to the plurality of multi-modal sample images, and each scene corresponds to one or more multi-modal sample images. In different face recognition scenes, different types of biological attacks can be encountered. For example, in scene 1, a paper type biological attack can be encountered; in scene 2, a screen type biological attack can be encountered; and in scene 3, a mask type biological attack can be encountered. The paper type biological attack is to use a paper-printed picture to fake a face feature; the screen type biological attack is to use an image displayed on an electronic device screen to fake a face feature; and the mask type biological attack is to use a mask to fake a face feature. Different types of biological attacks have different characteristics and are presented differently in different modalities. For example, the paper type biological attack can present a special effect in an infrared modality, and the screen type biological attack can present a special effect in a depth of field modality. By obtaining multi-modal sample images of a plurality of scenes and training a biological detection model using the multi-modal sample images of the plurality of scenes, the model can have good detection effects on different types of biological attacks in different scenes, and can effectively cope with different types of biological attack methods in different scenes.

[0050] The multi-modal sample image is an image fused from images of multiple modalities. In an implementation, a plurality of preset scenes can be determined, and for each scene in the plurality of preset scenes, a plurality of multi-modal sample images of the scene can be obtained from data sources of the plurality of modalities. The preset scenes can be various types of face recognition scenes, such as a face recognition for access control, a face recognition for taking express delivery, and the like. In the preset scenes, a plurality of image acquisition devices can be provided, such as an RGB camera, an infrared camera, and a three-dimensional depth camera. Different types of image acquisition devices can be used to acquire face image information of different modalities, and the acquired face image information of different modalities can be stored as historical image information in the data sources of the corresponding modalities.

[0051] In the embodiments of the present application, for each scene in the plurality of preset scenes, sample images of each modality of the scene can be obtained from the data sources corresponding to the plurality of modalities, respectively, and then the sample images of each modality can be fused to obtain a multi-modal sample image of the scene. The data sources of the plurality of modalities can include at least two of an RGB data source, an infrared data source, and a three-dimensional depth data source. The RGB data source is used to store RGB images acquired by an RGB camera, the infrared data source is used to store infrared images acquired by an infrared camera, and the three-dimensional depth data source is used to store three-dimensional depth images acquired by a three-dimensional depth camera. The RGB images contain apparent color, texture, shadow, and the like of a face as a biological detection target, the infrared images contain heat distribution and the like of the face as the biological detection target, and the three-dimensional depth images contain three-dimensional information of the face as the biological detection target. By obtaining images of multiple modalities and fusing the images of multiple modalities to obtain a multi-modal image as a training sample, the training sample can contain various types of biological attack features, and the model can better learn various types of biological attacks.

[0052] For example, as shown in FIG. 1, a first modality image can be obtained from a first data source, a second modality image can be obtained from a second data source, and a third modality image can be obtained from a third data source. The first modality image, the second modality image, and the third modality image can be input into an image fusion network to obtain a multi-modal sample image. The first modality image is an RGB image, the second modality image is an infrared image, and the third modality image is a three-dimensional depth image. The first modality image, the second modality image, and the third modality image all contain image information of a same sample target to be detected in a same scene. Figure 2

[0053] ​S104, input the plurality of multi-modal sample images into a feature extraction network of the training model, and extract general features and unique features of each multi-modal sample image in the plurality of multi-modal sample images.

[0054] The general features are used to represent general attributes of the multi-modal sample images, such as semantic features, human attribute features, etc., and the unique features are used to represent distinguishable attributes of the multi-modal sample images, such as scene difference features, biological attack type difference features, etc.

[0055] In an implementation manner, the feature extraction network comprises a general feature extraction network and a unique feature extraction network. After the multi-modal sample image is input into the feature extraction network, the general feature extraction network can be used to extract the general features of the multi-modal sample image, and the unique feature extraction network can be used to extract the unique features of the multi-modal sample.

[0056] The general feature extraction network is used to extract the general features of the multi-modal sample image from the multi-modal sample image. Specifically, although the multi-modal sample images are sample images of different biological attack types acquired from different scenes, they all include face regions, different multi-modal sample images share a semantic feature space, and whether it is a real human face image or a fake human face image, their shapes and sizes are similar, so the general feature extraction network can be used to extract the general features.

[0057] The unique feature extraction network is used to extract the unique features of the multi-modal sample image from the multi-modal sample image. The unique feature extraction network can adopt a self-attention structure to extract the unique features of the multi-modal sample image from the multi-modal sample image, and enhance the distinguishability of the unique features and the general features.

[0058] In an implementation manner, the feature extraction network can further comprise a basic feature extraction network. The basic feature extraction network can adopt ResNet18 to extract basic features from the multi-modal sample image. The general feature extraction network can extract the general features from the basic features, and the unique feature extraction network can extract the unique features from the basic features. In this way, the efficiency of feature extraction can be further improved.

[0059] S106, input the general features into a scene classification network of the training model, and calculate an adversarial loss value of the generated general features.

[0060] The scene classification network can be used as a scene discriminator, and the general feature extraction network can be used as a feature generator. The two form an adversarial learning network, and the generation of the general features is subjected to adversarial learning, so that the generated general features are not specific to a specific scene.

[0061] In training, the multi-modal sample image can be input into a general feature extraction network to obtain a general feature of the multi-modal sample image, and then the general feature of the multi-modal sample image is input into a scene classification network to predict a scene category of the general feature. Then, a scene classification loss value of the general feature can be calculated according to the scene category prediction result of the general feature, and a scene discriminator loss value and an adversarial loss value can be calculated according to the scene classification loss value of the general feature. The network parameters of the adversarial learning network are updated through the scene discriminator loss value and the adversarial loss value to locally optimize the training model. The parameters of the adversarial learning network can be optimized by maximizing the adversarial loss value and minimizing the scene discriminator loss, so that the general feature generated by the general feature generator is difficult to distinguish different scenes.

[0062] In S108, the general feature and the unique feature are input into a feature reorganization network of the training model to reorganize the general feature and the unique feature to obtain a plurality of reorganized features.

[0063] The general feature and the unique feature of each multi-modal sample image in the plurality of multi-modal sample images can be input into the feature reorganization network to randomly mix and reorganize the plurality of general features and the plurality of unique features to obtain a plurality of reorganized features, wherein each reorganized feature is reorganized based on one general feature and one unique feature. The reorganized feature is a feature randomly combined from the general feature and the unique feature, so for a reorganized feature, the general feature and the unique feature in the reorganized feature can come from the same multi-modal sample image or from different multi-modal sample images.

[0064] In implementation, the general feature of each multi-modal sample image in the plurality of multi-modal sample images can be put into a general feature set, and the unique feature of each multi-modal sample image can be put into a unique feature set. Then, one general feature is randomly extracted from the general feature set, and one unique feature is randomly extracted from the unique feature set. The randomly extracted general feature and the unique feature are reorganized to obtain one reorganized feature. By repeatedly performing the above extraction and reorganization operation, a plurality of reorganized features can be obtained.

[0065] In S110, the self-organizing feature in the plurality of reorganized features is input into an attack recognition network of the training model to calculate a classification loss value of recognizing the biological attack type of the self-organizing feature.

[0066] The general feature and the unique feature on which the self-organizing feature is reorganized come from the same multi-modal sample image, that is, the self-organizing feature is a reorganized feature reorganized from the general feature and the unique feature of the same multi-modal sample image.

[0067] In implementation, a self-organizing feature can be obtained from the plurality of reorganized features, the self-organizing feature is input into the attack identification network, the biological attack type of the self-organizing feature is identified, and then, based on a preset classification loss function, a classification loss value of identifying the biological attack type of the self-organizing feature is calculated according to the biological attack type prediction result of the self-organizing feature, that is, the classification loss value corresponding to the self-organizing feature. The preset classification loss function can use an existing classification loss function, such as a cross-entropy loss function.

[0068] The attack identification network is used to identify the biological attack type, and can use a classification learning network. Since the sample image of the embodiment of the present application is a multi-modal sample image, compared with a traditional two-dimensional RGB image, it not only includes RGB image information, but also includes heat distribution information and three-dimensional depth information. According to these information in the multi-modal sample image, not only can it be identified whether the target to be detected in the multi-modal sample image belongs to a biological attack or a non-biological attack, but also can further detect the type of biological attack when the target to be detected belongs to a biological attack, so as to facilitate subsequent taking corresponding security protection measures according to different types of biological attacks.

[0069] In an implementation manner, the biological attack type can include a biological attack and a non-biological attack, and correspondingly, the classification task of the attack identification network can be a binary classification task, and the classification learning network used can be a binary classification learning network. Through the attack identification network, it can be predicted whether the self-organizing feature belongs to a biological attack or a non-biological attack.

[0070] In an implementation manner, the biological attack type can include a paper-based biological attack, an electronic screen-based biological attack, a mask-based biological attack, and a non-biological attack, and correspondingly, the classification task of the attack identification network can be a multi-classification task, and the classification learning network used can be a multi-classification learning network. Through the attack identification network, it can be predicted which type the self-organizing feature belongs to among the paper-based biological attack, the electronic screen-based biological attack, the mask-based biological attack, and the non-biological attack.

[0071] Of course, it can be understood that the biological attack type is not limited to necessarily including the paper-based biological attack, the electronic screen-based biological attack, the mask-based biological attack, and the non-biological attack, and it can include one or more of the paper-based biological attack, the electronic screen-based biological attack, the mask-based biological attack, and the non-biological attack, or can include other biological attack types, for example, a newly emerging biological attack type.

[0072] S112, inputting the plurality of reorganized features into the contrast network of the training model to calculate a contrast loss value of the reorganized features and the self-organizing feature in the reorganized features.

[0073] The multiple recombination features include self-organizing features and mixed features. The self-organizing features are recombination features of the same multi-modal sample image, and the mixed features are recombination features of different multi-modal sample images.

[0074] In implementation, the mixed features and the self-organizing features can be compared and learned by the comparison network and the attack identification network, the distance between the recombination features of different general features but the same scene is narrowed, and the distance between the recombination features of the same general feature but different biological attack types is widened, so as to strengthen the features related to the biological attack type in the specific features and inhibit the specific features related to the scene, thereby making the model have better generalization performance and being suitable for new scenes.

[0075] In an implementation, the multiple recombination features can be input into the comparison network, and a first recombination feature pair is extracted from the multiple recombination features by the comparison network. The first recombination feature pair includes two recombination features. One of the two recombination features is a first self-organizing feature, which is the feature input into the attack identification network for biological attack identification. The other is a mixed feature corresponding to the first self-organizing feature. The general feature of the mixed feature and the general feature of the first self-organizing feature are from the same multi-modal sample image, and the specific features of the two are from different multi-modal sample images. Then, a comparison loss value corresponding to the recombination feature pair can be calculated according to the similarity between the two recombination features in the recombination feature pair. A multiple recombination feature pair can be extracted, each recombination feature pair including a first self-organizing feature and a mixed feature corresponding to the first self-organizing feature. The comparison loss value corresponding to each recombination feature pair in the multiple recombination feature pairs can be calculated, and a comparison loss value corresponding to the first self-organizing feature can be calculated according to the sum of the comparison loss values corresponding to the multiple recombination feature pairs. For a recombination feature pair, the similarity between the two recombination features can be calculated based on a cosine similarity function.

[0076] In S114, a total loss value is calculated according to the adversarial loss value, the classification loss value, and the comparison loss value. The training model is trained according to the total loss value. The trained biological detection model is obtained according to the feature extraction network, the feature recombination network, and the attack identification network in the trained training model.

[0077] The total loss value corresponding to the first multi-modal sample image can be calculated according to the adversarial loss value, the classification loss value and the contrast loss value corresponding to the first multi-modal sample image; and the network parameters of each network in the training model are adjusted by using a back propagation algorithm according to the total loss value of the first multi-modal sample image. The first multi-modal sample image is any one of the plurality of multi-modal sample images. Through iterative optimization of the network parameters of each network in the training model according to the total loss values corresponding to the plurality of multi-modal sample images, the trained training model can be finally obtained.

[0078] The training method of the biological detection model provided in the embodiments of the present application is further described below by taking one iteration training as an example. In this example, it is assumed that there are two scenes, and each scene corresponds to a multi-modal sample image. For one iteration, the following processing can be included:

[0079] Processing one, a first multi-modal sample image x1 corresponding to scene 1 and a second multi-modal sample image x2 corresponding to scene 2 are obtained;

[0080] Processing two, the first multi-modal sample image x1 and the second multi-modal sample image x2 are input into a feature extraction network to obtain the general feature f c (x1) and the unique feature f s (x1) of the first multi-modal sample image x1, and the general feature f c (x2) and the unique feature f s (x2) of the second multi-modal sample image x2;

[0081] Processing three, the general feature f c (x1) of the first multi-modal sample image x1 is input into a scene classification network to predict the scene category of the general feature f c (x1), and a general feature adversarial loss value Loss1(x1) of the general feature f c (x1) is calculated according to the scene category prediction result of the general feature f c (x1), which is the adversarial loss value Loss1(x1) corresponding to the first multi-modal sample image x1;

[0082] Processing four, the general feature f c (x1) and the unique feature f s (x1) of the first multi-modal sample image x1, the general feature f c (x2) and the unique feature f s (x2) of the second multi-modal sample image x2 are input into a feature recombination network to obtain a self-organizing feature [f c (x1), f s(x1), f c (x1), f s (x2);

[0083] Processing five, inputting the self-organizing feature [f c (x1), f s (x1)] corresponding to the first multi-modal sample image x1 into the attack identification network, predicting the biological attack type of the self-organizing feature [f c (x1), f s (x1)], and calculating the classification loss value Loss2(x1) of the biological attack type of the self-organizing feature [f c (x1), f s (x1)], which is the classification loss value Loss2(x1) corresponding to the first multi-modal sample image x1; c (x1), f s (x1)];

[0084] Processing six, inputting the self-organizing feature [f c (x1), f s (x1)] corresponding to the first multi-modal sample image x1 and the mixed feature [f c (x1), f s (x1)] corresponding to the self-organizing feature [f c (x1), f s (x2)] into the contrast learning network, calculating the similarity between the self-organizing feature [f c (x1), f s (x1)] and the mixed feature [f c (x1), f s (x1)] corresponding thereto, and according to the self-organizing feature [f c (x1), f s (x2)] and the mixed feature [f c (x1), f s (x1)] corresponding thereto, calculating the classification loss value Loss2(x1) of the biological attack type of the self-organizing feature [f c (x1), f sThe similarity S(x1, x2) between the self-organizing feature x1 and the mixed feature corresponding to the self-organizing feature x1 is calculated as the adversarial loss value Loss3(x1) corresponding to the first multi-modal sample image x1. The adversarial loss value corresponding to the multi-modal sample image x1 is the sum of the similarities of the self-organizing feature x1 and the mixed features corresponding thereto. The mixed feature corresponding to the self-organizing feature x1 is reorganized based on a general feature and a specific feature. The general feature based on which the mixed feature is reorganized and the general feature based on which the self-organizing feature corresponding to the mixed feature is reorganized come from the same multi-modal sample image, and the general features in the two are the same. The specific feature based on which the mixed feature is reorganized and the specific feature based on which the self-organizing feature corresponding to the mixed feature is reorganized come from different multi-modal sample images, and thus the specific features in the two can be different.

[0085] Processing seven, the overall loss value Loss(x1) corresponding to the first multi-modal sample image x1 is calculated according to the adversarial loss value Loss1(x1), the classification loss value Loss2(x1) and the contrast loss value Loss3(x1) corresponding to the first multi-modal sample image x1. The overall loss value can be obtained based on the weighted sum of the adversarial loss value, the classification loss value and the contrast loss value.

[0086] Processing eight, the network parameters of each network in the training model are adjusted by using the back propagation algorithm according to the overall loss value Loss(x1) corresponding to the first multi-modal sample image x1.

[0087] The embodiments of the present application take multi-modal sample images in different scenes as input, try to separate general features and specific features from the features of the multi-modal sample images, randomly combine the two features separated above by mixed integration, thereby enriching the feature diversity between the two features, and finally make the biological detection model have better adaptability to different scenes and biological attack types through the joint optimization of adversarial learning and comparison learning, thereby improving the generalization of the model to different scenes and different biological attack types. The biological detection model trained based on the training method provided by the embodiments of the present application can accurately detect and identify complex biological attacks in new scenes.

[0088] Further, the embodiments of the present application also provide a biological detection method for biological attack detection, Figure 3 A flowchart of a biological detection method provided by the embodiments of the present application is shown, as shown in Figure 3 The biological detection method provided by the embodiments of the present application can include the following steps:

[0089] S202, a multi-modal target image of a target scene is obtained.

[0090] The target multi-modal image is an image to be biologically identified in a target scene, and contains target information of an attack type to be identified, such as face image information.

[0091] The target scene can be a newly emerging, unknown scene to be biologically attacked and detected, and a plurality of face image acquisition devices, such as an RGB camera, an infrared camera, and a three-dimensional depth camera, are arranged in the scene. When a target to be biologically detected performs face recognition, a plurality of modal target images in the target scene can be simultaneously acquired by the plurality of face image acquisition devices, such as an RGB face image, an infrared face image, and a three-dimensional depth face image. Then, the plurality of modal target images in the target scene are fused to obtain a multi-modal target image of the target scene.

[0092] In an implementation manner, the biometric detection model can further include an image fusion network. The target image of each modal can be acquired from the plurality of image acquisition devices of the target scene, and the target images of the plurality of modalities are input into the image fusion network to obtain the multi-modal target image.

[0093] S204, inputting the multi-modal target image into the biometric detection model to obtain the biometric attack type of the target in the multi-modal target image.

[0094] The biometric detection model includes a feature extraction network, a feature reorganization network, and an attack recognition network. After the multi-modal target image is input into the biometric detection model, the common features and unique features of the multi-modal target image can be extracted by the feature extraction network, the common features and unique features of the multi-modal target image are reorganized by the feature mixing network to obtain the self-organizing features of the multi-modal target image, and the biometric attack type of the self-organizing features is recognized by the attack recognition network. Therefore, the biometric attack type of the target in the multi-modal target image is predicted by the biometric attack type of the self-organizing features, and the biometric attack detection of the target is performed.

[0095] It can be understood that the self-organizing features of the multi-modal target image are a representation form of the multi-modal target image, and the biometric attack type of the self-organizing features is consistent with the attack type of the multi-modal target image. The biometric attack type of the self-organizing features of the multi-modal target image is the biometric attack type of the target in the multi-modal target image.

[0096] The trained biometric detection model used in the embodiments of the present application can be trained by the training method of the biometric detection model provided in the above embodiments. The specific training method can refer to the foregoing embodiments, and will not be described here.

[0097] Based on the same technical concept, the embodiments of the present application further provide a training device of a biological detection model, which is used to execute the training method of the biological detection model provided by the above embodiments. Figure 4 As shown in the module schematic diagram of the training device of the biological detection model provided by the embodiments of the present application, Figure 4 the training device comprises a sample acquisition module 402, a feature extraction module 404, an adversarial loss calculation module 406, a feature recombination module 408, a classification loss calculation module 410, a contrast loss calculation module 412, and a training module 414.

[0098] The sample acquisition module 402 is configured to acquire a plurality of multi-modal sample images corresponding to a plurality of scenes, wherein the multi-modal sample images contain image information of a sample target to be detected.

[0099] The feature extraction module 404 is configured to input the plurality of multi-modal sample images into a feature extraction network of a training model, and extract a common feature and a unique feature of each multi-modal sample image in the plurality of multi-modal sample images.

[0100] The adversarial loss calculation module 406 is configured to input the common feature into a scene classification network of the training model, and calculate an adversarial loss value of the common feature.

[0101] The feature recombination module 408 is configured to input the common feature and the unique feature into a feature recombination network of the training model, recombine the common feature and the unique feature, and obtain a plurality of recombined features; wherein each recombined feature is recombined based on one common feature and one unique feature.

[0102] The classification loss module 410 is configured to input a self-recombined feature in the plurality of recombined features into an attack recognition network of the training model, and calculate a classification loss value of a biological attack type of the self-recombined feature; wherein the common feature and the unique feature based on which the self-recombined feature is recombined come from the same multi-modal sample image.

[0103] The contrast loss calculation module 412 is configured to input the plurality of recombined features into a contrast network of the training model, and calculate a contrast loss value between a mixed feature in the plurality of recombined features and the self-recombined feature; wherein the common feature and the unique feature based on which the mixed feature is recombined come from different multi-modal sample images.

[0104] The training module 414 is configured to calculate a total loss value according to the adversarial loss value, the classification loss value and the contrast loss value, train the training model according to the total loss value, and obtain the trained biological detection model according to the feature extraction network, the feature reorganization network and the attack identification network in the trained training model.

[0105] In an implementation manner, the obtaining of the plurality of multi-modal sample images corresponding to the plurality of scenes comprises:

[0106] For each scene in the plurality of preset scenes, the plurality of multi-modal sample images of the scene are obtained from the plurality of modal data sources.

[0107] The plurality of multi-modal sample images of the scene are obtained by performing image fusion on the plurality of multi-modal sample images.

[0108] In an implementation manner, the plurality of modal data sources comprises at least two data sources in the following data sources:

[0109] The plurality of modal data sources comprises at least two data sources in the following data sources:

[0110] In an implementation manner, the biological attack type comprises at least one of the following biological attack types:

[0111] The biological attack type comprises at least one of the following biological attack types:

[0112] In an implementation manner, the feature extraction network comprises a general feature extraction network.

[0113] The inputting of the plurality of multi-modal sample images into the feature extraction network of the training model comprises:

[0114] The inputting of the multi-modal sample image into the feature extraction network comprises the inputting of the multi-modal sample image into the general feature extraction network in the feature extraction network to extract the general feature of the multi-modal sample image.

[0115] The inputting of the general feature into the scene classification network of the training model comprises:

[0116] The inputting of the general feature into the scene classification network comprises the inputting of the general feature into the scene classification network to predict the scene category of the general feature.

[0117] The inputting of the general feature into the scene classification network comprises the inputting of the general feature into the scene classification network to predict the scene category of the general feature.

[0118] In an implementation manner, the feature extraction network further comprises: a unique feature extraction network.

[0119] The inputting the multi-modal sample image of each scene into the feature extraction network, extracting the general feature and the unique feature of the multi-modal sample of each scene, further comprises:

[0120] The unique feature of the multi-modal sample image is extracted from the multi-modal sample image by the unique feature extraction network in the feature extraction network.

[0121] In an implementation manner, the inputting the self-organizing feature in the plurality of reorganized features into the attack identification network of the training model, and calculating a classification loss value of the biological attack type of the self-organizing feature, comprises:

[0122] The self-organizing feature is inputted into the attack identification network to predict the biological attack type of the self-organizing feature.

[0123] Based on a preset classification loss function, a classification loss value of the biological attack type of the self-organizing feature is calculated according to the biological attack type prediction result of the self-organizing feature.

[0124] In an implementation manner, the inputting the plurality of reorganized features into the contrast network of the training model, and calculating a contrast loss value of the reorganized feature and the self-organizing feature in the reorganized feature, comprises:

[0125] A first reorganized feature pair is extracted from the plurality of reorganized features by the contrast network; wherein the first reorganized feature pair comprises two reorganized features, one of the two reorganized features is the self-organizing feature, and the other is a mixed feature corresponding to the self-organizing feature, and the mixed feature corresponding to the self-organizing feature is the same as the general feature of the self-organizing feature.

[0126] According to the similarity between the two reorganized features in the first reorganized feature pair, a contrast loss value corresponding to the first reorganized feature pair is calculated.

[0127] According to the contrast loss value corresponding to the first reorganized feature pair, a contrast loss value of the reorganized feature and the self-organizing feature in the reorganized feature is calculated.

[0128] In an implementation manner, the training the training model according to the total loss value comprises:

[0129] According to the total loss value, the network parameters of each network in the training model are adjusted by using a back propagation algorithm.

[0130] The embodiment of the application obtains a multi-modal image as a sample image, separates the features of the multi-modal sample image into general features and unique features, uses the general features to represent general elements of the multi-modal sample image, such as semantic features, target attribute features, etc., uses the unique features to represent exclusive elements of the multi-modal sample image, such as scene-related features, biological attack type-related features, etc., and then jointly optimizes the training model through adversarial learning and contrastive learning, wherein the adversarial learning is used to strengthen the general features, so that the general features are not specific to a specific scene, and the contrastive learning is used to strengthen the biological attack type-related features in the unique features and suppress the scene-related features in the unique features, so that the biological detection model obtained after training has better scene generalization and can better adapt to a new scene. When a new scene occurs, the biological detection model provided by the embodiment of the application can effectively detect and identify various types of complex biological attack types in the new scene.

[0131] It should be noted that the training device of the biological detection model provided by the embodiment of the application is based on the same inventive concept as the training method of the biological detection model provided by the embodiment of the application, and therefore the specific implementation of the training device of the biological detection model can be referred to the implementation of the corresponding training method of the biological detection model described above, and the repeated parts will not be described again.

[0132] Corresponding to the biological detection method provided by the above-mentioned embodiment, based on the same technical concept, the embodiment of the application further provides a biological detection device for executing the biological detection method provided by the above-mentioned embodiment. Figure 5 is a module schematic diagram of the biological detection device provided by the embodiment of the application, as Figure 5 shown, the biological detection device includes an image acquisition module 502 and a prediction module 504, wherein:

[0133] The image acquisition module 502 is configured to acquire a multi-modal target image of a target scene, wherein the multi-modal target image contains image information of a target to be biologically detected.

[0134] The prediction module 504 is configured to input the multi-modal target image into the trained biological detection model to predict a biological attack type of the target in the multi-modal target image.

[0135] The biological detection model can include a feature extraction network, a feature recombination network, and an attack identification network. After a multi-modal target image is input into the biological detection model, the feature extraction network can extract common features and unique features of the multi-modal target image, the feature mixing network can recombine the common features and the unique features of the multi-modal target image to obtain self-organizing features of the multi-modal target image, and the attack identification network can identify a biological attack type of the self-organizing features. Thus, the biological attack type of the target in the multi-modal target image is predicted by the biological attack type of the self-organizing features, and biological attack detection is performed on the target.

[0136] The biological detection model is trained by the training method of the biological detection model provided in the above embodiments.

[0137] It should be noted that the biological detection device provided in the embodiments of the present application is based on the same inventive concept as the training method of the biological detection model provided in the embodiments of the present application. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding training method of the biological detection model described above, and the repeated parts will not be described again.

[0138] Further, corresponding to the training method of the biological detection model and the biological detection method provided in the above embodiments, based on the same technical concept, the embodiments of the present application also provide a computer device, which is used to execute the training method of the biological detection model or is used to execute the biological detection method as shown in Figure 6 .

[0139] The computer device can have great differences due to different configurations or performances, and can include one or more processors and memories. The memory can store one or more storage applications or data. The memory can be temporary storage or persistent storage. The storage application stored in the memory can include one or more modules, and each module can include a series of computer executable instructions in the computer device. Further, the processor can be configured to communicate with the memory and execute a series of computer executable instructions in the memory on the computer device. The computer device can also include one or more power supplies, one or more wired or wireless network interfaces, one or more input and output interfaces, one or more keyboards, etc.

[0140] In a specific embodiment, the computer device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and the one or more programs can include one or more modules, and each module can include a series of computer executable instructions in the computer device, and the one or more processors are configured to execute the one or more programs including the following computer executable instructions:

[0141] obtain a plurality of multi-modal sample images corresponding to a plurality of scenes, the multi-modal sample images containing image information of a sample target to be biologically detected;

[0142] input the plurality of multi-modal sample images into a feature extraction network of a training model, extract a common feature and a unique feature of each multi-modal sample image in the plurality of multi-modal sample images;

[0143] input the common feature into a scene classification network of the training model, and calculate an adversarial loss value of the common feature;

[0144] input the common feature and the unique feature into a feature reorganization network of the training model, reorganize the common feature and the unique feature to obtain a plurality of reorganized features; wherein each reorganized feature is reorganized based on a common feature and a unique feature;

[0145] input a self-organizing feature in the plurality of reorganized features into an attack recognition network of the training model, and calculate a classification loss value of a biological attack type of the self-organizing feature; wherein the common feature and the unique feature based on which the self-organizing feature is reorganized come from the same multi-modal sample image;

[0146] input the plurality of reorganized features into a comparison network of the training model, and calculate a comparison loss value between a mixed feature in the plurality of reorganized features and the self-organizing feature, wherein the common feature and the unique feature based on which the mixed feature is reorganized come from different multi-modal sample images;

[0147] calculate a total loss value according to the adversarial loss value, the classification loss value and the comparison loss value, train the training model according to the total loss value, and obtain a trained biological detection model according to the feature extraction network, the feature reorganization network and the attack recognition network in the trained training model.

[0148] In an implementation manner, the obtaining a plurality of multi-modal sample images corresponding to a plurality of scenes comprises:

[0149] for each scene in a plurality of preset scenes, obtaining a plurality of multi-modal sample images of the scene from a plurality of modal data sources;

[0150] performing image fusion on the plurality of multi-modal sample images to obtain a multi-modal sample image of the scene.

[0151] In an implementation manner, the plurality of modal data sources comprises at least two data sources in the following:

[0152] an RGB data source, an infrared data source and a three-dimensional depth data source.

[0153] In an implementation manner, the biological attack type includes at least one of the following:

[0154] a paper class biological attack, an electronic screen class biological attack, a mask class biological attack, and a non-biological attack.

[0155] In an implementation manner, the feature extraction network includes a general feature extraction network;

[0156] The feature extraction network inputs the plurality of multi-modal sample images into a training model, extracts general features and unique features of each multi-modal sample image in the plurality of multi-modal sample images, and includes:

[0157] The multi-modal sample image is input into the feature extraction network, and the general features of the multi-modal sample image are extracted by a general feature extraction network in the feature extraction network.

[0158] The general features are input into a scene classification network of the training model, and an adversarial loss value of the general features is calculated and generated, and includes:

[0159] The general features are input into the scene classification network, and a scene category of the general features is predicted.

[0160] The scene classification loss value of the general features is calculated according to the scene category prediction result of the general features, and the adversarial loss value between the scene classification network and the general feature extraction network is calculated according to the scene classification loss value of the general features.

[0161] In an implementation manner, the feature extraction network further includes a unique feature extraction network.

[0162] The multi-modal sample image of the at least one scene is input into the feature extraction network, and the general features and the unique features of the multi-modal sample of each scene are extracted, and further includes:

[0163] The unique features of the multi-modal sample image are extracted from the multi-modal sample image by a unique feature extraction network in the feature extraction network.

[0164] In an implementation manner, the attack recognition network inputs the self-organizing features in the plurality of reorganized features into the training model, calculates a classification loss value of a biological attack type of the self-organizing features, and includes:

[0165] The self-organizing features are input into the attack recognition network, and a biological attack type of the self-organizing features is predicted.

[0166] Based on a preset classification loss function, a classification loss value of a biological attack type of the self-organizing features is calculated according to a biological attack type prediction result of the self-organizing features.

[0167] In an implementation manner, the contrast network inputs the plurality of reorganized features into the training model, and calculates a contrast loss value of the reorganized features and the self-organizing features in the reorganized features, including:

[0168] extracting a first reorganized feature pair from the plurality of reorganized features through the contrast network; wherein the first reorganized feature pair includes two reorganized features, one of the two reorganized features is the self-organizing feature, and the other is a mixed feature corresponding to the self-organizing feature, and the mixed feature corresponding to the self-organizing feature is the same as the general feature of the self-organizing feature;

[0169] According to the similarity between the two reorganized features in the first reorganized feature pair, a contrast loss value corresponding to the first reorganized feature pair is calculated;

[0170] According to the contrast loss value corresponding to the first reorganized feature pair, the contrast loss value of the reorganized features and the self-organizing features in the reorganized features is calculated.

[0171] In an implementation manner, the training of the training model according to the total loss value includes:

[0172] According to the total loss value, the network parameters of each network in the training model are adjusted by using a back propagation algorithm.

[0173] In another specific embodiment, a computer device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs can include one or more modules, and each module can include a series of computer executable instructions in the computer device, and is configured to be executed by one or more processors. The one or more programs include computer executable instructions for:

[0174] Obtain a multi-modal target image of a target scene, wherein the multi-modal target image contains image information of a target to be biologically detected;

[0175] Input the multi-modal target image into the trained biological detection model to predict the biological attack type of the target in the multi-modal target image;

[0176] The biological detection model is trained by the training method of the biological detection model provided in the above embodiments.

[0177] It should be noted that the embodiments of the computer device in the present application and the embodiments of the training method of the biological detection model in the present application are based on the same inventive concept, and therefore the specific implementation of this embodiment can be referred to the foregoing implementation of the corresponding training method of the biological detection model, and the repeated parts will not be described again.

[0178] Further, corresponding to the training method of the biological detection model and the biological detection method provided by the embodiments of the present application, based on the same technical concept, the embodiments of the present application also provide a storage medium for storing computer executable instructions;

[0179] In a specific embodiment, the storage medium can be a U disk, an optical disk, a hard disk, etc. The computer executable instructions stored in the storage medium can implement the following processes when executed by a processor:

[0180] Obtain a plurality of multi-modal sample images corresponding to a plurality of scenes, wherein the multi-modal sample images contain image information of sample targets to be biologically detected;

[0181] Input the plurality of multi-modal sample images into a feature extraction network of a training model, extract general features and unique features of each multi-modal sample image in the plurality of multi-modal sample images;

[0182] Input the general features into a scene classification network of the training model, and calculate an adversarial loss value of the general features;

[0183] Input the general features and the unique features into a feature reorganization network of the training model, reorganize the general features and the unique features to obtain a plurality of reorganized features; wherein each reorganized feature is reorganized based on one general feature and one unique feature;

[0184] Input a self-organizing feature in the plurality of reorganized features into an attack recognition network of the training model, calculate a classification loss value of a biological attack type of the self-organizing feature; wherein the general feature and the unique feature based on which the self-organizing feature is reorganized come from the same multi-modal sample image;

[0185] Input the plurality of reorganized features into a comparison network of the training model, calculate a comparison loss value between a mixed feature in the plurality of reorganized features and the self-organizing feature, wherein the general feature and the unique feature based on which the mixed feature is reorganized come from different multi-modal sample images;

[0186] Calculate a total loss value according to the adversarial loss value, the classification loss value and the comparison loss value, train the training model according to the total loss value, and obtain a trained biological detection model according to the feature extraction network, the feature reorganization network and the attack recognition network in the trained training model.

[0187] In an implementation manner, the obtaining the plurality of multi-modal sample images corresponding to the plurality of scenes comprises:

[0188] For each scene in the plurality of preset scenes, obtaining a plurality of modal sample images of the scene from a plurality of modal data sources;

[0189] Performing image fusion on the plurality of modal sample images to obtain a multi-modal sample image of the scene.

[0190] In an implementation manner, the plurality of modal data sources comprises at least two data sources from the following:

[0191] An RGB data source, an infrared data source, and a three-dimensional depth data source.

[0192] In an implementation manner, the biological attack type comprises at least one of the following:

[0193] Paper-based biological attacks, electronic screen-based biological attacks, mask-based biological attacks, and non-biological attacks.

[0194] In an implementation manner, the feature extraction network comprises a general feature extraction network;

[0195] The inputting the plurality of multi-modal sample images into the feature extraction network of the training model to extract general features and unique features of each multi-modal sample image in the plurality of multi-modal sample images comprises:

[0196] The inputting the multi-modal sample image into the feature extraction network, and extracting general features of the multi-modal sample image through a general feature extraction network in the feature extraction network;

[0197] The inputting the general features into the scene classification network of the training model to calculate an adversarial loss value of the general features comprises:

[0198] The inputting the general features into the scene classification network to predict a scene category of the general features;

[0199] Calculating a scene classification loss value of the general features according to the scene category prediction result of the general features, and calculating an adversarial loss value between the scene classification network and the general feature extraction network according to the scene classification loss value of the general features.

[0200] In an implementation manner, the feature extraction network further comprises a unique feature extraction network;

[0201] The inputting the multi-modal sample image of the at least one scene into the feature extraction network to extract general features and unique features of the multi-modal sample of each scene further comprises:

[0202] extracting, by a unique feature extraction network in the feature extraction network, unique features of the multi-modal sample image from the multi-modal sample image.

[0203] In an implementation manner, the inputting the self-organizing feature in the plurality of reorganized features into the attack identification network of the training model, calculating a classification loss value of a biological attack type of the self-organizing feature, comprises:

[0204] inputting the self-organizing feature into the attack identification network, and predicting a biological attack type of the self-organizing feature;

[0205] calculating the classification loss value of the biological attack type of the self-organizing feature based on a preset classification loss function according to a biological attack type prediction result of the self-organizing feature.

[0206] In an implementation manner, the inputting the self-organizing feature in the plurality of reorganized features into the attack identification network of the training model, calculating a classification loss value of a biological attack type of the self-organizing feature, comprises:

[0207] extracting a first reorganized feature pair from the plurality of reorganized features by the contrast network, wherein the first reorganized feature pair comprises two reorganized features, one of the two reorganized features is the self-organizing feature, and the other is a mixed feature corresponding to the self-organizing feature, and the mixed feature corresponding to the self-organizing feature is the same as the universal feature of the self-organizing feature;

[0208] calculating a contrast loss value corresponding to the first reorganized feature pair according to a similarity between the two reorganized features in the first reorganized feature pair;

[0209] calculating the contrast loss value of the reorganized feature and the self-organizing feature in the reorganized feature according to the contrast loss value corresponding to the first reorganized feature pair.

[0210] In an implementation manner, the training the training model according to the total loss value comprises:

[0211] adjusting network parameters of each network in the training model by using a back propagation algorithm according to the total loss value.

[0212] In another specific embodiment, the storage medium can be a U disk, an optical disk, a hard disk, etc., and the computer executable instructions stored in the storage medium can realize the following flow when executed by the processor:

[0213] obtaining a multi-modal target image of a target scene, wherein the multi-modal target image comprises image information of a target to be biologically detected;

[0214] input the multi-modal target image into the trained biological detection model, and predict a biological attack type of the target in the multi-modal target image,

[0215] The biological detection model is trained by the biological detection model training method provided in the embodiments of the present application.

[0216] It should be noted that the embodiments of the storage medium in the present application are based on the same inventive concept as the embodiments of the biological detection model training method or the biological detection method in the present application, and therefore the specific implementation of this embodiment can be referred to the implementation of the corresponding biological detection model training method described above, and the repeated parts will not be described herein.

[0217] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0218] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, system or computer program product. Therefore, the embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) containing computer usable program code.

[0219] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems) and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions executed by the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The device that implements the functions specified in one or more flows and / or blocks

[0220] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0221] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0222] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0223] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0224] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic disks storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that is accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0225] It is also to be noted that the terms "comprising", "including", and "having" or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0226] Embodiments of the present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. One or more embodiments of the present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0227] Embodiments of the present application are described with reference to the attached figures, wherein the same reference numerals are used to denote the same elements throughout the figures. The embodiments of the present application can be implemented in any number of hardware, software, or combinations of hardware and software. The embodiments of the present application can be implemented in one or more computer programs or code that can be executed by a computer or processor, whether program modules, objects, or the like. The embodiments of the present application can be implemented in one or more computer programs or code that can be executed by a computer or processor, whether program modules, objects, or the like. The computer programs or code can be stored in any type of computer readable medium or memory device(s) or storage media suitable for the use in a computer program implementation.

[0228] The foregoing is merely illustrative of the principles of the application and various modifications can be made by persons skilled in the art without departing from the scope and nature of the application as disclosed in the foregoing. The above description is therefore not to be taken in a limiting sense, and the scope of the application is to be determined solely by the claims.

Claims

1. A training method of a biological detection model, the method comprising: obtaining a plurality of multi-modal sample images corresponding to a plurality of scenes; wherein the multi-modal sample images contain image information of sample targets to be biologically detected; inputting the plurality of multi-modal sample images into a feature extraction network of a training model to extract a common feature and a unique feature of each multi-modal sample image in the plurality of multi-modal sample images; inputting the common feature into a scene classification network of the training model to calculate an adversarial loss value of the common feature; inputting the common feature and the unique feature into a feature recombination network of the training model to recombine the common feature and the unique feature to obtain a plurality of recombined features; wherein each recombined feature is recombined based on one common feature and one unique feature; inputting a self-recombined feature in the plurality of recombined features into an attack recognition network of the training model to calculate a classification loss value of a biological attack type of the self-recombined feature; wherein the common feature and the unique feature on which the self-recombined feature is recombined come from the same multi-modal sample image; inputting the plurality of recombined features into a comparison network of the training model to calculate a comparison loss value between a mixed feature in the plurality of recombined features and the self-recombined feature; wherein the common feature and the unique feature on which the mixed feature is recombined come from different multi-modal sample images; calculating an overall loss value according to the adversarial loss value, the classification loss value, and the comparison loss value, training the training model according to the overall loss value, and obtaining a trained biological detection model according to the feature extraction network, the feature recombination network, and the attack recognition network in the trained training model.

2. The training method of claim 1, wherein the obtaining a plurality of multi-modal sample images corresponding to a plurality of scenes comprises: for each scene in a plurality of preset scenes, obtaining a plurality of modal sample images of the scene from a plurality of modal data sources; performing image fusion on the plurality of modal sample images to obtain a multi-modal sample image of the scene.

3. The training method of claim 2, wherein the plurality of modal data sources comprise at least two of the following data sources: an RGB data source, an infrared data source, and a three-dimensional depth data source.

4. The training method of claim 1, wherein the biological attack type comprises at least one of the following: a paper-based biological attack, an electronic screen-based biological attack, a mask-based biological attack, and a non-biological attack.

5. The training method of claim 1, the feature extraction network comprising: a common feature extraction network; the inputting the plurality of multi-modal sample images into a feature extraction network of a training model to extract a common feature and a unique feature of each multi-modal sample image in the plurality of multi-modal sample images comprises: inputting the multi-modal sample image into the feature extraction network to extract the common feature of the multi-modal sample image by a common feature extraction network in the feature extraction network; the inputting the common feature into a scene classification network of the training model to calculate an adversarial loss value of the common feature comprises: inputting the common feature into the scene classification network to predict a scene category of the common feature; and The scene classification loss value of the general feature is calculated according to the scene category prediction result of the general feature, and the adversarial loss value between the scene classification network and the general feature extraction network is calculated according to the scene classification loss value of the general feature.

6. The training method of claim 5, the feature extraction network further comprising: The specific feature extraction network; The inputting the multi-modal sample image of the at least one scene into the feature extraction network, and extracting the general feature and the specific feature of the multi-modal sample of each scene further includes: The specific feature extraction network in the feature extraction network extracts the specific feature of the multi-modal sample image from the multi-modal sample image.

7. The training method of claim 1, wherein the inputting the self-assembly feature in the plurality of recombined features into the attack identification network of the training model to calculate a classification loss value of the biological attack type of the self-assembly feature comprises: inputting the self-assembly feature into the attack identification network to predict the biological attack type of the self-assembly feature; calculating the classification loss value of the biological attack type of the self-assembly feature according to the biological attack type prediction result of the self-assembly feature based on a preset classification loss function.

8. The training method of claim 1, wherein the inputting the plurality of recombined features into the contrast network of the training model to calculate a contrast loss value of the recombined feature and the self-assembly feature in the recombined feature comprises: extracting a first recombined feature pair from the plurality of recombined features through the contrast network; wherein the first recombined feature pair includes two recombined features, one of the two recombined features is the self-assembly feature, and the other is a mixed feature corresponding to the self-assembly feature, and the mixed feature corresponding to the self-assembly feature is the same as the general feature of the self-assembly feature; calculating a contrast loss value corresponding to the first recombined feature pair according to the similarity between the two recombined features in the first recombined feature pair; calculating the contrast loss value of the recombined feature and the self-assembly feature in the recombined feature according to the contrast loss value corresponding to the first recombined feature pair.

9. The training method of claim 1, wherein the training the training model according to the total loss value comprises: adjusting the network parameters of each network in the training model using a back propagation algorithm according to the total loss value.

10. A biological detection method, the method comprising: obtaining a multi-modal target image of a target scene, wherein the multi-modal target image contains image information of a target to be biologically detected; inputting the multi-modal target image into a trained biological detection model to predict the biological attack type of the target in the multi-modal target image; wherein the biological detection model is trained by the biological detection model training method of any one of claims 1-9.

11. A biological detection model training device, the device comprising: a sample acquisition module configured to obtain a plurality of multi-modal sample images corresponding to a plurality of scenes, wherein the multi-modal sample images contain image information of sample targets to be biologically detected; a feature extraction network configured to extract general features and specific features of the multi-modal sample of each scene from the multi-modal sample image; The feature extraction module is configured to input the plurality of multi-modal sample images into a feature extraction network of a training model, and extract common features and unique features of each multi-modal sample image in the plurality of multi-modal sample images. The adversarial loss calculation module is configured to input the common features into a scene classification network of the training model, and calculate an adversarial loss value of the common features. The feature recombination module is configured to input the common features and the unique features into a feature recombination network of the training model, recombine the common features and the unique features, and obtain a plurality of recombined features; each recombined feature is recombined based on one common feature and one unique feature. The classification loss module is configured to input a self-recombined feature in the plurality of recombined features into an attack recognition network of the training model, and calculate a classification loss value of a biological attack type of the self-recombined feature; the common feature and the unique feature on which the self-recombined feature is recombined come from the same multi-modal sample image. The contrast loss calculation module is configured to input the plurality of recombined features into a contrast network of the training model, and calculate a contrast loss value between a mixed feature in the plurality of recombined features and the self-recombined feature; the common feature and the unique feature on which the mixed feature is recombined come from different multi-modal sample images. The training module is configured to calculate a total loss value according to the adversarial loss value, the classification loss value, and the contrast loss value, train the training model according to the total loss value, and obtain a trained biological detection model according to a feature extraction network, a feature recombination network, and an attack recognition network in the trained training model.

12. A biological detection device, the device comprising: An image acquisition module configured to acquire a multi-modal target image of a target scene, wherein the multi-modal target image comprises image information of a target to be biologically detected; A prediction module configured to input the multi-modal target image into a trained biological detection model, and predict a biological attack type of the target in the multi-modal target image. The biological detection model is trained by the biological detection model training method of any one of claims 1-9.

13. A computer device, comprising: The device comprises: a processor; and a memory arranged to store computer-executable instructions configured to be executed by the processor, the computer-executable instructions comprising instructions for performing steps in the method of any one of claims 1-9 or 10.

14. A storage medium, characterized by The storage medium is configured to store computer-executable instructions, which cause a computer to perform the method of any one of claims 1-9 or 10.

Images