Attack detection model training method and system and cross-device performance evaluation method
By calculating the similarity between the training image and multiple real text/misreal texts and determining the comprehensive loss function, the problem that the model in the prior art is difficult to understand the potential connection between the image and different description texts is solved, and the accuracy and generalization performance of the attack detection model are improved.
Patent Information
- Application Number
- CN202411810748.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-04-01
AI Technical Summary
Existing attack detection methods are difficult to fully explore the potential connection between images and different description texts, which leads to difficulties in the model in understanding the semantic information contained in the text, limiting the accuracy of the model.
By calculating the similarity between the training image and multiple real text/misreal text, the comprehensive loss function is determined, and the attack detection model is trained using this loss function to enhance the model's understanding of the semantic association between the image and the text.
It improves the detection ability of the model in complex situations, enhances alertness to carefully constructed deceptive texts, improves generalization performance, and makes the model training process clearer and more efficient.
Smart Images
Figure CN119740042A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of multimodal learning, and in particular relates to a training method and system for an attack detection model and a cross-device performance evaluation method. Background Art
[0002] Among biometric technologies, iris recognition is widely used in key areas such as identity authentication and security monitoring due to its high accuracy and stability. However, with the continuous evolution of forgery technology and attack methods, iris presentation attack detection faces increasing challenges.
[0003] Existing attack detection methods attempt to improve detection results by combining text and image information. Specifically, these methods first extract features of text and image separately, and then use contrastive learning to let the model learn the matching relationship between text and image. However, this method still faces many challenges in practical applications.
[0004] However, although the model can extract features from text and images, it is often difficult to fully explore the potential connections between images and different description texts. Due to the diversity and complexity of text descriptions, the model has certain difficulties in understanding the semantic information contained in the text, which makes it difficult for the model to learn sufficiently rich semantic information during training, thus limiting the accuracy of the model. Summary of the invention
[0005] To solve the above problems, the present invention discloses a training method and system for an attack detection model and a cross-device performance evaluation method.
[0006] The present invention discloses a training method for an attack detection model, comprising the following steps:
[0007] Obtain training images, a real text set containing M real texts, and a false text set containing M false texts;
[0008] Determine a first loss function according to the training image, any true text, and any distorted text;
[0009] According to the authenticity of the training image, respectively determine the first similarities between the training image and the N real texts / N distorted texts, determine a second loss function according to the N first similarities and a preset label matrix, and determine a third loss function according to the N first similarities;
[0010] A comprehensive loss function is determined according to the first loss function, the second loss function and the third loss function, and the attack detection model is trained using the comprehensive loss function.
[0011] Preferably, the first loss function is determined according to the training image, any real text and any distorted text, specifically:
[0012] Respectively extracting image features of the training image, real text features of any real text, and distorted text features of any distorted text;
[0013] Determine a second similarity based on the image feature and the real text feature of any real text;
[0014] Determine a third similarity based on the image feature and the distorted text feature of any distorted text;
[0015] A first loss function is determined according to the second similarity and the third similarity.
[0016] Preferably, the first loss function is determined according to the second similarity and the third similarity, specifically:
[0017] Determining a model probability prediction distribution according to the second similarity and the third similarity;
[0018] A first loss function is determined based on the model probability prediction distribution.
[0019] Preferably, according to the authenticity of the training image, the first similarity between the training image and the N real texts / N false texts is determined respectively, specifically:
[0020] Extract X projection features of the training image;
[0021] According to the authenticity of the training images, extract the target text features of N real texts / N false texts;
[0022] By using matrix multiplication, the first similarity between each projection feature and each target text feature is determined one by one.
[0023] Preferably, the second loss function is determined according to the N first similarities and a preset label matrix, specifically:
[0024] Determine a prediction matrix according to the N first similarities;
[0025] A second loss function is determined according to the prediction matrix and the label matrix.
[0026] Preferably, the third loss function is determined according to the N first similarities, specifically:
[0027] One by one, obtain the square of the difference between any two first similarities;
[0028] A third loss function is determined based on the plurality of squared differences.
[0029] Optionally, the first loss function is a cross entropy loss function, the second loss function is a self-supervised contrastive learning loss function, and the third loss function is a mean square error loss function.
[0030] The present invention also discloses a training system for an attack detection model, comprising:
[0031] An image and text acquisition module is used to acquire training images, a real text set containing M real texts, and a false text set containing M false texts;
[0032] A first loss function determination module, used to determine a first loss function according to the training image, any real text and any distorted text;
[0033] A second loss function and a third loss function determination module, used to determine the first similarities between the training image and N real texts / N distorted texts respectively according to the authenticity of the training image, and determine the second loss function according to the N first similarities and a preset label matrix, and determine the third loss function according to the N first similarities;
[0034] A model training module is used to determine a comprehensive loss function based on the first loss function, the second loss function and the third loss function, and use the comprehensive loss function to train the attack detection model.
[0035] The present invention also discloses a cross-device performance evaluation method of an attack detection model, comprising the following steps:
[0036] Acquire a first preset number of data sets, each data set coming from a different public database;
[0037] Each data set is used as a test set in turn, and the remaining data sets are combined into a joint training set accordingly, to obtain a first preset number of training-test sets, each training-test set including a test set and a joint training set;
[0038] Using the training method of the attack detection model and the joint training set in each training-test set, the attack detection model is trained;
[0039] Using the test set in each training-test set and the corresponding trained attack detection model to perform attack detection, and obtain a first preset number of attack detection results;
[0040] Based on a first preset number of attack detection results, the cross-device performance of the attack training model is evaluated.
[0041] Compared with the prior art, the present invention has the following beneficial effects:
[0042] (1) By calculating the similarity between the training image and multiple real texts / false texts, the model can better understand the semantic association between the image and multiple texts. This deep semantic understanding helps the model to more accurately determine whether the text truly describes the image content, thereby enhancing its detection ability in complex situations, which helps the model remain highly alert when faced with carefully constructed deceptive texts;
[0043] (2) By introducing multiple similarity calculations between real texts and distorted texts and images, the model can learn richer feature representations. These feature representations not only cover the direct relationship between images and texts, but also the potential relationship between them. This enables the model to better use the learned knowledge to make judgments when faced with new and unseen images and texts, thereby improving its generalization performance;
[0044] (3) The present invention involves multiple loss functions and constitutes a comprehensive loss function, which makes the model training process clearer and more organized, helps to reduce the uncertainty in the training process and improve the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a flow chart of the training method of the attack detection model disclosed in the present invention;
[0046] Figure 2 It is a schematic diagram of the third loss function determination process disclosed in the present invention. DETAILED DESCRIPTION
[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.
[0048] like Figure 1 As shown, the present invention discloses a training method for an attack detection model, comprising the following steps:
[0049] S1, obtaining training images, a real text set containing M real texts, and a false text set containing M false texts;
[0050] In one embodiment, M is 6, that is, the true text set and the false text set each contain six corresponding texts (as shown in Table 1). The descriptions of these texts make the iris image consistent with the natural language category description so that the model can learn clues of specific categories.
[0051] Table 1: True text set and false text set
[0052]
[0053] S2, determining a first loss function according to the training image, any true text, and any false text, and determining a second loss function according to the training image, each true text, and each false text;
[0054] Preferably, the first loss function is determined according to the training image, any real text and any distorted text, specifically:
[0055] Extracting image features of training images, real text features of any real text, and distorted text features of any distorted text respectively;
[0056] Determine a second similarity based on the image feature and the real text feature of any real text;
[0057] Determine a third similarity based on the image feature and the distorted text feature of any distorted text;
[0058] A first loss function is determined according to the second similarity and the third similarity.
[0059] Preferably, the first loss function is determined according to the second similarity and the third similarity, specifically:
[0060] Determining a model probability prediction distribution according to the second similarity and the third similarity;
[0061] The first loss function is determined based on the model probability prediction distribution.
[0062] Preferably, the first loss function is determined according to the model probability prediction distribution, specifically: the predicted category label is determined according to the model probability prediction distribution; the first loss function is determined according to the predicted category label and the correct category label, and the correct category label is known during the model training process.
[0063] In one embodiment, the first loss function is a cross entropy loss function, and the determination process thereof is as follows:
[0064] Use the image encoder to extract the image features of the training image, and use the text encoder to calculate the real text features z of any real text r and the false text feature z of any false text s , and calculate the cosine similarity between the image features and the two different categories of text features, namely the second similarity and the third similarity mentioned above.
[0065] Cosine similarity is regarded as class logic and is passed to cross entropy loss calculation; specifically, in the model reasoning stage, the cosine similarity between the image feature and the two text features is input into the model, and the model internally outputs the model probability prediction distribution corresponding to the training image and each category of text, a total of two, and the category corresponding to the larger model probability prediction distribution is taken as the predicted category label. The model used in the present invention is the CLIP model. In other embodiments, common classification models can also be used, such as logistic regression models, decision tree models, support vector machines, random forest models, naive Bayes classification, neural network models, etc.
[0066] That is to say, in the inference phase, during the complete calculation process, the model will obtain two model probability prediction distributions for the two text categories. Formula (1) only takes the calculation process of the larger model probability prediction distribution as an example for explanation [correspondingly, the model probability prediction distribution in formula (1) corresponds to the predicted category label]:
[0067]
[0068] In formula 1, is the model probability prediction distribution of the predicted category label; τ is the temperature parameter; is a preset value; n is the number of categories; in this embodiment, n=2; is the predicted category label; is the cosine similarity between the predicted category label and the image feature; sim(x, z r ) is the image feature x and the real text feature z of any real text r The cosine similarity of sim(x, z s ) is the image feature x and the distorted text feature z of any distorted text s The cosine similarity of .
[0069] The calculation process of the first loss function is shown in formula (2):
[0070]
[0071] In formula (2), L ce is the first loss function; y i is the correct category label, is the known value; is the predicted category label, which is obtained in the model inference phase; MERGEFORMAT is the merging function.
[0072] S3. Determine first similarities between the training image and the N real texts / N false texts respectively according to the authenticity of the training image;
[0073] According to the authenticity of the training image, the first similarity between the training image and the N real texts / N false texts is determined respectively, specifically:
[0074] Extract X projection features of the training image;
[0075] According to the authenticity of the training images, extract the target text features of N real texts / N false texts;
[0076] By using matrix multiplication, the first similarity between each projection feature and each target text feature is determined one by one.
[0077] The second loss function is determined according to the N first similarities and the preset label matrix, specifically:
[0078] Determine a prediction matrix according to the N first similarities;
[0079] The second loss function is determined based on the prediction matrix and the label matrix.
[0080] In one embodiment, the second loss function is a self-supervised contrastive learning loss function, and the values of X and N are both 2. The determination process is as follows:
[0081] 1. Calculate the eigenvector
[0082] First, by performing different transformations on the given image I, two views are generated, which are denoted as and The view features of the two transformed images are then extracted through an image encoder. These features are then further processed using a nonlinear projection network H, and finally, a contrastive loss is applied to these projected features.
[0083]
[0084] In formula (3), x v1 For View I v1 View features of x v2 For View The view feature of h1 is the view feature x v1 The projection feature; h2 is the view feature x v 2 projection features.
[0085] Then, according to the authenticity of the training images, two real texts / two false texts are randomly selected; each real text and each false text are tokenized and encoded to obtain the corresponding two target text features.
[0086] 2. Normalized feature vector
[0087] The projected features and each target text feature are normalized separately to ensure that they are compared at the same scale.
[0088] 3. Calculate the similarity matrix
[0089] Using the normalized results, the cosine similarity between each projection feature and each target text feature is calculated one by one through matrix multiplication to obtain the cosine similarity matrix.
[0090] 4. Build tags
[0091] Construct a label matrix to identify which features are positive label samples (training images and real text / distorted text are of the same category) and which are negative label samples (training images and real text / distorted text are of different categories). The label matrix contains positive label samples and negative label samples, which contain the correct correspondence between the training image and the two real texts / two distorted texts mentioned above.
[0092] 5. Select positive and negative samples
[0093] Positive and negative samples are extracted from the cosine similarity matrix and combined into the final prediction matrix.
[0094] Generally, the value of cosine similarity is (-1, 1). The closer to -1, the lower the similarity between the image and text, and the closer to the negative sample; the closer to 1, the higher the similarity between the image and text, and the closer to the positive sample. In this step, according to the preset threshold and each cosine similarity in the cosine similarity matrix, the model divides and extracts the predicted positive samples and the predicted negative samples to form a prediction matrix.
[0095] 6. Calculate the loss
[0096] Finally, the second loss function is obtained by calculating the prediction matrix and the label matrix. Specifically, in this embodiment, the second loss function is InfoNCE (info_nce_loss) loss, and InfoNCE loss is one of the self-supervised contrastive learning loss functions.
[0097] The InfoNCE loss function implements a similarity matrix-based method to calculate the cosine similarity between images and texts, and optimizes the model by constructing positive and negative samples to better align the feature representations of images and texts.
[0098] Preferably, according to the authenticity of the training image, the similarities between the training image and the N real texts / N false texts are determined respectively, and the third loss function is determined according to the N similarities, specifically:
[0099] Extract the projection features of the training images,
[0100] According to the authenticity of the training images, target text features of N real texts / N false texts are extracted; whether the training images are real biological images or forged (false) biological images is known during the training process, so in the process of determining the third loss function, it is only necessary to extract the text corresponding to the correct category of the training images.
[0101] Furthermore, the text features of any real text, the text features of any distorted text, and the target text features mentioned in the present invention all represent the text features of the real text / distorted text extracted by the text encoder. The different names are only for the convenience of reference or description, and the meaning of the text features is the same.
[0102] Using matrix multiplication, the first similarity between the projection feature and each target text feature is determined one by one.
[0103] Preferably, the third loss function is determined according to the multiple first similarities, specifically:
[0104] One by one, obtain the square of the difference between any two first similarities;
[0105] A third loss function is determined based on the plurality of squared differences.
[0106] In one embodiment, N=2, such as Figure 2 As shown, the third loss function is the mean square error loss function, and its determination process is as follows:
[0107] The training image is transformed into two views, and the projection features of the two views are determined respectively. The process is similar to the related process in the self-supervised contrastive learning loss function and will not be described in detail.
[0108] For the mean square error loss, when there are two target text features (recorded as the first label text feature and the second label text feature) and two projection features (recorded as the first projection feature and the second projection feature), the first similarity between each projection feature and each target text feature is determined one by one, that is, the cosine similarity between each projection feature and each text feature, and each two cosine similarities are aligned so that the two views and The projection features can correspond to text features respectively and
[0109] In this embodiment, a preset number of first similarities are aligned in pairs, specifically, the square of the difference between each two first similarities is calculated, as shown in formula (4):
[0110]
[0111] In formula (4), sim(xv1 , z v1 ) is the cosine similarity between the first projection feature and the first label text feature; sim(x v1 , z v2 ) is the cosine similarity between the first projection feature and the second label text feature; sim(x v2 , z v1 ) is the cosine similarity between the second projection feature and the first label text feature; sim(x v2 , z v2 ) is the cosine similarity between the second projection feature and the second label text feature. mse1 is sim(x v1 , z v1 ) and sim(x v1 , z v2 ) squared; l mse2 is sim(x v2 , z v1 ) and sim(x v2 , z v2 ) squared; l mse3 is sim(x v1 , z v1 ) and sim(x v2 , z v2 ) square of the difference; L MSEs is the mean square error loss function.
[0112] In the above embodiment, the one-to-one alignment between all target text features and all projection features has been covered in the processing of the preset number of first similarities, so all the first similarities are not aligned pairwise; in other embodiments, all the first similarities can also be aligned pairwise, and then the square of the difference between each two first similarities is calculated, and then the weighted average is taken to obtain the third loss function.
[0113] The present invention introduces a third loss function, so that the model can simultaneously consider the similarity between an image and multiple texts during training, thereby optimizing its parameters more comprehensively. This multi-level optimization helps the model converge to the optimal solution more quickly, reducing training time and resource consumption.
[0114] S4. Determine a comprehensive loss function according to the first loss function, the second loss function and the third loss function, and use the comprehensive loss function to train the attack detection model; the specific determination process of the comprehensive loss function is shown in formula (5):
[0115] L itma =L ce +L simCLR +L MSEs \*MERGEFORMAT(5)
[0116] In formula (5), L itma is the comprehensive loss function; L ce is the cross entropy loss function; L simCLR is the self-supervised contrastive learning loss function; L MSEs is the mean square error loss function.
[0117] By extracting the image features of the training images and the features of the real / distorted texts respectively and calculating the similarity between them, the present invention can more accurately capture the correlation between the images and the texts. This fine-grained feature matching helps the model better identify the mismatch between the distorted texts and the real images, thereby improving the accuracy of attack detection.
[0118] The present invention not only considers the similarity between a single real text and a distorted text and an image, but also comprehensively evaluates the relationship between all real texts and distorted texts and images through a second loss function. This comprehensive evaluation method helps the model learn more complex feature patterns, making it more robust and reliable in the face of diverse attack methods.
[0119] The present invention also discloses a training system for an attack detection model, comprising:
[0120] An image and text acquisition module is used to acquire training images, a real text set containing M real texts, and a false text set containing M false texts;
[0121] A first loss function determination module, used to determine a first loss function according to a training image, any real text, and any distorted text;
[0122] A second loss function and a third loss function determination module, used to determine the first similarities between the training image and the N real texts / N distorted texts respectively according to the authenticity of the training image, and determine the second loss function according to the N first similarities and a preset label matrix, and determine the third loss function according to the N first similarities;
[0123] The model training module is used to determine a comprehensive loss function based on the first loss function, the second loss function and the third loss function, and use the comprehensive loss function to train the attack detection model.
[0124] The present invention also discloses a cross-device performance evaluation method of an attack detection model, comprising the following steps:
[0125] Acquire a first preset number of data sets, each data set coming from a different public database;
[0126] Each data set is used as a test set in turn, and the remaining data sets are combined into a joint training set accordingly, to obtain a first preset number of training-test sets, each training-test set including a test set and a joint training set;
[0127] Using the training method of the attack detection model and the joint training set in each training-test set, the attack detection model is trained;
[0128] Using the test set in each training-test set and the corresponding trained attack detection model to perform attack detection, and obtain a first preset number of attack detection results;
[0129] Based on a first preset number of attack detection results, the cross-device performance of the attack training model is evaluated.
[0130] The cross-device performance evaluation method of the attack detection model of the present invention can reflect the adaptability of the model under different conditions through multi-device testing, provide more comprehensive performance feedback, and thus more accurately reflect the cross-device performance of the model and the actual deployment effect, thereby enhancing the generalization performance of the model.
[0131] In one embodiment, nine datasets from widely recognized public iris image databases are used for testing, and these datasets are collected by different devices. Table 2 shows the structure of each test-training set, and the letters in the first row represent the names of each dataset.
[0132] One of the datasets is selected as the test set, and the other eight datasets are used as joint training sets for cross-device experiments. To facilitate experimental comparison, each dataset is treated as an independent domain and tested separately. "√" means that the dataset is used as the test set, and "×" means that the dataset is used as the training set.
[0133] Table 2 Structure of each test-training set
[0134]
[0135]
[0136] Compared with the prior art, the present invention has the following beneficial effects:
[0137] (1) By calculating the similarity between the training image and multiple real texts / false texts, the model can better understand the semantic association between the image and multiple texts. This deep semantic understanding helps the model to more accurately determine whether the text truly describes the image content, thereby enhancing its detection ability in complex situations, which helps the model remain highly alert when faced with carefully constructed deceptive texts;
[0138] (2) By introducing multiple similarity calculations between real texts and distorted texts and images, the model can learn richer feature representations. These feature representations not only cover the direct relationship between images and texts, but also the potential relationship between them. This enables the model to better use the learned knowledge to make judgments when faced with new and unseen images and texts, thereby improving its generalization performance;
[0139] (3) The present invention involves multiple loss functions and constitutes a comprehensive loss function, which makes the model training process clearer and more organized, helps to reduce the uncertainty in the training process and improve the training efficiency.
[0140] The above are only a few embodiments of the present application and do not constitute any form of limitation to the present application. Although the present application is disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the profession, without departing from the scope of the technical solution of the present application, using the technical content disclosed above to make slight changes or modifications are equivalent to equivalent implementation cases and fall within the scope of the technical solution.
Claims
1. A training method for an attack detection model, characterized in that: The following steps are involved: Obtain training images, a real text set containing M real texts, and a false text set containing M false texts; Determine a first loss function according to the training image, any true text, and any distorted text; According to the authenticity of the training image, respectively determine the first similarities between the training image and the N real texts / N distorted texts, determine a second loss function according to the N first similarities and a preset label matrix, and determine a third loss function according to the N first similarities; A comprehensive loss function is determined according to the first loss function, the second loss function and the third loss function, and the attack detection model is trained using the comprehensive loss function.
2. The attack detection model training method according to claim 1, characterized in that: Determine a first loss function according to the training image, any real text and any distorted text, specifically: Respectively extracting image features of the training image, real text features of any real text, and distorted text features of any distorted text; Determine a second similarity based on the image feature and the real text feature of any real text; Determine a third similarity based on the image feature and the distorted text feature of any distorted text; A first loss function is determined according to the second similarity and the third similarity.
3. The attack detection model training method according to claim 2, characterized in that: The first loss function is determined according to the second similarity and the third similarity, specifically: Determining a model probability prediction distribution according to the second similarity and the third similarity; A first loss function is determined based on the model probability prediction distribution.
4. The training method of the attack detection model according to claim 1, characterized in that: According to the authenticity of the training image, the first similarity between the training image and the N real texts / N false texts is determined respectively, specifically: Extract X projection features of the training image; According to the authenticity of the training images, extract the target text features of N real texts / N false texts; By using matrix multiplication, the first similarity between each projection feature and each target text feature is determined one by one.
5. The attack detection model training method according to claim 4, characterized in that: The second loss function is determined according to the N first similarities and the preset label matrix, specifically: Determine a prediction matrix according to the N first similarities; A second loss function is determined according to the prediction matrix and the label matrix.
6. The training method of the attack detection model according to claim 4, characterized in that: The third loss function is determined according to the N first similarities, specifically: One by one, obtain the square of the difference between any two first similarities; A third loss function is determined based on the plurality of squared differences.
7. The method for training an attack detection model according to any one of claims 1 to 6, characterized in that: The first loss function is a cross entropy loss function, the second loss function is a self-supervised contrastive learning loss function, and the third loss function is a mean square error loss function.
8. A training system for an attack detection model, characterized in that: include: An image and text acquisition module is used to acquire training images, a real text set containing M real texts, and a false text set containing M false texts; A first loss function determination module, used to determine a first loss function according to the training image, any real text and any distorted text; A second loss function and a third loss function determination module, used to determine the first similarities between the training image and N real texts / N distorted texts respectively according to the authenticity of the training image, and determine the second loss function according to the N first similarities and a preset label matrix, and determine the third loss function according to the N first similarities; A model training module is used to determine a comprehensive loss function based on the first loss function, the second loss function and the third loss function, and use the comprehensive loss function to train the attack detection model.
9. A cross-device performance evaluation method for an attack detection model, characterized in that: The following steps are involved: Acquire a first preset number of data sets, each data set coming from a different public database; Each data set is used as a test set in turn, and the remaining data sets are combined into a joint training set accordingly, to obtain a first preset number of training-test sets, each training-test set including a test set and a joint training set; Using the attack detection model training method according to any one of claims 1 to 7 and the joint training set in each training-test set to train the attack detection model; Using the test set in each training-test set and the corresponding trained attack detection model to perform attack detection, and obtain a first preset number of attack detection results; Based on a first preset number of attack detection results, the cross-device performance of the attack training model is evaluated.